Merge pull request #78 from filoozom/patch-1

Update config.go to handle default slices correctly (fixes #76)

.gitignore

@@ -1,3 +1,4 @@
 syncthing
+syncthing.exe
 *.tar.gz
-dist
+*.zip

CONTRIBUTING.md

@@ -0,0 +1,22 @@
+Please do contribute!
+
+## Building
+
+[See the wiki](https://github.com/calmh/syncthing/wiki/Building)
+
+## Tests
+
+Yes please!
+
+## Style
+
+`go fmt`
+
+## Documentation
+
+[Hack it here](https://github.com/calmh/syncthing/wiki)
+
+## License
+
+MIT
+

LICENSE

@@ -1,4 +1,4 @@
-Copyright (C) 2013 Jakob Borg
+Copyright (C) 2013-2014 Jakob Borg
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of
 this software and associated documentation files (the "Software"), to deal in

README.md

@@ -1,18 +1,18 @@
-syncthing
+syncthing [![Build Status](https://drone.io/github.com/calmh/syncthing/status.png)](https://drone.io/github.com/calmh/syncthing/latest)
 =========
 
-This is `syncthing`, an open BitTorrent Sync alternative. It is
-currently far from ready for mass consumption, but it is a usable proof
-of concept and tech demo. The following are the project goals:
+This is the `syncthing` project. The following are the project goals:
 
- 1. Define an open, secure, language neutral protocol usable for
-    efficient synchronization of a file repository between an arbitrary
-    number of nodes. This is the [Block Exchange
-    Protocol](https://github.com/calmh/syncthing/blob/master/protocol/PROTOCOL.md)
-    (BEP).
+ 1. Define a protocol for synchronization of a file repository between a
+    number of collaborating nodes. The protocol should be well defined,
+    unambigous, easily understood, free to use, efficient, secure and
+    languange neutral. This is the [Block Exchange
+    Protocol](https://github.com/calmh/syncthing/blob/master/protocol/PROTOCOL.md).
 
  2. Provide the reference implementation to demonstrate the usability of
-    said protocol. This is the `syncthing` utility.
+    said protocol. This is the `syncthing` utility. It is the hope that
+    alternative, compatible implementations of the protocol will come to
+    exist.
 
 The two are evolving together; the protocol is not to be considered
 stable until syncthing 1.0 is released, at which point it is locked down
@@ -25,181 +25,18 @@ making sure large swarms of selfish agents behave and somehow work
 towards a common goal. Here we have a much smaller swarm of cooperative
 agents and a simpler approach will suffice.
 
-Features
---------
+Documentation
+=============
 
-> To request features and file bugs, see [the issue tracker][issues].
-
-The following features are _currently implemented and working_:
-
- * The formation of a cluster of nodes, certificate authenticated and
-   communicating over TLS over TCP.
-
- * Synchronization of a single directory among the cluster nodes.
-
- * Change detection by periodic scanning of the local repository.
-
- * Static configuration of cluster nodes.
-
- * Automatic discovery of cluster nodes. See [discover.go][discover.go]
-   for the protocol specification. Discovery on the LAN is performed by
-   broadcasts, Internet wide discovery is performed with the assistance
-   of a global server.
-
- * Handling of deleted files. Deletes can be propagated or ignored per
-   client.
-
- * Synchronizing multiple unrelated directory trees by following
-   symlinks directly below the repository level.
-
- * HTTP GUI.
-
-The following features are _not yet implemented but planned_:
-
- * Change detection by listening to file system notifications instead of
-   periodic scanning.
-
-The following features are _not implemented but may be implemented_ in
-the future:
-
- * Syncing multiple directories from the same syncthing instance.
-
- * Automatic NAT handling via UPNP.
-
- * Conflict resolution. Currently whichever file has the newest
-   modification time "wins". The correct behavior in the face of
-   conflicts is open for discussion.
-
-[discover.go]: https://github.com/calmh/syncthing/blob/master/discover/discover.go
-[issues]: https://github.com/calmh/syncthing/issues
-
-Security
---------
-
-Security is one of the primary project goals. This means that it should
-not be possible for an attacker to join a cluster uninvited, and it
-should not be possible to extract private information from intercepted
-traffic. Currently this is implemented as follows.
-
-All traffic is protected by TLS. To prevent uninvited nodes from joining
-a cluster, the certificate fingerprint of each node is compared to a
-preset list of acceptable nodes at connection establishment. The
-fingerprint is computed as the SHA-1 hash of the certificate and
-displayed in BASE32 encoding to form a compact yet convenient string.
-Currently SHA-1 is deemed secure against preimage attacks.
-
-Incoming requests for file data are verified to the extent that the
-requested file name must exist in the local index and the global model.
-
-Installing
-==========
-
-Download the appropriate precompiled binary from the
-[releases](https://github.com/calmh/syncthing/releases) page. Untar and
-put the `syncthing` binary somewhere convenient in your `$PATH`.
-
-If you are a developer and have Go 1.2 installed you can also install
-the latest version from source. `go get` works as expected but builds
-a binary without GUI capabilities. Use the included `build.sh` script
-without parameters to build a syncthing with GUI.
-
-Usage
-=====
-
-Check out the options:
-
-```
-$ syncthing --help
-Usage:
-  syncthing [options]
-
-...
-```
-
-Run syncthing to let it create it's config directory and certificate:
-
-```
-$ syncthing
-11:34:13 main.go:85: INFO: Version v0.1-40-gbb0fd87
-11:34:13 tls.go:61: OK: Created TLS certificate file
-11:34:13 tls.go:67: OK: Created TLS key file
-11:34:13 main.go:66: INFO: My ID: NCTBZAAHXR6ZZP3D7SL3DLYFFQERMW4Q
-11:34:13 main.go:90: FATAL: No config file
-```
-
-Take note of the "My ID: ..." line. Perform the same operation on
-another computer to create another node. Take note of that ID as well,
-and create a config file `~/.syncthing/syncthing.ini` looking something
-like this:
-
-```
-[repository]
-dir = /Users/jb/Synced
-
-[nodes]
-NCTBZAAHXR6ZZP3D7SL3DLYFFQERMW4Q = 172.16.32.1:22000 192.23.34.56:22000
-CUGAE43Y5N64CRJU26YFH6MTWPSBLSUL = dynamic
-```
-
-This assumes that the first node is reachable on either of the two
-addresses listed (perhaps one internal and one port-forwarded external)
-and that the other node is not normally reachable from the outside. Save
-this config file, identically, to both nodes.
-
-If the nodes are running on the same network, or reachable on port 22000
-from the outside world, you can set all addresses to "dynamic" and they
-will find each other using automatic discovery. (This discovery,
-including port numbers, can be tweaked or disabled using command line
-options.)
-
-Start syncthing on both nodes. For the cautious, one side can be set to
-be read only.
-
-```
-$ syncthing --ro
-13:30:55 main.go:85: INFO: Version v0.1-40-gbb0fd87
-13:30:55 main.go:102: INFO: My ID: NCTBZAAHXR6ZZP3D7SL3DLYFFQERMW4Q
-13:30:55 main.go:149: INFO: Initial repository scan in progress
-13:30:59 main.go:153: INFO: Listening for incoming connections
-13:30:59 main.go:157: INFO: Attempting to connect to other nodes
-13:30:59 main.go:247: INFO: Starting local discovery
-13:30:59 main.go:165: OK: Ready to synchronize
-13:31:04 discover.go:113: INFO: Discovered node CUGAE43Y5N64CRJU26YFH6MTWPSBLSUL at 172.16.32.24:22000
-13:31:14 main.go:296: INFO: Connected to node CUGAE43Y5N64CRJU26YFH6MTWPSBLSUL
-13:31:19 main.go:345: INFO: Transferred 139 KiB in (14 KiB/s), 139 KiB out (14 KiB/s)
-13:32:20 model.go:94: INFO: CUGAE43Y5N64CRJU26YFH6MTWPSBLSUL: 263.4 KB/s in, 69.1 KB/s out
-13:32:20 model.go:104: INFO:  18289 files,  24.24 GB in cluster
-13:32:20 model.go:111: INFO:  17132 files,  22.39 GB in local repo
-13:32:20 model.go:117: INFO:   1157 files,   1.84 GB to synchronize
-...
-```
-You should see the synchronization start and then finish a short while
-later. Add nodes to taste.
-
-GUI
----
-
-The web based GUI is disabled per default. To enable and access it you
-must start syncthing with the `--gui` command line option, giving a
-listen address. For example:
-
-```
-$ syncthing --gui 127.0.0.1:8080
-```
-
-You then point your browser to the given address.
-
-Excluding Files
----------------
-
-syncthing looks for files named `.stignore` while walking the
-repository. The file is expected to contain glob patterns of file names
-to ignore. Patterns are matched on file name only and apply to files in
-the same directory as the `.stignore` file and in directories lower down
-in the hierarchy.
+The syncthing documentation is kept on the
+[GitHub Wiki](https://github.com/calmh/syncthing/wiki).
 
 License
 =======
 
-MIT
+All documentation and protocol specifications are licensed
+under the [Creative Commons Attribution 4.0 International
+License](http://creativecommons.org/licenses/by/4.0/).
 
+All code is licensed under the [MIT
+License](https://github.com/calmh/syncthing/blob/master/LICENSE).

assets.sh

@@ -0,0 +1,27 @@
+#!/bin/bash
+
+cat <<EOT
+package auto
+
+import "compress/gzip"
+import "bytes"
+import "io/ioutil"
+
+var Assets = make(map[string][]byte)
+
+func init() {
+	var data []byte
+	var gr *gzip.Reader
+EOT
+
+cd gui
+for f in $(find . -type f) ; do 
+	f="${f#./}"
+	echo "gr, _ = gzip.NewReader(bytes.NewBuffer([]byte{"
+	gzip -n -c $f | od -vt x1 | sed 's/^[0-9a-f]*//' | sed 's/\([0-9a-f][0-9a-f]\)/0x\1,/g'
+	echo "}))"
+	echo "data, _ = ioutil.ReadAll(gr)"
+	echo "Assets[\"$f\"] = data"
+done
+echo "}"
+

build.sh

@@ -1,48 +1,91 @@
 #!/bin/bash
 
+export COPYFILE_DISABLE=true
+
+distFiles=(README.md LICENSE) # apart from the binary itself
 version=$(git describe --always)
-buildDir=dist
 
-if [[ -z $1 ]] ; then
+build() {
+	go build -ldflags "-w -X main.Version $version" ./cmd/syncthing	
+}
+
+prepare() {
+	./assets.sh | gofmt > auto/gui.files.go
+	go get -d
+}
+
+test() {
 	go test ./...
-	go build -ldflags "-X main.Version $version" \
-	&& nrsc syncthing gui
-else
-	go test ./... || exit 1
+}	
 
-	rm -rf "$buildDir"
-	mkdir -p "$buildDir" || exit 1
+tarDist() {
+	name="$1"
+	mkdir -p "$name"
+	cp syncthing "${distFiles[@]}" "$name"
+	tar zcvf "$name.tar.gz" "$name"
+	rm -rf "$name"
+}
 
-	for goos in darwin linux freebsd ; do
-		for goarch in amd64 386 ; do
-			echo "$goos-$goarch"
-			export GOOS="$goos"
-			export GOARCH="$goarch"
-			export name="syncthing-$goos-$goarch"
-			go build -ldflags "-X main.Version $version" \
-				&& nrsc syncthing gui \
-				&& mkdir -p "$name" \
-				&& cp syncthing "$buildDir/$name" \
-				&& cp README.md LICENSE "$name" \
-				&& mv syncthing "$name" \
-				&& tar zcf "$buildDir/$name.tar.gz" "$name" \
-				&& rm -r  "$name"
+zipDist() {
+	name="$1"
+	mkdir -p "$name"
+	cp syncthing.exe "${distFiles[@]}" "$name"
+	zip -r "$name.zip" "$name"
+	rm -rf "$name"
+}
+
+case "$1" in
+	"")
+		build
+		;;
+
+	tar)
+		rm -f *.tar.gz *.zip
+		prepare
+		test || exit 1
+		build
+
+		eval $(go env)
+		name="syncthing-$GOOS-$GOARCH-$version"
+
+		tarDist "$name"
+		;;
+
+	all)
+		rm -f *.tar.gz *.zip
+		prepare
+		test || exit 1
+
+		export GOARM=7
+		for os in darwin-amd64 linux-amd64 linux-arm freebsd-amd64 windows-amd64 ; do
+			export GOOS=${os%-*}
+			export GOARCH=${os#*-}
+
+			build
+			
+			name="syncthing-$os-$version"
+			case $GOOS in
+				windows)
+					zipDist "$name"
+					rm -f syncthing.exe
+					;;
+				*)
+					tarDist "$name"
+					rm -f syncthing
+					;;
+			esac
 		done
-	done
+		;;
 
-	for goos in windows ; do
-		for goarch in amd64 386 ; do
-			echo "$goos-$goarch"
-			export GOOS="$goos"
-			export GOARCH="$goarch"
-			export name="syncthing-$goos-$goarch"
-			go build -ldflags "-X main.Version $version" \
-				&& nrsc syncthing.exe gui \
-				&& mkdir -p "$name" \
-				&& cp syncthing.exe "$buildDir/$name.exe" \
-				&& cp README.md LICENSE "$name" \
-				&& zip -qr "$buildDir/$name.zip" "$name" \
-				&& rm -r  "$name"
+	upload)
+		tag=$(git describe)
+		shopt -s nullglob
+		for f in *gz *zip ; do
+			relup calmh/syncthing "$tag" "$f"
 		done
-	done
-fi
+		;;
+
+	*)
+		echo "Unknown build parameter $1"
+		;;
+esac

cid/cid.go

@@ -0,0 +1,42 @@
+package cid
+
+type Map struct {
+	toCid  map[string]int
+	toName []string
+}
+
+func NewMap() *Map {
+	return &Map{
+		toCid: make(map[string]int),
+	}
+}
+
+func (m *Map) Get(name string) int {
+	cid, ok := m.toCid[name]
+	if ok {
+		return cid
+	}
+
+	// Find a free slot to get a new ID
+	for i, n := range m.toName {
+		if n == "" {
+			m.toName[i] = name
+			m.toCid[name] = i
+			return i
+		}
+	}
+
+	// Add it to the end since we didn't find a free slot
+	m.toName = append(m.toName, name)
+	cid = len(m.toName) - 1
+	m.toCid[name] = cid
+	return cid
+}
+
+func (m *Map) Clear(name string) {
+	cid, ok := m.toCid[name]
+	if ok {
+		m.toName[cid] = ""
+		delete(m.toCid, name)
+	}
+}

cmd/.gitignore

@@ -0,0 +1 @@
+!syncthing

cmd/syncthing/blocks.go

@@ -1,4 +1,4 @@
-package model
+package main
 
 import (
 	"bytes"
@@ -7,17 +7,17 @@ import (
 )
 
 type Block struct {
-	Offset uint64
-	Length uint32
+	Offset int64
+	Size   uint32
 	Hash   []byte
 }
 
 // Blocks returns the blockwise hash of the reader.
 func Blocks(r io.Reader, blocksize int) ([]Block, error) {
 	var blocks []Block
-	var offset uint64
+	var offset int64
 	for {
-		lr := &io.LimitedReader{r, int64(blocksize)}
+		lr := &io.LimitedReader{R: r, N: int64(blocksize)}
 		hf := sha256.New()
 		n, err := io.Copy(hf, lr)
 		if err != nil {
@@ -30,11 +30,20 @@ func Blocks(r io.Reader, blocksize int) ([]Block, error) {
 
 		b := Block{
 			Offset: offset,
-			Length: uint32(n),
+			Size:   uint32(n),
 			Hash:   hf.Sum(nil),
 		}
 		blocks = append(blocks, b)
-		offset += uint64(n)
+		offset += int64(n)
+	}
+
+	if len(blocks) == 0 {
+		// Empty file
+		blocks = append(blocks, Block{
+			Offset: 0,
+			Size:   0,
+			Hash:   []uint8{0xe3, 0xb0, 0xc4, 0x42, 0x98, 0xfc, 0x1c, 0x14, 0x9a, 0xfb, 0xf4, 0xc8, 0x99, 0x6f, 0xb9, 0x24, 0x27, 0xae, 0x41, 0xe4, 0x64, 0x9b, 0x93, 0x4c, 0xa4, 0x95, 0x99, 0x1b, 0x78, 0x52, 0xb8, 0x55},
+		})
 	}
 
 	return blocks, nil

cmd/syncthing/blocks_test.go

@@ -1,4 +1,4 @@
-package model
+package main
 
 import (
 	"bytes"
@@ -11,7 +11,8 @@ var blocksTestData = []struct {
 	blocksize int
 	hash      []string
 }{
-	{[]byte(""), 1024, []string{}},
+	{[]byte(""), 1024, []string{
+		"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"}},
 	{[]byte("contents"), 1024, []string{
 		"d1b2a59fbea7e20077af9f91b27e95e865061b270be03ff539ab3b73587882e8"}},
 	{[]byte("contents"), 9, []string{
@@ -52,7 +53,7 @@ func TestBlocks(t *testing.T) {
 			t.Fatalf("Incorrect number of blocks %d != %d", l, len(test.hash))
 		} else {
 			i := 0
-			for off := uint64(0); off < uint64(len(test.data)); off += uint64(test.blocksize) {
+			for off := int64(0); off < int64(len(test.data)); off += int64(test.blocksize) {
 				if blocks[i].Offset != off {
 					t.Errorf("Incorrect offset for block %d: %d != %d", i, blocks[i].Offset, off)
 				}
@@ -61,8 +62,8 @@ func TestBlocks(t *testing.T) {
 				if rem := len(test.data) - int(off); bs > rem {
 					bs = rem
 				}
-				if int(blocks[i].Length) != bs {
-					t.Errorf("Incorrect length for block %d: %d != %d", i, blocks[i].Length, bs)
+				if int(blocks[i].Size) != bs {
+					t.Errorf("Incorrect length for block %d: %d != %d", i, blocks[i].Size, bs)
 				}
 				if h := fmt.Sprintf("%x", blocks[i].Hash); h != test.hash[i] {
 					t.Errorf("Incorrect block hash %q != %q", h, test.hash[i])
@@ -86,7 +87,7 @@ var diffTestData = []struct {
 	{"contents", "cantents", 3, []Block{{0, 3, nil}}},
 	{"contents", "contants", 3, []Block{{3, 3, nil}}},
 	{"contents", "cantants", 3, []Block{{0, 3, nil}, {3, 3, nil}}},
-	{"contents", "", 3, nil},
+	{"contents", "", 3, []Block{{0, 0, nil}}},
 	{"", "contents", 3, []Block{{0, 3, nil}, {3, 3, nil}, {6, 2, nil}}},
 	{"con", "contents", 3, []Block{{3, 3, nil}, {6, 2, nil}}},
 	{"contents", "con", 3, nil},
@@ -106,8 +107,8 @@ func TestDiff(t *testing.T) {
 				if d[j].Offset != test.d[j].Offset {
 					t.Errorf("Incorrect offset for diff %d block %d; %d != %d", i, j, d[j].Offset, test.d[j].Offset)
 				}
-				if d[j].Length != test.d[j].Length {
-					t.Errorf("Incorrect length for diff %d block %d; %d != %d", i, j, d[j].Length, test.d[j].Length)
+				if d[j].Size != test.d[j].Size {
+					t.Errorf("Incorrect length for diff %d block %d; %d != %d", i, j, d[j].Size, test.d[j].Size)
 				}
 			}
 		}

cmd/syncthing/config.go

@@ -0,0 +1,210 @@
+package main
+
+import (
+	"crypto/sha256"
+	"encoding/xml"
+	"fmt"
+	"io"
+	"reflect"
+	"sort"
+	"strconv"
+	"strings"
+)
+
+type Configuration struct {
+	Version      int                       `xml:"version,attr" default:"1"`
+	Repositories []RepositoryConfiguration `xml:"repository"`
+	Options      OptionsConfiguration      `xml:"options"`
+	XMLName      xml.Name                  `xml:"configuration" json:"-"`
+}
+
+type RepositoryConfiguration struct {
+	Directory string              `xml:"directory,attr"`
+	Nodes     []NodeConfiguration `xml:"node"`
+}
+
+type NodeConfiguration struct {
+	NodeID    string   `xml:"id,attr"`
+	Name      string   `xml:"name,attr"`
+	Addresses []string `xml:"address"`
+}
+
+type OptionsConfiguration struct {
+	ListenAddress      []string `xml:"listenAddress" default:":22000" ini:"listen-address"`
+	ReadOnly           bool     `xml:"readOnly" ini:"read-only"`
+	AllowDelete        bool     `xml:"allowDelete" default:"true" ini:"allow-delete"`
+	FollowSymlinks     bool     `xml:"followSymlinks" default:"true" ini:"follow-symlinks"`
+	GUIEnabled         bool     `xml:"guiEnabled" default:"true" ini:"gui-enabled"`
+	GUIAddress         string   `xml:"guiAddress" default:"127.0.0.1:8080" ini:"gui-address"`
+	GlobalAnnServer    string   `xml:"globalAnnounceServer" default:"announce.syncthing.net:22025" ini:"global-announce-server"`
+	GlobalAnnEnabled   bool     `xml:"globalAnnounceEnabled" default:"true" ini:"global-announce-enabled"`
+	LocalAnnEnabled    bool     `xml:"localAnnounceEnabled" default:"true" ini:"local-announce-enabled"`
+	ParallelRequests   int      `xml:"parallelRequests" default:"16" ini:"parallel-requests"`
+	MaxSendKbps        int      `xml:"maxSendKbps" ini:"max-send-kbps"`
+	RescanIntervalS    int      `xml:"rescanIntervalS" default:"60" ini:"rescan-interval"`
+	ReconnectIntervalS int      `xml:"reconnectionIntervalS" default:"60" ini:"reconnection-interval"`
+	MaxChangeKbps      int      `xml:"maxChangeKbps" default:"1000" ini:"max-change-bw"`
+}
+
+func setDefaults(data interface{}, setEmptySlices bool) error {
+	s := reflect.ValueOf(data).Elem()
+	t := s.Type()
+
+	for i := 0; i < s.NumField(); i++ {
+		f := s.Field(i)
+		tag := t.Field(i).Tag
+
+		v := tag.Get("default")
+		if len(v) > 0 {
+			if f.Kind().String() == "slice" && f.Len() != 0 {
+				continue
+			}
+
+			switch f.Interface().(type) {
+			case string:
+				f.SetString(v)
+
+			case []string:
+				if setEmptySlices {
+					rv := reflect.MakeSlice(reflect.TypeOf([]string{}), 1, 1)
+					rv.Index(0).SetString(v)
+					f.Set(rv)
+				}
+
+			case int:
+				i, err := strconv.ParseInt(v, 10, 64)
+				if err != nil {
+					return err
+				}
+				f.SetInt(i)
+
+			case bool:
+				f.SetBool(v == "true")
+
+			default:
+				panic(f.Type())
+			}
+		}
+	}
+	return nil
+}
+
+func readConfigINI(m map[string]string, data interface{}) error {
+	s := reflect.ValueOf(data).Elem()
+	t := s.Type()
+
+	for i := 0; i < s.NumField(); i++ {
+		f := s.Field(i)
+		tag := t.Field(i).Tag
+
+		name := tag.Get("ini")
+		if len(name) == 0 {
+			name = strings.ToLower(t.Field(i).Name)
+		}
+
+		if v, ok := m[name]; ok {
+			switch f.Interface().(type) {
+			case string:
+				f.SetString(v)
+
+			case int:
+				i, err := strconv.ParseInt(v, 10, 64)
+				if err == nil {
+					f.SetInt(i)
+				}
+
+			case bool:
+				f.SetBool(v == "true")
+
+			default:
+				panic(f.Type())
+			}
+		}
+	}
+	return nil
+}
+
+func writeConfigXML(wr io.Writer, cfg Configuration) error {
+	e := xml.NewEncoder(wr)
+	e.Indent("", "    ")
+	err := e.Encode(cfg)
+	if err != nil {
+		return err
+	}
+	_, err = wr.Write([]byte("\n"))
+	return err
+}
+
+func uniqueStrings(ss []string) []string {
+	var m = make(map[string]bool, len(ss))
+	for _, s := range ss {
+		m[s] = true
+	}
+
+	var us = make([]string, 0, len(m))
+	for k := range m {
+		us = append(us, k)
+	}
+
+	return us
+}
+
+func readConfigXML(rd io.Reader) (Configuration, error) {
+	var cfg Configuration
+
+	setDefaults(&cfg, false)
+	setDefaults(&cfg.Options, false)
+
+	var err error
+	if rd != nil {
+		err = xml.NewDecoder(rd).Decode(&cfg)
+	}
+
+	setDefaults(&cfg.Options, true)
+
+	cfg.Options.ListenAddress = uniqueStrings(cfg.Options.ListenAddress)
+	return cfg, err
+}
+
+type NodeConfigurationList []NodeConfiguration
+
+func (l NodeConfigurationList) Less(a, b int) bool {
+	return l[a].NodeID < l[b].NodeID
+}
+func (l NodeConfigurationList) Swap(a, b int) {
+	l[a], l[b] = l[b], l[a]
+}
+func (l NodeConfigurationList) Len() int {
+	return len(l)
+}
+
+func clusterHash(nodes []NodeConfiguration) string {
+	sort.Sort(NodeConfigurationList(nodes))
+	h := sha256.New()
+	for _, n := range nodes {
+		h.Write([]byte(n.NodeID))
+	}
+	return fmt.Sprintf("%x", h.Sum(nil))
+}
+
+func cleanNodeList(nodes []NodeConfiguration, myID string) []NodeConfiguration {
+	var myIDExists bool
+	for _, node := range nodes {
+		if node.NodeID == myID {
+			myIDExists = true
+			break
+		}
+	}
+
+	if !myIDExists {
+		nodes = append(nodes, NodeConfiguration{
+			NodeID:    myID,
+			Addresses: []string{"dynamic"},
+			Name:      "",
+		})
+	}
+
+	sort.Sort(NodeConfigurationList(nodes))
+
+	return nodes
+}

cmd/syncthing/filemonitor.go

@@ -0,0 +1,173 @@
+package main
+
+import (
+	"bytes"
+	"errors"
+	"fmt"
+	"log"
+	"os"
+	"path"
+	"sync"
+	"time"
+
+	"github.com/calmh/syncthing/buffers"
+)
+
+type fileMonitor struct {
+	name        string // in-repo name
+	path        string // full path
+	writeDone   sync.WaitGroup
+	model       *Model
+	global      File
+	localBlocks []Block
+	copyError   error
+	writeError  error
+}
+
+func (m *fileMonitor) FileBegins(cc <-chan content) error {
+	if m.model.trace["file"] {
+		log.Printf("FILE: FileBegins: " + m.name)
+	}
+
+	tmp := tempName(m.path, m.global.Modified)
+
+	dir := path.Dir(tmp)
+	_, err := os.Stat(dir)
+	if err != nil && os.IsNotExist(err) {
+		err = os.MkdirAll(dir, 0777)
+		if err != nil {
+			return err
+		}
+	}
+
+	outFile, err := os.Create(tmp)
+	if err != nil {
+		return err
+	}
+
+	m.writeDone.Add(1)
+
+	var writeWg sync.WaitGroup
+	if len(m.localBlocks) > 0 {
+		writeWg.Add(1)
+		inFile, err := os.Open(m.path)
+		if err != nil {
+			return err
+		}
+
+		// Copy local blocks, close infile when done
+		go m.copyLocalBlocks(inFile, outFile, &writeWg)
+	}
+
+	// Write remote blocks,
+	writeWg.Add(1)
+	go m.copyRemoteBlocks(cc, outFile, &writeWg)
+
+	// Wait for both writing routines, then close the outfile
+	go func() {
+		writeWg.Wait()
+		outFile.Close()
+		m.writeDone.Done()
+	}()
+
+	return nil
+}
+
+func (m *fileMonitor) copyLocalBlocks(inFile, outFile *os.File, writeWg *sync.WaitGroup) {
+	defer inFile.Close()
+	defer writeWg.Done()
+
+	var buf = buffers.Get(BlockSize)
+	defer buffers.Put(buf)
+
+	for _, lb := range m.localBlocks {
+		buf = buf[:lb.Size]
+		_, err := inFile.ReadAt(buf, lb.Offset)
+		if err != nil {
+			m.copyError = err
+			return
+		}
+		_, err = outFile.WriteAt(buf, lb.Offset)
+		if err != nil {
+			m.copyError = err
+			return
+		}
+	}
+}
+
+func (m *fileMonitor) copyRemoteBlocks(cc <-chan content, outFile *os.File, writeWg *sync.WaitGroup) {
+	defer writeWg.Done()
+
+	for content := range cc {
+		_, err := outFile.WriteAt(content.data, content.offset)
+		buffers.Put(content.data)
+		if err != nil {
+			m.writeError = err
+			return
+		}
+	}
+}
+
+func (m *fileMonitor) FileDone() error {
+	if m.model.trace["file"] {
+		log.Printf("FILE: FileDone: " + m.name)
+	}
+
+	m.writeDone.Wait()
+
+	tmp := tempName(m.path, m.global.Modified)
+	defer os.Remove(tmp)
+
+	if m.copyError != nil {
+		return m.copyError
+	}
+	if m.writeError != nil {
+		return m.writeError
+	}
+
+	err := hashCheck(tmp, m.global.Blocks)
+	if err != nil {
+		return err
+	}
+
+	err = os.Chtimes(tmp, time.Unix(m.global.Modified, 0), time.Unix(m.global.Modified, 0))
+	if err != nil {
+		return err
+	}
+
+	err = os.Chmod(tmp, os.FileMode(m.global.Flags&0777))
+	if err != nil {
+		return err
+	}
+
+	err = os.Rename(tmp, m.path)
+	if err != nil {
+		return err
+	}
+
+	m.model.updateLocal(m.global)
+	return nil
+}
+
+func hashCheck(name string, correct []Block) error {
+	rf, err := os.Open(name)
+	if err != nil {
+		return err
+	}
+	defer rf.Close()
+
+	current, err := Blocks(rf, BlockSize)
+	if err != nil {
+		return err
+	}
+	if len(current) != len(correct) {
+		return errors.New("incorrect number of blocks")
+	}
+	for i := range current {
+		if bytes.Compare(current[i].Hash, correct[i].Hash) != 0 {
+			return fmt.Errorf("hash mismatch: %x != %x", current[i], correct[i])
+		}
+	}
+
+	return nil
+}

cmd/syncthing/filequeue.go

@@ -0,0 +1,239 @@
+package main
+
+import (
+	"log"
+	"sort"
+	"sync"
+	"time"
+)
+
+type Monitor interface {
+	FileBegins(<-chan content) error
+	FileDone() error
+}
+
+type FileQueue struct {
+	files        queuedFileList
+	sorted       bool
+	fmut         sync.Mutex // protects files and sorted
+	availability map[string][]string
+	amut         sync.Mutex // protects availability
+	queued       map[string]bool
+}
+
+type queuedFile struct {
+	name         string
+	blocks       []Block
+	activeBlocks []bool
+	given        int
+	remaining    int
+	channel      chan content
+	nodes        []string
+	nodesChecked time.Time
+	monitor      Monitor
+}
+
+type content struct {
+	offset int64
+	data   []byte
+}
+
+type queuedFileList []queuedFile
+
+func (l queuedFileList) Len() int { return len(l) }
+
+func (l queuedFileList) Swap(a, b int) { l[a], l[b] = l[b], l[a] }
+
+func (l queuedFileList) Less(a, b int) bool {
+	// Sort by most blocks already given out, then alphabetically
+	if l[a].given != l[b].given {
+		return l[a].given > l[b].given
+	}
+	return l[a].name < l[b].name
+}
+
+type queuedBlock struct {
+	name  string
+	block Block
+	index int
+}
+
+func NewFileQueue() *FileQueue {
+	return &FileQueue{
+		availability: make(map[string][]string),
+		queued:       make(map[string]bool),
+	}
+}
+
+func (q *FileQueue) Add(name string, blocks []Block, monitor Monitor) {
+	q.fmut.Lock()
+	defer q.fmut.Unlock()
+
+	if q.queued[name] {
+		return
+	}
+
+	q.files = append(q.files, queuedFile{
+		name:         name,
+		blocks:       blocks,
+		activeBlocks: make([]bool, len(blocks)),
+		remaining:    len(blocks),
+		channel:      make(chan content),
+		monitor:      monitor,
+	})
+	q.queued[name] = true
+	q.sorted = false
+}
+
+func (q *FileQueue) Len() int {
+	q.fmut.Lock()
+	defer q.fmut.Unlock()
+
+	return len(q.files)
+}
+
+func (q *FileQueue) Get(nodeID string) (queuedBlock, bool) {
+	q.fmut.Lock()
+	defer q.fmut.Unlock()
+
+	if !q.sorted {
+		sort.Sort(q.files)
+		q.sorted = true
+	}
+
+	for i := range q.files {
+		qf := &q.files[i]
+
+		q.amut.Lock()
+		av := q.availability[qf.name]
+		q.amut.Unlock()
+
+		if len(av) == 0 {
+			// Noone has the file we want; abort.
+			if qf.remaining != len(qf.blocks) {
+				// We have already started on this file; close it down
+				close(qf.channel)
+				if mon := qf.monitor; mon != nil {
+					mon.FileDone()
+				}
+			}
+			delete(q.queued, qf.name)
+			q.deleteAt(i)
+			return queuedBlock{}, false
+		}
+
+		for _, ni := range av {
+			// Find and return the next block in the queue
+			if ni == nodeID {
+				for j, b := range qf.blocks {
+					if !qf.activeBlocks[j] {
+						qf.activeBlocks[j] = true
+						qf.given++
+						return queuedBlock{
+							name:  qf.name,
+							block: b,
+							index: j,
+						}, true
+					}
+				}
+				break
+			}
+		}
+	}
+
+	// We found nothing to do
+	return queuedBlock{}, false
+}
+
+func (q *FileQueue) Done(file string, offset int64, data []byte) {
+	q.fmut.Lock()
+	defer q.fmut.Unlock()
+
+	c := content{
+		offset: offset,
+		data:   data,
+	}
+	for i := range q.files {
+		qf := &q.files[i]
+
+		if qf.name == file {
+			if qf.monitor != nil && qf.remaining == len(qf.blocks) {
+				err := qf.monitor.FileBegins(qf.channel)
+				if err != nil {
+					log.Printf("WARNING: %s: %v (not synced)", qf.name, err)
+					delete(q.queued, qf.name)
+					q.deleteAt(i)
+					return
+				}
+			}
+
+			qf.channel <- c
+			qf.remaining--
+
+			if qf.remaining == 0 {
+				close(qf.channel)
+				if qf.monitor != nil {
+					err := qf.monitor.FileDone()
+					if err != nil {
+						log.Printf("WARNING: %s: %v", qf.name, err)
+					}
+				}
+				delete(q.queued, qf.name)
+				q.deleteAt(i)
+			}
+			return
+		}
+	}
+
+	// We found nothing, might have errored out already
+}
+
+func (q *FileQueue) QueuedFiles() (files []string) {
+	q.fmut.Lock()
+	defer q.fmut.Unlock()
+
+	for _, qf := range q.files {
+		files = append(files, qf.name)
+	}
+	return
+}
+
+func (q *FileQueue) deleteAt(i int) {
+	q.files = append(q.files[:i], q.files[i+1:]...)
+}
+
+func (q *FileQueue) deleteFile(n string) {
+	for i, file := range q.files {
+		if n == file.name {
+			q.deleteAt(i)
+			delete(q.queued, file.name)
+			return
+		}
+	}
+}
+
+func (q *FileQueue) SetAvailable(file string, nodes []string) {
+	q.amut.Lock()
+	defer q.amut.Unlock()
+
+	q.availability[file] = nodes
+}
+
+func (q *FileQueue) RemoveAvailable(toRemove string) {
+	q.fmut.Lock()
+	q.amut.Lock()
+	defer q.amut.Unlock()
+	defer q.fmut.Unlock()
+
+	for file, nodes := range q.availability {
+		for i, node := range nodes {
+			if node == toRemove {
+				q.availability[file] = nodes[:i+copy(nodes[i:], nodes[i+1:])]
+				if len(q.availability[file]) == 0 {
+					q.deleteFile(file)
+				}
+			}
+			break
+		}
+	}
+}

cmd/syncthing/filequeue_test.go

@@ -0,0 +1,295 @@
+package main
+
+import (
+	"reflect"
+	"sync"
+	"sync/atomic"
+	"testing"
+)
+
+func TestFileQueueAdd(t *testing.T) {
+	q := NewFileQueue()
+	q.Add("foo", nil, nil)
+}
+
+func TestFileQueueAddSorting(t *testing.T) {
+	q := NewFileQueue()
+	q.SetAvailable("zzz", []string{"nodeID"})
+	q.SetAvailable("aaa", []string{"nodeID"})
+
+	q.Add("zzz", []Block{{Offset: 0, Size: 128}, {Offset: 128, Size: 128}}, nil)
+	q.Add("aaa", []Block{{Offset: 0, Size: 128}, {Offset: 128, Size: 128}}, nil)
+	b, _ := q.Get("nodeID")
+	if b.name != "aaa" {
+		t.Errorf("Incorrectly sorted get: %+v", b)
+	}
+
+	q = NewFileQueue()
+	q.SetAvailable("zzz", []string{"nodeID"})
+	q.SetAvailable("aaa", []string{"nodeID"})
+
+	q.Add("zzz", []Block{{Offset: 0, Size: 128}, {Offset: 128, Size: 128}}, nil)
+	b, _ = q.Get("nodeID") // Start on zzzz
+	if b.name != "zzz" {
+		t.Errorf("Incorrectly sorted get: %+v", b)
+	}
+	q.Add("aaa", []Block{{Offset: 0, Size: 128}, {Offset: 128, Size: 128}}, nil)
+	b, _ = q.Get("nodeID")
+	if b.name != "zzz" {
+		// Continue rather than starting a new file
+		t.Errorf("Incorrectly sorted get: %+v", b)
+	}
+}
+
+func TestFileQueueLen(t *testing.T) {
+	q := NewFileQueue()
+	q.Add("foo", nil, nil)
+	q.Add("bar", nil, nil)
+
+	if l := q.Len(); l != 2 {
+		t.Errorf("Incorrect len %d != 2 after adds", l)
+	}
+}
+
+func TestFileQueueGet(t *testing.T) {
+	q := NewFileQueue()
+	q.SetAvailable("foo", []string{"nodeID"})
+	q.SetAvailable("bar", []string{"nodeID"})
+
+	q.Add("foo", []Block{
+		{Offset: 0, Size: 128, Hash: []byte("some foo hash bytes")},
+		{Offset: 128, Size: 128, Hash: []byte("some other foo hash bytes")},
+		{Offset: 256, Size: 128, Hash: []byte("more foo hash bytes")},
+	}, nil)
+	q.Add("bar", []Block{
+		{Offset: 0, Size: 128, Hash: []byte("some bar hash bytes")},
+		{Offset: 128, Size: 128, Hash: []byte("some other bar hash bytes")},
+	}, nil)
+
+	// First get should return the first block of the first file
+
+	expected := queuedBlock{
+		name: "bar",
+		block: Block{
+			Offset: 0,
+			Size:   128,
+			Hash:   []byte("some bar hash bytes"),
+		},
+	}
+	actual, ok := q.Get("nodeID")
+
+	if !ok {
+		t.Error("Unexpected non-OK Get()")
+	}
+	if !reflect.DeepEqual(expected, actual) {
+		t.Errorf("Incorrect block returned (first)\n  E: %+v\n  A: %+v", expected, actual)
+	}
+
+	// Second get should return the next block of the first file
+
+	expected = queuedBlock{
+		name: "bar",
+		block: Block{
+			Offset: 128,
+			Size:   128,
+			Hash:   []byte("some other bar hash bytes"),
+		},
+		index: 1,
+	}
+	actual, ok = q.Get("nodeID")
+
+	if !ok {
+		t.Error("Unexpected non-OK Get()")
+	}
+	if !reflect.DeepEqual(expected, actual) {
+		t.Errorf("Incorrect block returned (second)\n  E: %+v\n  A: %+v", expected, actual)
+	}
+
+	// Third get should return the first block of the second file
+
+	expected = queuedBlock{
+		name: "foo",
+		block: Block{
+			Offset: 0,
+			Size:   128,
+			Hash:   []byte("some foo hash bytes"),
+		},
+	}
+	actual, ok = q.Get("nodeID")
+
+	if !ok {
+		t.Error("Unexpected non-OK Get()")
+	}
+	if !reflect.DeepEqual(expected, actual) {
+		t.Errorf("Incorrect block returned (third)\n  E: %+v\n  A: %+v", expected, actual)
+	}
+}
+
+/*
+func TestFileQueueDone(t *testing.T) {
+	ch := make(chan content)
+	var recv sync.WaitGroup
+	recv.Add(1)
+	go func() {
+		content := <-ch
+		if bytes.Compare(content.data, []byte("first block bytes")) != 0 {
+			t.Error("Incorrect data in first content block")
+		}
+
+		content = <-ch
+		if bytes.Compare(content.data, []byte("second block bytes")) != 0 {
+			t.Error("Incorrect data in second content block")
+		}
+
+		_, ok := <-ch
+		if ok {
+			t.Error("Content channel not closed")
+		}
+
+		recv.Done()
+	}()
+
+	q := FileQueue{resolver: fakeResolver{}}
+	q.Add("foo", []Block{
+		{Offset: 0, Length: 128, Hash: []byte("some foo hash bytes")},
+		{Offset: 128, Length: 128, Hash: []byte("some other foo hash bytes")},
+	}, ch)
+
+	b0, _ := q.Get("nodeID")
+	b1, _ := q.Get("nodeID")
+
+	q.Done(b0.name, b0.block.Offset, []byte("first block bytes"))
+	q.Done(b1.name, b1.block.Offset, []byte("second block bytes"))
+
+	recv.Wait()
+
+	// Queue should now have one file less
+
+	if l := q.Len(); l != 0 {
+		t.Error("Queue not empty")
+	}
+
+	_, ok := q.Get("nodeID")
+	if ok {
+		t.Error("Unexpected OK Get()")
+	}
+}
+*/
+
+func TestFileQueueGetNodeIDs(t *testing.T) {
+	q := NewFileQueue()
+	q.SetAvailable("a-foo", []string{"nodeID", "a"})
+	q.SetAvailable("b-bar", []string{"nodeID", "b"})
+
+	q.Add("a-foo", []Block{
+		{Offset: 0, Size: 128, Hash: []byte("some foo hash bytes")},
+		{Offset: 128, Size: 128, Hash: []byte("some other foo hash bytes")},
+		{Offset: 256, Size: 128, Hash: []byte("more foo hash bytes")},
+	}, nil)
+	q.Add("b-bar", []Block{
+		{Offset: 0, Size: 128, Hash: []byte("some bar hash bytes")},
+		{Offset: 128, Size: 128, Hash: []byte("some other bar hash bytes")},
+	}, nil)
+
+	expected := queuedBlock{
+		name: "b-bar",
+		block: Block{
+			Offset: 0,
+			Size:   128,
+			Hash:   []byte("some bar hash bytes"),
+		},
+	}
+	actual, ok := q.Get("b")
+	if !ok {
+		t.Error("Unexpected non-OK Get()")
+	}
+	if !reflect.DeepEqual(expected, actual) {
+		t.Errorf("Incorrect block returned\n  E: %+v\n  A: %+v", expected, actual)
+	}
+
+	expected = queuedBlock{
+		name: "a-foo",
+		block: Block{
+			Offset: 0,
+			Size:   128,
+			Hash:   []byte("some foo hash bytes"),
+		},
+	}
+	actual, ok = q.Get("a")
+	if !ok {
+		t.Error("Unexpected non-OK Get()")
+	}
+	if !reflect.DeepEqual(expected, actual) {
+		t.Errorf("Incorrect block returned\n  E: %+v\n  A: %+v", expected, actual)
+	}
+
+	expected = queuedBlock{
+		name: "a-foo",
+		block: Block{
+			Offset: 128,
+			Size:   128,
+			Hash:   []byte("some other foo hash bytes"),
+		},
+		index: 1,
+	}
+	actual, ok = q.Get("nodeID")
+	if !ok {
+		t.Error("Unexpected non-OK Get()")
+	}
+	if !reflect.DeepEqual(expected, actual) {
+		t.Errorf("Incorrect block returned\n  E: %+v\n  A: %+v", expected, actual)
+	}
+}
+
+func TestFileQueueThreadHandling(t *testing.T) {
+	// This should pass with go test -race
+
+	const n = 100
+	var total int
+	var blocks []Block
+	for i := 1; i <= n; i++ {
+		blocks = append(blocks, Block{Offset: int64(i), Size: 1})
+		total += i
+	}
+
+	q := NewFileQueue()
+	q.Add("foo", blocks, nil)
+	q.SetAvailable("foo", []string{"nodeID"})
+
+	var start = make(chan bool)
+	var gotTot uint32
+	var wg sync.WaitGroup
+	wg.Add(n)
+	for i := 1; i <= n; i++ {
+		go func() {
+			<-start
+			b, _ := q.Get("nodeID")
+			atomic.AddUint32(&gotTot, uint32(b.block.Offset))
+			wg.Done()
+		}()
+	}
+
+	close(start)
+	wg.Wait()
+	if int(gotTot) != total {
+		t.Errorf("Total mismatch; %d != %d", gotTot, total)
+	}
+}
+
+func TestDeleteAt(t *testing.T) {
+	q := FileQueue{}
+
+	for i := 0; i < 4; i++ {
+		q.files = queuedFileList{{name: "a"}, {name: "b"}, {name: "c"}, {name: "d"}}
+		q.deleteAt(i)
+		if l := len(q.files); l != 3 {
+			t.Fatalf("deleteAt(%d) failed; %d != 3", i, l)
+		}
+	}
+
+	q.files = queuedFileList{{name: "a"}}
+	q.deleteAt(0)
+	if l := len(q.files); l != 0 {
+		t.Fatalf("deleteAt(only) failed; %d != 0", l)
+	}
+}

cmd/syncthing/gui.go

@@ -0,0 +1,172 @@
+package main
+
+import (
+	"encoding/json"
+	"io/ioutil"
+	"log"
+	"net/http"
+	"runtime"
+	"sync"
+	"time"
+
+	"github.com/codegangsta/martini"
+)
+
+type guiError struct {
+	Time  time.Time
+	Error string
+}
+
+var (
+	configInSync = true
+	guiErrors    = []guiError{}
+	guiErrorsMut sync.Mutex
+)
+
+func startGUI(addr string, m *Model) {
+	router := martini.NewRouter()
+	router.Get("/", getRoot)
+	router.Get("/rest/version", restGetVersion)
+	router.Get("/rest/model", restGetModel)
+	router.Get("/rest/connections", restGetConnections)
+	router.Get("/rest/config", restGetConfig)
+	router.Get("/rest/config/sync", restGetConfigInSync)
+	router.Get("/rest/need", restGetNeed)
+	router.Get("/rest/system", restGetSystem)
+	router.Get("/rest/errors", restGetErrors)
+
+	router.Post("/rest/config", restPostConfig)
+	router.Post("/rest/restart", restPostRestart)
+	router.Post("/rest/error", restPostError)
+
+	go func() {
+		mr := martini.New()
+		mr.Use(embeddedStatic())
+		mr.Use(martini.Recovery())
+		mr.Action(router.Handle)
+		mr.Map(m)
+		err := http.ListenAndServe(addr, mr)
+		if err != nil {
+			warnln("GUI not possible:", err)
+		}
+	}()
+}
+
+func getRoot(w http.ResponseWriter, r *http.Request) {
+	http.Redirect(w, r, "/index.html", 302)
+}
+
+func restGetVersion() string {
+	return Version
+}
+
+func restGetModel(m *Model, w http.ResponseWriter) {
+	var res = make(map[string]interface{})
+
+	globalFiles, globalDeleted, globalBytes := m.GlobalSize()
+	res["globalFiles"], res["globalDeleted"], res["globalBytes"] = globalFiles, globalDeleted, globalBytes
+
+	localFiles, localDeleted, localBytes := m.LocalSize()
+	res["localFiles"], res["localDeleted"], res["localBytes"] = localFiles, localDeleted, localBytes
+
+	inSyncFiles, inSyncBytes := m.InSyncSize()
+	res["inSyncFiles"], res["inSyncBytes"] = inSyncFiles, inSyncBytes
+
+	files, total := m.NeedFiles()
+	res["needFiles"], res["needBytes"] = len(files), total
+
+	w.Header().Set("Content-Type", "application/json")
+	json.NewEncoder(w).Encode(res)
+}
+
+func restGetConnections(m *Model, w http.ResponseWriter) {
+	var res = m.ConnectionStats()
+	w.Header().Set("Content-Type", "application/json")
+	json.NewEncoder(w).Encode(res)
+}
+
+func restGetConfig(w http.ResponseWriter) {
+	json.NewEncoder(w).Encode(cfg)
+}
+
+func restPostConfig(req *http.Request) {
+	err := json.NewDecoder(req.Body).Decode(&cfg)
+	if err != nil {
+		log.Println(err)
+	} else {
+		saveConfig()
+		configInSync = false
+	}
+}
+
+func restGetConfigInSync(w http.ResponseWriter) {
+	json.NewEncoder(w).Encode(map[string]bool{"configInSync": configInSync})
+}
+
+func restPostRestart(req *http.Request) {
+	restart()
+}
+
+type guiFile File
+
+func (f guiFile) MarshalJSON() ([]byte, error) {
+	type t struct {
+		Name string
+		Size int64
+	}
+	return json.Marshal(t{
+		Name: f.Name,
+		Size: File(f).Size,
+	})
+}
+
+func restGetNeed(m *Model, w http.ResponseWriter) {
+	files, _ := m.NeedFiles()
+	gfs := make([]guiFile, len(files))
+	for i, f := range files {
+		gfs[i] = guiFile(f)
+	}
+	w.Header().Set("Content-Type", "application/json")
+	json.NewEncoder(w).Encode(gfs)
+}
+
+var cpuUsagePercent float64
+var cpuUsageLock sync.RWMutex
+
+func restGetSystem(w http.ResponseWriter) {
+	var m runtime.MemStats
+	runtime.ReadMemStats(&m)
+
+	res := make(map[string]interface{})
+	res["myID"] = myID
+	res["goroutines"] = runtime.NumGoroutine()
+	res["alloc"] = m.Alloc
+	res["sys"] = m.Sys
+	cpuUsageLock.RLock()
+	res["cpuPercent"] = cpuUsagePercent
+	cpuUsageLock.RUnlock()
+
+	w.Header().Set("Content-Type", "application/json")
+	json.NewEncoder(w).Encode(res)
+}
+
+func restGetErrors(w http.ResponseWriter) {
+	guiErrorsMut.Lock()
+	json.NewEncoder(w).Encode(guiErrors)
+	guiErrorsMut.Unlock()
+}
+
+func restPostError(req *http.Request) {
+	bs, _ := ioutil.ReadAll(req.Body)
+	req.Body.Close()
+	showGuiError(string(bs))
+}
+
+func showGuiError(err string) {
+	guiErrorsMut.Lock()
+	guiErrors = append(guiErrors, guiError{time.Now(), err})
+	if len(guiErrors) > 5 {
+		guiErrors = guiErrors[len(guiErrors)-5:]
+	}
+	guiErrorsMut.Unlock()
+}

cmd/syncthing/gui_development.go

@@ -0,0 +1,9 @@
+//+build guidev
+
+package main
+
+import "github.com/codegangsta/martini"
+
+func embeddedStatic() interface{} {
+	return martini.Static("gui")
+}

cmd/syncthing/gui_embedded.go

@@ -0,0 +1,40 @@
+//+build !guidev
+
+package main
+
+import (
+	"fmt"
+	"log"
+	"mime"
+	"net/http"
+	"path/filepath"
+	"time"
+
+	"github.com/calmh/syncthing/auto"
+)
+
+func embeddedStatic() interface{} {
+	var modt = time.Now().UTC().Format(http.TimeFormat)
+
+	return func(res http.ResponseWriter, req *http.Request, log *log.Logger) {
+		file := req.URL.Path
+
+		if file[0] == '/' {
+			file = file[1:]
+		}
+
+		bs, ok := auto.Assets[file]
+		if !ok {
+			return
+		}
+
+		mtype := mime.TypeByExtension(filepath.Ext(req.URL.Path))
+		if len(mtype) != 0 {
+			res.Header().Set("Content-Type", mtype)
+		}
+		res.Header().Set("Content-Size", fmt.Sprintf("%d", len(bs)))
+		res.Header().Set("Last-Modified", modt)
+
+		res.Write(bs)
+	}
+}

cmd/syncthing/gui_unix.go

@@ -0,0 +1,31 @@
+//+build !windows,!solaris
+
+package main
+
+import (
+	"syscall"
+	"time"
+)
+
+func init() {
+	go trackCPUUsage()
+}
+
+func trackCPUUsage() {
+	var prevUsage int64
+	var prevTime = time.Now().UnixNano()
+	var rusage syscall.Rusage
+	for {
+		time.Sleep(10 * time.Second)
+		syscall.Getrusage(syscall.RUSAGE_SELF, &rusage)
+		curTime := time.Now().UnixNano()
+		timeDiff := curTime - prevTime
+		curUsage := rusage.Utime.Nano() + rusage.Stime.Nano()
+		usageDiff := curUsage - prevUsage
+		cpuUsageLock.Lock()
+		cpuUsagePercent = 100 * float64(usageDiff) / float64(timeDiff)
+		cpuUsageLock.Unlock()
+		prevTime = curTime
+		prevUsage = curUsage
+	}
+}

cmd/syncthing/logger.go

@@ -6,7 +6,12 @@ import (
 	"os"
 )
 
-var logger = log.New(os.Stderr, "", log.Ltime)
+var logger *log.Logger
+
+func init() {
+	log.SetOutput(os.Stderr)
+	logger = log.New(os.Stderr, "", log.Flags())
+}
 
 func debugln(vals ...interface{}) {
 	s := fmt.Sprintln(vals...)
@@ -40,11 +45,13 @@ func okf(format string, vals ...interface{}) {
 
 func warnln(vals ...interface{}) {
 	s := fmt.Sprintln(vals...)
+	showGuiError(s)
 	logger.Output(2, "WARNING: "+s)
 }
 
 func warnf(format string, vals ...interface{}) {
 	s := fmt.Sprintf(format, vals...)
+	showGuiError(s)
 	logger.Output(2, "WARNING: "+s)
 }
 

cmd/syncthing/main.go

@@ -0,0 +1,598 @@
+package main
+
+import (
+	"compress/gzip"
+	"crypto/tls"
+	"flag"
+	"fmt"
+	"log"
+	"net"
+	"net/http"
+	_ "net/http/pprof"
+	"os"
+	"os/exec"
+	"path"
+	"runtime"
+	"runtime/debug"
+	"strconv"
+	"strings"
+	"time"
+
+	"github.com/calmh/ini"
+	"github.com/calmh/syncthing/discover"
+	"github.com/calmh/syncthing/protocol"
+)
+
+var cfg Configuration
+var Version = "unknown-dev"
+
+var (
+	myID string
+)
+
+var (
+	showVersion  bool
+	confDir      string
+	trace        string
+	profiler     string
+	verbose      bool
+	startupDelay int
+)
+
+func main() {
+	flag.StringVar(&confDir, "home", getDefaultConfDir(), "Set configuration directory")
+	flag.StringVar(&trace, "debug.trace", "", "(connect,net,idx,file,pull)")
+	flag.StringVar(&profiler, "debug.profiler", "", "(addr)")
+	flag.BoolVar(&showVersion, "version", false, "Show version")
+	flag.BoolVar(&verbose, "v", false, "Be more verbose")
+	flag.IntVar(&startupDelay, "delay", 0, "Startup delay (s)")
+	flag.Usage = usageFor(flag.CommandLine, "syncthing [options]")
+	flag.Parse()
+
+	if startupDelay > 0 {
+		time.Sleep(time.Duration(startupDelay) * time.Second)
+	}
+
+	if showVersion {
+		fmt.Println(Version)
+		os.Exit(0)
+	}
+
+	if len(os.Getenv("GOGC")) == 0 {
+		debug.SetGCPercent(25)
+	}
+
+	if len(os.Getenv("GOMAXPROCS")) == 0 {
+		runtime.GOMAXPROCS(runtime.NumCPU())
+	}
+
+	if len(trace) > 0 {
+		log.SetFlags(log.Lshortfile | log.Ldate | log.Ltime | log.Lmicroseconds)
+		logger.SetFlags(log.Lshortfile | log.Ldate | log.Ltime | log.Lmicroseconds)
+	}
+	confDir = expandTilde(confDir)
+
+	// Ensure that our home directory exists and that we have a certificate and key.
+
+	ensureDir(confDir, 0700)
+	cert, err := loadCert(confDir)
+	if err != nil {
+		newCertificate(confDir)
+		cert, err = loadCert(confDir)
+		fatalErr(err)
+	}
+
+	myID = string(certID(cert.Certificate[0]))
+	log.SetPrefix("[" + myID[0:5] + "] ")
+	logger.SetPrefix("[" + myID[0:5] + "] ")
+
+	infoln("Version", Version)
+	infoln("My ID:", myID)
+
+	// Prepare to be able to save configuration
+
+	cfgFile := path.Join(confDir, "config.xml")
+	go saveConfigLoop(cfgFile)
+
+	// Load the configuration file, if it exists.
+	// If it does not, create a template.
+
+	cf, err := os.Open(cfgFile)
+	if err == nil {
+		// Read config.xml
+		cfg, err = readConfigXML(cf)
+		if err != nil {
+			fatalln(err)
+		}
+		cf.Close()
+	} else {
+		// No config.xml, let's try the old syncthing.ini
+		iniFile := path.Join(confDir, "syncthing.ini")
+		cf, err := os.Open(iniFile)
+		if err == nil {
+			infoln("Migrating syncthing.ini to config.xml")
+			iniCfg := ini.Parse(cf)
+			cf.Close()
+			os.Rename(iniFile, path.Join(confDir, "migrated_syncthing.ini"))
+
+			cfg, _ = readConfigXML(nil)
+			cfg.Repositories = []RepositoryConfiguration{
+				{Directory: iniCfg.Get("repository", "dir")},
+			}
+			readConfigINI(iniCfg.OptionMap("settings"), &cfg.Options)
+			for name, addrs := range iniCfg.OptionMap("nodes") {
+				n := NodeConfiguration{
+					NodeID:    name,
+					Addresses: strings.Fields(addrs),
+				}
+				cfg.Repositories[0].Nodes = append(cfg.Repositories[0].Nodes, n)
+			}
+
+			saveConfig()
+		}
+	}
+
+	if len(cfg.Repositories) == 0 {
+		infoln("No config file; starting with empty defaults")
+
+		cfg, err = readConfigXML(nil)
+		cfg.Repositories = []RepositoryConfiguration{
+			{
+				Directory: path.Join(getHomeDir(), "Sync"),
+				Nodes: []NodeConfiguration{
+					{NodeID: myID, Addresses: []string{"dynamic"}},
+				},
+			},
+		}
+
+		saveConfig()
+		infof("Edit %s to taste or use the GUI\n", cfgFile)
+	}
+
+	// Make sure the local node is in the node list.
+	cfg.Repositories[0].Nodes = cleanNodeList(cfg.Repositories[0].Nodes, myID)
+
+	var dir = expandTilde(cfg.Repositories[0].Directory)
+
+	if len(profiler) > 0 {
+		go func() {
+			err := http.ListenAndServe(profiler, nil)
+			if err != nil {
+				warnln(err)
+			}
+		}()
+	}
+
+	// The TLS configuration is used for both the listening socket and outgoing
+	// connections.
+
+	tlsCfg := &tls.Config{
+		Certificates:           []tls.Certificate{cert},
+		NextProtos:             []string{"bep/1.0"},
+		ServerName:             myID,
+		ClientAuth:             tls.RequestClientCert,
+		SessionTicketsDisabled: true,
+		InsecureSkipVerify:     true,
+		MinVersion:             tls.VersionTLS12,
+	}
+
+	ensureDir(dir, -1)
+	m := NewModel(dir, cfg.Options.MaxChangeKbps*1000)
+	for _, t := range strings.Split(trace, ",") {
+		m.Trace(t)
+	}
+	if cfg.Options.MaxSendKbps > 0 {
+		m.LimitRate(cfg.Options.MaxSendKbps)
+	}
+
+	// GUI
+	if cfg.Options.GUIEnabled && cfg.Options.GUIAddress != "" {
+		addr, err := net.ResolveTCPAddr("tcp", cfg.Options.GUIAddress)
+		if err != nil {
+			warnf("Cannot start GUI on %q: %v", cfg.Options.GUIAddress, err)
+		} else {
+			var hostOpen, hostShow string
+			switch {
+			case addr.IP == nil:
+				hostOpen = "localhost"
+				hostShow = "0.0.0.0"
+			case addr.IP.IsUnspecified():
+				hostOpen = "localhost"
+				hostShow = addr.IP.String()
+			default:
+				hostOpen = addr.IP.String()
+				hostShow = hostOpen
+			}
+
+			infof("Starting web GUI on http://%s:%d/", hostShow, addr.Port)
+			startGUI(cfg.Options.GUIAddress, m)
+			openURL(fmt.Sprintf("http://%s:%d", hostOpen, addr.Port))
+		}
+	}
+
+	// Walk the repository and update the local model before establishing any
+	// connections to other nodes.
+
+	if verbose {
+		infoln("Populating repository index")
+	}
+	loadIndex(m)
+	updateLocalModel(m)
+
+	connOpts := map[string]string{
+		"clientId":      "syncthing",
+		"clientVersion": Version,
+		"clusterHash":   clusterHash(cfg.Repositories[0].Nodes),
+	}
+
+	// Routine to listen for incoming connections
+	if verbose {
+		infoln("Listening for incoming connections")
+	}
+	for _, addr := range cfg.Options.ListenAddress {
+		go listen(myID, addr, m, tlsCfg, connOpts)
+	}
+
+	// Routine to connect out to configured nodes
+	if verbose {
+		infoln("Attempting to connect to other nodes")
+	}
+	disc := discovery(cfg.Options.ListenAddress[0])
+	go connect(myID, disc, m, tlsCfg, connOpts)
+
+	// Routine to pull blocks from other nodes to synchronize the local
+	// repository. Does not run when we are in read only (publish only) mode.
+	if !cfg.Options.ReadOnly {
+		if verbose {
+			if cfg.Options.AllowDelete {
+				infoln("Deletes from peer nodes are allowed")
+			} else {
+				infoln("Deletes from peer nodes will be ignored")
+			}
+			okln("Ready to synchronize (read-write)")
+		}
+		m.StartRW(cfg.Options.AllowDelete, cfg.Options.ParallelRequests)
+	} else if verbose {
+		okln("Ready to synchronize (read only; no external updates accepted)")
+	}
+
+	// Periodically scan the repository and update the local
+	// XXX: Should use some fsnotify mechanism.
+	go func() {
+		td := time.Duration(cfg.Options.RescanIntervalS) * time.Second
+		for {
+			time.Sleep(td)
+			if m.LocalAge() > (td / 2).Seconds() {
+				updateLocalModel(m)
+			}
+		}
+	}()
+
+	if verbose {
+		// Periodically print statistics
+		go printStatsLoop(m)
+	}
+
+	select {}
+}
+
+func restart() {
+	infoln("Restarting")
+	args := os.Args
+	doAppend := true
+	for _, arg := range args {
+		if arg == "-delay" {
+			doAppend = false
+			break
+		}
+	}
+	if doAppend {
+		args = append(args, "-delay", "2")
+	}
+	pgm, err := exec.LookPath(os.Args[0])
+	if err != nil {
+		warnln(err)
+		return
+	}
+	proc, err := os.StartProcess(pgm, args, &os.ProcAttr{
+		Env:   os.Environ(),
+		Files: []*os.File{os.Stdin, os.Stdout, os.Stderr},
+	})
+	if err != nil {
+		fatalln(err)
+	}
+	proc.Release()
+	os.Exit(0)
+}
+
+var saveConfigCh = make(chan struct{})
+
+func saveConfigLoop(cfgFile string) {
+	for _ = range saveConfigCh {
+		fd, err := os.Create(cfgFile + ".tmp")
+		if err != nil {
+			warnln(err)
+			continue
+		}
+
+		err = writeConfigXML(fd, cfg)
+		if err != nil {
+			warnln(err)
+			fd.Close()
+			continue
+		}
+
+		err = fd.Close()
+		if err != nil {
+			warnln(err)
+			continue
+		}
+
+		if runtime.GOOS == "windows" {
+			err := os.Remove(cfgFile)
+			if err != nil && !os.IsNotExist(err) {
+				warnln(err)
+			}
+		}
+
+		err = os.Rename(cfgFile+".tmp", cfgFile)
+		if err != nil {
+			warnln(err)
+		}
+	}
+}
+
+func saveConfig() {
+	saveConfigCh <- struct{}{}
+}
+
+func printStatsLoop(m *Model) {
+	var lastUpdated int64
+	var lastStats = make(map[string]ConnectionInfo)
+
+	for {
+		time.Sleep(60 * time.Second)
+
+		for node, stats := range m.ConnectionStats() {
+			secs := time.Since(lastStats[node].At).Seconds()
+			inbps := 8 * int(float64(stats.InBytesTotal-lastStats[node].InBytesTotal)/secs)
+			outbps := 8 * int(float64(stats.OutBytesTotal-lastStats[node].OutBytesTotal)/secs)
+
+			if inbps+outbps > 0 {
+				infof("%s: %sb/s in, %sb/s out", node[0:5], MetricPrefix(int64(inbps)), MetricPrefix(int64(outbps)))
+			}
+
+			lastStats[node] = stats
+		}
+
+		if lu := m.Generation(); lu > lastUpdated {
+			lastUpdated = lu
+			files, _, bytes := m.GlobalSize()
+			infof("%6d files, %9sB in cluster", files, BinaryPrefix(bytes))
+			files, _, bytes = m.LocalSize()
+			infof("%6d files, %9sB in local repo", files, BinaryPrefix(bytes))
+			needFiles, bytes := m.NeedFiles()
+			infof("%6d files, %9sB to synchronize", len(needFiles), BinaryPrefix(bytes))
+		}
+	}
+}
+
+func listen(myID string, addr string, m *Model, tlsCfg *tls.Config, connOpts map[string]string) {
+	if strings.Contains(trace, "connect") {
+		debugln("NET: Listening on", addr)
+	}
+	l, err := tls.Listen("tcp", addr, tlsCfg)
+	fatalErr(err)
+
+listen:
+	for {
+		conn, err := l.Accept()
+		if err != nil {
+			warnln(err)
+			continue
+		}
+
+		if strings.Contains(trace, "connect") {
+			debugln("NET: Connect from", conn.RemoteAddr())
+		}
+
+		tc := conn.(*tls.Conn)
+		err = tc.Handshake()
+		if err != nil {
+			warnln(err)
+			tc.Close()
+			continue
+		}
+
+		remoteID := certID(tc.ConnectionState().PeerCertificates[0].Raw)
+
+		if remoteID == myID {
+			warnf("Connect from myself (%s) - should not happen", remoteID)
+			conn.Close()
+			continue
+		}
+
+		if m.ConnectedTo(remoteID) {
+			warnf("Connect from connected node (%s)", remoteID)
+		}
+
+		for _, nodeCfg := range cfg.Repositories[0].Nodes {
+			if nodeCfg.NodeID == remoteID {
+				protoConn := protocol.NewConnection(remoteID, conn, conn, m, connOpts)
+				m.AddConnection(conn, protoConn)
+				continue listen
+			}
+		}
+		conn.Close()
+	}
+}
+
+func discovery(addr string) *discover.Discoverer {
+	_, portstr, err := net.SplitHostPort(addr)
+	fatalErr(err)
+	port, _ := strconv.Atoi(portstr)
+
+	if !cfg.Options.LocalAnnEnabled {
+		port = -1
+	} else if verbose {
+		infoln("Sending local discovery announcements")
+	}
+
+	if !cfg.Options.GlobalAnnEnabled {
+		cfg.Options.GlobalAnnServer = ""
+	} else if verbose {
+		infoln("Sending external discovery announcements")
+	}
+
+	disc, err := discover.NewDiscoverer(myID, port, cfg.Options.GlobalAnnServer)
+
+	if err != nil {
+		warnf("No discovery possible (%v)", err)
+	}
+
+	return disc
+}
+
+func connect(myID string, disc *discover.Discoverer, m *Model, tlsCfg *tls.Config, connOpts map[string]string) {
+	for {
+	nextNode:
+		for _, nodeCfg := range cfg.Repositories[0].Nodes {
+			if nodeCfg.NodeID == myID {
+				continue
+			}
+			if m.ConnectedTo(nodeCfg.NodeID) {
+				continue
+			}
+			for _, addr := range nodeCfg.Addresses {
+				if addr == "dynamic" {
+					if disc != nil {
+						t := disc.Lookup(nodeCfg.NodeID)
+						if len(t) == 0 {
+							continue
+						}
+						addr = t[0] //XXX: Handle all of them
+					}
+				}
+
+				if strings.Contains(trace, "connect") {
+					debugln("NET: Dial", nodeCfg.NodeID, addr)
+				}
+				conn, err := tls.Dial("tcp", addr, tlsCfg)
+				if err != nil {
+					if strings.Contains(trace, "connect") {
+						debugln("NET:", err)
+					}
+					continue
+				}
+
+				remoteID := certID(conn.ConnectionState().PeerCertificates[0].Raw)
+				if remoteID != nodeCfg.NodeID {
+					warnln("Unexpected nodeID", remoteID, "!=", nodeCfg.NodeID)
+					conn.Close()
+					continue
+				}
+
+				protoConn := protocol.NewConnection(remoteID, conn, conn, m, connOpts)
+				m.AddConnection(conn, protoConn)
+				continue nextNode
+			}
+		}
+
+		time.Sleep(time.Duration(cfg.Options.ReconnectIntervalS) * time.Second)
+	}
+}
+
+func updateLocalModel(m *Model) {
+	files, _ := m.Walk(cfg.Options.FollowSymlinks)
+	m.ReplaceLocal(files)
+	saveIndex(m)
+}
+
+func saveIndex(m *Model) {
+	name := m.RepoID() + ".idx.gz"
+	fullName := path.Join(confDir, name)
+	idxf, err := os.Create(fullName + ".tmp")
+	if err != nil {
+		return
+	}
+
+	gzw := gzip.NewWriter(idxf)
+
+	protocol.IndexMessage{
+		Repository: "local",
+		Files:      m.ProtocolIndex(),
+	}.EncodeXDR(gzw)
+	gzw.Close()
+	idxf.Close()
+	os.Rename(fullName+".tmp", fullName)
+}
+
+func loadIndex(m *Model) {
+	name := m.RepoID() + ".idx.gz"
+	idxf, err := os.Open(path.Join(confDir, name))
+	if err != nil {
+		return
+	}
+	defer idxf.Close()
+
+	gzr, err := gzip.NewReader(idxf)
+	if err != nil {
+		return
+	}
+	defer gzr.Close()
+
+	var im protocol.IndexMessage
+	err = im.DecodeXDR(gzr)
+	if err != nil || im.Repository != "local" {
+		return
+	}
+	m.SeedLocal(im.Files)
+}
+
+func ensureDir(dir string, mode int) {
+	fi, err := os.Stat(dir)
+	if os.IsNotExist(err) {
+		err := os.MkdirAll(dir, 0700)
+		fatalErr(err)
+	} else if mode >= 0 && err == nil && int(fi.Mode()&0777) != mode {
+		err := os.Chmod(dir, os.FileMode(mode))
+		fatalErr(err)
+	}
+}
+
+func expandTilde(p string) string {
+	if runtime.GOOS == "windows" {
+		return p
+	}
+
+	if strings.HasPrefix(p, "~/") {
+		return strings.Replace(p, "~", getUnixHomeDir(), 1)
+	}
+	return p
+}
+
+func getUnixHomeDir() string {
+	home := os.Getenv("HOME")
+	if home == "" {
+		fatalln("No home directory?")
+	}
+	return home
+}
+
+func getHomeDir() string {
+	if runtime.GOOS == "windows" {
+		home := os.Getenv("HOMEDRIVE") + os.Getenv("HOMEPATH")
+		if home == "" {
+			home = os.Getenv("USERPROFILE")
+		}
+		return home
+	}
+	return getUnixHomeDir()
+}
+
+func getDefaultConfDir() string {
+	if runtime.GOOS == "windows" {
+		return path.Join(os.Getenv("AppData"), "syncthing")
+	}
+	return expandTilde("~/.syncthing")
+}

cmd/syncthing/model.go

@@ -0,0 +1,917 @@
+package main
+
+import (
+	"crypto/sha1"
+	"errors"
+	"fmt"
+	"io"
+	"net"
+	"os"
+	"path"
+	"sync"
+	"time"
+
+	"github.com/calmh/syncthing/buffers"
+	"github.com/calmh/syncthing/protocol"
+)
+
+type Model struct {
+	dir string
+
+	global    map[string]File // the latest version of each file as it exists in the cluster
+	gmut      sync.RWMutex    // protects global
+	local     map[string]File // the files we currently have locally on disk
+	lmut      sync.RWMutex    // protects local
+	remote    map[string]map[string]File
+	rmut      sync.RWMutex // protects remote
+	protoConn map[string]Connection
+	rawConn   map[string]io.Closer
+	pmut      sync.RWMutex // protects protoConn and rawConn
+
+	// Queue for files to fetch. fq can call back into the model, so we must ensure
+	// to hold no locks when calling methods on fq.
+	fq *FileQueue
+	dq chan File // queue for files to delete
+
+	updatedLocal        int64 // timestamp of last update to local
+	updateGlobal        int64 // timestamp of last update to remote
+	lastIdxBcast        time.Time
+	lastIdxBcastRequest time.Time
+	umut                sync.RWMutex // provides updated* and lastIdx*
+
+	rwRunning bool
+	delete    bool
+	initmut   sync.Mutex // protects rwRunning and delete
+
+	trace map[string]bool
+
+	sup suppressor
+
+	parallelRequests int
+	limitRequestRate chan struct{}
+
+	imut sync.Mutex // protects Index
+}
+
+type Connection interface {
+	ID() string
+	Index(string, []protocol.FileInfo)
+	Request(repo, name string, offset int64, size int) ([]byte, error)
+	Statistics() protocol.Statistics
+	Option(key string) string
+}
+
+const (
+	idxBcastHoldtime = 15 * time.Second  // Wait at least this long after the last index modification
+	idxBcastMaxDelay = 120 * time.Second // Unless we've already waited this long
+)
+
+var (
+	ErrNoSuchFile = errors.New("no such file")
+	ErrInvalid    = errors.New("file is invalid")
+)
+
+// NewModel creates and starts a new model. The model starts in read-only mode,
+// where it sends index information to connected peers and responds to requests
+// for file data without altering the local repository in any way.
+func NewModel(dir string, maxChangeBw int) *Model {
+	m := &Model{
+		dir:          dir,
+		global:       make(map[string]File),
+		local:        make(map[string]File),
+		remote:       make(map[string]map[string]File),
+		protoConn:    make(map[string]Connection),
+		rawConn:      make(map[string]io.Closer),
+		lastIdxBcast: time.Now(),
+		trace:        make(map[string]bool),
+		sup:          suppressor{threshold: int64(maxChangeBw)},
+		fq:           NewFileQueue(),
+		dq:           make(chan File),
+	}
+
+	go m.broadcastIndexLoop()
+	return m
+}
+
+func (m *Model) LimitRate(kbps int) {
+	m.limitRequestRate = make(chan struct{}, kbps)
+	n := kbps/10 + 1
+	go func() {
+		for {
+			time.Sleep(100 * time.Millisecond)
+			for i := 0; i < n; i++ {
+				select {
+				case m.limitRequestRate <- struct{}{}:
+				}
+			}
+		}
+	}()
+}
+
+// Trace enables trace logging of the given facility. This is a debugging function; grep for m.trace.
+func (m *Model) Trace(t string) {
+	m.trace[t] = true
+}
+
+// StartRW starts read/write processing on the current model. When in
+// read/write mode the model will attempt to keep in sync with the cluster by
+// pulling needed files from peer nodes.
+func (m *Model) StartRW(del bool, threads int) {
+	m.initmut.Lock()
+	defer m.initmut.Unlock()
+
+	if m.rwRunning {
+		panic("starting started model")
+	}
+
+	m.rwRunning = true
+	m.delete = del
+	m.parallelRequests = threads
+
+	go m.cleanTempFiles()
+	if del {
+		go m.deleteLoop()
+	}
+}
+
+// Generation returns an opaque integer that is guaranteed to increment on
+// every change to the local repository or global model.
+func (m *Model) Generation() int64 {
+	m.umut.RLock()
+	defer m.umut.RUnlock()
+
+	return m.updatedLocal + m.updateGlobal
+}
+
+func (m *Model) LocalAge() float64 {
+	m.umut.RLock()
+	defer m.umut.RUnlock()
+
+	return time.Since(time.Unix(m.updatedLocal, 0)).Seconds()
+}
+
+type ConnectionInfo struct {
+	protocol.Statistics
+	Address       string
+	ClientID      string
+	ClientVersion string
+	Completion    int
+}
+
+// ConnectionStats returns a map with connection statistics for each connected node.
+func (m *Model) ConnectionStats() map[string]ConnectionInfo {
+	type remoteAddrer interface {
+		RemoteAddr() net.Addr
+	}
+
+	m.gmut.RLock()
+	m.pmut.RLock()
+	m.rmut.RLock()
+
+	var tot int64
+	for _, f := range m.global {
+		tot += f.Size
+	}
+
+	var res = make(map[string]ConnectionInfo)
+	for node, conn := range m.protoConn {
+		ci := ConnectionInfo{
+			Statistics:    conn.Statistics(),
+			ClientID:      conn.Option("clientId"),
+			ClientVersion: conn.Option("clientVersion"),
+		}
+		if nc, ok := m.rawConn[node].(remoteAddrer); ok {
+			ci.Address = nc.RemoteAddr().String()
+		}
+
+		var have int64
+		for _, f := range m.remote[node] {
+			if f.Equals(m.global[f.Name]) {
+				have += f.Size
+			}
+		}
+
+		ci.Completion = 100
+		if tot != 0 {
+			ci.Completion = int(100 * have / tot)
+		}
+
+		res[node] = ci
+	}
+
+	m.rmut.RUnlock()
+	m.pmut.RUnlock()
+	m.gmut.RUnlock()
+	return res
+}
+
+// GlobalSize returns the number of files, deleted files and total bytes for all
+// files in the global model.
+func (m *Model) GlobalSize() (files, deleted int, bytes int64) {
+	m.gmut.RLock()
+
+	for _, f := range m.global {
+		if f.Flags&protocol.FlagDeleted == 0 {
+			files++
+			bytes += f.Size
+		} else {
+			deleted++
+		}
+	}
+
+	m.gmut.RUnlock()
+	return
+}
+
+// LocalSize returns the number of files, deleted files and total bytes for all
+// files in the local repository.
+func (m *Model) LocalSize() (files, deleted int, bytes int64) {
+	m.lmut.RLock()
+
+	for _, f := range m.local {
+		if f.Flags&protocol.FlagDeleted == 0 {
+			files++
+			bytes += f.Size
+		} else {
+			deleted++
+		}
+	}
+
+	m.lmut.RUnlock()
+	return
+}
+
+// InSyncSize returns the number and total byte size of the local files that
+// are in sync with the global model.
+func (m *Model) InSyncSize() (files, bytes int64) {
+	m.gmut.RLock()
+	m.lmut.RLock()
+
+	for n, f := range m.local {
+		if gf, ok := m.global[n]; ok && f.Equals(gf) {
+			files++
+			bytes += f.Size
+		}
+	}
+
+	m.lmut.RUnlock()
+	m.gmut.RUnlock()
+	return
+}
+
+// NeedFiles returns the list of currently needed files and the total size.
+func (m *Model) NeedFiles() (files []File, bytes int64) {
+	qf := m.fq.QueuedFiles()
+
+	m.gmut.RLock()
+
+	for _, n := range qf {
+		f := m.global[n]
+		files = append(files, f)
+		bytes += f.Size
+	}
+
+	m.gmut.RUnlock()
+	return
+}
+
+// Index is called when a new node is connected and we receive their full index.
+// Implements the protocol.Model interface.
+func (m *Model) Index(nodeID string, fs []protocol.FileInfo) {
+	var files = make([]File, len(fs))
+	for i := range fs {
+		files[i] = fileFromFileInfo(fs[i])
+	}
+
+	m.imut.Lock()
+	defer m.imut.Unlock()
+
+	if m.trace["net"] {
+		debugf("NET IDX(in): %s: %d files", nodeID, len(fs))
+	}
+
+	repo := make(map[string]File)
+	for _, f := range files {
+		m.indexUpdate(repo, f)
+	}
+
+	m.rmut.Lock()
+	m.remote[nodeID] = repo
+	m.rmut.Unlock()
+
+	m.recomputeGlobal()
+	m.recomputeNeedForFiles(files)
+}
+
+// IndexUpdate is called for incremental updates to connected nodes' indexes.
+// Implements the protocol.Model interface.
+func (m *Model) IndexUpdate(nodeID string, fs []protocol.FileInfo) {
+	var files = make([]File, len(fs))
+	for i := range fs {
+		files[i] = fileFromFileInfo(fs[i])
+	}
+
+	m.imut.Lock()
+	defer m.imut.Unlock()
+
+	if m.trace["net"] {
+		debugf("NET IDXUP(in): %s: %d files", nodeID, len(files))
+	}
+
+	m.rmut.Lock()
+	repo, ok := m.remote[nodeID]
+	if !ok {
+		warnf("Index update from node %s that does not have an index", nodeID)
+		m.rmut.Unlock()
+		return
+	}
+
+	for _, f := range files {
+		m.indexUpdate(repo, f)
+	}
+	m.rmut.Unlock()
+
+	m.recomputeGlobal()
+	m.recomputeNeedForFiles(files)
+}
+
+func (m *Model) indexUpdate(repo map[string]File, f File) {
+	if m.trace["idx"] {
+		var flagComment string
+		if f.Flags&protocol.FlagDeleted != 0 {
+			flagComment = " (deleted)"
+		}
+		debugf("IDX(in): %q m=%d f=%o%s v=%d (%d blocks)", f.Name, f.Modified, f.Flags, flagComment, f.Version, len(f.Blocks))
+	}
+
+	if extraFlags := f.Flags &^ (protocol.FlagInvalid | protocol.FlagDeleted | 0xfff); extraFlags != 0 {
+		warnf("IDX(in): Unknown flags 0x%x in index record %+v", extraFlags, f)
+		return
+	}
+
+	repo[f.Name] = f
+}
+
+// Close removes the peer from the model and closes the underlying connection if possible.
+// Implements the protocol.Model interface.
+func (m *Model) Close(node string, err error) {
+	if m.trace["net"] {
+		debugf("NET: %s: %v", node, err)
+	}
+	if err == protocol.ErrClusterHash {
+		warnf("Connection to %s closed due to mismatched cluster hash. Ensure that the configured cluster members are identical on both nodes.", node)
+	} else if err != io.EOF {
+		warnf("Connection to %s closed: %v", node, err)
+	}
+
+	m.fq.RemoveAvailable(node)
+
+	m.pmut.Lock()
+	m.rmut.Lock()
+
+	conn, ok := m.rawConn[node]
+	if ok {
+		conn.Close()
+	}
+
+	delete(m.remote, node)
+	delete(m.protoConn, node)
+	delete(m.rawConn, node)
+
+	m.rmut.Unlock()
+	m.pmut.Unlock()
+
+	m.recomputeGlobal()
+	m.recomputeNeedForGlobal()
+}
+
+// Request returns the specified data segment by reading it from local disk.
+// Implements the protocol.Model interface.
+func (m *Model) Request(nodeID, repo, name string, offset int64, size int) ([]byte, error) {
+	// Verify that the requested file exists in the local and global model.
+	m.lmut.RLock()
+	lf, localOk := m.local[name]
+	m.lmut.RUnlock()
+
+	m.gmut.RLock()
+	_, globalOk := m.global[name]
+	m.gmut.RUnlock()
+
+	if !localOk || !globalOk {
+		warnf("SECURITY (nonexistent file) REQ(in): %s: %q o=%d s=%d", nodeID, name, offset, size)
+		return nil, ErrNoSuchFile
+	}
+	if lf.Flags&protocol.FlagInvalid != 0 {
+		return nil, ErrInvalid
+	}
+
+	if m.trace["net"] && nodeID != "<local>" {
+		debugf("NET REQ(in): %s: %q o=%d s=%d", nodeID, name, offset, size)
+	}
+	fn := path.Join(m.dir, name)
+	fd, err := os.Open(fn) // XXX: Inefficient, should cache fd?
+	if err != nil {
+		return nil, err
+	}
+	defer fd.Close()
+
+	buf := buffers.Get(int(size))
+	_, err = fd.ReadAt(buf, offset)
+	if err != nil {
+		return nil, err
+	}
+
+	if m.limitRequestRate != nil {
+		for s := 0; s < len(buf); s += 1024 {
+			<-m.limitRequestRate
+		}
+	}
+
+	return buf, nil
+}
+
+// ReplaceLocal replaces the local repository index with the given list of files.
+func (m *Model) ReplaceLocal(fs []File) {
+	var updated bool
+	var newLocal = make(map[string]File)
+
+	m.lmut.RLock()
+	for _, f := range fs {
+		newLocal[f.Name] = f
+		if ef := m.local[f.Name]; !ef.Equals(f) {
+			updated = true
+		}
+	}
+	m.lmut.RUnlock()
+
+	if m.markDeletedLocals(newLocal) {
+		updated = true
+	}
+
+	m.lmut.RLock()
+	if len(newLocal) != len(m.local) {
+		updated = true
+	}
+	m.lmut.RUnlock()
+
+	if updated {
+		m.lmut.Lock()
+		m.local = newLocal
+		m.lmut.Unlock()
+
+		m.recomputeGlobal()
+		m.recomputeNeedForGlobal()
+
+		m.umut.Lock()
+		m.updatedLocal = time.Now().Unix()
+		m.lastIdxBcastRequest = time.Now()
+		m.umut.Unlock()
+	}
+}
+
+// SeedLocal replaces the local repository index with the given list of files,
+// in protocol data types. Does not track deletes, should only be used to seed
+// the local index from a cache file at startup.
+func (m *Model) SeedLocal(fs []protocol.FileInfo) {
+	m.lmut.Lock()
+	m.local = make(map[string]File)
+	for _, f := range fs {
+		m.local[f.Name] = fileFromFileInfo(f)
+	}
+	m.lmut.Unlock()
+
+	m.recomputeGlobal()
+	m.recomputeNeedForGlobal()
+}
+
+// ConnectedTo returns true if we are connected to the named node.
+func (m *Model) ConnectedTo(nodeID string) bool {
+	m.pmut.RLock()
+	_, ok := m.protoConn[nodeID]
+	m.pmut.RUnlock()
+	return ok
+}
+
+// RepoID returns a unique ID representing the current repository location.
+func (m *Model) RepoID() string {
+	return fmt.Sprintf("%x", sha1.Sum([]byte(m.dir)))
+}
+
+// AddConnection adds a new peer connection to the model. An initial index will
+// be sent to the connected peer, thereafter index updates whenever the local
+// repository changes.
+func (m *Model) AddConnection(rawConn io.Closer, protoConn Connection) {
+	nodeID := protoConn.ID()
+	m.pmut.Lock()
+	m.protoConn[nodeID] = protoConn
+	m.rawConn[nodeID] = rawConn
+	m.pmut.Unlock()
+
+	go func() {
+		idx := m.ProtocolIndex()
+		protoConn.Index("default", idx)
+	}()
+
+	m.initmut.Lock()
+	rw := m.rwRunning
+	m.initmut.Unlock()
+	if !rw {
+		return
+	}
+
+	for i := 0; i < m.parallelRequests; i++ {
+		i := i
+		go func() {
+			if m.trace["pull"] {
+				debugln("PULL: Starting", nodeID, i)
+			}
+			for {
+				m.pmut.RLock()
+				if _, ok := m.protoConn[nodeID]; !ok {
+					if m.trace["pull"] {
+						debugln("PULL: Exiting", nodeID, i)
+					}
+					m.pmut.RUnlock()
+					return
+				}
+				m.pmut.RUnlock()
+
+				qb, ok := m.fq.Get(nodeID)
+				if ok {
+					if m.trace["pull"] {
+						debugln("PULL: Request", nodeID, i, qb.name, qb.block.Offset)
+					}
+					data, _ := protoConn.Request("default", qb.name, qb.block.Offset, int(qb.block.Size))
+					m.fq.Done(qb.name, qb.block.Offset, data)
+				} else {
+					time.Sleep(1 * time.Second)
+				}
+			}
+		}()
+	}
+}
+
+// ProtocolIndex returns the current local index in protocol data types.
+// Must be called with the read lock held.
+func (m *Model) ProtocolIndex() []protocol.FileInfo {
+	var index []protocol.FileInfo
+
+	m.lmut.RLock()
+
+	for _, f := range m.local {
+		mf := fileInfoFromFile(f)
+		if m.trace["idx"] {
+			var flagComment string
+			if mf.Flags&protocol.FlagDeleted != 0 {
+				flagComment = " (deleted)"
+			}
+			debugf("IDX(out): %q m=%d f=%o%s v=%d (%d blocks)", mf.Name, mf.Modified, mf.Flags, flagComment, mf.Version, len(mf.Blocks))
+		}
+		index = append(index, mf)
+	}
+
+	m.lmut.RUnlock()
+	return index
+}
+
+func (m *Model) requestGlobal(nodeID, name string, offset int64, size int, hash []byte) ([]byte, error) {
+	m.pmut.RLock()
+	nc, ok := m.protoConn[nodeID]
+	m.pmut.RUnlock()
+
+	if !ok {
+		return nil, fmt.Errorf("requestGlobal: no such node: %s", nodeID)
+	}
+
+	if m.trace["net"] {
+		debugf("NET REQ(out): %s: %q o=%d s=%d h=%x", nodeID, name, offset, size, hash)
+	}
+
+	return nc.Request("default", name, offset, size)
+}
+
+func (m *Model) broadcastIndexLoop() {
+	for {
+		m.umut.RLock()
+		bcastRequested := m.lastIdxBcastRequest.After(m.lastIdxBcast)
+		holdtimeExceeded := time.Since(m.lastIdxBcastRequest) > idxBcastHoldtime
+		m.umut.RUnlock()
+
+		maxDelayExceeded := time.Since(m.lastIdxBcast) > idxBcastMaxDelay
+		if bcastRequested && (holdtimeExceeded || maxDelayExceeded) {
+			idx := m.ProtocolIndex()
+
+			var indexWg sync.WaitGroup
+			indexWg.Add(len(m.protoConn))
+
+			m.umut.Lock()
+			m.lastIdxBcast = time.Now()
+			m.umut.Unlock()
+
+			m.pmut.RLock()
+			for _, node := range m.protoConn {
+				node := node
+				if m.trace["net"] {
+					debugf("NET IDX(out/loop): %s: %d files", node.ID(), len(idx))
+				}
+				go func() {
+					node.Index("default", idx)
+					indexWg.Done()
+				}()
+			}
+			m.pmut.RUnlock()
+
+			indexWg.Wait()
+		}
+		time.Sleep(idxBcastHoldtime)
+	}
+}
+
+// markDeletedLocals sets the deleted flag on files that have gone missing locally.
+func (m *Model) markDeletedLocals(newLocal map[string]File) bool {
+	// For every file in the existing local table, check if they are also
+	// present in the new local table. If they are not, check that we already
+	// had the newest version available according to the global table and if so
+	// note the file as having been deleted.
+	var updated bool
+
+	m.gmut.RLock()
+	m.lmut.RLock()
+
+	for n, f := range m.local {
+		if _, ok := newLocal[n]; !ok {
+			if gf := m.global[n]; !gf.NewerThan(f) {
+				if f.Flags&protocol.FlagDeleted == 0 {
+					f.Flags = protocol.FlagDeleted
+					f.Version++
+					f.Blocks = nil
+					updated = true
+				}
+				newLocal[n] = f
+			}
+		}
+	}
+
+	m.lmut.RUnlock()
+	m.gmut.RUnlock()
+
+	return updated
+}
+
+func (m *Model) updateLocal(f File) {
+	var updated bool
+
+	m.lmut.Lock()
+	if ef, ok := m.local[f.Name]; !ok || !ef.Equals(f) {
+		m.local[f.Name] = f
+		updated = true
+	}
+	m.lmut.Unlock()
+
+	if updated {
+		m.recomputeGlobal()
+		// We don't recomputeNeed here for two reasons:
+		// - a need shouldn't have arisen due to having a newer local file
+		// - recomputeNeed might call into fq.Add but we might have been called by
+		//   fq which would be a deadlock on fq
+
+		m.umut.Lock()
+		m.updatedLocal = time.Now().Unix()
+		m.lastIdxBcastRequest = time.Now()
+		m.umut.Unlock()
+	}
+}
+
+/*
+XXX: Not done, needs elegant handling of availability
+
+func (m *Model) recomputeGlobalFor(files []File) bool {
+	m.gmut.Lock()
+	defer m.gmut.Unlock()
+
+	var updated bool
+	for _, f := range files {
+		if gf, ok := m.global[f.Name]; !ok || f.NewerThan(gf) {
+			m.global[f.Name] = f
+			updated = true
+			// Fix availability
+		}
+	}
+	return updated
+}
+*/
+
+func (m *Model) recomputeGlobal() {
+	var newGlobal = make(map[string]File)
+
+	m.lmut.RLock()
+	for n, f := range m.local {
+		newGlobal[n] = f
+	}
+	m.lmut.RUnlock()
+
+	var available = make(map[string][]string)
+
+	m.rmut.RLock()
+	var highestMod int64
+	for nodeID, fs := range m.remote {
+		for n, nf := range fs {
+			if lf, ok := newGlobal[n]; !ok || nf.NewerThan(lf) {
+				newGlobal[n] = nf
+				available[n] = []string{nodeID}
+				if nf.Modified > highestMod {
+					highestMod = nf.Modified
+				}
+			} else if lf.Equals(nf) {
+				available[n] = append(available[n], nodeID)
+			}
+		}
+	}
+	m.rmut.RUnlock()
+
+	for f, ns := range available {
+		m.fq.SetAvailable(f, ns)
+	}
+
+	// Figure out if anything actually changed
+
+	m.gmut.RLock()
+	var updated bool
+	if highestMod > m.updateGlobal || len(newGlobal) != len(m.global) {
+		updated = true
+	} else {
+		for n, f0 := range newGlobal {
+			if f1, ok := m.global[n]; !ok || !f0.Equals(f1) {
+				updated = true
+				break
+			}
+		}
+	}
+	m.gmut.RUnlock()
+
+	if updated {
+		m.gmut.Lock()
+		m.umut.Lock()
+		m.global = newGlobal
+		m.updateGlobal = time.Now().Unix()
+		m.umut.Unlock()
+		m.gmut.Unlock()
+	}
+}
+
+type addOrder struct {
+	n      string
+	remote []Block
+	fm     *fileMonitor
+}
+
+func (m *Model) recomputeNeedForGlobal() {
+	var toDelete []File
+	var toAdd []addOrder
+
+	m.gmut.RLock()
+
+	for _, gf := range m.global {
+		toAdd, toDelete = m.recomputeNeedForFile(gf, toAdd, toDelete)
+	}
+
+	m.gmut.RUnlock()
+
+	for _, ao := range toAdd {
+		m.fq.Add(ao.n, ao.remote, ao.fm)
+	}
+	for _, gf := range toDelete {
+		m.dq <- gf
+	}
+}
+
+func (m *Model) recomputeNeedForFiles(files []File) {
+	var toDelete []File
+	var toAdd []addOrder
+
+	m.gmut.RLock()
+
+	for _, gf := range files {
+		toAdd, toDelete = m.recomputeNeedForFile(gf, toAdd, toDelete)
+	}
+
+	m.gmut.RUnlock()
+
+	for _, ao := range toAdd {
+		m.fq.Add(ao.n, ao.remote, ao.fm)
+	}
+	for _, gf := range toDelete {
+		m.dq <- gf
+	}
+}
+
+func (m *Model) recomputeNeedForFile(gf File, toAdd []addOrder, toDelete []File) ([]addOrder, []File) {
+	m.lmut.RLock()
+	lf, ok := m.local[gf.Name]
+	m.lmut.RUnlock()
+
+	if !ok || gf.NewerThan(lf) {
+		if gf.Flags&protocol.FlagInvalid != 0 {
+			// Never attempt to sync invalid files
+			return toAdd, toDelete
+		}
+		if gf.Flags&protocol.FlagDeleted != 0 && !m.delete {
+			// Don't want to delete files, so forget this need
+			return toAdd, toDelete
+		}
+		if gf.Flags&protocol.FlagDeleted != 0 && !ok {
+			// Don't have the file, so don't need to delete it
+			return toAdd, toDelete
+		}
+		if m.trace["need"] {
+			debugf("NEED: lf:%v gf:%v", lf, gf)
+		}
+
+		if gf.Flags&protocol.FlagDeleted != 0 {
+			toDelete = append(toDelete, gf)
+		} else {
+			local, remote := BlockDiff(lf.Blocks, gf.Blocks)
+			fm := fileMonitor{
+				name:        gf.Name,
+				path:        path.Clean(path.Join(m.dir, gf.Name)),
+				global:      gf,
+				model:       m,
+				localBlocks: local,
+			}
+			toAdd = append(toAdd, addOrder{gf.Name, remote, &fm})
+		}
+	}
+
+	return toAdd, toDelete
+}
+
+func (m *Model) WhoHas(name string) []string {
+	var remote []string
+
+	m.gmut.RLock()
+	m.rmut.RLock()
+
+	gf := m.global[name]
+	for node, files := range m.remote {
+		if file, ok := files[name]; ok && file.Equals(gf) {
+			remote = append(remote, node)
+		}
+	}
+
+	m.rmut.RUnlock()
+	m.gmut.RUnlock()
+	return remote
+}
+
+func (m *Model) deleteLoop() {
+	for file := range m.dq {
+		if m.trace["file"] {
+			debugln("FILE: Delete", file.Name)
+		}
+		path := path.Clean(path.Join(m.dir, file.Name))
+		err := os.Remove(path)
+		if err != nil {
+			warnf("%s: %v", file.Name, err)
+		}
+
+		m.updateLocal(file)
+	}
+}
+
+func fileFromFileInfo(f protocol.FileInfo) File {
+	var blocks = make([]Block, len(f.Blocks))
+	var offset int64
+	for i, b := range f.Blocks {
+		blocks[i] = Block{
+			Offset: offset,
+			Size:   b.Size,
+			Hash:   b.Hash,
+		}
+		offset += int64(b.Size)
+	}
+	return File{
+		Name:     f.Name,
+		Size:     offset,
+		Flags:    f.Flags,
+		Modified: f.Modified,
+		Version:  f.Version,
+		Blocks:   blocks,
+	}
+}
+
+func fileInfoFromFile(f File) protocol.FileInfo {
+	var blocks = make([]protocol.BlockInfo, len(f.Blocks))
+	for i, b := range f.Blocks {
+		blocks[i] = protocol.BlockInfo{
+			Size: b.Size,
+			Hash: b.Hash,
+		}
+	}
+	return protocol.FileInfo{
+		Name:     f.Name,
+		Flags:    f.Flags,
+		Modified: f.Modified,
+		Version:  f.Version,
+		Blocks:   blocks,
+	}
+}

cmd/syncthing/model_test.go

@@ -1,7 +1,8 @@
-package model
+package main
 
 import (
 	"bytes"
+	"fmt"
 	"os"
 	"reflect"
 	"testing"
@@ -11,13 +12,13 @@ import (
 )
 
 func TestNewModel(t *testing.T) {
-	m := NewModel("foo")
+	m := NewModel("foo", 1e6)
 
 	if m == nil {
 		t.Fatalf("NewModel returned nil")
 	}
 
-	if len(m.need) > 0 {
+	if fs, _ := m.NeedFiles(); len(fs) > 0 {
 		t.Errorf("New model should have no Need")
 	}
 
@@ -31,19 +32,22 @@ var testDataExpected = map[string]File{
 		Name:     "foo",
 		Flags:    0,
 		Modified: 0,
-		Blocks:   []Block{{Offset: 0x0, Length: 0x7, Hash: []uint8{0xae, 0xc0, 0x70, 0x64, 0x5f, 0xe5, 0x3e, 0xe3, 0xb3, 0x76, 0x30, 0x59, 0x37, 0x61, 0x34, 0xf0, 0x58, 0xcc, 0x33, 0x72, 0x47, 0xc9, 0x78, 0xad, 0xd1, 0x78, 0xb6, 0xcc, 0xdf, 0xb0, 0x1, 0x9f}}},
+		Size:     7,
+		Blocks:   []Block{{Offset: 0x0, Size: 0x7, Hash: []uint8{0xae, 0xc0, 0x70, 0x64, 0x5f, 0xe5, 0x3e, 0xe3, 0xb3, 0x76, 0x30, 0x59, 0x37, 0x61, 0x34, 0xf0, 0x58, 0xcc, 0x33, 0x72, 0x47, 0xc9, 0x78, 0xad, 0xd1, 0x78, 0xb6, 0xcc, 0xdf, 0xb0, 0x1, 0x9f}}},
+	},
+	"empty": File{
+		Name:     "empty",
+		Flags:    0,
+		Modified: 0,
+		Size:     0,
+		Blocks:   []Block{{Offset: 0x0, Size: 0x0, Hash: []uint8{0xe3, 0xb0, 0xc4, 0x42, 0x98, 0xfc, 0x1c, 0x14, 0x9a, 0xfb, 0xf4, 0xc8, 0x99, 0x6f, 0xb9, 0x24, 0x27, 0xae, 0x41, 0xe4, 0x64, 0x9b, 0x93, 0x4c, 0xa4, 0x95, 0x99, 0x1b, 0x78, 0x52, 0xb8, 0x55}}},
 	},
 	"bar": File{
 		Name:     "bar",
 		Flags:    0,
 		Modified: 0,
-		Blocks:   []Block{{Offset: 0x0, Length: 0xa, Hash: []uint8{0x2f, 0x72, 0xcc, 0x11, 0xa6, 0xfc, 0xd0, 0x27, 0x1e, 0xce, 0xf8, 0xc6, 0x10, 0x56, 0xee, 0x1e, 0xb1, 0x24, 0x3b, 0xe3, 0x80, 0x5b, 0xf9, 0xa9, 0xdf, 0x98, 0xf9, 0x2f, 0x76, 0x36, 0xb0, 0x5c}}},
-	},
-	"baz/quux": File{
-		Name:     "baz/quux",
-		Flags:    0,
-		Modified: 0,
-		Blocks:   []Block{{Offset: 0x0, Length: 0x9, Hash: []uint8{0xc1, 0x54, 0xd9, 0x4e, 0x94, 0xba, 0x72, 0x98, 0xa6, 0xad, 0xb0, 0x52, 0x3a, 0xfe, 0x34, 0xd1, 0xb6, 0xa5, 0x81, 0xd6, 0xb8, 0x93, 0xa7, 0x63, 0xd4, 0x5d, 0xdc, 0x5e, 0x20, 0x9d, 0xcb, 0x83}}},
+		Size:     10,
+		Blocks:   []Block{{Offset: 0x0, Size: 0xa, Hash: []uint8{0x2f, 0x72, 0xcc, 0x11, 0xa6, 0xfc, 0xd0, 0x27, 0x1e, 0xce, 0xf8, 0xc6, 0x10, 0x56, 0xee, 0x1e, 0xb1, 0x24, 0x3b, 0xe3, 0x80, 0x5b, 0xf9, 0xa9, 0xdf, 0x98, 0xf9, 0x2f, 0x76, 0x36, 0xb0, 0x5c}}},
 	},
 }
 
@@ -58,11 +62,11 @@ func init() {
 }
 
 func TestUpdateLocal(t *testing.T) {
-	m := NewModel("testdata")
+	m := NewModel("testdata", 1e6)
 	fs, _ := m.Walk(false)
 	m.ReplaceLocal(fs)
 
-	if len(m.need) > 0 {
+	if fs, _ := m.NeedFiles(); len(fs) > 0 {
 		t.Fatalf("Model with only local data should have no need")
 	}
 
@@ -100,7 +104,7 @@ func TestUpdateLocal(t *testing.T) {
 }
 
 func TestRemoteUpdateExisting(t *testing.T) {
-	m := NewModel("testdata")
+	m := NewModel("testdata", 1e6)
 	fs, _ := m.Walk(false)
 	m.ReplaceLocal(fs)
 
@@ -111,13 +115,13 @@ func TestRemoteUpdateExisting(t *testing.T) {
 	}
 	m.Index("42", []protocol.FileInfo{newFile})
 
-	if l := len(m.need); l != 1 {
-		t.Errorf("Model missing Need for one file (%d != 1)", l)
+	if fs, _ := m.NeedFiles(); len(fs) != 1 {
+		t.Errorf("Model missing Need for one file (%d != 1)", len(fs))
 	}
 }
 
 func TestRemoteAddNew(t *testing.T) {
-	m := NewModel("testdata")
+	m := NewModel("testdata", 1e6)
 	fs, _ := m.Walk(false)
 	m.ReplaceLocal(fs)
 
@@ -128,13 +132,13 @@ func TestRemoteAddNew(t *testing.T) {
 	}
 	m.Index("42", []protocol.FileInfo{newFile})
 
-	if l1, l2 := len(m.need), 1; l1 != l2 {
-		t.Errorf("Model len(m.need) incorrect (%d != %d)", l1, l2)
+	if fs, _ := m.NeedFiles(); len(fs) != 1 {
+		t.Errorf("Model len(m.need) incorrect (%d != 1)", len(fs))
 	}
 }
 
 func TestRemoteUpdateOld(t *testing.T) {
-	m := NewModel("testdata")
+	m := NewModel("testdata", 1e6)
 	fs, _ := m.Walk(false)
 	m.ReplaceLocal(fs)
 
@@ -146,13 +150,13 @@ func TestRemoteUpdateOld(t *testing.T) {
 	}
 	m.Index("42", []protocol.FileInfo{newFile})
 
-	if l1, l2 := len(m.need), 0; l1 != l2 {
-		t.Errorf("Model len(need) incorrect (%d != %d)", l1, l2)
+	if fs, _ := m.NeedFiles(); len(fs) != 0 {
+		t.Errorf("Model len(need) incorrect (%d != 0)", len(fs))
 	}
 }
 
 func TestRemoteIndexUpdate(t *testing.T) {
-	m := NewModel("testdata")
+	m := NewModel("testdata", 1e6)
 	fs, _ := m.Walk(false)
 	m.ReplaceLocal(fs)
 
@@ -170,22 +174,22 @@ func TestRemoteIndexUpdate(t *testing.T) {
 
 	m.Index("42", []protocol.FileInfo{foo})
 
-	if _, ok := m.need["foo"]; !ok {
+	if fs, _ := m.NeedFiles(); fs[0].Name != "foo" {
 		t.Error("Model doesn't need 'foo'")
 	}
 
 	m.IndexUpdate("42", []protocol.FileInfo{bar})
 
-	if _, ok := m.need["foo"]; !ok {
+	if fs, _ := m.NeedFiles(); fs[0].Name != "foo" {
 		t.Error("Model doesn't need 'foo'")
 	}
-	if _, ok := m.need["bar"]; !ok {
+	if fs, _ := m.NeedFiles(); fs[1].Name != "bar" {
 		t.Error("Model doesn't need 'bar'")
 	}
 }
 
 func TestDelete(t *testing.T) {
-	m := NewModel("testdata")
+	m := NewModel("testdata", 1e6)
 	fs, _ := m.Walk(false)
 	m.ReplaceLocal(fs)
 
@@ -228,9 +232,12 @@ func TestDelete(t *testing.T) {
 	if len(m.local["a new file"].Blocks) != 0 {
 		t.Error("Unexpected non-zero blocks for deleted file in local")
 	}
-	if ft := m.local["a new file"].Modified; ft != ot+1 {
+	if ft := m.local["a new file"].Modified; ft != ot {
 		t.Errorf("Unexpected time %d != %d for deleted file in local", ft, ot+1)
 	}
+	if fv := m.local["a new file"].Version; fv != 1 {
+		t.Errorf("Unexpected version %d != 1 for deleted file in local", fv)
+	}
 
 	if m.global["a new file"].Flags&(1<<12) == 0 {
 		t.Error("Unexpected deleted flag = 0 in global table")
@@ -238,8 +245,11 @@ func TestDelete(t *testing.T) {
 	if len(m.global["a new file"].Blocks) != 0 {
 		t.Error("Unexpected non-zero blocks for deleted file in global")
 	}
-	if ft := m.local["a new file"].Modified; ft != ot+1 {
-		t.Errorf("Unexpected time %d != %d for deleted file in local", ft, ot+1)
+	if ft := m.global["a new file"].Modified; ft != ot {
+		t.Errorf("Unexpected time %d != %d for deleted file in global", ft, ot+1)
+	}
+	if fv := m.local["a new file"].Version; fv != 1 {
+		t.Errorf("Unexpected version %d != 1 for deleted file in global", fv)
 	}
 
 	// Another update should change nothing
@@ -259,8 +269,11 @@ func TestDelete(t *testing.T) {
 	if len(m.local["a new file"].Blocks) != 0 {
 		t.Error("Unexpected non-zero blocks for deleted file in local")
 	}
-	if ft := m.local["a new file"].Modified; ft != ot+1 {
-		t.Errorf("Unexpected time %d != %d for deleted file in local", ft, ot+1)
+	if ft := m.local["a new file"].Modified; ft != ot {
+		t.Errorf("Unexpected time %d != %d for deleted file in local", ft, ot)
+	}
+	if fv := m.local["a new file"].Version; fv != 1 {
+		t.Errorf("Unexpected version %d != 1 for deleted file in local", fv)
 	}
 
 	if m.global["a new file"].Flags&(1<<12) == 0 {
@@ -269,13 +282,16 @@ func TestDelete(t *testing.T) {
 	if len(m.global["a new file"].Blocks) != 0 {
 		t.Error("Unexpected non-zero blocks for deleted file in global")
 	}
-	if ft := m.local["a new file"].Modified; ft != ot+1 {
-		t.Errorf("Unexpected time %d != %d for deleted file in local", ft, ot+1)
+	if ft := m.global["a new file"].Modified; ft != ot {
+		t.Errorf("Unexpected time %d != %d for deleted file in global", ft, ot)
+	}
+	if fv := m.local["a new file"].Version; fv != 1 {
+		t.Errorf("Unexpected version %d != 1 for deleted file in global", fv)
 	}
 }
 
 func TestForgetNode(t *testing.T) {
-	m := NewModel("testdata")
+	m := NewModel("testdata", 1e6)
 	fs, _ := m.Walk(false)
 	m.ReplaceLocal(fs)
 
@@ -285,8 +301,8 @@ func TestForgetNode(t *testing.T) {
 	if l1, l2 := len(m.global), len(fs); l1 != l2 {
 		t.Errorf("Model len(global) incorrect (%d != %d)", l1, l2)
 	}
-	if l1, l2 := len(m.need), 0; l1 != l2 {
-		t.Errorf("Model len(need) incorrect (%d != %d)", l1, l2)
+	if fs, _ := m.NeedFiles(); len(fs) != 0 {
+		t.Errorf("Model len(need) incorrect (%d != 0)", len(fs))
 	}
 
 	newFile := protocol.FileInfo{
@@ -296,14 +312,21 @@ func TestForgetNode(t *testing.T) {
 	}
 	m.Index("42", []protocol.FileInfo{newFile})
 
+	newFile = protocol.FileInfo{
+		Name:     "new file 2",
+		Modified: time.Now().Unix(),
+		Blocks:   []protocol.BlockInfo{{100, []byte("some hash bytes")}},
+	}
+	m.Index("43", []protocol.FileInfo{newFile})
+
 	if l1, l2 := len(m.local), len(fs); l1 != l2 {
 		t.Errorf("Model len(local) incorrect (%d != %d)", l1, l2)
 	}
-	if l1, l2 := len(m.global), len(fs)+1; l1 != l2 {
+	if l1, l2 := len(m.global), len(fs)+2; l1 != l2 {
 		t.Errorf("Model len(global) incorrect (%d != %d)", l1, l2)
 	}
-	if l1, l2 := len(m.need), 1; l1 != l2 {
-		t.Errorf("Model len(need) incorrect (%d != %d)", l1, l2)
+	if fs, _ := m.NeedFiles(); len(fs) != 2 {
+		t.Errorf("Model len(need) incorrect (%d != 2)", len(fs))
 	}
 
 	m.Close("42", nil)
@@ -311,20 +334,21 @@ func TestForgetNode(t *testing.T) {
 	if l1, l2 := len(m.local), len(fs); l1 != l2 {
 		t.Errorf("Model len(local) incorrect (%d != %d)", l1, l2)
 	}
-	if l1, l2 := len(m.global), len(fs); l1 != l2 {
+	if l1, l2 := len(m.global), len(fs)+1; l1 != l2 {
 		t.Errorf("Model len(global) incorrect (%d != %d)", l1, l2)
 	}
-	if l1, l2 := len(m.need), 0; l1 != l2 {
-		t.Errorf("Model len(need) incorrect (%d != %d)", l1, l2)
+
+	if fs, _ := m.NeedFiles(); len(fs) != 1 {
+		t.Errorf("Model len(need) incorrect (%d != 1)", len(fs))
 	}
 }
 
 func TestRequest(t *testing.T) {
-	m := NewModel("testdata")
+	m := NewModel("testdata", 1e6)
 	fs, _ := m.Walk(false)
 	m.ReplaceLocal(fs)
 
-	bs, err := m.Request("some node", "foo", 0, 6, nil)
+	bs, err := m.Request("some node", "default", "foo", 0, 6)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -332,7 +356,7 @@ func TestRequest(t *testing.T) {
 		t.Errorf("Incorrect data from request: %q", string(bs))
 	}
 
-	bs, err = m.Request("some node", "../walk.go", 0, 6, nil)
+	bs, err = m.Request("some node", "default", "../walk.go", 0, 6)
 	if err == nil {
 		t.Error("Unexpected nil error on insecure file read")
 	}
@@ -340,3 +364,177 @@ func TestRequest(t *testing.T) {
 		t.Errorf("Unexpected non nil data on insecure file read: %q", string(bs))
 	}
 }
+
+func TestIgnoreWithUnknownFlags(t *testing.T) {
+	m := NewModel("testdata", 1e6)
+	fs, _ := m.Walk(false)
+	m.ReplaceLocal(fs)
+
+	valid := protocol.FileInfo{
+		Name:     "valid",
+		Modified: time.Now().Unix(),
+		Blocks:   []protocol.BlockInfo{{100, []byte("some hash bytes")}},
+		Flags:    protocol.FlagDeleted | 0755,
+	}
+
+	invalid := protocol.FileInfo{
+		Name:     "invalid",
+		Modified: time.Now().Unix(),
+		Blocks:   []protocol.BlockInfo{{100, []byte("some hash bytes")}},
+		Flags:    1<<27 | protocol.FlagDeleted | 0755,
+	}
+
+	m.Index("42", []protocol.FileInfo{valid, invalid})
+
+	if _, ok := m.global[valid.Name]; !ok {
+		t.Error("Model should include", valid)
+	}
+	if _, ok := m.global[invalid.Name]; ok {
+		t.Error("Model not should include", invalid)
+	}
+}
+
+func genFiles(n int) []protocol.FileInfo {
+	files := make([]protocol.FileInfo, n)
+	t := time.Now().Unix()
+	for i := 0; i < n; i++ {
+		files[i] = protocol.FileInfo{
+			Name:     fmt.Sprintf("file%d", i),
+			Modified: t,
+			Blocks:   []protocol.BlockInfo{{100, []byte("some hash bytes")}},
+		}
+	}
+
+	return files
+}
+
+func BenchmarkIndex10000(b *testing.B) {
+	m := NewModel("testdata", 1e6)
+	fs, _ := m.Walk(false)
+	m.ReplaceLocal(fs)
+	files := genFiles(10000)
+
+	b.ResetTimer()
+	for i := 0; i < b.N; i++ {
+		m.Index("42", files)
+	}
+}
+
+func BenchmarkIndex00100(b *testing.B) {
+	m := NewModel("testdata", 1e6)
+	fs, _ := m.Walk(false)
+	m.ReplaceLocal(fs)
+	files := genFiles(100)
+
+	b.ResetTimer()
+	for i := 0; i < b.N; i++ {
+		m.Index("42", files)
+	}
+}
+
+func BenchmarkIndexUpdate10000f10000(b *testing.B) {
+	m := NewModel("testdata", 1e6)
+	fs, _ := m.Walk(false)
+	m.ReplaceLocal(fs)
+	files := genFiles(10000)
+	m.Index("42", files)
+
+	b.ResetTimer()
+	for i := 0; i < b.N; i++ {
+		m.IndexUpdate("42", files)
+	}
+}
+
+func BenchmarkIndexUpdate10000f00100(b *testing.B) {
+	m := NewModel("testdata", 1e6)
+	fs, _ := m.Walk(false)
+	m.ReplaceLocal(fs)
+	files := genFiles(10000)
+	m.Index("42", files)
+
+	ufiles := genFiles(100)
+	b.ResetTimer()
+	for i := 0; i < b.N; i++ {
+		m.IndexUpdate("42", ufiles)
+	}
+}
+
+func BenchmarkIndexUpdate10000f00001(b *testing.B) {
+	m := NewModel("testdata", 1e6)
+	fs, _ := m.Walk(false)
+	m.ReplaceLocal(fs)
+	files := genFiles(10000)
+	m.Index("42", files)
+
+	ufiles := genFiles(1)
+	b.ResetTimer()
+	for i := 0; i < b.N; i++ {
+		m.IndexUpdate("42", ufiles)
+	}
+}
+
+type FakeConnection struct {
+	id          string
+	requestData []byte
+}
+
+func (FakeConnection) Close() error {
+	return nil
+}
+
+func (f FakeConnection) ID() string {
+	return string(f.id)
+}
+
+func (f FakeConnection) Option(string) string {
+	return ""
+}
+
+func (FakeConnection) Index(string, []protocol.FileInfo) {}
+
+func (f FakeConnection) Request(repo, name string, offset int64, size int) ([]byte, error) {
+	return f.requestData, nil
+}
+
+func (FakeConnection) Ping() bool {
+	return true
+}
+
+func (FakeConnection) Statistics() protocol.Statistics {
+	return protocol.Statistics{}
+}
+
+func BenchmarkRequest(b *testing.B) {
+	m := NewModel("testdata", 1e6)
+	fs, _ := m.Walk(false)
+	m.ReplaceLocal(fs)
+
+	const n = 1000
+	files := make([]protocol.FileInfo, n)
+	t := time.Now().Unix()
+	for i := 0; i < n; i++ {
+		files[i] = protocol.FileInfo{
+			Name:     fmt.Sprintf("file%d", i),
+			Modified: t,
+			Blocks:   []protocol.BlockInfo{{100, []byte("some hash bytes")}},
+		}
+	}
+
+	fc := FakeConnection{
+		id:          "42",
+		requestData: []byte("some data to return"),
+	}
+	m.AddConnection(fc, fc)
+	m.Index("42", files)
+
+	b.ResetTimer()
+	for i := 0; i < b.N; i++ {
+		data, err := m.requestGlobal("42", files[i%n].Name, 0, 32, nil)
+		if err != nil {
+			b.Error(err)
+		}
+		if data == nil {
+			b.Error("nil data")
+		}
+	}
+}

cmd/syncthing/openurl.go

@@ -0,0 +1,34 @@
+/*
+Copyright 2011 Google Inc.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package main
+
+import (
+	"os/exec"
+	"runtime"
+)
+
+func openURL(url string) error {
+	if runtime.GOOS == "windows" {
+		return exec.Command("cmd.exe", "/C", "start "+url).Run()
+	}
+
+	if runtime.GOOS == "darwin" {
+		return exec.Command("open", url).Run()
+	}
+
+	return exec.Command("xdg-open", url).Run()
+}

cmd/syncthing/suppressor.go

@@ -0,0 +1,72 @@
+package main
+
+import (
+	"sync"
+	"time"
+)
+
+const (
+	MaxChangeHistory = 4
+)
+
+type change struct {
+	size int64
+	when time.Time
+}
+
+type changeHistory struct {
+	changes []change
+	next    int64
+	prevSup bool
+}
+
+type suppressor struct {
+	sync.Mutex
+	changes   map[string]changeHistory
+	threshold int64 // bytes/s
+}
+
+func (h changeHistory) bandwidth(t time.Time) int64 {
+	if len(h.changes) == 0 {
+		return 0
+	}
+
+	var t0 = h.changes[0].when
+	if t == t0 {
+		return 0
+	}
+
+	var bw float64
+	for _, c := range h.changes {
+		bw += float64(c.size)
+	}
+	return int64(bw / t.Sub(t0).Seconds())
+}
+
+func (h *changeHistory) append(size int64, t time.Time) {
+	c := change{size, t}
+	if len(h.changes) == MaxChangeHistory {
+		h.changes = h.changes[1:MaxChangeHistory]
+	}
+	h.changes = append(h.changes, c)
+}
+
+func (s *suppressor) suppress(name string, size int64, t time.Time) (bool, bool) {
+	s.Lock()
+
+	if s.changes == nil {
+		s.changes = make(map[string]changeHistory)
+	}
+	h := s.changes[name]
+	sup := h.bandwidth(t) > s.threshold
+	prevSup := h.prevSup
+	h.prevSup = sup
+	if !sup {
+		h.append(size, t)
+	}
+	s.changes[name] = h
+
+	s.Unlock()
+
+	return sup, prevSup
+}

cmd/syncthing/suppressor_test.go

@@ -0,0 +1,113 @@
+package main
+
+import (
+	"testing"
+	"time"
+)
+
+func TestSuppressor(t *testing.T) {
+	s := suppressor{threshold: 10000}
+	t0 := time.Now()
+
+	t1 := t0
+	sup, prev := s.suppress("foo", 10000, t1)
+	if sup {
+		t.Fatal("Never suppress first change")
+	}
+	if prev {
+		t.Fatal("Incorrect prev status")
+	}
+
+	// bw is 10000 / 10 = 1000
+	t1 = t0.Add(10 * time.Second)
+	if bw := s.changes["foo"].bandwidth(t1); bw != 1000 {
+		t.Errorf("Incorrect bw %d", bw)
+	}
+	sup, prev = s.suppress("foo", 10000, t1)
+	if sup {
+		t.Fatal("Should still be fine")
+	}
+	if prev {
+		t.Fatal("Incorrect prev status")
+	}
+
+	// bw is (10000 + 10000) / 11 = 1818
+	t1 = t0.Add(11 * time.Second)
+	if bw := s.changes["foo"].bandwidth(t1); bw != 1818 {
+		t.Errorf("Incorrect bw %d", bw)
+	}
+	sup, prev = s.suppress("foo", 100500, t1)
+	if sup {
+		t.Fatal("Should still be fine")
+	}
+	if prev {
+		t.Fatal("Incorrect prev status")
+	}
+
+	// bw is (10000 + 10000 + 100500) / 12 = 10041
+	t1 = t0.Add(12 * time.Second)
+	if bw := s.changes["foo"].bandwidth(t1); bw != 10041 {
+		t.Errorf("Incorrect bw %d", bw)
+	}
+	sup, prev = s.suppress("foo", 10000000, t1) // value will be ignored
+	if !sup {
+		t.Fatal("Should be over threshold")
+	}
+	if prev {
+		t.Fatal("Incorrect prev status")
+	}
+
+	// bw is (10000 + 10000 + 100500) / 15 = 8033
+	t1 = t0.Add(15 * time.Second)
+	if bw := s.changes["foo"].bandwidth(t1); bw != 8033 {
+		t.Errorf("Incorrect bw %d", bw)
+	}
+	sup, prev = s.suppress("foo", 10000000, t1)
+	if sup {
+		t.Fatal("Should be Ok")
+	}
+	if !prev {
+		t.Fatal("Incorrect prev status")
+	}
+}
+
+func TestHistory(t *testing.T) {
+	h := changeHistory{}
+
+	t0 := time.Now()
+	h.append(40, t0)
+
+	if l := len(h.changes); l != 1 {
+		t.Errorf("Incorrect history length %d", l)
+	}
+	if s := h.changes[0].size; s != 40 {
+		t.Errorf("Incorrect first record size %d", s)
+	}
+
+	for i := 1; i < MaxChangeHistory; i++ {
+		h.append(int64(40+i), t0.Add(time.Duration(i)*time.Second))
+	}
+
+	if l := len(h.changes); l != MaxChangeHistory {
+		t.Errorf("Incorrect history length %d", l)
+	}
+	if s := h.changes[0].size; s != 40 {
+		t.Errorf("Incorrect first record size %d", s)
+	}
+	if s := h.changes[MaxChangeHistory-1].size; s != 40+MaxChangeHistory-1 {
+		t.Errorf("Incorrect last record size %d", s)
+	}
+
+	h.append(999, t0.Add(time.Duration(999)*time.Second))
+
+	if l := len(h.changes); l != MaxChangeHistory {
+		t.Errorf("Incorrect history length %d", l)
+	}
+	if s := h.changes[0].size; s != 41 {
+		t.Errorf("Incorrect first record size %d", s)
+	}
+	if s := h.changes[MaxChangeHistory-1].size; s != 999 {
+		t.Errorf("Incorrect last record size %d", s)
+	}
+
+}

cmd/syncthing/tls.go

@@ -3,7 +3,7 @@ package main
 import (
 	"crypto/rand"
 	"crypto/rsa"
-	"crypto/sha1"
+	"crypto/sha256"
 	"crypto/tls"
 	"crypto/x509"
 	"crypto/x509/pkix"
@@ -12,11 +12,12 @@ import (
 	"math/big"
 	"os"
 	"path"
+	"strings"
 	"time"
 )
 
 const (
-	tlsRSABits = 2048
+	tlsRSABits = 3072
 	tlsName    = "syncthing"
 )
 
@@ -24,14 +25,16 @@ func loadCert(dir string) (tls.Certificate, error) {
 	return tls.LoadX509KeyPair(path.Join(dir, "cert.pem"), path.Join(dir, "key.pem"))
 }
 
-func certId(bs []byte) string {
-	hf := sha1.New()
+func certID(bs []byte) string {
+	hf := sha256.New()
 	hf.Write(bs)
 	id := hf.Sum(nil)
-	return base32.StdEncoding.EncodeToString(id)
+	return strings.Trim(base32.StdEncoding.EncodeToString(id), "=")
 }
 
 func newCertificate(dir string) {
+	infoln("Generating RSA certificate and key...")
+
 	priv, err := rsa.GenerateKey(rand.Reader, tlsRSABits)
 	fatalErr(err)
 
@@ -47,7 +50,7 @@ func newCertificate(dir string) {
 		NotAfter:  notAfter,
 
 		KeyUsage:              x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature,
-		ExtKeyUsage:           []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
+		ExtKeyUsage:           []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth, x509.ExtKeyUsageClientAuth},
 		BasicConstraintsValid: true,
 	}
 
@@ -58,11 +61,11 @@ func newCertificate(dir string) {
 	fatalErr(err)
 	pem.Encode(certOut, &pem.Block{Type: "CERTIFICATE", Bytes: derBytes})
 	certOut.Close()
-	okln("Created TLS certificate file")
+	okln("Created RSA certificate file")
 
 	keyOut, err := os.OpenFile(path.Join(dir, "key.pem"), os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600)
 	fatalErr(err)
 	pem.Encode(keyOut, &pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(priv)})
 	keyOut.Close()
-	okln("Created TLS key file")
+	okln("Created RSA key file")
 }

cmd/syncthing/usage.go

@@ -0,0 +1,52 @@
+package main
+
+import (
+	"bytes"
+	"flag"
+	"fmt"
+	"io"
+	"text/tabwriter"
+)
+
+func optionTable(w io.Writer, rows [][]string) {
+	tw := tabwriter.NewWriter(w, 2, 4, 2, ' ', 0)
+	for _, row := range rows {
+		for i, cell := range row {
+			if i > 0 {
+				tw.Write([]byte("\t"))
+			}
+			tw.Write([]byte(cell))
+		}
+		tw.Write([]byte("\n"))
+	}
+	tw.Flush()
+}
+
+func usageFor(fs *flag.FlagSet, usage string) func() {
+	return func() {
+		var b bytes.Buffer
+		b.WriteString("Usage:\n  " + usage + "\n")
+
+		var options [][]string
+		fs.VisitAll(func(f *flag.Flag) {
+			var dash = "-"
+			if len(f.Name) > 1 {
+				dash = "--"
+			}
+			var opt = "  " + dash + f.Name
+
+			if f.DefValue != "false" {
+				opt += "=" + f.DefValue
+			}
+
+			options = append(options, []string{opt, f.Usage})
+		})
+
+		if len(options) > 0 {
+			b.WriteString("\nOptions:\n")
+			optionTable(&b, options)
+		}
+
+		fmt.Println(b.String())
+	}
+}

cmd/syncthing/util.go

@@ -2,7 +2,7 @@ package main
 
 import "fmt"
 
-func MetricPrefix(n int) string {
+func MetricPrefix(n int64) string {
 	if n > 1e9 {
 		return fmt.Sprintf("%.02f G", float64(n)/1e9)
 	}
@@ -15,7 +15,7 @@ func MetricPrefix(n int) string {
 	return fmt.Sprintf("%d ", n)
 }
 
-func BinaryPrefix(n int) string {
+func BinaryPrefix(n int64) string {
 	if n > 1<<30 {
 		return fmt.Sprintf("%.02f Gi", float64(n)/(1<<30))
 	}

cmd/syncthing/walk.go

@@ -0,0 +1,238 @@
+package main
+
+import (
+	"bytes"
+	"fmt"
+	"io/ioutil"
+	"log"
+	"os"
+	"path"
+	"path/filepath"
+	"strings"
+	"time"
+
+	"github.com/calmh/syncthing/protocol"
+)
+
+const BlockSize = 128 * 1024
+
+type File struct {
+	Name     string
+	Flags    uint32
+	Modified int64
+	Version  uint32
+	Size     int64
+	Blocks   []Block
+}
+
+func (f File) String() string {
+	return fmt.Sprintf("File{Name:%q, Flags:0x%x, Modified:%d, Version:%d, Size:%d, NumBlocks:%d}",
+		f.Name, f.Flags, f.Modified, f.Version, f.Size, len(f.Blocks))
+}
+
+func (f File) Equals(o File) bool {
+	return f.Modified == o.Modified && f.Version == o.Version
+}
+
+func (f File) NewerThan(o File) bool {
+	return f.Modified > o.Modified || (f.Modified == o.Modified && f.Version > o.Version)
+}
+
+func isTempName(name string) bool {
+	return strings.HasPrefix(path.Base(name), ".syncthing.")
+}
+
+func tempName(name string, modified int64) string {
+	tdir := path.Dir(name)
+	tname := fmt.Sprintf(".syncthing.%s.%d", path.Base(name), modified)
+	return path.Join(tdir, tname)
+}
+
+func (m *Model) loadIgnoreFiles(ign map[string][]string) filepath.WalkFunc {
+	return func(p string, info os.FileInfo, err error) error {
+		if err != nil {
+			return nil
+		}
+
+		rn, err := filepath.Rel(m.dir, p)
+		if err != nil {
+			return nil
+		}
+
+		if pn, sn := path.Split(rn); sn == ".stignore" {
+			pn := strings.Trim(pn, "/")
+			bs, _ := ioutil.ReadFile(p)
+			lines := bytes.Split(bs, []byte("\n"))
+			var patterns []string
+			for _, line := range lines {
+				if len(line) > 0 {
+					patterns = append(patterns, string(line))
+				}
+			}
+			ign[pn] = patterns
+		}
+
+		return nil
+	}
+}
+
+func (m *Model) walkAndHashFiles(res *[]File, ign map[string][]string) filepath.WalkFunc {
+	return func(p string, info os.FileInfo, err error) error {
+		if err != nil {
+			if m.trace["file"] {
+				log.Printf("FILE: %q: %v", p, err)
+			}
+			return nil
+		}
+
+		if isTempName(p) {
+			return nil
+		}
+
+		rn, err := filepath.Rel(m.dir, p)
+		if err != nil {
+			return nil
+		}
+
+		if _, sn := path.Split(rn); sn == ".stignore" {
+			// We never sync the .stignore files
+			return nil
+		}
+
+		if ignoreFile(ign, rn) {
+			if m.trace["file"] {
+				log.Println("FILE: IGNORE:", rn)
+			}
+			return nil
+		}
+
+		if info.Mode()&os.ModeType == 0 {
+			modified := info.ModTime().Unix()
+
+			m.lmut.RLock()
+			lf, ok := m.local[rn]
+			m.lmut.RUnlock()
+
+			if ok && lf.Modified == modified {
+				if nf := uint32(info.Mode()); nf != lf.Flags {
+					lf.Flags = nf
+					lf.Version++
+				}
+				*res = append(*res, lf)
+			} else {
+				if cur, prev := m.sup.suppress(rn, info.Size(), time.Now()); cur {
+					if m.trace["file"] {
+						log.Printf("FILE: SUPPRESS: %q change bw over threshold", rn)
+					}
+					if !prev {
+						log.Printf("INFO: Changes to %q are being temporarily suppressed because it changes too frequently.", rn)
+					}
+
+					if ok {
+						lf.Flags = protocol.FlagInvalid
+						lf.Version++
+						*res = append(*res, lf)
+					}
+					return nil
+				} else if prev && !cur {
+					log.Printf("INFO: Changes to %q are no longer suppressed.", rn)
+				}
+
+				if m.trace["file"] {
+					log.Printf("FILE: Hash %q", p)
+				}
+				fd, err := os.Open(p)
+				if err != nil {
+					if m.trace["file"] {
+						log.Printf("FILE: %q: %v", p, err)
+					}
+					return nil
+				}
+				defer fd.Close()
+
+				blocks, err := Blocks(fd, BlockSize)
+				if err != nil {
+					if m.trace["file"] {
+						log.Printf("FILE: %q: %v", p, err)
+					}
+					return nil
+				}
+				f := File{
+					Name:     rn,
+					Size:     info.Size(),
+					Flags:    uint32(info.Mode()),
+					Modified: modified,
+					Blocks:   blocks,
+				}
+				*res = append(*res, f)
+			}
+		}
+
+		return nil
+	}
+}
+
+// Walk returns the list of files found in the local repository by scanning the
+// file system. Files are blockwise hashed.
+func (m *Model) Walk(followSymlinks bool) (files []File, ignore map[string][]string) {
+	ignore = make(map[string][]string)
+
+	hashFiles := m.walkAndHashFiles(&files, ignore)
+
+	filepath.Walk(m.dir, m.loadIgnoreFiles(ignore))
+	filepath.Walk(m.dir, hashFiles)
+
+	if followSymlinks {
+		d, err := os.Open(m.dir)
+		if err != nil {
+			return
+		}
+		defer d.Close()
+
+		fis, err := d.Readdir(-1)
+		if err != nil {
+			return
+		}
+
+		for _, info := range fis {
+			if info.Mode()&os.ModeSymlink != 0 {
+				dir := path.Join(m.dir, info.Name()) + "/"
+				filepath.Walk(dir, m.loadIgnoreFiles(ignore))
+				filepath.Walk(dir, hashFiles)
+			}
+		}
+	}
+
+	return
+}
+
+func (m *Model) cleanTempFile(path string, info os.FileInfo, err error) error {
+	if err != nil {
+		return err
+	}
+	if info.Mode()&os.ModeType == 0 && isTempName(path) {
+		if m.trace["file"] {
+			log.Printf("FILE: Remove %q", path)
+		}
+		os.Remove(path)
+	}
+	return nil
+}
+
+func (m *Model) cleanTempFiles() {
+	filepath.Walk(m.dir, m.cleanTempFile)
+}
+
+func ignoreFile(patterns map[string][]string, file string) bool {
+	first, last := path.Split(file)
+	for prefix, pats := range patterns {
+		if len(prefix) == 0 || prefix == first || strings.HasPrefix(first, prefix+"/") {
+			for _, pattern := range pats {
+				if match, _ := path.Match(pattern, last); match {
+					return true
+				}
+			}
+		}
+	}
+	return false
+}

cmd/syncthing/walk_test.go

@@ -1,4 +1,4 @@
-package model
+package main
 
 import (
 	"fmt"
@@ -13,7 +13,7 @@ var testdata = []struct {
 	hash string
 }{
 	{"bar", 10, "2f72cc11a6fcd0271ecef8c61056ee1eb1243be3805bf9a9df98f92f7636b05c"},
-	{"baz/quux", 9, "c154d94e94ba7298a6adb0523afe34d1b6a581d6b893a763d45ddc5e209dcb83"},
+	{"empty", 0, "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"},
 	{"foo", 7, "aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f"},
 }
 
@@ -22,7 +22,7 @@ var correctIgnores = map[string][]string{
 }
 
 func TestWalk(t *testing.T) {
-	m := NewModel("testdata")
+	m := NewModel("testdata", 1e6)
 	files, ignores := m.Walk(false)
 
 	if l1, l2 := len(files), len(testdata); l1 != l2 {
@@ -50,53 +50,34 @@ func TestWalk(t *testing.T) {
 	}
 }
 
-func TestFilteredWalk(t *testing.T) {
-	m := NewModel("testdata")
-	files := m.FilteredWalk(false)
-
-	if len(files) != 2 {
-		t.Fatalf("Incorrect number of walked filtered files %d != 2", len(files))
-	}
-	if files[0].Name != "bar" {
-		t.Error("Incorrect first file", files[0])
-	}
-	if files[1].Name != "foo" {
-		t.Error("Incorrect second file", files[1])
-	}
-}
-
 func TestIgnore(t *testing.T) {
 	var patterns = map[string][]string{
 		"":        {"t2"},
 		"foo":     {"bar", "z*"},
 		"foo/baz": {"quux", ".*"},
 	}
-	var files = []File{
-		{Name: "foo/bar"},
-		{Name: "foo/quux"},
-		{Name: "foo/zuux"},
-		{Name: "foo/qzuux"},
-		{Name: "foo/baz/t1"},
-		{Name: "foo/baz/t2"},
-		{Name: "foo/baz/bar"},
-		{Name: "foo/baz/quuxa"},
-		{Name: "foo/baz/aquux"},
-		{Name: "foo/baz/.quux"},
-		{Name: "foo/baz/zquux"},
-		{Name: "foo/baz/quux"},
-		{Name: "foo/bazz/quux"},
-	}
-	var remaining = []File{
-		{Name: "foo/quux"},
-		{Name: "foo/qzuux"},
-		{Name: "foo/baz/t1"},
-		{Name: "foo/baz/quuxa"},
-		{Name: "foo/baz/aquux"},
-		{Name: "foo/bazz/quux"},
+	var tests = []struct {
+		f string
+		r bool
+	}{
+		{"foo/bar", true},
+		{"foo/quux", false},
+		{"foo/zuux", true},
+		{"foo/qzuux", false},
+		{"foo/baz/t1", false},
+		{"foo/baz/t2", true},
+		{"foo/baz/bar", true},
+		{"foo/baz/quuxa", false},
+		{"foo/baz/aquux", false},
+		{"foo/baz/.quux", true},
+		{"foo/baz/zquux", true},
+		{"foo/baz/quux", true},
+		{"foo/bazz/quux", false},
 	}
 
-	var filtered = ignoreFilter(patterns, files)
-	if !reflect.DeepEqual(filtered, remaining) {
-		t.Errorf("Filtering mismatch\n  %v\n  %v", remaining, filtered)
+	for i, tc := range tests {
+		if r := ignoreFile(patterns, tc.f); r != tc.r {
+			t.Errorf("Incorrect ignoreFile() #%d; E: %v, A: %v", i, tc.r, r)
+		}
 	}
 }

discover/PROTOCOL.md

@@ -0,0 +1,115 @@
+Node Discovery Protocol v2
+==========================
+
+Mode of Operation
+-----------------
+
+There are two distinct modes: "local discovery", performed on a LAN
+segment (broadcast domain) and "global discovery" performed over the
+Internet in general with the support of a well known server.
+
+Local discovery does not use Query packets. Instead Announcement packets
+are sent periodically and each participating node keeps a table of the
+announcements it has seen. On multihomed hosts the announcement packets
+should be sent on each interface that syncthing will accept connections.
+
+It is recommended that local discovery Announcement packets are sent on
+a 30 to 60 second interval, possibly with forced transmissions when a
+previously unknown node is discovered.
+
+Global discovery is made possible by periodically updating a global server
+using Announcement packets indentical to those transmitted for local
+discovery. The node performing discovery will transmit a Query packet to
+the global server and expect an Announcement packet in response. In case
+the global server has no knowledge of the queried node ID, there will be
+no response. A timeout is to be used to determine lookup failure.
+
+There is no message to unregister from the global server; instead
+registrations are forgotten after 60 minutes. It is recommended to
+send Announcement packets to the global server on a 30 minute interval.
+
+Packet Formats
+--------------
+
+The Announcement packet has the following structure:
+
+     0                   1                   2                   3
+     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+    |                   Magic Number (0x029E4C77)                   |
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+    |                       Length of Node ID                       |
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+    /                                                               /
+    \                   Node ID (variable length)                   \
+    /                                                               /
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+    |                      Number of Addresses                      |
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+    /                                                               /
+    \                Zero or more Address Structures                \
+    /                                                               /
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+    Address Structure:
+
+     0                   1                   2                   3
+     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+    |                         Length of IP                          |
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+    /                                                               /
+    \                     IP (variable length)                      \
+    /                                                               /
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+    |          Port Number          |            0x0000             |
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+This is the XDR encoding of:
+
+    struct Announcement {
+        unsigned int MagicNumber;
+        string NodeID<>;
+        Address Addresses<>;
+    }
+
+    struct Address {
+        opaque IP<>;
+        unsigned short PortNumber;
+    }
+
+NodeID is padded to a multiple of 32 bits and all fields are in sent in
+network (big endian) byte order. In the Address structure, the IP field
+can be of three differnt kinds;
+
+ - A zero length indicates that the IP address should be taken from the
+   source address of the announcement packet, be it IPv4 or IPv6. The
+   source address must be a valid unicast address.
+
+ - A four byte length indicates that the address is an IPv4 unicast
+   address.
+
+ - A sixteen byte length indicates that the address is an IPv6 unicast
+   address.
+
+The Query packet has the following structure:
+
+     0                   1                   2                   3
+     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+    |                   Magic Number (0x23D63A9A)                   |
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+    |                       Length of Node ID                       |
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+    /                                                               /
+    \                   Node ID (variable length)                   \
+    /                                                               /
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+This is the XDR encoding of:
+
+    struct Announcement {
+        unsigned int MagicNumber;
+        string NodeID<>;
+    }
+

discover/cmd/discosrv/main.go

@@ -1,74 +1,237 @@
 package main
 
 import (
+	"encoding/binary"
+	"encoding/hex"
+	"flag"
 	"log"
 	"net"
+	"os"
 	"sync"
+	"time"
 
 	"github.com/calmh/syncthing/discover"
 )
 
 type Node struct {
+	Addresses []Address
+	Updated   time.Time
+}
+
+type Address struct {
 	IP   []byte
 	Port uint16
 }
 
 var (
-	nodes = make(map[string]Node)
-	lock  sync.Mutex
+	nodes    = make(map[string]Node)
+	lock     sync.Mutex
+	queries  = 0
+	answered = 0
 )
 
 func main() {
-	addr, _ := net.ResolveUDPAddr("udp", ":22025")
+	var debug bool
+	var listen string
+	var timestamp bool
+
+	flag.StringVar(&listen, "listen", ":22025", "Listen address")
+	flag.BoolVar(&debug, "debug", false, "Enable debug output")
+	flag.BoolVar(&timestamp, "timestamp", true, "Timestamp the log output")
+	flag.Parse()
+
+	log.SetOutput(os.Stdout)
+	if !timestamp {
+		log.SetFlags(0)
+	}
+
+	addr, _ := net.ResolveUDPAddr("udp", listen)
 	conn, err := net.ListenUDP("udp", addr)
 	if err != nil {
 		panic(err)
 	}
 
+	go func() {
+		for {
+			time.Sleep(600 * time.Second)
+
+			lock.Lock()
+
+			var deleted = 0
+			for id, node := range nodes {
+				if time.Since(node.Updated) > 60*time.Minute {
+					delete(nodes, id)
+					deleted++
+				}
+			}
+			log.Printf("Expired %d nodes; %d nodes in registry; %d queries (%d answered)", deleted, len(nodes), queries, answered)
+			queries = 0
+			answered = 0
+
+			lock.Unlock()
+		}
+	}()
+
 	var buf = make([]byte, 1024)
 	for {
+		buf = buf[:cap(buf)]
 		n, addr, err := conn.ReadFromUDP(buf)
 		if err != nil {
 			panic(err)
 		}
-		pkt, err := discover.DecodePacket(buf[:n])
-		if err != nil {
-			log.Println("Warning:", err)
+		if n < 4 {
+			log.Printf("Received short packet (%d bytes)", n)
 			continue
 		}
 
-		switch pkt.Magic {
-		case 0x20121025:
-			// Announcement
-			//lock.Lock()
+		buf = buf[:n]
+		magic := binary.BigEndian.Uint32(buf)
+
+		switch magic {
+		case discover.AnnouncementMagicV1:
+			var pkt discover.AnnounceV1
+			err := pkt.UnmarshalXDR(buf)
+			if err != nil {
+				log.Println("AnnounceV1 Unmarshal:", err)
+				log.Println(hex.Dump(buf))
+				continue
+			}
+			if debug {
+				log.Printf("<- %v %#v", addr, pkt)
+			}
+
 			ip := addr.IP.To4()
 			if ip == nil {
 				ip = addr.IP.To16()
 			}
-			node := Node{ip, uint16(pkt.Port)}
-			log.Println("<-", pkt.ID, node)
-			nodes[pkt.ID] = node
-			//lock.Unlock()
-		case 0x19760309:
-			// Query
-			//lock.Lock()
-			node, ok := nodes[pkt.ID]
-			//lock.Unlock()
-			if ok {
-				pkt := discover.Packet{
-					Magic: 0x20121025,
-					ID:    pkt.ID,
-					Port:  node.Port,
-					IP:    node.IP,
+			node := Node{
+				Addresses: []Address{{
+					IP:   ip,
+					Port: pkt.Port,
+				}},
+				Updated: time.Now(),
+			}
+
+			lock.Lock()
+			nodes[pkt.NodeID] = node
+			lock.Unlock()
+
+		case discover.QueryMagicV1:
+			var pkt discover.QueryV1
+			err := pkt.UnmarshalXDR(buf)
+			if err != nil {
+				log.Println("QueryV1 Unmarshal:", err)
+				log.Println(hex.Dump(buf))
+				continue
+			}
+			if debug {
+				log.Printf("<- %v %#v", addr, pkt)
+			}
+
+			lock.Lock()
+			node, ok := nodes[pkt.NodeID]
+			queries++
+			lock.Unlock()
+
+			if ok && len(node.Addresses) > 0 {
+				pkt := discover.AnnounceV1{
+					Magic:  discover.AnnouncementMagicV1,
+					NodeID: pkt.NodeID,
+					Port:   node.Addresses[0].Port,
+					IP:     node.Addresses[0].IP,
 				}
-				_, _, err = conn.WriteMsgUDP(discover.EncodePacket(pkt), nil, addr)
+				if debug {
+					log.Printf("-> %v %#v", addr, pkt)
+				}
+
+				tb := pkt.MarshalXDR()
+				_, _, err = conn.WriteMsgUDP(tb, nil, addr)
 				if err != nil {
-					log.Println("Warning:", err)
-				} else {
-					log.Println("->", pkt.ID, node)
+					log.Println("QueryV1 response write:", err)
 				}
+
+				lock.Lock()
+				answered++
+				lock.Unlock()
+			}
+
+		case discover.AnnouncementMagicV2:
+			var pkt discover.AnnounceV2
+			err := pkt.UnmarshalXDR(buf)
+			if err != nil {
+				log.Println("AnnounceV2 Unmarshal:", err)
+				log.Println(hex.Dump(buf))
+				continue
+			}
+			if debug {
+				log.Printf("<- %v %#v", addr, pkt)
+			}
+
+			ip := addr.IP.To4()
+			if ip == nil {
+				ip = addr.IP.To16()
+			}
+
+			var addrs []Address
+			for _, addr := range pkt.Addresses {
+				tip := addr.IP
+				if len(tip) == 0 {
+					tip = ip
+				}
+				addrs = append(addrs, Address{
+					IP:   tip,
+					Port: addr.Port,
+				})
+			}
+
+			node := Node{
+				Addresses: addrs,
+				Updated:   time.Now(),
+			}
+
+			lock.Lock()
+			nodes[pkt.NodeID] = node
+			lock.Unlock()
+
+		case discover.QueryMagicV2:
+			var pkt discover.QueryV2
+			err := pkt.UnmarshalXDR(buf)
+			if err != nil {
+				log.Println("QueryV2 Unmarshal:", err)
+				log.Println(hex.Dump(buf))
+				continue
+			}
+			if debug {
+				log.Printf("<- %v %#v", addr, pkt)
+			}
+
+			lock.Lock()
+			node, ok := nodes[pkt.NodeID]
+			queries++
+			lock.Unlock()
+
+			if ok && len(node.Addresses) > 0 {
+				pkt := discover.AnnounceV2{
+					Magic:  discover.AnnouncementMagicV2,
+					NodeID: pkt.NodeID,
+				}
+				for _, addr := range node.Addresses {
+					pkt.Addresses = append(pkt.Addresses, discover.Address{IP: addr.IP, Port: addr.Port})
+				}
+				if debug {
+					log.Printf("-> %v %#v", addr, pkt)
+				}
+
+				tb := pkt.MarshalXDR()
+				_, _, err = conn.WriteMsgUDP(tb, nil, addr)
+				if err != nil {
+					log.Println("QueryV2 response write:", err)
+				}
+
+				lock.Lock()
+				answered++
+				lock.Unlock()
 			}
 		}
-
 	}
 }

discover/discover.go

@@ -1,89 +1,12 @@
-/*
-This is the local node discovery protocol. In principle we might be better
-served by something more standardized, such as mDNS / DNS-SD. In practice, this
-was much easier and quicker to get up and running.
-
-The mode of operation is to periodically (currently once every 30 seconds)
-broadcast an Announcement packet to UDP port 21025. The packet has the
-following format:
-
-     0                   1                   2                   3
-     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
-    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
-    |                   Magic Number (0x20121025)                   |
-    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
-    |          Port Number          |           Reserved            |
-    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
-    |                        Length of NodeID                       |
-    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
-    /                                                               /
-    \                   NodeID (variable length)                    \
-    /                                                               /
-    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
-    |                          Length of IP                         |
-    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
-    /                                                               /
-    \                     IP (variable length)                      \
-    /                                                               /
-    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
-
-This is the XDR encoding of:
-
-struct Announcement {
-	unsigned int Magic;
-	unsigned short Port;
-	string NodeID<>;
-}
-
-(Hence NodeID is padded to a multiple of 32 bits)
-
-The sending node's address is not encoded in local announcement -- the Length
-of IP field is set to zero and the address is taken to be the source address of
-the announcement. In announcement packets sent by a discovery server in
-response to a query, the IP is present and the length is either 4 (IPv4) or 16
-(IPv6).
-
-Every time such a packet is received, a local table that maps NodeID to Address
-is updated. When the local node wants to connect to another node with the
-address specification 'dynamic', this table is consulted.
-
-For external discovery, an identical packet is sent every 30 minutes to the
-external discovery server. The server keeps information for up to 60 minutes.
-To query the server, and UDP packet with the format below is sent.
-
-     0                   1                   2                   3
-     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
-    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
-    |                   Magic Number (0x19760309)                   |
-    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
-    |                        Length of NodeID                       |
-    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
-    /                                                               /
-    \                   NodeID (variable length)                    \
-    /                                                               /
-    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
-
-This is the XDR encoding of:
-
-struct Announcement {
-	unsigned int Magic;
-	string NodeID<>;
-}
-
-(Hence NodeID is padded to a multiple of 32 bits)
-
-It is answered with an announcement packet for the queried node ID if the
-information is available. There is no answer for queries about unknown nodes. A
-reasonable timeout is recommended instead. (This, combined with server side
-rate limits for packets per source IP and queries per node ID, prevents the
-server from being used as an amplifier in a DDoS attack.)
-*/
 package discover
 
 import (
+	"encoding/hex"
+	"errors"
 	"fmt"
 	"log"
 	"net"
+	"strings"
 	"sync"
 	"time"
 
@@ -91,32 +14,37 @@ import (
 )
 
 const (
-	AnnouncementPort  = 21025
-	AnnouncementMagic = 0x20121025
-	QueryMagic        = 0x19760309
+	AnnouncementPort = 21025
+	Debug            = false
 )
 
 type Discoverer struct {
 	MyID             string
 	ListenPort       int
 	BroadcastIntv    time.Duration
-	ExtListenPort    int
 	ExtBroadcastIntv time.Duration
 
 	conn         *net.UDPConn
-	registry     map[string]string
+	registry     map[string][]string
 	registryLock sync.RWMutex
 	extServer    string
+
+	localBroadcastTick  <-chan time.Time
+	forcedBroadcastTick chan time.Time
 }
 
+var (
+	ErrIncorrectMagic = errors.New("incorrect magic number")
+)
+
 // We tolerate a certain amount of errors because we might be running on
 // laptops that sleep and wake, have intermittent network connectivity, etc.
 // When we hit this many errors in succession, we stop.
 const maxErrors = 30
 
-func NewDiscoverer(id string, port int, extPort int, extServer string) (*Discoverer, error) {
-	local4 := &net.UDPAddr{IP: net.IP{0, 0, 0, 0}, Port: AnnouncementPort}
-	conn, err := net.ListenUDP("udp4", local4)
+func NewDiscoverer(id string, port int, extServer string) (*Discoverer, error) {
+	local := &net.UDPAddr{IP: nil, Port: AnnouncementPort}
+	conn, err := net.ListenUDP("udp", local)
 	if err != nil {
 		return nil, err
 	}
@@ -125,58 +53,139 @@ func NewDiscoverer(id string, port int, extPort int, extServer string) (*Discove
 		MyID:             id,
 		ListenPort:       port,
 		BroadcastIntv:    30 * time.Second,
-		ExtListenPort:    extPort,
 		ExtBroadcastIntv: 1800 * time.Second,
 
 		conn:      conn,
-		registry:  make(map[string]string),
+		registry:  make(map[string][]string),
 		extServer: extServer,
 	}
 
 	go disc.recvAnnouncements()
 
 	if disc.ListenPort > 0 {
-		disc.sendAnnouncements()
+		disc.localBroadcastTick = time.Tick(disc.BroadcastIntv)
+		disc.forcedBroadcastTick = make(chan time.Time)
+		go disc.sendAnnouncements()
 	}
-	if len(disc.extServer) > 0 && disc.ExtListenPort > 0 {
-		disc.sendExtAnnouncements()
+	if len(disc.extServer) > 0 {
+		go disc.sendExtAnnouncements()
 	}
 
 	return disc, nil
 }
 
 func (d *Discoverer) sendAnnouncements() {
-	remote4 := &net.UDPAddr{IP: net.IP{255, 255, 255, 255}, Port: AnnouncementPort}
+	var pkt = AnnounceV2{AnnouncementMagicV2, d.MyID, []Address{{nil, 22000}}}
+	var buf = pkt.MarshalXDR()
+	var errCounter = 0
+	var err error
 
-	buf := EncodePacket(Packet{AnnouncementMagic, uint16(d.ListenPort), d.MyID, nil})
-	go d.writeAnnouncements(buf, remote4, d.BroadcastIntv)
+	remote := &net.UDPAddr{
+		IP:   net.IP{255, 255, 255, 255},
+		Port: AnnouncementPort,
+	}
+
+	for errCounter < maxErrors {
+		intfs, err := net.Interfaces()
+		if err != nil {
+			log.Printf("discover/listInterfaces: %v; no local announcements", err)
+			return
+		}
+
+		for _, intf := range intfs {
+			if intf.Flags&(net.FlagBroadcast|net.FlagLoopback) == net.FlagBroadcast {
+				addrs, err := intf.Addrs()
+				if err != nil {
+					log.Println("discover/listAddrs: warning:", err)
+					errCounter++
+					continue
+				}
+
+				var srcAddr string
+				for _, addr := range addrs {
+					if strings.Contains(addr.String(), ".") {
+						// Found an IPv4 adress
+						parts := strings.Split(addr.String(), "/")
+						srcAddr = parts[0]
+						break
+					}
+				}
+				if len(srcAddr) == 0 {
+					if Debug {
+						log.Println("discover: debug: no source address found on interface", intf.Name)
+					}
+					continue
+				}
+
+				iaddr, err := net.ResolveUDPAddr("udp4", srcAddr+":0")
+				if err != nil {
+					log.Println("discover/resolve: warning:", err)
+					errCounter++
+					continue
+				}
+
+				conn, err := net.ListenUDP("udp4", iaddr)
+				if err != nil {
+					log.Println("discover/listen: warning:", err)
+					errCounter++
+					continue
+				}
+
+				if Debug {
+					log.Println("discover: debug: send announcement from", conn.LocalAddr(), "to", remote, "on", intf.Name)
+				}
+
+				_, err = conn.WriteTo(buf, remote)
+				if err != nil {
+					// Some interfaces don't seem to support broadcast even though the flags claims they do, i.e. vmnet
+					conn.Close()
+
+					if Debug {
+						log.Println("discover/write: debug:", err)
+					}
+
+					errCounter++
+					continue
+				}
+
+				conn.Close()
+				errCounter = 0
+			}
+		}
+
+		select {
+		case <-d.localBroadcastTick:
+		case <-d.forcedBroadcastTick:
+		}
+	}
+	log.Println("discover/write: local: stopping due to too many errors:", err)
 }
 
 func (d *Discoverer) sendExtAnnouncements() {
-	extIP, err := net.ResolveUDPAddr("udp", d.extServer+":22025")
+	remote, err := net.ResolveUDPAddr("udp", d.extServer)
 	if err != nil {
 		log.Printf("discover/external: %v; no external announcements", err)
 		return
 	}
 
-	buf := EncodePacket(Packet{AnnouncementMagic, uint16(d.ExtListenPort), d.MyID, nil})
-	go d.writeAnnouncements(buf, extIP, d.ExtBroadcastIntv)
-}
-
-func (d *Discoverer) writeAnnouncements(buf []byte, remote *net.UDPAddr, intv time.Duration) {
+	var pkt = AnnounceV2{AnnouncementMagicV2, d.MyID, []Address{{nil, 22000}}}
+	var buf = pkt.MarshalXDR()
 	var errCounter = 0
-	var err error
+
 	for errCounter < maxErrors {
-		_, _, err = d.conn.WriteMsgUDP(buf, nil, remote)
+		if Debug {
+			log.Println("send announcement -> ", remote)
+		}
+		_, err = d.conn.WriteTo(buf, remote)
 		if err != nil {
 			log.Println("discover/write: warning:", err)
 			errCounter++
 		} else {
 			errCounter = 0
 		}
-		time.Sleep(intv)
+		time.Sleep(d.ExtBroadcastIntv)
 	}
-	log.Println("discover/write: %v: stopping due to too many errors:", remote, err)
+	log.Printf("discover/write: %v: stopping due to too many errors: %v", remote, err)
 }
 
 func (d *Discoverer) recvAnnouncements() {
@@ -191,90 +200,143 @@ func (d *Discoverer) recvAnnouncements() {
 			continue
 		}
 
-		pkt, err := DecodePacket(buf[:n])
-		if err != nil || pkt.Magic != AnnouncementMagic {
+		if Debug {
+			log.Printf("read announcement:\n%s", hex.Dump(buf[:n]))
+		}
+
+		var pkt AnnounceV2
+		err = pkt.UnmarshalXDR(buf[:n])
+		if err != nil {
 			errCounter++
 			time.Sleep(time.Second)
 			continue
 		}
 
+		if Debug {
+			log.Printf("read announcement: %#v", pkt)
+		}
+
 		errCounter = 0
 
-		if pkt.ID != d.MyID {
-			nodeAddr := fmt.Sprintf("%s:%d", addr.IP.String(), pkt.Port)
-			d.registryLock.Lock()
-			if d.registry[pkt.ID] != nodeAddr {
-				d.registry[pkt.ID] = nodeAddr
+		if pkt.NodeID != d.MyID {
+			var addrs []string
+			for _, a := range pkt.Addresses {
+				var nodeAddr string
+				if len(a.IP) > 0 {
+					nodeAddr = fmt.Sprintf("%s:%d", ipStr(a.IP), a.Port)
+				} else {
+					nodeAddr = fmt.Sprintf("%s:%d", addr.IP.String(), a.Port)
+				}
+				addrs = append(addrs, nodeAddr)
 			}
+			if Debug {
+				log.Printf("register: %#v", addrs)
+			}
+			d.registryLock.Lock()
+			_, seen := d.registry[pkt.NodeID]
+			if !seen {
+				select {
+				case d.forcedBroadcastTick <- time.Now():
+				}
+			}
+			d.registry[pkt.NodeID] = addrs
 			d.registryLock.Unlock()
 		}
 	}
 	log.Println("discover/read: stopping due to too many errors:", err)
 }
 
-func (d *Discoverer) externalLookup(node string) (string, bool) {
-	extIP, err := net.ResolveUDPAddr("udp", d.extServer+":22025")
+func (d *Discoverer) externalLookup(node string) []string {
+	extIP, err := net.ResolveUDPAddr("udp", d.extServer)
 	if err != nil {
 		log.Printf("discover/external: %v; no external lookup", err)
-		return "", false
+		return nil
 	}
 
 	conn, err := net.DialUDP("udp", nil, extIP)
 	if err != nil {
 		log.Printf("discover/external: %v; no external lookup", err)
-		return "", false
+		return nil
 	}
 	defer conn.Close()
 
 	err = conn.SetDeadline(time.Now().Add(5 * time.Second))
 	if err != nil {
 		log.Printf("discover/external: %v; no external lookup", err)
-		return "", false
+		return nil
 	}
 
-	_, err = conn.Write(EncodePacket(Packet{QueryMagic, 0, node, nil}))
+	buf := QueryV2{QueryMagicV2, node}.MarshalXDR()
+	_, err = conn.Write(buf)
 	if err != nil {
 		log.Printf("discover/external: %v; no external lookup", err)
-		return "", false
+		return nil
 	}
+	buffers.Put(buf)
 
-	var buf = buffers.Get(256)
+	buf = buffers.Get(256)
 	defer buffers.Put(buf)
 
 	n, err := conn.Read(buf)
 	if err != nil {
 		if err, ok := err.(net.Error); ok && err.Timeout() {
 			// Expected if the server doesn't know about requested node ID
-			return "", false
+			return nil
 		}
 		log.Printf("discover/external/read: %v; no external lookup", err)
-		return "", false
+		return nil
 	}
 
-	pkt, err := DecodePacket(buf[:n])
+	if Debug {
+		log.Printf("read external:\n%s", hex.Dump(buf[:n]))
+	}
+
+	var pkt AnnounceV2
+	err = pkt.UnmarshalXDR(buf[:n])
 	if err != nil {
-		log.Printf("discover/external/read: %v; no external lookup", err)
-		return "", false
+		log.Println("discover/external/decode:", err)
+		return nil
 	}
 
-	if pkt.Magic != AnnouncementMagic {
-		log.Printf("discover/external/read: bad magic; no external lookup", err)
-		return "", false
+	if Debug {
+		log.Printf("read external: %#v", pkt)
 	}
 
-	return fmt.Sprintf("%s:%d", ipStr(pkt.IP), pkt.Port), true
+	var addrs []string
+	for _, a := range pkt.Addresses {
+		var nodeAddr string
+		if len(a.IP) > 0 {
+			nodeAddr = fmt.Sprintf("%s:%d", ipStr(a.IP), a.Port)
+		}
+		addrs = append(addrs, nodeAddr)
+	}
+	return addrs
 }
 
-func (d *Discoverer) Lookup(node string) (string, bool) {
+func (d *Discoverer) Lookup(node string) []string {
 	d.registryLock.Lock()
 	addr, ok := d.registry[node]
 	d.registryLock.Unlock()
 
 	if ok {
-		return addr, true
+		return addr
 	} else if len(d.extServer) != 0 {
 		// We might want to cache this, but not permanently so it needs some intelligence
 		return d.externalLookup(node)
 	}
-	return "", false
+	return nil
+}
+
+func ipStr(ip []byte) string {
+	var f = "%d"
+	var s = "."
+	if len(ip) > 4 {
+		f = "%x"
+		s = ":"
+	}
+	var ss = make([]string, len(ip))
+	for i := range ip {
+		ss[i] = fmt.Sprintf(f, ip[i])
+	}
+	return strings.Join(ss, s)
 }

discover/encoding.go

@@ -1,160 +0,0 @@
-package discover
-
-import (
-	"encoding/binary"
-	"errors"
-	"fmt"
-)
-
-type Packet struct {
-	Magic uint32 // AnnouncementMagic or QueryMagic
-	Port  uint16 // unset if magic == QueryMagic
-	ID    string
-	IP    []byte // zero length in local announcements
-}
-
-var (
-	errBadMagic = errors.New("bad magic")
-	errFormat   = errors.New("incorrect packet format")
-)
-
-func EncodePacket(pkt Packet) []byte {
-	if l := len(pkt.IP); l != 0 && l != 4 && l != 16 {
-		// bad ip format
-		return nil
-	}
-
-	var idbs = []byte(pkt.ID)
-	var l = 4 + 4 + len(idbs) + pad(len(idbs))
-	if pkt.Magic == AnnouncementMagic {
-		l += 4 + 4 + len(pkt.IP)
-	}
-
-	var buf = make([]byte, l)
-	var offset = 0
-
-	binary.BigEndian.PutUint32(buf[offset:], pkt.Magic)
-	offset += 4
-
-	if pkt.Magic == AnnouncementMagic {
-		binary.BigEndian.PutUint16(buf[offset:], uint16(pkt.Port))
-		offset += 4
-	}
-
-	binary.BigEndian.PutUint32(buf[offset:], uint32(len(idbs)))
-	offset += 4
-	copy(buf[offset:], idbs)
-	offset += len(idbs) + pad(len(idbs))
-
-	if pkt.Magic == AnnouncementMagic {
-		binary.BigEndian.PutUint32(buf[offset:], uint32(len(pkt.IP)))
-		offset += 4
-		copy(buf[offset:], pkt.IP)
-		offset += len(pkt.IP)
-	}
-
-	return buf
-}
-
-func DecodePacket(buf []byte) (*Packet, error) {
-	var p Packet
-	var offset int
-
-	if len(buf) < 4 {
-		// short packet
-		return nil, errFormat
-	}
-	p.Magic = binary.BigEndian.Uint32(buf[offset:])
-	offset += 4
-
-	if p.Magic != AnnouncementMagic && p.Magic != QueryMagic {
-		return nil, errBadMagic
-	}
-
-	if p.Magic == AnnouncementMagic {
-		// Port Number
-
-		if len(buf) < offset+4 {
-			// short packet
-			return nil, errFormat
-		}
-		p.Port = binary.BigEndian.Uint16(buf[offset:])
-		offset += 2
-		reserved := binary.BigEndian.Uint16(buf[offset:])
-		if reserved != 0 {
-			return nil, errFormat
-		}
-		offset += 2
-	}
-
-	// Node ID
-
-	if len(buf) < offset+4 {
-		// short packet
-		return nil, errFormat
-	}
-	l := binary.BigEndian.Uint32(buf[offset:])
-	offset += 4
-
-	if len(buf) < offset+int(l)+pad(int(l)) {
-		// short packet
-		return nil, errFormat
-	}
-	idbs := buf[offset : offset+int(l)]
-	p.ID = string(idbs)
-	offset += int(l) + pad(int(l))
-
-	if p.Magic == AnnouncementMagic {
-		// IP
-
-		if len(buf) < offset+4 {
-			// short packet
-			return nil, errFormat
-		}
-		l = binary.BigEndian.Uint32(buf[offset:])
-		offset += 4
-
-		if l != 0 && l != 4 && l != 16 {
-			// weird ip length
-			return nil, errFormat
-		}
-		if len(buf) < offset+int(l) {
-			// short packet
-			return nil, errFormat
-		}
-		if l > 0 {
-			p.IP = buf[offset : offset+int(l)]
-			offset += int(l)
-		}
-	}
-
-	if len(buf[offset:]) > 0 {
-		// extra data
-		return nil, errFormat
-	}
-
-	return &p, nil
-}
-
-func pad(l int) int {
-	d := l % 4
-	if d == 0 {
-		return 0
-	}
-	return 4 - d
-}
-
-func ipStr(ip []byte) string {
-	switch len(ip) {
-	case 4:
-		return fmt.Sprintf("%d.%d.%d.%d", ip[0], ip[1], ip[2], ip[3])
-	case 16:
-		return fmt.Sprintf("%02x%02x:%02x%02x:%02x%02x:%02x%02x:%02x%02x:%02x%02x:%02x%02x:%02x%02x",
-			ip[0], ip[1], ip[2], ip[3],
-			ip[4], ip[5], ip[6], ip[7],
-			ip[8], ip[9], ip[10], ip[11],
-			ip[12], ip[13], ip[14], ip[15])
-	default:
-		return ""
-	}
-}

discover/encoding_test.go

@@ -1,138 +0,0 @@
-package discover
-
-import (
-	"bytes"
-	"reflect"
-	"testing"
-)
-
-var testdata = []struct {
-	data   []byte
-	packet *Packet
-	err    error
-}{
-	{
-		[]byte{0x20, 0x12, 0x10, 0x25,
-			0x12, 0x34, 0x00, 0x00,
-			0x00, 0x00, 0x00, 0x05,
-			0x68, 0x65, 0x6c, 0x6c, 0x6f, 0x00, 0x00, 0x00,
-			0x00, 0x00, 0x00, 0x00},
-		&Packet{
-			Magic: 0x20121025,
-			Port:  0x1234,
-			ID:    "hello",
-		},
-		nil,
-	},
-	{
-		[]byte{0x20, 0x12, 0x10, 0x25,
-			0x34, 0x56, 0x00, 0x00,
-			0x00, 0x00, 0x00, 0x08,
-			0x68, 0x65, 0x6c, 0x6c, 0x6f, 0x21, 0x21, 0x21,
-			0x00, 0x00, 0x00, 0x04,
-			0x01, 0x02, 0x03, 0x04},
-		&Packet{
-			Magic: 0x20121025,
-			Port:  0x3456,
-			ID:    "hello!!!",
-			IP:    []byte{1, 2, 3, 4},
-		},
-		nil,
-	},
-	{
-		[]byte{0x19, 0x76, 0x03, 0x09,
-			0x00, 0x00, 0x00, 0x06,
-			0x68, 0x65, 0x6c, 0x6c, 0x6f, 0x21, 0x00, 0x00},
-		&Packet{
-			Magic: 0x19760309,
-			ID:    "hello!",
-		},
-		nil,
-	},
-	{
-		[]byte{0x20, 0x12, 0x10, 0x25,
-			0x12, 0x34, 0x12, 0x34, // reserved bits not set to zero
-			0x00, 0x00, 0x00, 0x06,
-			0x68, 0x65, 0x6c, 0x6c, 0x6f, 0x21, 0x00, 0x00,
-			0x00, 0x00, 0x00, 0x00},
-		nil,
-		errFormat,
-	},
-	{
-		[]byte{0x20, 0x12, 0x10, 0x25,
-			0x12, 0x34, 0x00, 0x00,
-			0x00, 0x00, 0x00, 0x06,
-			0x68, 0x65, 0x6c, 0x6c, 0x6f, 0x21, // missing padding
-			0x00, 0x00, 0x00, 0x00},
-		nil,
-		errFormat,
-	},
-	{
-		[]byte{0x19, 0x77, 0x03, 0x09, // incorrect Magic
-			0x00, 0x00, 0x00, 0x06,
-			0x68, 0x65, 0x6c, 0x6c, 0x6f, 0x21, 0x00, 0x00},
-		nil,
-		errBadMagic,
-	},
-	{
-		[]byte{0x19, 0x76, 0x03, 0x09,
-			0x6c, 0x6c, 0x6c, 0x6c, // length exceeds packet size
-			0x68, 0x65, 0x6c, 0x6c, 0x6f, 0x21, 0x00, 0x00},
-		nil,
-		errFormat,
-	},
-	{
-		[]byte{0x19, 0x76, 0x03, 0x09,
-			0x00, 0x00, 0x00, 0x06,
-			0x68, 0x65, 0x6c, 0x6c, 0x6f, 0x21, 0x00, 0x00,
-			0x23}, // extra data at the end
-		nil,
-		errFormat,
-	},
-}
-
-func TestDecodePacket(t *testing.T) {
-	for i, test := range testdata {
-		p, err := DecodePacket(test.data)
-		if err != test.err {
-			t.Errorf("%d: unexpected error %v", i, err)
-		} else {
-			if !reflect.DeepEqual(p, test.packet) {
-				t.Errorf("%d: incorrect packet\n%v\n%v", i, test.packet, p)
-			}
-		}
-	}
-}
-
-func TestEncodePacket(t *testing.T) {
-	for i, test := range testdata {
-		if test.err != nil {
-			continue
-		}
-		buf := EncodePacket(*test.packet)
-		if bytes.Compare(buf, test.data) != 0 {
-			t.Errorf("%d: incorrect encoded packet\n% x\n% 0x", i, test.data, buf)
-		}
-	}
-}
-
-var ipstrTests = []struct {
-	d []byte
-	s string
-}{
-	{[]byte{192, 168, 34}, ""},
-	{[]byte{192, 168, 0, 34}, "192.168.0.34"},
-	{[]byte{0x20, 0x01, 0x12, 0x34,
-		0x34, 0x56, 0x56, 0x78,
-		0x78, 0x00, 0x00, 0xdc,
-		0x00, 0x00, 0x43, 0x54}, "2001:1234:3456:5678:7800:00dc:0000:4354"},
-}
-
-func TestIPStr(t *testing.T) {
-	for _, tc := range ipstrTests {
-		s1 := ipStr(tc.d)
-		if s1 != tc.s {
-			t.Errorf("Incorrect ipstr %q != %q", tc.s, s1)
-		}
-	}
-}

discover/packets.go

@@ -0,0 +1,39 @@
+package discover
+
+const (
+	AnnouncementMagicV1 = 0x20121025
+	QueryMagicV1        = 0x19760309
+)
+
+type QueryV1 struct {
+	Magic  uint32
+	NodeID string // max:64
+}
+
+type AnnounceV1 struct {
+	Magic  uint32
+	Port   uint16
+	NodeID string // max:64
+	IP     []byte // max:16
+}
+
+const (
+	AnnouncementMagicV2 = 0x029E4C77
+	QueryMagicV2        = 0x23D63A9A
+)
+
+type QueryV2 struct {
+	Magic  uint32
+	NodeID string // max:64
+}
+
+type AnnounceV2 struct {
+	Magic     uint32
+	NodeID    string    // max:64
+	Addresses []Address // max:16
+}
+
+type Address struct {
+	IP   []byte // max:16
+	Port uint16
+}

discover/packets_xdr.go

@@ -0,0 +1,220 @@
+package discover
+
+import (
+	"bytes"
+	"io"
+
+	"github.com/calmh/syncthing/xdr"
+)
+
+func (o QueryV1) EncodeXDR(w io.Writer) (int, error) {
+	var xw = xdr.NewWriter(w)
+	return o.encodeXDR(xw)
+}
+
+func (o QueryV1) MarshalXDR() []byte {
+	var buf bytes.Buffer
+	var xw = xdr.NewWriter(&buf)
+	o.encodeXDR(xw)
+	return buf.Bytes()
+}
+
+func (o QueryV1) encodeXDR(xw *xdr.Writer) (int, error) {
+	xw.WriteUint32(o.Magic)
+	if len(o.NodeID) > 64 {
+		return xw.Tot(), xdr.ErrElementSizeExceeded
+	}
+	xw.WriteString(o.NodeID)
+	return xw.Tot(), xw.Error()
+}
+
+func (o *QueryV1) DecodeXDR(r io.Reader) error {
+	xr := xdr.NewReader(r)
+	return o.decodeXDR(xr)
+}
+
+func (o *QueryV1) UnmarshalXDR(bs []byte) error {
+	var buf = bytes.NewBuffer(bs)
+	var xr = xdr.NewReader(buf)
+	return o.decodeXDR(xr)
+}
+
+func (o *QueryV1) decodeXDR(xr *xdr.Reader) error {
+	o.Magic = xr.ReadUint32()
+	o.NodeID = xr.ReadStringMax(64)
+	return xr.Error()
+}
+
+func (o AnnounceV1) EncodeXDR(w io.Writer) (int, error) {
+	var xw = xdr.NewWriter(w)
+	return o.encodeXDR(xw)
+}
+
+func (o AnnounceV1) MarshalXDR() []byte {
+	var buf bytes.Buffer
+	var xw = xdr.NewWriter(&buf)
+	o.encodeXDR(xw)
+	return buf.Bytes()
+}
+
+func (o AnnounceV1) encodeXDR(xw *xdr.Writer) (int, error) {
+	xw.WriteUint32(o.Magic)
+	xw.WriteUint16(o.Port)
+	if len(o.NodeID) > 64 {
+		return xw.Tot(), xdr.ErrElementSizeExceeded
+	}
+	xw.WriteString(o.NodeID)
+	if len(o.IP) > 16 {
+		return xw.Tot(), xdr.ErrElementSizeExceeded
+	}
+	xw.WriteBytes(o.IP)
+	return xw.Tot(), xw.Error()
+}
+
+func (o *AnnounceV1) DecodeXDR(r io.Reader) error {
+	xr := xdr.NewReader(r)
+	return o.decodeXDR(xr)
+}
+
+func (o *AnnounceV1) UnmarshalXDR(bs []byte) error {
+	var buf = bytes.NewBuffer(bs)
+	var xr = xdr.NewReader(buf)
+	return o.decodeXDR(xr)
+}
+
+func (o *AnnounceV1) decodeXDR(xr *xdr.Reader) error {
+	o.Magic = xr.ReadUint32()
+	o.Port = xr.ReadUint16()
+	o.NodeID = xr.ReadStringMax(64)
+	o.IP = xr.ReadBytesMax(16)
+	return xr.Error()
+}
+
+func (o QueryV2) EncodeXDR(w io.Writer) (int, error) {
+	var xw = xdr.NewWriter(w)
+	return o.encodeXDR(xw)
+}
+
+func (o QueryV2) MarshalXDR() []byte {
+	var buf bytes.Buffer
+	var xw = xdr.NewWriter(&buf)
+	o.encodeXDR(xw)
+	return buf.Bytes()
+}
+
+func (o QueryV2) encodeXDR(xw *xdr.Writer) (int, error) {
+	xw.WriteUint32(o.Magic)
+	if len(o.NodeID) > 64 {
+		return xw.Tot(), xdr.ErrElementSizeExceeded
+	}
+	xw.WriteString(o.NodeID)
+	return xw.Tot(), xw.Error()
+}
+
+func (o *QueryV2) DecodeXDR(r io.Reader) error {
+	xr := xdr.NewReader(r)
+	return o.decodeXDR(xr)
+}
+
+func (o *QueryV2) UnmarshalXDR(bs []byte) error {
+	var buf = bytes.NewBuffer(bs)
+	var xr = xdr.NewReader(buf)
+	return o.decodeXDR(xr)
+}
+
+func (o *QueryV2) decodeXDR(xr *xdr.Reader) error {
+	o.Magic = xr.ReadUint32()
+	o.NodeID = xr.ReadStringMax(64)
+	return xr.Error()
+}
+
+func (o AnnounceV2) EncodeXDR(w io.Writer) (int, error) {
+	var xw = xdr.NewWriter(w)
+	return o.encodeXDR(xw)
+}
+
+func (o AnnounceV2) MarshalXDR() []byte {
+	var buf bytes.Buffer
+	var xw = xdr.NewWriter(&buf)
+	o.encodeXDR(xw)
+	return buf.Bytes()
+}
+
+func (o AnnounceV2) encodeXDR(xw *xdr.Writer) (int, error) {
+	xw.WriteUint32(o.Magic)
+	if len(o.NodeID) > 64 {
+		return xw.Tot(), xdr.ErrElementSizeExceeded
+	}
+	xw.WriteString(o.NodeID)
+	if len(o.Addresses) > 16 {
+		return xw.Tot(), xdr.ErrElementSizeExceeded
+	}
+	xw.WriteUint32(uint32(len(o.Addresses)))
+	for i := range o.Addresses {
+		o.Addresses[i].encodeXDR(xw)
+	}
+	return xw.Tot(), xw.Error()
+}
+
+func (o *AnnounceV2) DecodeXDR(r io.Reader) error {
+	xr := xdr.NewReader(r)
+	return o.decodeXDR(xr)
+}
+
+func (o *AnnounceV2) UnmarshalXDR(bs []byte) error {
+	var buf = bytes.NewBuffer(bs)
+	var xr = xdr.NewReader(buf)
+	return o.decodeXDR(xr)
+}
+
+func (o *AnnounceV2) decodeXDR(xr *xdr.Reader) error {
+	o.Magic = xr.ReadUint32()
+	o.NodeID = xr.ReadStringMax(64)
+	_AddressesSize := int(xr.ReadUint32())
+	if _AddressesSize > 16 {
+		return xdr.ErrElementSizeExceeded
+	}
+	o.Addresses = make([]Address, _AddressesSize)
+	for i := range o.Addresses {
+		(&o.Addresses[i]).decodeXDR(xr)
+	}
+	return xr.Error()
+}
+
+func (o Address) EncodeXDR(w io.Writer) (int, error) {
+	var xw = xdr.NewWriter(w)
+	return o.encodeXDR(xw)
+}
+
+func (o Address) MarshalXDR() []byte {
+	var buf bytes.Buffer
+	var xw = xdr.NewWriter(&buf)
+	o.encodeXDR(xw)
+	return buf.Bytes()
+}
+
+func (o Address) encodeXDR(xw *xdr.Writer) (int, error) {
+	if len(o.IP) > 16 {
+		return xw.Tot(), xdr.ErrElementSizeExceeded
+	}
+	xw.WriteBytes(o.IP)
+	xw.WriteUint16(o.Port)
+	return xw.Tot(), xw.Error()
+}
+
+func (o *Address) DecodeXDR(r io.Reader) error {
+	xr := xdr.NewReader(r)
+	return o.decodeXDR(xr)
+}
+
+func (o *Address) UnmarshalXDR(bs []byte) error {
+	var buf = bytes.NewBuffer(bs)
+	var xr = xdr.NewReader(buf)
+	return o.decodeXDR(xr)
+}
+
+func (o *Address) decodeXDR(xr *xdr.Reader) error {
+	o.IP = xr.ReadBytesMax(16)
+	o.Port = xr.ReadUint16()
+	return xr.Error()
+}

files/set.go

@@ -0,0 +1,173 @@
+package fileset
+
+import "sync"
+
+type File struct {
+	Key      Key
+	Modified int64
+	Flags    uint32
+	Data     interface{}
+}
+
+type Key struct {
+	Name    string
+	Version uint32
+}
+
+type fileRecord struct {
+	Usage int
+	File  File
+}
+
+type bitset uint64
+
+func (a Key) newerThan(b Key) bool {
+	return a.Version > b.Version
+}
+
+type Set struct {
+	mutex              sync.RWMutex
+	files              map[Key]fileRecord
+	remoteKey          [64]map[string]Key
+	globalAvailability map[string]bitset
+	globalKey          map[string]Key
+}
+
+func NewSet() *Set {
+	var m = Set{
+		files:              make(map[Key]fileRecord),
+		globalAvailability: make(map[string]bitset),
+		globalKey:          make(map[string]Key),
+	}
+	return &m
+}
+
+func (m *Set) AddLocal(fs []File) {
+	m.mutex.Lock()
+	m.unlockedAddRemote(0, fs)
+	m.mutex.Unlock()
+}
+
+func (m *Set) SetLocal(fs []File) {
+	m.mutex.Lock()
+	m.unlockedSetRemote(0, fs)
+	m.mutex.Unlock()
+}
+
+func (m *Set) AddRemote(cid uint, fs []File) {
+	if cid < 1 || cid > 63 {
+		panic("Connection ID must be in the range 1 - 63 inclusive")
+	}
+	m.mutex.Lock()
+	m.unlockedAddRemote(cid, fs)
+	m.mutex.Unlock()
+}
+
+func (m *Set) SetRemote(cid uint, fs []File) {
+	if cid < 1 || cid > 63 {
+		panic("Connection ID must be in the range 1 - 63 inclusive")
+	}
+	m.mutex.Lock()
+	m.unlockedSetRemote(cid, fs)
+	m.mutex.Unlock()
+}
+
+func (m *Set) unlockedAddRemote(cid uint, fs []File) {
+	remFiles := m.remoteKey[cid]
+	for _, f := range fs {
+		n := f.Key.Name
+
+		if ck, ok := remFiles[n]; ok && ck == f.Key {
+			// The remote already has exactly this file, skip it
+			continue
+		}
+
+		remFiles[n] = f.Key
+
+		// Keep the block list or increment the usage
+		if br, ok := m.files[f.Key]; !ok {
+			m.files[f.Key] = fileRecord{
+				Usage: 1,
+				File:  f,
+			}
+		} else {
+			br.Usage++
+			m.files[f.Key] = br
+		}
+
+		// Update global view
+		gk, ok := m.globalKey[n]
+		switch {
+		case ok && f.Key == gk:
+			av := m.globalAvailability[n]
+			av |= 1 << cid
+			m.globalAvailability[n] = av
+		case f.Key.newerThan(gk):
+			m.globalKey[n] = f.Key
+			m.globalAvailability[n] = 1 << cid
+		}
+	}
+}
+
+func (m *Set) unlockedSetRemote(cid uint, fs []File) {
+	// Decrement usage for all files belonging to this remote, and remove
+	// those that are no longer needed.
+	for _, fk := range m.remoteKey[cid] {
+		br, ok := m.files[fk]
+		switch {
+		case ok && br.Usage == 1:
+			delete(m.files, fk)
+		case ok && br.Usage > 1:
+			br.Usage--
+			m.files[fk] = br
+		}
+	}
+
+	// Clear existing remote remoteKey
+	m.remoteKey[cid] = make(map[string]Key)
+
+	// Recalculate global based on all remaining remoteKey
+	for n := range m.globalKey {
+		var nk Key    // newest key
+		var na bitset // newest availability
+
+		for i, rem := range m.remoteKey {
+			if rk, ok := rem[n]; ok {
+				switch {
+				case rk == nk:
+					na |= 1 << uint(i)
+				case rk.newerThan(nk):
+					nk = rk
+					na = 1 << uint(i)
+				}
+			}
+		}
+
+		if na != 0 {
+			// Someone had the file
+			m.globalKey[n] = nk
+			m.globalAvailability[n] = na
+		} else {
+			// Noone had the file
+			delete(m.globalKey, n)
+			delete(m.globalAvailability, n)
+		}
+	}
+
+	// Add new remote remoteKey to the mix
+	m.unlockedAddRemote(cid, fs)
+}
+
+func (m *Set) Need(cid uint) []File {
+	var fs []File
+	m.mutex.Lock()
+
+	for name, gk := range m.globalKey {
+		if gk.newerThan(m.remoteKey[cid][name]) {
+			fs = append(fs, m.files[gk].File)
+		}
+	}
+
+	m.mutex.Unlock()
+	return fs
+}

files/set_test.go

@@ -0,0 +1,207 @@
+package fileset
+
+import (
+	"fmt"
+	"reflect"
+	"testing"
+)
+
+func TestGlobalSet(t *testing.T) {
+	m := NewSet()
+
+	local := []File{
+		File{Key{"a", 1000}, 0, 0, nil},
+		File{Key{"b", 1000}, 0, 0, nil},
+		File{Key{"c", 1000}, 0, 0, nil},
+		File{Key{"d", 1000}, 0, 0, nil},
+	}
+
+	remote := []File{
+		File{Key{"a", 1000}, 0, 0, nil},
+		File{Key{"b", 1001}, 0, 0, nil},
+		File{Key{"c", 1002}, 0, 0, nil},
+		File{Key{"e", 1000}, 0, 0, nil},
+	}
+
+	expectedGlobal := map[string]Key{
+		"a": local[0].Key,
+		"b": remote[1].Key,
+		"c": remote[2].Key,
+		"d": local[3].Key,
+		"e": remote[3].Key,
+	}
+
+	m.SetLocal(local)
+	m.SetRemote(1, remote)
+
+	if !reflect.DeepEqual(m.globalKey, expectedGlobal) {
+		t.Errorf("Global incorrect;\n%v !=\n%v", m.globalKey, expectedGlobal)
+	}
+
+	if lb := len(m.files); lb != 7 {
+		t.Errorf("Num files incorrect %d != 7\n%v", lb, m.files)
+	}
+}
+
+func BenchmarkSetLocal10k(b *testing.B) {
+	m := NewSet()
+
+	var local []File
+	for i := 0; i < 10000; i++ {
+		local = append(local, File{Key{fmt.Sprintf("file%d"), 1000}, 0, 0, nil})
+	}
+
+	var remote []File
+	for i := 0; i < 10000; i++ {
+		remote = append(remote, File{Key{fmt.Sprintf("file%d"), 1000}, 0, 0, nil})
+	}
+
+	m.SetRemote(1, remote)
+
+	b.ResetTimer()
+	for i := 0; i < b.N; i++ {
+		m.SetLocal(local)
+	}
+}
+
+func BenchmarkSetLocal10(b *testing.B) {
+	m := NewSet()
+
+	var local []File
+	for i := 0; i < 10; i++ {
+		local = append(local, File{Key{fmt.Sprintf("file%d"), 1000}, 0, 0, nil})
+	}
+
+	var remote []File
+	for i := 0; i < 10000; i++ {
+		remote = append(remote, File{Key{fmt.Sprintf("file%d"), 1000}, 0, 0, nil})
+	}
+
+	m.SetRemote(1, remote)
+
+	b.ResetTimer()
+	for i := 0; i < b.N; i++ {
+		m.SetLocal(local)
+	}
+}
+
+func BenchmarkAddLocal10k(b *testing.B) {
+	m := NewSet()
+
+	var local []File
+	for i := 0; i < 10000; i++ {
+		local = append(local, File{Key{fmt.Sprintf("file%d"), 1000}, 0, 0, nil})
+	}
+
+	var remote []File
+	for i := 0; i < 10000; i++ {
+		remote = append(remote, File{Key{fmt.Sprintf("file%d"), 1000}, 0, 0, nil})
+	}
+
+	m.SetRemote(1, remote)
+	m.SetLocal(local)
+
+	b.ResetTimer()
+	for i := 0; i < b.N; i++ {
+		b.StopTimer()
+		for j := range local {
+			local[j].Key.Version++
+		}
+		b.StartTimer()
+		m.AddLocal(local)
+	}
+}
+
+func BenchmarkAddLocal10(b *testing.B) {
+	m := NewSet()
+
+	var local []File
+	for i := 0; i < 10; i++ {
+		local = append(local, File{Key{fmt.Sprintf("file%d"), 1000}, 0, 0, nil})
+	}
+
+	var remote []File
+	for i := 0; i < 10000; i++ {
+		remote = append(remote, File{Key{fmt.Sprintf("file%d"), 1000}, 0, 0, nil})
+	}
+
+	m.SetRemote(1, remote)
+	m.SetLocal(local)
+
+	b.ResetTimer()
+	for i := 0; i < b.N; i++ {
+		for j := range local {
+			local[j].Key.Version++
+		}
+		m.AddLocal(local)
+	}
+}
+
+func TestGlobalReset(t *testing.T) {
+	m := NewSet()
+
+	local := []File{
+		File{Key{"a", 1000}, 0, 0, nil},
+		File{Key{"b", 1000}, 0, 0, nil},
+		File{Key{"c", 1000}, 0, 0, nil},
+		File{Key{"d", 1000}, 0, 0, nil},
+	}
+
+	remote := []File{
+		File{Key{"a", 1000}, 0, 0, nil},
+		File{Key{"b", 1001}, 0, 0, nil},
+		File{Key{"c", 1002}, 0, 0, nil},
+		File{Key{"e", 1000}, 0, 0, nil},
+	}
+
+	expectedGlobalKey := map[string]Key{
+		"a": local[0].Key,
+		"b": local[1].Key,
+		"c": local[2].Key,
+		"d": local[3].Key,
+	}
+
+	m.SetLocal(local)
+	m.SetRemote(1, remote)
+	m.SetRemote(1, nil)
+
+	if !reflect.DeepEqual(m.globalKey, expectedGlobalKey) {
+		t.Errorf("Global incorrect;\n%v !=\n%v", m.globalKey, expectedGlobalKey)
+	}
+
+	if lb := len(m.files); lb != 4 {
+		t.Errorf("Num files incorrect %d != 4\n%v", lb, m.files)
+	}
+}
+
+func TestNeed(t *testing.T) {
+	m := NewSet()
+
+	local := []File{
+		File{Key{"a", 1000}, 0, 0, nil},
+		File{Key{"b", 1000}, 0, 0, nil},
+		File{Key{"c", 1000}, 0, 0, nil},
+		File{Key{"d", 1000}, 0, 0, nil},
+	}
+
+	remote := []File{
+		File{Key{"a", 1000}, 0, 0, nil},
+		File{Key{"b", 1001}, 0, 0, nil},
+		File{Key{"c", 1002}, 0, 0, nil},
+		File{Key{"e", 1000}, 0, 0, nil},
+	}
+
+	shouldNeed := []File{
+		File{Key{"b", 1001}, 0, 0, nil},
+		File{Key{"c", 1002}, 0, 0, nil},
+		File{Key{"e", 1000}, 0, 0, nil},
+	}
+
+	m.SetLocal(local)
+	m.SetRemote(1, remote)
+
+	need := m.Need(0)
+	if !reflect.DeepEqual(need, shouldNeed) {
+		t.Errorf("Need incorrect;\n%v !=\n%v", need, shouldNeed)
+	}
+}

github.com/jessevdk/go-flags/LICENSE

@@ -1,26 +0,0 @@
-Copyright (c) 2012 Jesse van den Kieboom. All rights reserved.
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are
-met:
-
-   * Redistributions of source code must retain the above copyright
-     notice, this list of conditions and the following disclaimer.
-   * Redistributions in binary form must reproduce the above
-     copyright notice, this list of conditions and the following disclaimer
-     in the documentation and/or other materials provided with the
-     distribution.
-   * Neither the name of Google Inc. nor the names of its
-     contributors may be used to endorse or promote products derived from
-     this software without specific prior written permission.
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

github.com/jessevdk/go-flags/README.md

@@ -1,128 +0,0 @@
-go-flags: a go library for parsing command line arguments
-=========================================================
-
-This library provides similar functionality to the builtin flag library of
-go, but provides much more functionality and nicer formatting. From the
-documentation:
-
-Package flags provides an extensive command line option parser.
-The flags package is similar in functionality to the go builtin flag package
-but provides more options and uses reflection to provide a convenient and
-succinct way of specifying command line options.
-
-Supported features:
-* Options with short names (-v)
-* Options with long names (--verbose)
-* Options with and without arguments (bool v.s. other type)
-* Options with optional arguments and default values
-* Multiple option groups each containing a set of options
-* Generate and print well-formatted help message
-* Passing remaining command line arguments after -- (optional)
-* Ignoring unknown command line options (optional)
-* Supports -I/usr/include -I=/usr/include -I /usr/include option argument specification
-* Supports multiple short options -aux
-* Supports all primitive go types (string, int{8..64}, uint{8..64}, float)
-* Supports same option multiple times (can store in slice or last option counts)
-* Supports maps
-* Supports function callbacks
-
-The flags package uses structs, reflection and struct field tags
-to allow users to specify command line options. This results in very simple
-and consise specification of your application options. For example:
-
-    type Options struct {
-        Verbose []bool `short:"v" long:"verbose" description:"Show verbose debug information"`
-    }
-
-This specifies one option with a short name -v and a long name --verbose.
-When either -v or --verbose is found on the command line, a 'true' value
-will be appended to the Verbose field. e.g. when specifying -vvv, the
-resulting value of Verbose will be {[true, true, true]}.
-
-Example:
---------
-	var opts struct {
-		// Slice of bool will append 'true' each time the option
-		// is encountered (can be set multiple times, like -vvv)
-		Verbose []bool `short:"v" long:"verbose" description:"Show verbose debug information"`
-
-		// Example of automatic marshalling to desired type (uint)
-		Offset uint `long:"offset" description:"Offset"`
-
-		// Example of a callback, called each time the option is found.
-		Call func(string) `short:"c" description:"Call phone number"`
-
-		// Example of a required flag
-		Name string `short:"n" long:"name" description:"A name" required:"true"`
-
-		// Example of a value name
-		File string `short:"f" long:"file" description:"A file" value-name:"FILE"`
-
-		// Example of a pointer
-		Ptr *int `short:"p" description:"A pointer to an integer"`
-
-		// Example of a slice of strings
-		StringSlice []string `short:"s" description:"A slice of strings"`
-
-		// Example of a slice of pointers
-		PtrSlice []*string `long:"ptrslice" description:"A slice of pointers to string"`
-
-		// Example of a map
-		IntMap map[string]int `long:"intmap" description:"A map from string to int"`
-	}
-
-	// Callback which will invoke callto:<argument> to call a number.
-	// Note that this works just on OS X (and probably only with
-	// Skype) but it shows the idea.
-	opts.Call = func(num string) {
-		cmd := exec.Command("open", "callto:"+num)
-		cmd.Start()
-		cmd.Process.Release()
-	}
-
-	// Make some fake arguments to parse.
-	args := []string{
-		"-vv",
-		"--offset=5",
-		"-n", "Me",
-		"-p", "3",
-		"-s", "hello",
-		"-s", "world",
-		"--ptrslice", "hello",
-		"--ptrslice", "world",
-		"--intmap", "a:1",
-		"--intmap", "b:5",
-		"arg1",
-		"arg2",
-		"arg3",
-	}
-
-	// Parse flags from `args'. Note that here we use flags.ParseArgs for
-	// the sake of making a working example. Normally, you would simply use
-	// flags.Parse(&opts) which uses os.Args
-	args, err := flags.ParseArgs(&opts, args)
-
-	if err != nil {
-		panic(err)
-		os.Exit(1)
-	}
-
-	fmt.Printf("Verbosity: %v\n", opts.Verbose)
-	fmt.Printf("Offset: %d\n", opts.Offset)
-	fmt.Printf("Name: %s\n", opts.Name)
-	fmt.Printf("Ptr: %d\n", *opts.Ptr)
-	fmt.Printf("StringSlice: %v\n", opts.StringSlice)
-	fmt.Printf("PtrSlice: [%v %v]\n", *opts.PtrSlice[0], *opts.PtrSlice[1])
-	fmt.Printf("IntMap: [a:%v b:%v]\n", opts.IntMap["a"], opts.IntMap["b"])
-	fmt.Printf("Remaining args: %s\n", strings.Join(args, " "))
-
-	// Output: Verbosity: [true true]
-	// Offset: 5
-	// Name: Me
-	// Ptr: 3
-	// StringSlice: [hello world]
-	// PtrSlice: [hello world]
-	// IntMap: [a:1 b:5]
-	// Remaining args: arg1 arg2 arg3
-
-More information can be found in the godocs: <http://godoc.org/github.com/jessevdk/go-flags>

github.com/jessevdk/go-flags/assert_test.go

@@ -1,82 +0,0 @@
-package flags
-
-import (
-	"testing"
-)
-
-func assertString(t *testing.T, a string, b string) {
-	if a != b {
-		t.Errorf("Expected %#v, but got %#v", b, a)
-	}
-}
-func assertStringArray(t *testing.T, a []string, b []string) {
-	if len(a) != len(b) {
-		t.Errorf("Expected %#v, but got %#v", b, a)
-		return
-	}
-
-	for i, v := range a {
-		if b[i] != v {
-			t.Errorf("Expected %#v, but got %#v", b, a)
-			return
-		}
-	}
-}
-
-func assertBoolArray(t *testing.T, a []bool, b []bool) {
-	if len(a) != len(b) {
-		t.Errorf("Expected %#v, but got %#v", b, a)
-		return
-	}
-
-	for i, v := range a {
-		if b[i] != v {
-			t.Errorf("Expected %#v, but got %#v", b, a)
-			return
-		}
-	}
-}
-
-func assertParserSuccess(t *testing.T, data interface{}, args ...string) (*Parser, []string) {
-	parser := NewParser(data, Default&^PrintErrors)
-	ret, err := parser.ParseArgs(args)
-
-	if err != nil {
-		t.Fatalf("Unexpected parse error: %s", err)
-		return nil, nil
-	}
-
-	return parser, ret
-}
-
-func assertParseSuccess(t *testing.T, data interface{}, args ...string) []string {
-	_, ret := assertParserSuccess(t, data, args...)
-	return ret
-}
-
-func assertError(t *testing.T, err error, typ ErrorType, msg string) {
-	if err == nil {
-		t.Fatalf("Expected error: %s", msg)
-		return
-	}
-
-	if e, ok := err.(*Error); !ok {
-		t.Fatalf("Expected Error type, but got %#v", err)
-		return
-	} else {
-		if e.Type != typ {
-			t.Errorf("Expected error type {%s}, but got {%s}", typ, e.Type)
-		}
-
-		if e.Message != msg {
-			t.Errorf("Expected error message %#v, but got %#v", msg, e.Message)
-		}
-	}
-}
-
-func assertParseFail(t *testing.T, typ ErrorType, msg string, data interface{}, args ...string) {
-	parser := NewParser(data, Default&^PrintErrors)
-	_, err := parser.ParseArgs(args)
-
-	assertError(t, err, typ, msg)
-}

github.com/jessevdk/go-flags/check_crosscompile.sh

@@ -1,16 +0,0 @@
-#!/bin/bash
-
-set -e
-
-echo '# linux arm7'
-GOARM=7 GOARCH=arm GOOS=linux go build
-echo '# linux arm5'
-GOARM=5 GOARCH=arm GOOS=linux go build
-echo '# windows 386'
-GOARCH=386 GOOS=windows go build
-echo '# windows amd64'
-GOARCH=amd64 GOOS=windows go build
-echo '# darwin'
-GOARCH=amd64 GOOS=darwin go build
-echo '# freebsd'
-GOARCH=amd64 GOOS=freebsd go build

github.com/jessevdk/go-flags/closest.go

@@ -1,61 +0,0 @@
-package flags
-
-func levenshtein(s string, t string) int {
-	if len(s) == 0 {
-		return len(t)
-	}
-
-	if len(t) == 0 {
-		return len(s)
-	}
-
-	var l1, l2, l3 int
-
-	if len(s) == 1 {
-		l1 = len(t) + 1
-	} else {
-		l1 = levenshtein(s[1:len(s)-1], t) + 1
-	}
-
-	if len(t) == 1 {
-		l2 = len(s) + 1
-	} else {
-		l2 = levenshtein(t[1:len(t)-1], s) + 1
-	}
-
-	l3 = levenshtein(s[1:len(s)], t[1:len(t)])
-
-	if s[0] != t[0] {
-		l3 += 1
-	}
-
-	if l2 < l1 {
-		l1 = l2
-	}
-
-	if l1 < l3 {
-		return l1
-	}
-
-	return l3
-}
-
-func closestChoice(cmd string, choices []string) (string, int) {
-	if len(choices) == 0 {
-		return "", 0
-	}
-
-	mincmd := -1
-	mindist := -1
-
-	for i, c := range choices {
-		l := levenshtein(cmd, c)
-
-		if mincmd < 0 || l < mindist {
-			mindist = l
-			mincmd = i
-		}
-	}
-
-	return choices[mincmd], mindist
-}

github.com/jessevdk/go-flags/command.go

@@ -1,84 +0,0 @@
-package flags
-
-// Command represents an application command. Commands can be added to the
-// parser (which itself is a command) and are selected/executed when its name
-// is specified on the command line. The Command type embeds a Group and
-// therefore also carries a set of command specific options.
-type Command struct {
-	// Embedded, see Group for more information
-	*Group
-
-	// The name by which the command can be invoked
-	Name string
-
-	// The active sub command (set by parsing) or nil
-	Active *Command
-
-	commands            []*Command
-	hasBuiltinHelpGroup bool
-}
-
-// Commander is an interface which can be implemented by any command added in
-// the options. When implemented, the Execute method will be called for the last
-// specified (sub)command providing the remaining command line arguments.
-type Commander interface {
-	// Execute will be called for the last active (sub)command. The
-	// args argument contains the remaining command line arguments. The
-	// error that Execute returns will be eventually passed out of the
-	// Parse method of the Parser.
-	Execute(args []string) error
-}
-
-// Usage is an interface which can be implemented to show a custom usage string
-// in the help message shown for a command.
-type Usage interface {
-	// Usage is called for commands to allow customized printing of command
-	// usage in the generated help message.
-	Usage() string
-}
-
-// AddCommand adds a new command to the parser with the given name and data. The
-// data needs to be a pointer to a struct from which the fields indicate which
-// options are in the command. The provided data can implement the Command and
-// Usage interfaces.
-func (c *Command) AddCommand(command string, shortDescription string, longDescription string, data interface{}) (*Command, error) {
-	cmd := newCommand(command, shortDescription, longDescription, data)
-
-	if err := cmd.scan(); err != nil {
-		return nil, err
-	}
-
-	c.commands = append(c.commands, cmd)
-	return cmd, nil
-}
-
-// AddGroup adds a new group to the command with the given name and data. The
-// data needs to be a pointer to a struct from which the fields indicate which
-// options are in the group.
-func (c *Command) AddGroup(shortDescription string, longDescription string, data interface{}) (*Group, error) {
-	group := newGroup(shortDescription, longDescription, data)
-
-	if err := group.scanType(c.scanSubCommandHandler(group)); err != nil {
-		return nil, err
-	}
-
-	c.groups = append(c.groups, group)
-	return group, nil
-}
-
-// Commands returns a list of subcommands of this command.
-func (c *Command) Commands() []*Command {
-	return c.commands
-}
-
-// Find locates the subcommand with the given name and returns it. If no such
-// command can be found Find will return nil.
-func (c *Command) Find(name string) *Command {
-	for _, cc := range c.commands {
-		if cc.Name == name {
-			return cc
-		}
-	}
-
-	return nil
-}

github.com/jessevdk/go-flags/command_private.go

@@ -1,161 +0,0 @@
-package flags
-
-import (
-	"reflect"
-	"sort"
-	"strings"
-	"unsafe"
-)
-
-type lookup struct {
-	shortNames map[string]*Option
-	longNames  map[string]*Option
-
-	required map[*Option]bool
-	commands map[string]*Command
-}
-
-func newCommand(name string, shortDescription string, longDescription string, data interface{}) *Command {
-	return &Command{
-		Group: newGroup(shortDescription, longDescription, data),
-		Name:  name,
-	}
-}
-
-func (c *Command) scanSubCommandHandler(parentg *Group) scanHandler {
-	f := func(realval reflect.Value, sfield *reflect.StructField) (bool, error) {
-		mtag := newMultiTag(string(sfield.Tag))
-
-		if err := mtag.Parse(); err != nil {
-			return true, err
-		}
-
-		subcommand := mtag.Get("command")
-
-		if len(subcommand) != 0 {
-			ptrval := reflect.NewAt(realval.Type(), unsafe.Pointer(realval.UnsafeAddr()))
-
-			shortDescription := mtag.Get("description")
-			longDescription := mtag.Get("long-description")
-
-			if _, err := c.AddCommand(subcommand, shortDescription, longDescription, ptrval.Interface()); err != nil {
-				return true, err
-			}
-
-			return true, nil
-		}
-
-		return parentg.scanSubGroupHandler(realval, sfield)
-	}
-
-	return f
-}
-
-func (c *Command) scan() error {
-	return c.scanType(c.scanSubCommandHandler(c.Group))
-}
-
-func (c *Command) eachCommand(f func(*Command), recurse bool) {
-	f(c)
-
-	for _, cc := range c.commands {
-		if recurse {
-			cc.eachCommand(f, true)
-		} else {
-			f(cc)
-		}
-	}
-}
-
-func (c *Command) eachActiveGroup(f func(g *Group)) {
-	c.eachGroup(f)
-
-	if c.Active != nil {
-		c.Active.eachActiveGroup(f)
-	}
-}
-
-func (c *Command) addHelpGroups(showHelp func() error) {
-	if !c.hasBuiltinHelpGroup {
-		c.addHelpGroup(showHelp)
-		c.hasBuiltinHelpGroup = true
-	}
-
-	for _, cc := range c.commands {
-		cc.addHelpGroups(showHelp)
-	}
-}
-
-func (c *Command) makeLookup() lookup {
-	ret := lookup{
-		shortNames: make(map[string]*Option),
-		longNames:  make(map[string]*Option),
-
-		required: make(map[*Option]bool),
-		commands: make(map[string]*Command),
-	}
-
-	c.eachGroup(func(g *Group) {
-		for _, option := range g.options {
-			if option.Required && option.canCli() {
-				ret.required[option] = true
-			}
-
-			if option.ShortName != 0 {
-				ret.shortNames[string(option.ShortName)] = option
-			}
-
-			if len(option.LongName) > 0 {
-				ret.longNames[option.LongName] = option
-			}
-		}
-	})
-
-	for _, subcommand := range c.commands {
-		ret.commands[subcommand.Name] = subcommand
-	}
-
-	return ret
-}
-
-func (c *Command) groupByName(name string) *Group {
-	if grp := c.Group.groupByName(name); grp != nil {
-		return grp
-	}
-
-	for _, subc := range c.commands {
-		prefix := subc.Name + "."
-
-		if strings.HasPrefix(name, prefix) {
-			if grp := subc.groupByName(name[len(prefix):]); grp != nil {
-				return grp
-			}
-		} else if name == subc.Name {
-			return subc.Group
-		}
-	}
-
-	return nil
-}
-
-type commandList []*Command
-
-func (c commandList) Less(i, j int) bool {
-	return c[i].Name < c[j].Name
-}
-
-func (c commandList) Len() int {
-	return len(c)
-}
-
-func (c commandList) Swap(i, j int) {
-	c[i], c[j] = c[j], c[i]
-}
-
-func (c *Command) sortedCommands() []*Command {
-	ret := make(commandList, len(c.commands))
-	copy(ret, c.commands)
-
-	sort.Sort(ret)
-	return []*Command(ret)
-}

github.com/jessevdk/go-flags/command_test.go

@@ -1,255 +0,0 @@
-package flags
-
-import (
-	"testing"
-)
-
-func TestCommandInline(t *testing.T) {
-	var opts = struct {
-		Value bool `short:"v"`
-
-		Command struct {
-			G bool `short:"g"`
-		} `command:"cmd"`
-	}{}
-
-	p, ret := assertParserSuccess(t, &opts, "-v", "cmd", "-g")
-
-	assertStringArray(t, ret, []string{})
-
-	if p.Active == nil {
-		t.Errorf("Expected active command")
-	}
-
-	if !opts.Value {
-		t.Errorf("Expected Value to be true")
-	}
-
-	if !opts.Command.G {
-		t.Errorf("Expected Command.G to be true")
-	}
-
-	if p.Command.Find("cmd") != p.Active {
-		t.Errorf("Expected to find command `cmd' to be active")
-	}
-}
-
-func TestCommandInlineMulti(t *testing.T) {
-	var opts = struct {
-		Value bool `short:"v"`
-
-		C1 struct {
-		} `command:"c1"`
-
-		C2 struct {
-			G bool `short:"g"`
-		} `command:"c2"`
-	}{}
-
-	p, ret := assertParserSuccess(t, &opts, "-v", "c2", "-g")
-
-	assertStringArray(t, ret, []string{})
-
-	if p.Active == nil {
-		t.Errorf("Expected active command")
-	}
-
-	if !opts.Value {
-		t.Errorf("Expected Value to be true")
-	}
-
-	if !opts.C2.G {
-		t.Errorf("Expected C2.G to be true")
-	}
-
-	if p.Command.Find("c1") == nil {
-		t.Errorf("Expected to find command `c1'")
-	}
-
-	if c2 := p.Command.Find("c2"); c2 == nil {
-		t.Errorf("Expected to find command `c2'")
-	} else if c2 != p.Active {
-		t.Errorf("Expected to find command `c2' to be active")
-	}
-}
-
-func TestCommandFlagOrder1(t *testing.T) {
-	var opts = struct {
-		Value bool `short:"v"`
-
-		Command struct {
-			G bool `short:"g"`
-		} `command:"cmd"`
-	}{}
-
-	assertParseFail(t, ErrUnknownFlag, "unknown flag `g'", &opts, "-v", "-g", "cmd")
-}
-
-func TestCommandFlagOrder2(t *testing.T) {
-	var opts = struct {
-		Value bool `short:"v"`
-
-		Command struct {
-			G bool `short:"g"`
-		} `command:"cmd"`
-	}{}
-
-	assertParseFail(t, ErrUnknownFlag, "unknown flag `v'", &opts, "cmd", "-v", "-g")
-}
-
-func TestCommandEstimate(t *testing.T) {
-	var opts = struct {
-		Value bool `short:"v"`
-
-		Cmd1 struct {
-		} `command:"remove"`
-
-		Cmd2 struct {
-		} `command:"add"`
-	}{}
-
-	p := NewParser(&opts, None)
-	_, err := p.ParseArgs([]string{})
-
-	assertError(t, err, ErrRequired, "Please specify one command of: add or remove")
-}
-
-type testCommand struct {
-	G        bool `short:"g"`
-	Executed bool
-	EArgs    []string
-}
-
-func (c *testCommand) Execute(args []string) error {
-	c.Executed = true
-	c.EArgs = args
-
-	return nil
-}
-
-func TestCommandExecute(t *testing.T) {
-	var opts = struct {
-		Value bool `short:"v"`
-
-		Command testCommand `command:"cmd"`
-	}{}
-
-	assertParseSuccess(t, &opts, "-v", "cmd", "-g", "a", "b")
-
-	if !opts.Value {
-		t.Errorf("Expected Value to be true")
-	}
-
-	if !opts.Command.Executed {
-		t.Errorf("Did not execute command")
-	}
-
-	if !opts.Command.G {
-		t.Errorf("Expected Command.C to be true")
-	}
-
-	assertStringArray(t, opts.Command.EArgs, []string{"a", "b"})
-}
-
-func TestCommandClosest(t *testing.T) {
-	var opts = struct {
-		Value bool `short:"v"`
-
-		Cmd1 struct {
-		} `command:"remove"`
-
-		Cmd2 struct {
-		} `command:"add"`
-	}{}
-
-	assertParseFail(t, ErrRequired, "Unknown command `addd', did you mean `add'?", &opts, "-v", "addd")
-}
-
-func TestCommandAdd(t *testing.T) {
-	var opts = struct {
-		Value bool `short:"v"`
-	}{}
-
-	var cmd = struct {
-		G bool `short:"g"`
-	}{}
-
-	p := NewParser(&opts, Default)
-	c, err := p.AddCommand("cmd", "", "", &cmd)
-
-	if err != nil {
-		t.Fatalf("Unexpected error: %v", err)
-		return
-	}
-
-	ret, err := p.ParseArgs([]string{"-v", "cmd", "-g", "rest"})
-
-	if err != nil {
-		t.Fatalf("Unexpected error: %v", err)
-		return
-	}
-
-	assertStringArray(t, ret, []string{"rest"})
-
-	if !opts.Value {
-		t.Errorf("Expected Value to be true")
-	}
-
-	if !cmd.G {
-		t.Errorf("Expected Command.G to be true")
-	}
-
-	if p.Command.Find("cmd") != c {
-		t.Errorf("Expected to find command `cmd'")
-	}
-
-	if p.Commands()[0] != c {
-		t.Errorf("Espected command #v, but got #v", c, p.Commands()[0])
-	}
-
-	if c.Options()[0].ShortName != 'g' {
-		t.Errorf("Expected short name `g' but got %v", c.Options()[0].ShortName)
-	}
-}
-
-func TestCommandNestedInline(t *testing.T) {
-	var opts = struct {
-		Value bool `short:"v"`
-
-		Command struct {
-			G bool `short:"g"`
-
-			Nested struct {
-				N string `long:"n"`
-			} `command:"nested"`
-		} `command:"cmd"`
-	}{}
-
-	p, ret := assertParserSuccess(t, &opts, "-v", "cmd", "-g", "nested", "--n", "n", "rest")
-
-	assertStringArray(t, ret, []string{"rest"})
-
-	if !opts.Value {
-		t.Errorf("Expected Value to be true")
-	}
-
-	if !opts.Command.G {
-		t.Errorf("Expected Command.G to be true")
-	}
-
-	assertString(t, opts.Command.Nested.N, "n")
-
-	if c := p.Command.Find("cmd"); c == nil {
-		t.Errorf("Expected to find command `cmd'")
-	} else {
-		if c != p.Active {
-			t.Errorf("Expected `cmd' to be the active parser command")
-		}
-
-		if nested := c.Find("nested"); nested == nil {
-			t.Errorf("Expected to find command `nested'")
-		} else if nested != c.Active {
-			t.Errorf("Expected to find command `nested' to be the active `cmd' command")
-		}
-	}
-}

github.com/jessevdk/go-flags/convert.go

@@ -1,315 +0,0 @@
-// Copyright 2012 Jesse van den Kieboom. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package flags
-
-import (
-	"fmt"
-	"reflect"
-	"strconv"
-	"strings"
-	"time"
-)
-
-// Marshaler is the interface implemented by types that can marshal themselves
-// to a string representation of the flag.
-type Marshaler interface {
-	// MarshalFlag marshals a flag value to its string representation.
-	MarshalFlag() (string, error)
-}
-
-// Unmarshaler is the interface implemented by types that can unmarshal a flag
-// argument to themselves. The provided value is directly passed from the
-// command line.
-type Unmarshaler interface {
-	// UnmarshalFlag unmarshals a string value representation to the flag
-	// value (which therefore needs to be a pointer receiver).
-	UnmarshalFlag(value string) error
-}
-
-func getBase(options multiTag, base int) (int, error) {
-	sbase := options.Get("base")
-
-	var err error
-	var ivbase int64
-
-	if sbase != "" {
-		ivbase, err = strconv.ParseInt(sbase, 10, 32)
-		base = int(ivbase)
-	}
-
-	return base, err
-}
-
-func convertMarshal(val reflect.Value) (bool, string, error) {
-	// Check first for the Marshaler interface
-	if val.Type().NumMethod() > 0 && val.CanInterface() {
-		if marshaler, ok := val.Interface().(Marshaler); ok {
-			ret, err := marshaler.MarshalFlag()
-			return true, ret, err
-		}
-	}
-
-	return false, "", nil
-}
-
-func convertToString(val reflect.Value, options multiTag) (string, error) {
-	if ok, ret, err := convertMarshal(val); ok {
-		return ret, err
-	}
-
-	tp := val.Type()
-
-	// Support for time.Duration
-	if tp == reflect.TypeOf((*time.Duration)(nil)).Elem() {
-		stringer := val.Interface().(fmt.Stringer)
-		return stringer.String(), nil
-	}
-
-	switch tp.Kind() {
-	case reflect.String:
-		return val.String(), nil
-	case reflect.Bool:
-		if val.Bool() {
-			return "true", nil
-		}
-
-		return "false", nil
-	case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
-		base, _ := getBase(options, 10)
-		return strconv.FormatInt(val.Int(), base), nil
-	case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
-		base, _ := getBase(options, 10)
-		return strconv.FormatUint(val.Uint(), base), nil
-	case reflect.Float32, reflect.Float64:
-		return strconv.FormatFloat(val.Float(), 'g', -1, tp.Bits()), nil
-	case reflect.Slice:
-		if val.Len() == 0 {
-			return "", nil
-		}
-
-		ret := "["
-
-		for i := 0; i < val.Len(); i++ {
-			if i != 0 {
-				ret += ", "
-			}
-
-			item, err := convertToString(val.Index(i), options)
-
-			if err != nil {
-				return "", err
-			}
-
-			ret += item
-		}
-
-		return ret + "]", nil
-	case reflect.Map:
-		ret := "{"
-
-		for i, key := range val.MapKeys() {
-			if i != 0 {
-				ret += ", "
-			}
-
-			item, err := convertToString(val.MapIndex(key), options)
-
-			if err != nil {
-				return "", err
-			}
-
-			ret += item
-		}
-
-		return ret + "}", nil
-	case reflect.Ptr:
-		return convertToString(reflect.Indirect(val), options)
-	case reflect.Interface:
-		if !val.IsNil() {
-			return convertToString(val.Elem(), options)
-		}
-	}
-
-	return "", nil
-}
-
-func convertUnmarshal(val string, retval reflect.Value) (bool, error) {
-	if retval.Type().NumMethod() > 0 && retval.CanInterface() {
-		if unmarshaler, ok := retval.Interface().(Unmarshaler); ok {
-			return true, unmarshaler.UnmarshalFlag(val)
-		}
-	}
-
-	if retval.Type().Kind() != reflect.Ptr && retval.CanAddr() {
-		return convertUnmarshal(val, retval.Addr())
-	}
-
-	if retval.Type().Kind() == reflect.Interface && !retval.IsNil() {
-		return convertUnmarshal(val, retval.Elem())
-	}
-
-	return false, nil
-}
-
-func convert(val string, retval reflect.Value, options multiTag) error {
-	if ok, err := convertUnmarshal(val, retval); ok {
-		return err
-	}
-
-	tp := retval.Type()
-
-	// Support for time.Duration
-	if tp == reflect.TypeOf((*time.Duration)(nil)).Elem() {
-		parsed, err := time.ParseDuration(val)
-
-		if err != nil {
-			return err
-		}
-
-		retval.SetInt(int64(parsed))
-		return nil
-	}
-
-	switch tp.Kind() {
-	case reflect.String:
-		retval.SetString(val)
-	case reflect.Bool:
-		if val == "" {
-			retval.SetBool(true)
-		} else {
-			b, err := strconv.ParseBool(val)
-
-			if err != nil {
-				return err
-			}
-
-			retval.SetBool(b)
-		}
-	case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
-		base, err := getBase(options, 10)
-
-		if err != nil {
-			return err
-		}
-
-		parsed, err := strconv.ParseInt(val, base, tp.Bits())
-
-		if err != nil {
-			return err
-		}
-
-		retval.SetInt(parsed)
-	case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
-		base, err := getBase(options, 10)
-
-		if err != nil {
-			return err
-		}
-
-		parsed, err := strconv.ParseUint(val, base, tp.Bits())
-
-		if err != nil {
-			return err
-		}
-
-		retval.SetUint(parsed)
-	case reflect.Float32, reflect.Float64:
-		parsed, err := strconv.ParseFloat(val, tp.Bits())
-
-		if err != nil {
-			return err
-		}
-
-		retval.SetFloat(parsed)
-	case reflect.Slice:
-		elemtp := tp.Elem()
-
-		elemvalptr := reflect.New(elemtp)
-		elemval := reflect.Indirect(elemvalptr)
-
-		if err := convert(val, elemval, options); err != nil {
-			return err
-		}
-
-		retval.Set(reflect.Append(retval, elemval))
-	case reflect.Map:
-		parts := strings.SplitN(val, ":", 2)
-
-		key := parts[0]
-		var value string
-
-		if len(parts) == 2 {
-			value = parts[1]
-		}
-
-		keytp := tp.Key()
-		keyval := reflect.New(keytp)
-
-		if err := convert(key, keyval, options); err != nil {
-			return err
-		}
-
-		valuetp := tp.Elem()
-		valueval := reflect.New(valuetp)
-
-		if err := convert(value, valueval, options); err != nil {
-			return err
-		}
-
-		if retval.IsNil() {
-			retval.Set(reflect.MakeMap(tp))
-		}
-
-		retval.SetMapIndex(reflect.Indirect(keyval), reflect.Indirect(valueval))
-	case reflect.Ptr:
-		if retval.IsNil() {
-			retval.Set(reflect.New(retval.Type().Elem()))
-		}
-
-		return convert(val, reflect.Indirect(retval), options)
-	case reflect.Interface:
-		if !retval.IsNil() {
-			return convert(val, retval.Elem(), options)
-		}
-	}
-
-	return nil
-}
-
-func wrapText(s string, l int, prefix string) string {
-	// Basic text wrapping of s at spaces to fit in l
-	var ret string
-
-	s = strings.TrimSpace(s)
-
-	for len(s) > l {
-		// Try to split on space
-		suffix := ""
-
-		pos := strings.LastIndex(s[:l], " ")
-
-		if pos < 0 {
-			pos = l - 1
-			suffix = "-\n"
-		}
-
-		if len(ret) != 0 {
-			ret += "\n" + prefix
-		}
-
-		ret += strings.TrimSpace(s[:pos]) + suffix
-		s = strings.TrimSpace(s[pos:])
-	}
-
-	if len(s) > 0 {
-		if len(ret) != 0 {
-			ret += "\n" + prefix
-		}
-
-		return ret + s
-	}
-
-	return ret
-}

github.com/jessevdk/go-flags/error.go

@@ -1,113 +0,0 @@
-package flags
-
-import (
-	"fmt"
-)
-
-// ErrorType represents the type of error.
-type ErrorType uint
-
-const (
-	// ErrUnknown indicates a generic error.
-	ErrUnknown ErrorType = iota
-
-	// ErrExpectedArgument indicates that an argument was expected.
-	ErrExpectedArgument
-
-	// ErrUnknownFlag indicates an unknown flag.
-	ErrUnknownFlag
-
-	// ErrUnknownGroup indicates an unknown group.
-	ErrUnknownGroup
-
-	// ErrMarshal indicates a marshalling error while converting values.
-	ErrMarshal
-
-	// ErrHelp indicates that the builtin help was shown (the error
-	// contains the help message).
-	ErrHelp
-
-	// ErrNoArgumentForBool indicates that an argument was given for a
-	// boolean flag (which don't not take any arguments).
-	ErrNoArgumentForBool
-
-	// ErrRequired indicates that a required flag was not provided.
-	ErrRequired
-
-	// ErrShortNameTooLong indicates that a short flag name was specified,
-	// longer than one character.
-	ErrShortNameTooLong
-
-	// ErrDuplicatedFlag indicates that a short or long flag has been
-	// defined more than once
-	ErrDuplicatedFlag
-
-	// ErrTag indicates an error while parsing flag tags.
-	ErrTag
-)
-
-// String returns a string representation of the error type.
-func (e ErrorType) String() string {
-	switch e {
-	case ErrUnknown:
-		return "unknown"
-	case ErrExpectedArgument:
-		return "expected argument"
-	case ErrUnknownFlag:
-		return "unknown flag"
-	case ErrUnknownGroup:
-		return "unknown group"
-	case ErrMarshal:
-		return "marshal"
-	case ErrHelp:
-		return "help"
-	case ErrNoArgumentForBool:
-		return "no argument for bool"
-	case ErrRequired:
-		return "required"
-	case ErrShortNameTooLong:
-		return "short name too long"
-	case ErrDuplicatedFlag:
-		return "duplicated flag"
-	case ErrTag:
-		return "tag"
-	}
-
-	return "unknown"
-}
-
-// Error represents a parser error. The error returned from Parse is of this
-// type. The error contains both a Type and Message.
-type Error struct {
-	// The type of error
-	Type ErrorType
-
-	// The error message
-	Message string
-}
-
-// Error returns the error's message
-func (e *Error) Error() string {
-	return e.Message
-}
-
-func newError(tp ErrorType, message string) *Error {
-	return &Error{
-		Type:    tp,
-		Message: message,
-	}
-}
-
-func newErrorf(tp ErrorType, format string, args ...interface{}) *Error {
-	return newError(tp, fmt.Sprintf(format, args...))
-}
-
-func wrapError(err error) *Error {
-	ret, ok := err.(*Error)
-
-	if !ok {
-		return newError(ErrUnknown, err.Error())
-	}
-
-	return ret
-}

github.com/jessevdk/go-flags/example_test.go

@@ -1,95 +0,0 @@
-// Example of use of the flags package.
-package flags
-
-import (
-	"fmt"
-	"os"
-	"os/exec"
-	"strings"
-)
-
-func Example() {
-	var opts struct {
-		// Slice of bool will append 'true' each time the option
-		// is encountered (can be set multiple times, like -vvv)
-		Verbose []bool `short:"v" long:"verbose" description:"Show verbose debug information"`
-
-		// Example of automatic marshalling to desired type (uint)
-		Offset uint `long:"offset" description:"Offset"`
-
-		// Example of a callback, called each time the option is found.
-		Call func(string) `short:"c" description:"Call phone number"`
-
-		// Example of a required flag
-		Name string `short:"n" long:"name" description:"A name" required:"true"`
-
-		// Example of a value name
-		File string `short:"f" long:"file" description:"A file" value-name:"FILE"`
-
-		// Example of a pointer
-		Ptr *int `short:"p" description:"A pointer to an integer"`
-
-		// Example of a slice of strings
-		StringSlice []string `short:"s" description:"A slice of strings"`
-
-		// Example of a slice of pointers
-		PtrSlice []*string `long:"ptrslice" description:"A slice of pointers to string"`
-
-		// Example of a map
-		IntMap map[string]int `long:"intmap" description:"A map from string to int"`
-	}
-
-	// Callback which will invoke callto:<argument> to call a number.
-	// Note that this works just on OS X (and probably only with
-	// Skype) but it shows the idea.
-	opts.Call = func(num string) {
-		cmd := exec.Command("open", "callto:"+num)
-		cmd.Start()
-		cmd.Process.Release()
-	}
-
-	// Make some fake arguments to parse.
-	args := []string{
-		"-vv",
-		"--offset=5",
-		"-n", "Me",
-		"-p", "3",
-		"-s", "hello",
-		"-s", "world",
-		"--ptrslice", "hello",
-		"--ptrslice", "world",
-		"--intmap", "a:1",
-		"--intmap", "b:5",
-		"arg1",
-		"arg2",
-		"arg3",
-	}
-
-	// Parse flags from `args'. Note that here we use flags.ParseArgs for
-	// the sake of making a working example. Normally, you would simply use
-	// flags.Parse(&opts) which uses os.Args
-	args, err := ParseArgs(&opts, args)
-
-	if err != nil {
-		panic(err)
-		os.Exit(1)
-	}
-
-	fmt.Printf("Verbosity: %v\n", opts.Verbose)
-	fmt.Printf("Offset: %d\n", opts.Offset)
-	fmt.Printf("Name: %s\n", opts.Name)
-	fmt.Printf("Ptr: %d\n", *opts.Ptr)
-	fmt.Printf("StringSlice: %v\n", opts.StringSlice)
-	fmt.Printf("PtrSlice: [%v %v]\n", *opts.PtrSlice[0], *opts.PtrSlice[1])
-	fmt.Printf("IntMap: [a:%v b:%v]\n", opts.IntMap["a"], opts.IntMap["b"])
-	fmt.Printf("Remaining args: %s\n", strings.Join(args, " "))
-
-	// Output: Verbosity: [true true]
-	// Offset: 5
-	// Name: Me
-	// Ptr: 3
-	// StringSlice: [hello world]
-	// PtrSlice: [hello world]
-	// IntMap: [a:1 b:5]
-	// Remaining args: arg1 arg2 arg3
-}

github.com/jessevdk/go-flags/examples/add.go

@@ -1,23 +0,0 @@
-package main
-
-import (
-	"fmt"
-)
-
-type AddCommand struct {
-	All bool `short:"a" long:"all" description:"Add all files"`
-}
-
-var addCommand AddCommand
-
-func (x *AddCommand) Execute(args []string) error {
-	fmt.Printf("Adding (all=%v): %#v\n", x.All, args)
-	return nil
-}
-
-func init() {
-	parser.AddCommand("add",
-		"Add a file",
-		"The add command adds a file to the repository. Use -a to add all files.",
-		&addCommand)
-}

github.com/jessevdk/go-flags/examples/main.go

@@ -1,75 +0,0 @@
-package main
-
-import (
-	"errors"
-	"fmt"
-	"github.com/calmh/syncthing/github.com/jessevdk/go-flags"
-	"os"
-	"strconv"
-	"strings"
-)
-
-type EditorOptions struct {
-	Input	string	`short:"i" long:"input" description:"Input file" default:"-"`
-	Output	string	`short:"o" long:"output" description:"Output file" default:"-"`
-}
-
-type Point struct {
-	X, Y int
-}
-
-func (p *Point) UnmarshalFlag(value string) error {
-	parts := strings.Split(value, ",")
-
-	if len(parts) != 2 {
-		return errors.New("Expected two numbers separated by a ,")
-	}
-
-	x, err := strconv.ParseInt(parts[0], 10, 32)
-
-	if err != nil {
-		return err
-	}
-
-	y, err := strconv.ParseInt(parts[1], 10, 32)
-
-	if err != nil {
-		return err
-	}
-
-	p.X = int(x)
-	p.Y = int(y)
-
-	return nil
-}
-
-func (p Point) MarshalFlag() (string, error) {
-	return fmt.Sprintf("%d,%d", p.X, p.Y), nil
-}
-
-type Options struct {
-	// Example of verbosity with level
-	Verbose	[]bool	`short:"v" long:"verbose" description:"Verbose output"`
-
-	// Example of optional value
-	User	string	`short:"u" long:"user" description:"User name" optional:"yes" optional-value:"pancake"`
-
-	// Example of map with multiple default values
-	Users	map[string]string	`long:"users" description:"User e-mail map" default:"system:system@example.org" default:"admin:admin@example.org"`
-
-	// Example of option group
-	Editor	EditorOptions	`group:"Editor Options"`
-
-	// Example of custom type Marshal/Unmarshal
-	Point	Point	`long:"point" description:"A x,y point" default:"1,2"`
-}
-
-var options Options
-
-var parser = flags.NewParser(&options, flags.Default)
-
-func main() {
-	if _, err := parser.Parse(); err != nil {
-		os.Exit(1)
-	}
-}

github.com/jessevdk/go-flags/examples/rm.go

@@ -1,23 +0,0 @@
-package main
-
-import (
-	"fmt"
-)
-
-type RmCommand struct {
-	Force bool `short:"f" long:"force" description:"Force removal of files"`
-}
-
-var rmCommand RmCommand
-
-func (x *RmCommand) Execute(args []string) error {
-	fmt.Printf("Removing (force=%v): %#v\n", x.Force, args)
-	return nil
-}
-
-func init() {
-	parser.AddCommand("rm",
-		"Remove a file",
-		"The rm command removes a file to the repository. Use -f to force removal of files.",
-		&rmCommand)
-}

github.com/jessevdk/go-flags/flags.go

@@ -1,141 +0,0 @@
-// Copyright 2012 Jesse van den Kieboom. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-// Package flags provides an extensive command line option parser.
-// The flags package is similar in functionality to the go builtin flag package
-// but provides more options and uses reflection to provide a convenient and
-// succinct way of specifying command line options.
-//
-// Supported features:
-//     Options with short names (-v)
-//     Options with long names (--verbose)
-//     Options with and without arguments (bool v.s. other type)
-//     Options with optional arguments and default values
-//     Multiple option groups each containing a set of options
-//     Generate and print well-formatted help message
-//     Passing remaining command line arguments after -- (optional)
-//     Ignoring unknown command line options (optional)
-//     Supports -I/usr/include -I=/usr/include -I /usr/include option argument specification
-//     Supports multiple short options -aux
-//     Supports all primitive go types (string, int{8..64}, uint{8..64}, float)
-//     Supports same option multiple times (can store in slice or last option counts)
-//     Supports maps
-//     Supports function callbacks
-//
-// Additional features specific to Windows:
-//     Options with short names (/v)
-//     Options with long names (/verbose)
-//     Windows-style options with arguments use a colon as the delimiter
-//     Modify generated help message with Windows-style / options
-//
-// The flags package uses structs, reflection and struct field tags
-// to allow users to specify command line options. This results in very simple
-// and consise specification of your application options. For example:
-//
-//     type Options struct {
-//         Verbose []bool `short:"v" long:"verbose" description:"Show verbose debug information"`
-//     }
-//
-// This specifies one option with a short name -v and a long name --verbose.
-// When either -v or --verbose is found on the command line, a 'true' value
-// will be appended to the Verbose field. e.g. when specifying -vvv, the
-// resulting value of Verbose will be {[true, true, true]}.
-//
-// Slice options work exactly the same as primitive type options, except that
-// whenever the option is encountered, a value is appended to the slice.
-//
-// Map options from string to primitive type are also supported. On the command
-// line, you specify the value for such an option as key:value. For example
-//
-//     type Options struct {
-//         AuthorInfo string[string] `short:"a"`
-//     }
-//
-// Then, the AuthorInfo map can be filled with something like
-// -a name:Jesse -a "surname:van den Kieboom".
-//
-// Finally, for full control over the conversion between command line argument
-// values and options, user defined types can choose to implement the Marshaler
-// and Unmarshaler interfaces.
-//
-// Available field tags:
-//     short:          the short name of the option (single character)
-//     long:           the long name of the option
-//     description:    the description of the option (optional)
-//     optional:       whether an argument of the option is optional (optional)
-//     optional-value: the value of an optional option when the option occurs
-//                     without an argument. This tag can be specified multiple
-//                     times in the case of maps or slices (optional)
-//     default:        the default value of an option. This tag can be specified
-//                     multiple times in the case of slices or maps (optional).
-//     default-mask:   when specified, this value will be displayed in the help
-//                     instead of the actual default value. This is useful
-//                     mostly for hiding otherwise sensitive information from
-//                     showing up in the help. If default-mask takes the special
-//                     value "-", then no default value will be shown at all
-//                     (optional)
-//     required:       whether an option is required to appear on the command
-//                     line. If a required option is not present, the parser
-//                     will return ErrRequired.
-//     base:           a base (radix) used to convert strings to integer values,
-//                     the default base is 10 (i.e. decimal) (optional)
-//     value-name:     the name of the argument value (to be shown in the help,
-//                     (optional)
-//     group:          when specified on a struct field, makes the struct field
-//                     a separate group with the given name (optional).
-//     command:        when specified on a struct field, makes the struct field
-//                     a (sub)command with the given name (optional).
-//
-// Either short: or long: must be specified to make the field eligible as an
-// option.
-//
-//
-// Option groups:
-//
-// Option groups are a simple way to semantically separate your options. The
-// only real difference is in how your options will appear in the builtin
-// generated help. All options in a particular group are shown together in the
-// help under the name of the group.
-//
-// There are currently three ways to specify option groups.
-//
-//     1. Use NewNamedParser specifying the various option groups.
-//     2. Use AddGroup to add a group to an existing parser.
-//     3. Add a struct field to the toplevel options annotated with the
-//        group:"group-name" tag.
-//
-//
-//
-// Commands:
-//
-// The flags package also has basic support for commands. Commands are often
-// used in monolithic applications that support various commands or actions.
-// Take git for example, all of the add, commit, checkout, etc. are called
-// commands. Using commands you can easily separate multiple functions of your
-// application.
-//
-// There are currently two ways to specifiy a command.
-//
-//     1. Use AddCommand on an existing parser.
-//     2. Add a struct field to your options struct annotated with the
-//        command:"command-name" tag.
-//
-// The most common, idiomatic way to implement commands is to define a global
-// parser instance and implement each command in a separate file. These
-// command files should define a go init function which calls AddCommand on
-// the global parser.
-//
-// When parsing ends and there is an active command and that command implements
-// the Commander interface, then its Execute method will be run with the
-// remaining command line arguments.
-//
-// Command structs can have options which become valid to parse after the
-// command has been specified on the command line. It is currently not valid
-// to specify options from the parent level of the command after the command
-// name has occurred. Thus, given a toplevel option "-v" and a command "add":
-//
-//     Valid:   ./app -v add
-//     Invalid: ./app add -v
-//
-package flags

github.com/jessevdk/go-flags/group.go

@@ -1,80 +0,0 @@
-// Copyright 2012 Jesse van den Kieboom. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package flags
-
-import (
-	"errors"
-	"strings"
-)
-
-// ErrNotPointerToStruct indicates that a provided data container is not
-// a pointer to a struct. Only pointers to structs are valid data containers
-// for options.
-var ErrNotPointerToStruct = errors.New("provided data is not a pointer to struct")
-
-// Group represents an option group. Option groups can be used to logically
-// group options together under a description. Groups are only used to provide
-// more structure to options both for the user (as displayed in the help message)
-// and for you, since groups can be nested.
-type Group struct {
-	// A short description of the group. The
-	// short description is primarily used in the builtin generated help
-	// message
-	ShortDescription string
-
-	// A long description of the group. The long
-	// description is primarily used to present information on commands
-	// (Command embeds Group) in the builtin generated help and man pages.
-	LongDescription string
-
-	// All the options in the group
-	options []*Option
-
-	// All the subgroups
-	groups []*Group
-
-	data interface{}
-}
-
-// AddGroup adds a new group to the command with the given name and data. The
-// data needs to be a pointer to a struct from which the fields indicate which
-// options are in the group.
-func (g *Group) AddGroup(shortDescription string, longDescription string, data interface{}) (*Group, error) {
-	group := newGroup(shortDescription, longDescription, data)
-
-	if err := group.scan(); err != nil {
-		return nil, err
-	}
-
-	g.groups = append(g.groups, group)
-	return group, nil
-}
-
-// Groups returns the list of groups embedded in this group.
-func (g *Group) Groups() []*Group {
-	return g.groups
-}
-
-// Options returns the list of options in this group.
-func (g *Group) Options() []*Option {
-	return g.options
-}
-
-// Find locates the subgroup with the given short description and returns it.
-// If no such group can be found Find will return nil. Note that the description
-// is matched case insensitively.
-func (g *Group) Find(shortDescription string) *Group {
-	lshortDescription := strings.ToLower(shortDescription)
-
-	var ret *Group
-
-	g.eachGroup(func(gg *Group) {
-		if gg != g && strings.ToLower(gg.ShortDescription) == lshortDescription {
-			ret = gg
-		}
-	})
-
-	return ret
-}

github.com/jessevdk/go-flags/group_private.go

@@ -1,263 +0,0 @@
-package flags
-
-import (
-	"reflect"
-	"unicode/utf8"
-	"unsafe"
-)
-
-type scanHandler func(reflect.Value, *reflect.StructField) (bool, error)
-
-func newGroup(shortDescription string, longDescription string, data interface{}) *Group {
-	return &Group{
-		ShortDescription: shortDescription,
-		LongDescription:  longDescription,
-
-		data: data,
-	}
-}
-
-func (g *Group) optionByName(name string, namematch func(*Option, string) bool) *Option {
-	prio := 0
-	var retopt *Option
-
-	for _, opt := range g.options {
-		if namematch != nil && namematch(opt, name) && prio < 4 {
-			retopt = opt
-			prio = 4
-		}
-
-		if name == opt.field.Name && prio < 3 {
-			retopt = opt
-			prio = 3
-		}
-
-		if name == opt.LongName && prio < 2 {
-			retopt = opt
-			prio = 2
-		}
-
-		if opt.ShortName != 0 && name == string(opt.ShortName) && prio < 1 {
-			retopt = opt
-			prio = 1
-		}
-	}
-
-	return retopt
-}
-
-func (g *Group) storeDefaults() {
-	for _, option := range g.options {
-		// First. empty out the value
-		if len(option.Default) > 0 {
-			option.clear()
-		}
-
-		for _, d := range option.Default {
-			option.set(&d)
-		}
-
-		if !option.value.CanSet() {
-			continue
-		}
-
-		option.defaultValue = reflect.ValueOf(option.value.Interface())
-	}
-}
-
-func (g *Group) eachGroup(f func(*Group)) {
-	f(g)
-
-	for _, gg := range g.groups {
-		gg.eachGroup(f)
-	}
-}
-
-func (g *Group) scanStruct(realval reflect.Value, sfield *reflect.StructField, handler scanHandler) error {
-	stype := realval.Type()
-
-	if sfield != nil {
-		if ok, err := handler(realval, sfield); err != nil {
-			return err
-		} else if ok {
-			return nil
-		}
-	}
-
-	for i := 0; i < stype.NumField(); i++ {
-		field := stype.Field(i)
-
-		// PkgName is set only for non-exported fields, which we ignore
-		if field.PkgPath != "" {
-			continue
-		}
-
-		mtag := newMultiTag(string(field.Tag))
-
-		if err := mtag.Parse(); err != nil {
-			return err
-		}
-
-		// Skip fields with the no-flag tag
-		if mtag.Get("no-flag") != "" {
-			continue
-		}
-
-		// Dive deep into structs or pointers to structs
-		kind := field.Type.Kind()
-		fld := realval.Field(i)
-
-		if kind == reflect.Struct {
-			if err := g.scanStruct(fld, &field, handler); err != nil {
-				return err
-			}
-		} else if kind == reflect.Ptr && field.Type.Elem().Kind() == reflect.Struct {
-			if fld.IsNil() {
-				fld.Set(reflect.New(fld.Type().Elem()))
-			}
-
-			if err := g.scanStruct(reflect.Indirect(fld), &field, handler); err != nil {
-				return err
-			}
-		}
-
-		longname := mtag.Get("long")
-		shortname := mtag.Get("short")
-
-		// Need at least either a short or long name
-		if longname == "" && shortname == "" && mtag.Get("ini-name") == "" {
-			continue
-		}
-
-		short := rune(0)
-		rc := utf8.RuneCountInString(shortname)
-
-		if rc > 1 {
-			return newErrorf(ErrShortNameTooLong,
-				"short names can only be 1 character long, not `%s'",
-				shortname)
-
-		} else if rc == 1 {
-			short, _ = utf8.DecodeRuneInString(shortname)
-		}
-
-		description := mtag.Get("description")
-		def := mtag.GetMany("default")
-		optionalValue := mtag.GetMany("optional-value")
-		valueName := mtag.Get("value-name")
-		defaultMask := mtag.Get("default-mask")
-
-		optional := (mtag.Get("optional") != "")
-		required := (mtag.Get("required") != "")
-
-		option := &Option{
-			Description:      description,
-			ShortName:        short,
-			LongName:         longname,
-			Default:          def,
-			OptionalArgument: optional,
-			OptionalValue:    optionalValue,
-			Required:         required,
-			ValueName:        valueName,
-			DefaultMask:      defaultMask,
-
-			field: field,
-			value: realval.Field(i),
-			tag:   mtag,
-		}
-
-		g.options = append(g.options, option)
-	}
-
-	return nil
-}
-
-func (g *Group) checkForDuplicateFlags() *Error {
-	shortNames := make(map[rune]*Option)
-	longNames := make(map[string]*Option)
-
-	var duplicateError *Error
-
-	g.eachGroup(func(g *Group) {
-		for _, option := range g.options {
-			if option.LongName != "" {
-				if otherOption, ok := longNames[option.LongName]; ok {
-					duplicateError = newErrorf(ErrDuplicatedFlag, "option `%s' uses the same long name as option `%s'", option, otherOption)
-					return
-				}
-				longNames[option.LongName] = option
-			}
-			if option.ShortName != 0 {
-				if otherOption, ok := shortNames[option.ShortName]; ok {
-					duplicateError = newErrorf(ErrDuplicatedFlag, "option `%s' uses the same short name as option `%s'", option, otherOption)
-					return
-				}
-				shortNames[option.ShortName] = option
-			}
-		}
-	})
-
-	return duplicateError
-}
-
-func (g *Group) scanSubGroupHandler(realval reflect.Value, sfield *reflect.StructField) (bool, error) {
-	mtag := newMultiTag(string(sfield.Tag))
-
-	if err := mtag.Parse(); err != nil {
-		return true, err
-	}
-
-	subgroup := mtag.Get("group")
-
-	if len(subgroup) != 0 {
-		ptrval := reflect.NewAt(realval.Type(), unsafe.Pointer(realval.UnsafeAddr()))
-		description := mtag.Get("description")
-
-		if _, err := g.AddGroup(subgroup, description, ptrval.Interface()); err != nil {
-			return true, err
-		}
-
-		return true, nil
-	}
-
-	return false, nil
-}
-
-func (g *Group) scanType(handler scanHandler) error {
-	// Get all the public fields in the data struct
-	ptrval := reflect.ValueOf(g.data)
-
-	if ptrval.Type().Kind() != reflect.Ptr {
-		panic(ErrNotPointerToStruct)
-	}
-
-	stype := ptrval.Type().Elem()
-
-	if stype.Kind() != reflect.Struct {
-		panic(ErrNotPointerToStruct)
-	}
-
-	realval := reflect.Indirect(ptrval)
-
-	if err := g.scanStruct(realval, nil, handler); err != nil {
-		return err
-	}
-
-	if err := g.checkForDuplicateFlags(); err != nil {
-		return err
-	}
-
-	return nil
-}
-
-func (g *Group) scan() error {
-	return g.scanType(g.scanSubGroupHandler)
-}
-
-func (g *Group) groupByName(name string) *Group {
-	if len(name) == 0 {
-		return g
-	}
-
-	return g.Find(name)
-}

github.com/jessevdk/go-flags/group_test.go

@@ -1,160 +0,0 @@
-package flags
-
-import (
-	"testing"
-)
-
-func TestGroupInline(t *testing.T) {
-	var opts = struct {
-		Value bool `short:"v"`
-
-		Group struct {
-			G bool `short:"g"`
-		} `group:"Grouped Options"`
-	}{}
-
-	p, ret := assertParserSuccess(t, &opts, "-v", "-g")
-
-	assertStringArray(t, ret, []string{})
-
-	if !opts.Value {
-		t.Errorf("Expected Value to be true")
-	}
-
-	if !opts.Group.G {
-		t.Errorf("Expected Group.G to be true")
-	}
-
-	if p.Command.Group.Find("Grouped Options") == nil {
-		t.Errorf("Expected to find group `Grouped Options'")
-	}
-}
-
-func TestGroupAdd(t *testing.T) {
-	var opts = struct {
-		Value bool `short:"v"`
-	}{}
-
-	var grp = struct {
-		G bool `short:"g"`
-	}{}
-
-	p := NewParser(&opts, Default)
-	g, err := p.AddGroup("Grouped Options", "", &grp)
-
-	if err != nil {
-		t.Fatalf("Unexpected error: %v", err)
-		return
-	}
-
-	ret, err := p.ParseArgs([]string{"-v", "-g", "rest"})
-
-	if err != nil {
-		t.Fatalf("Unexpected error: %v", err)
-		return
-	}
-
-	assertStringArray(t, ret, []string{"rest"})
-
-	if !opts.Value {
-		t.Errorf("Expected Value to be true")
-	}
-
-	if !grp.G {
-		t.Errorf("Expected Group.G to be true")
-	}
-
-	if p.Command.Group.Find("Grouped Options") != g {
-		t.Errorf("Expected to find group `Grouped Options'")
-	}
-
-	if p.Groups()[1] != g {
-		t.Errorf("Espected group #v,	 but got #v", g, p.Groups()[0])
-	}
-
-	if g.Options()[0].ShortName != 'g' {
-		t.Errorf("Expected short name `g' but got %v", g.Options()[0].ShortName)
-	}
-}
-
-func TestGroupNestedInline(t *testing.T) {
-	var opts = struct {
-		Value bool `short:"v"`
-
-		Group struct {
-			G bool `short:"g"`
-
-			Nested struct {
-				N string `long:"n"`
-			} `group:"Nested Options"`
-		} `group:"Grouped Options"`
-	}{}
-
-	p, ret := assertParserSuccess(t, &opts, "-v", "-g", "--n", "n", "rest")
-
-	assertStringArray(t, ret, []string{"rest"})
-
-	if !opts.Value {
-		t.Errorf("Expected Value to be true")
-	}
-
-	if !opts.Group.G {
-		t.Errorf("Expected Group.G to be true")
-	}
-
-	assertString(t, opts.Group.Nested.N, "n")
-
-	if p.Command.Group.Find("Grouped Options") == nil {
-		t.Errorf("Expected to find group `Grouped Options'")
-	}
-
-	if p.Command.Group.Find("Nested Options") == nil {
-		t.Errorf("Expected to find group `Nested Options'")
-	}
-}
-
-func TestDuplicateShortFlags(t *testing.T) {
-	var opts struct {
-		Verbose   []bool   `short:"v" long:"verbose" description:"Show verbose debug information"`
-		Variables []string `short:"v" long:"variable" description:"Set a variable value."`
-	}
-
-	args := []string{
-		"--verbose",
-		"-v", "123",
-		"-v", "456",
-	}
-
-	_, err := ParseArgs(&opts, args)
-
-	if err == nil {
-		t.Errorf("Expected an error with type ErrDuplicatedFlag")
-	} else {
-		err2 := err.(*Error)
-		if err2.Type != ErrDuplicatedFlag {
-			t.Errorf("Expected an error with type ErrDuplicatedFlag")
-		}
-	}
-}
-
-func TestDuplicateLongFlags(t *testing.T) {
-	var opts struct {
-		Test1 []bool   `short:"a" long:"testing" description:"Test 1"`
-		Test2 []string `short:"b" long:"testing" description:"Test 2."`
-	}
-
-	args := []string{
-		"--testing",
-	}
-
-	_, err := ParseArgs(&opts, args)
-
-	if err == nil {
-		t.Errorf("Expected an error with type ErrDuplicatedFlag")
-	} else {
-		err2 := err.(*Error)
-		if err2.Type != ErrDuplicatedFlag {
-			t.Errorf("Expected an error with type ErrDuplicatedFlag")
-		}
-	}
-}

github.com/jessevdk/go-flags/help.go

@@ -1,275 +0,0 @@
-// Copyright 2012 Jesse van den Kieboom. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package flags
-
-import (
-	"bufio"
-	"bytes"
-	"fmt"
-	"io"
-	"reflect"
-	"strings"
-	"unicode/utf8"
-)
-
-type alignmentInfo struct {
-	maxLongLen      int
-	hasShort        bool
-	hasValueName    bool
-	terminalColumns int
-}
-
-func (p *Parser) getAlignmentInfo() alignmentInfo {
-	ret := alignmentInfo{
-		maxLongLen:      0,
-		hasShort:        false,
-		hasValueName:    false,
-		terminalColumns: getTerminalColumns(),
-	}
-
-	if ret.terminalColumns <= 0 {
-		ret.terminalColumns = 80
-	}
-
-	p.eachActiveGroup(func(grp *Group) {
-		for _, info := range grp.options {
-			if info.ShortName != 0 {
-				ret.hasShort = true
-			}
-
-			lv := utf8.RuneCountInString(info.ValueName)
-
-			if lv != 0 {
-				ret.hasValueName = true
-			}
-
-			l := utf8.RuneCountInString(info.LongName) + lv
-
-			if l > ret.maxLongLen {
-				ret.maxLongLen = l
-			}
-		}
-	})
-
-	return ret
-}
-
-func (p *Parser) writeHelpOption(writer *bufio.Writer, option *Option, info alignmentInfo) {
-	line := &bytes.Buffer{}
-
-	distanceBetweenOptionAndDescription := 2
-	paddingBeforeOption := 2
-
-	line.WriteString(strings.Repeat(" ", paddingBeforeOption))
-
-	if option.ShortName != 0 {
-		line.WriteRune(defaultShortOptDelimiter)
-		line.WriteRune(option.ShortName)
-	} else if info.hasShort {
-		line.WriteString("  ")
-	}
-
-	descstart := info.maxLongLen + paddingBeforeOption + distanceBetweenOptionAndDescription
-
-	if info.hasShort {
-		descstart += 2
-	}
-
-	if info.maxLongLen > 0 {
-		descstart += 4
-	}
-
-	if info.hasValueName {
-		descstart += 3
-	}
-
-	if len(option.LongName) > 0 {
-		if option.ShortName != 0 {
-			line.WriteString(", ")
-		} else if info.hasShort {
-			line.WriteString("  ")
-		}
-
-		line.WriteString(defaultLongOptDelimiter)
-		line.WriteString(option.LongName)
-	}
-
-	if option.canArgument() {
-		line.WriteRune(defaultNameArgDelimiter)
-
-		if len(option.ValueName) > 0 {
-			line.WriteString(option.ValueName)
-		}
-	}
-
-	written := line.Len()
-	line.WriteTo(writer)
-
-	if option.Description != "" {
-		dw := descstart - written
-		writer.WriteString(strings.Repeat(" ", dw))
-
-		def := ""
-		defs := option.Default
-
-		if len(option.DefaultMask) != 0 {
-			if option.DefaultMask != "-" {
-				def = option.DefaultMask
-			}
-		} else if len(defs) == 0 && option.canArgument() {
-			var showdef bool
-
-			switch option.field.Type.Kind() {
-			case reflect.Func, reflect.Ptr:
-				showdef = !option.value.IsNil()
-			case reflect.Slice, reflect.String, reflect.Array:
-				showdef = option.value.Len() > 0
-			case reflect.Map:
-				showdef = !option.value.IsNil() && option.value.Len() > 0
-			default:
-				zeroval := reflect.Zero(option.field.Type)
-				showdef = !reflect.DeepEqual(zeroval.Interface(), option.value.Interface())
-			}
-
-			if showdef {
-				def, _ = convertToString(option.value, option.tag)
-			}
-		} else if len(defs) != 0 {
-			def = strings.Join(defs, ", ")
-		}
-
-		var desc string
-
-		if def != "" {
-			desc = fmt.Sprintf("%s (%v)", option.Description, def)
-		} else {
-			desc = option.Description
-		}
-
-		writer.WriteString(wrapText(desc,
-			info.terminalColumns-descstart,
-			strings.Repeat(" ", descstart)))
-	}
-
-	writer.WriteString("\n")
-}
-
-func maxCommandLength(s []*Command) int {
-	if len(s) == 0 {
-		return 0
-	}
-
-	ret := len(s[0].Name)
-
-	for _, v := range s[1:] {
-		l := len(v.Name)
-
-		if l > ret {
-			ret = l
-		}
-	}
-
-	return ret
-}
-
-// WriteHelp writes a help message containing all the possible options and
-// their descriptions to the provided writer. Note that the HelpFlag parser
-// option provides a convenient way to add a -h/--help option group to the
-// command line parser which will automatically show the help messages using
-// this method.
-func (p *Parser) WriteHelp(writer io.Writer) {
-	if writer == nil {
-		return
-	}
-
-	wr := bufio.NewWriter(writer)
-	aligninfo := p.getAlignmentInfo()
-
-	cmd := p.Command
-
-	for cmd.Active != nil {
-		cmd = cmd.Active
-	}
-
-	if p.Name != "" {
-		wr.WriteString("Usage:\n")
-		wr.WriteString(" ")
-
-		allcmd := p.Command
-
-		for allcmd != nil {
-			var usage string
-
-			if allcmd == p.Command {
-				if len(p.Usage) != 0 {
-					usage = p.Usage
-				} else {
-					usage = "[OPTIONS]"
-				}
-			} else if us, ok := allcmd.data.(Usage); ok {
-				usage = us.Usage()
-			} else {
-				usage = fmt.Sprintf("[%s-OPTIONS]", allcmd.Name)
-			}
-
-			if len(usage) != 0 {
-				fmt.Fprintf(wr, " %s %s", allcmd.Name, usage)
-			} else {
-				fmt.Fprintf(wr, " %s", allcmd.Name)
-			}
-
-			allcmd = allcmd.Active
-		}
-
-		fmt.Fprintln(wr)
-
-		if len(cmd.LongDescription) != 0 {
-			fmt.Fprintln(wr)
-
-			t := wrapText(cmd.LongDescription,
-				aligninfo.terminalColumns,
-				"")
-
-			fmt.Fprintln(wr, t)
-		}
-	}
-
-	p.eachActiveGroup(func(grp *Group) {
-		first := true
-
-		for _, info := range grp.options {
-			if info.canCli() {
-				if first {
-					fmt.Fprintf(wr, "\n%s:\n", grp.ShortDescription)
-					first = false
-				}
-
-				p.writeHelpOption(wr, info, aligninfo)
-			}
-		}
-	})
-
-	scommands := cmd.sortedCommands()
-
-	if len(scommands) > 0 {
-		maxnamelen := maxCommandLength(scommands)
-
-		fmt.Fprintln(wr)
-		fmt.Fprintln(wr, "Available commands:")
-
-		for _, c := range scommands {
-			fmt.Fprintf(wr, "  %s", c.Name)
-
-			if len(c.ShortDescription) > 0 {
-				pad := strings.Repeat(" ", maxnamelen-len(c.Name))
-				fmt.Fprintf(wr, "%s  %s", pad, c.ShortDescription)
-			}
-
-			fmt.Fprintln(wr)
-		}
-	}
-
-	wr.Flush()
-}

github.com/jessevdk/go-flags/help_test.go

@@ -1,153 +0,0 @@
-package flags
-
-import (
-	"bytes"
-	"fmt"
-	"io"
-	"io/ioutil"
-	"os"
-	"os/exec"
-	"testing"
-	"time"
-)
-
-func helpDiff(a, b string) (string, error) {
-	atmp, err := ioutil.TempFile("", "help-diff")
-
-	if err != nil {
-		return "", err
-	}
-
-	btmp, err := ioutil.TempFile("", "help-diff")
-
-	if err != nil {
-		return "", err
-	}
-
-	if _, err := io.WriteString(atmp, a); err != nil {
-		return "", err
-	}
-
-	if _, err := io.WriteString(btmp, b); err != nil {
-		return "", err
-	}
-
-	ret, err := exec.Command("diff", "-u", "-d", "--label", "got", atmp.Name(), "--label", "expected", btmp.Name()).Output()
-
-	os.Remove(atmp.Name())
-	os.Remove(btmp.Name())
-
-	return string(ret), nil
-}
-
-type helpOptions struct {
-	Verbose  []bool       `short:"v" long:"verbose" description:"Show verbose debug information" ini-name:"verbose"`
-	Call     func(string) `short:"c" description:"Call phone number" ini-name:"call"`
-	PtrSlice []*string    `long:"ptrslice" description:"A slice of pointers to string"`
-
-	OnlyIni string `ini-name:"only-ini" description:"Option only available in ini"`
-
-	Other struct {
-		StringSlice []string       `short:"s" description:"A slice of strings"`
-		IntMap      map[string]int `long:"intmap" description:"A map from string to int" ini-name:"int-map"`
-	} `group:"Other Options"`
-}
-
-func TestHelp(t *testing.T) {
-	var opts helpOptions
-
-	p := NewNamedParser("TestHelp", HelpFlag)
-	p.AddGroup("Application Options", "The application options", &opts)
-
-	_, err := p.ParseArgs([]string{"--help"})
-
-	if err == nil {
-		t.Fatalf("Expected help error")
-	}
-
-	if e, ok := err.(*Error); !ok {
-		t.Fatalf("Expected flags.Error, but got %#T", err)
-	} else {
-		if e.Type != ErrHelp {
-			t.Errorf("Expected flags.ErrHelp type, but got %s", e.Type)
-		}
-
-		expected := `Usage:
-  TestHelp [OPTIONS]
-
-Application Options:
-  -v, --verbose   Show verbose debug information
-  -c=             Call phone number
-      --ptrslice= A slice of pointers to string
-
-Other Options:
-  -s=             A slice of strings
-      --intmap=   A map from string to int
-
-Help Options:
-  -h, --help      Show this help message
-`
-
-		if e.Message != expected {
-			ret, err := helpDiff(e.Message, expected)
-
-			if err != nil {
-				t.Errorf("Unexpected diff error: %s", err)
-				t.Errorf("Unexpected help message, expected:\n\n%s\n\nbut got\n\n%s", expected, e.Message)
-			} else {
-				t.Errorf("Unexpected help message:\n\n%s", ret)
-			}
-		}
-	}
-}
-
-func TestMan(t *testing.T) {
-	var opts helpOptions
-
-	p := NewNamedParser("TestMan", HelpFlag)
-	p.ShortDescription = "Test manpage generation"
-	p.LongDescription = "This is a somewhat longer description of what this does"
-	p.AddGroup("Application Options", "The application options", &opts)
-
-	var buf bytes.Buffer
-	p.WriteManPage(&buf)
-
-	got := buf.String()
-
-	tt := time.Now()
-
-	expected := fmt.Sprintf(`.TH TestMan 1 "%s"
-.SH NAME
-TestMan \- Test manpage generation
-.SH SYNOPSIS
-\fBTestMan\fP [OPTIONS]
-.SH DESCRIPTION
-This is a somewhat longer description of what this does
-.SH OPTIONS
-.TP
-\fB-v, --verbose\fP
-Show verbose debug information
-.TP
-\fB-c\fP
-Call phone number
-.TP
-\fB--ptrslice\fP
-A slice of pointers to string
-.TP
-\fB-s\fP
-A slice of strings
-.TP
-\fB--intmap\fP
-A map from string to int
-`, tt.Format("2 January 2006"))
-
-	if got != expected {
-		ret, err := helpDiff(got, expected)
-
-		if err != nil {
-			t.Errorf("Unexpected man page, expected:\n\n%s\n\nbut got\n\n%s", expected, got)
-		} else {
-			t.Errorf("Unexpected man page:\n\n%s", ret)
-		}
-	}
-}

github.com/jessevdk/go-flags/ini.go

@@ -1,146 +0,0 @@
-package flags
-
-import (
-	"fmt"
-	"io"
-	"os"
-)
-
-// IniError contains location information on where in the ini file an error
-// occured.
-type IniError struct {
-	// The error message.
-	Message    string
-
-	// The filename of the file in which the error occurred.
-	File       string
-
-	// The line number at which the error occurred.
-	LineNumber uint
-}
-
-// Error provides a "file:line: message" formatted message of the ini error.
-func (x *IniError) Error() string {
-	return fmt.Sprintf("%s:%d: %s",
-		x.File,
-		x.LineNumber,
-		x.Message)
-}
-
-// IniOptions for writing ini files
-type IniOptions uint
-
-const (
-	// IniNone indicates no options.
-	IniNone IniOptions = 0
-
-	// IniIncludeDefaults indicates that default values should be written
-	// when writing options to an ini file.
-	IniIncludeDefaults = 1 << iota
-
-	// IniIncludeComments indicates that comments containing the description
-	// of an option should be written when writing options to an ini file.
-	IniIncludeComments
-
-	// IniDefault provides a default set of options.
-	IniDefault = IniIncludeComments
-)
-
-// IniParser is a utility to read and write flags options from and to ini
-// files.
-type IniParser struct {
-	parser *Parser
-}
-
-// NewIniParser creates a new ini parser for a given Parser.
-func NewIniParser(p *Parser) *IniParser {
-	return &IniParser{
-		parser: p,
-	}
-}
-
-// IniParse is a convenience function to parse command line options with default
-// settings from an ini file. The provided data is a pointer to a struct
-// representing the default option group (named "Application Options"). For
-// more control, use flags.NewParser.
-func IniParse(filename string, data interface{}) error {
-	p := NewParser(data, Default)
-	return NewIniParser(p).ParseFile(filename)
-}
-
-// ParseFile parses flags from an ini formatted file. See Parse for more
-// information on the ini file foramt. The returned errors can be of the type
-// flags.Error or flags.IniError.
-func (i *IniParser) ParseFile(filename string) error {
-	i.parser.storeDefaults()
-
-	ini, err := readIniFromFile(filename)
-
-	if err != nil {
-		return err
-	}
-
-	return i.parse(ini)
-}
-
-// Parse parses flags from an ini format. You can use ParseFile as a
-// convenience function to parse from a filename instead of a general
-// io.Reader.
-//
-// The format of the ini file is as follows:
-//
-//     [Option group name]
-//     option = value
-//
-// Each section in the ini file represents an option group or command in the
-// flags parser. The default flags parser option group (i.e. when using
-// flags.Parse) is named 'Application Options'. The ini option name is matched
-// in the following order:
-//
-//     1. Compared to the ini-name tag on the option struct field (if present)
-//     2. Compared to the struct field name
-//     3. Compared to the option long name (if present)
-//     4. Compared to the option short name (if present)
-//
-// Sections for nested groups and commands can be addressed using a dot `.'
-// namespacing notation (i.e [subcommand.Options]). Group section names are
-// matched case insensitive.
-//
-// The returned errors can be of the type flags.Error or
-// flags.IniError.
-func (i *IniParser) Parse(reader io.Reader) error {
-	i.parser.storeDefaults()
-
-	ini, err := readIni(reader, "")
-
-	if err != nil {
-		return err
-	}
-
-	return i.parse(ini)
-}
-
-// WriteFile writes the flags as ini format into a file. See WriteIni
-// for more information. The returned error occurs when the specified file
-// could not be opened for writing.
-func (i *IniParser) WriteFile(filename string, options IniOptions) error {
-	file, err := os.Create(filename)
-
-	if err != nil {
-		return err
-	}
-
-	defer file.Close()
-	i.Write(file, options)
-
-	return nil
-}
-
-// Write writes the current values of all the flags to an ini format.
-// See Parse for more information on the ini file format. You typically
-// call this only after settings have been parsed since the default values of each
-// option are stored just before parsing the flags (this is only relevant when
-// IniIncludeDefaults is _not_ set in options).
-func (i *IniParser) Write(writer io.Writer, options IniOptions) {
-	writeIni(i, writer, options)
-}

github.com/jessevdk/go-flags/ini_private.go

@@ -1,333 +0,0 @@
-package flags
-
-import (
-	"bufio"
-	"fmt"
-	"io"
-	"os"
-	"reflect"
-	"strings"
-)
-
-type iniValue struct {
-	Name  string
-	Value string
-}
-
-type iniSection []iniValue
-type ini map[string]iniSection
-
-func readFullLine(reader *bufio.Reader) (string, error) {
-	var line []byte
-
-	for {
-		l, more, err := reader.ReadLine()
-
-		if err != nil {
-			return "", err
-		}
-
-		if line == nil && !more {
-			return string(l), nil
-		}
-
-		line = append(line, l...)
-
-		if !more {
-			break
-		}
-	}
-
-	return string(line), nil
-}
-
-func optionIniName(option *Option) string {
-	name := option.tag.Get("_read-ini-name")
-
-	if len(name) != 0 {
-		return name
-	}
-
-	name = option.tag.Get("ini-name")
-
-	if len(name) != 0 {
-		return name
-	}
-
-	return option.field.Name
-}
-
-func writeGroupIni(group *Group, namespace string, writer io.Writer, options IniOptions) {
-	var sname string
-
-	if len(namespace) != 0 {
-		sname = namespace + "." + group.ShortDescription
-	} else {
-		sname = group.ShortDescription
-	}
-
-	sectionwritten := false
-	comments := (options & IniIncludeComments) != IniNone
-
-	for _, option := range group.options {
-		if option.isFunc() {
-			continue
-		}
-
-		if len(option.tag.Get("no-ini")) != 0 {
-			continue
-		}
-
-		val := option.value
-
-		if (options&IniIncludeDefaults) == IniNone &&
-			reflect.DeepEqual(val, option.defaultValue) {
-			continue
-		}
-
-		if !sectionwritten {
-			fmt.Fprintf(writer, "[%s]\n", sname)
-			sectionwritten = true
-		}
-
-		if comments {
-			fmt.Fprintf(writer, "; %s\n", option.Description)
-		}
-
-		oname := optionIniName(option)
-
-		switch val.Type().Kind() {
-		case reflect.Slice:
-			for idx := 0; idx < val.Len(); idx++ {
-				v, _ := convertToString(val.Index(idx), option.tag)
-				fmt.Fprintf(writer, "%s = %s\n", oname, v)
-			}
-
-			if val.Len() == 0 {
-				fmt.Fprintf(writer, "; %s =\n", oname)
-			}
-		case reflect.Map:
-			for _, key := range val.MapKeys() {
-				k, _ := convertToString(key, option.tag)
-				v, _ := convertToString(val.MapIndex(key), option.tag)
-
-				fmt.Fprintf(writer, "%s = %s:%s\n", oname, k, v)
-			}
-
-			if val.Len() == 0 {
-				fmt.Fprintf(writer, "; %s =\n", oname)
-			}
-		default:
-			v, _ := convertToString(val, option.tag)
-
-			if len(v) != 0 {
-				fmt.Fprintf(writer, "%s = %s\n", oname, v)
-			} else {
-				fmt.Fprintf(writer, "%s =\n", oname)
-			}
-		}
-
-		if comments {
-			fmt.Fprintln(writer)
-		}
-	}
-
-	if sectionwritten && !comments {
-		fmt.Fprintln(writer)
-	}
-}
-
-func writeCommandIni(command *Command, namespace string, writer io.Writer, options IniOptions) {
-	command.eachGroup(func(group *Group) {
-		writeGroupIni(group, namespace, writer, options)
-	})
-
-	for _, c := range command.commands {
-		var nns string
-
-		if len(namespace) != 0 {
-			nns = c.Name + "." + nns
-		} else {
-			nns = c.Name
-		}
-
-		writeCommandIni(c, nns, writer, options)
-	}
-}
-
-func writeIni(parser *IniParser, writer io.Writer, options IniOptions) {
-	writeCommandIni(parser.parser.Command, "", writer, options)
-}
-
-func readIniFromFile(filename string) (ini, error) {
-	file, err := os.Open(filename)
-
-	if err != nil {
-		return nil, err
-	}
-
-	defer file.Close()
-
-	return readIni(file, filename)
-}
-
-func readIni(contents io.Reader, filename string) (ini, error) {
-	ret := make(ini)
-
-	reader := bufio.NewReader(contents)
-
-	// Empty global section
-	section := make(iniSection, 0, 10)
-	sectionname := ""
-
-	ret[sectionname] = section
-
-	var lineno uint
-
-	for {
-		line, err := readFullLine(reader)
-
-		if err == io.EOF {
-			break
-		}
-
-		if err != nil {
-			return nil, err
-		}
-
-		lineno++
-		line = strings.TrimSpace(line)
-
-		// Skip empty lines and lines starting with ; (comments)
-		if len(line) == 0 || line[0] == ';' {
-			continue
-		}
-
-		if line[0] == '[' {
-			if line[0] != '[' || line[len(line)-1] != ']' {
-				return nil, &IniError{
-					Message:    "malformed section header",
-					File:       filename,
-					LineNumber: lineno,
-				}
-			}
-
-			name := strings.TrimSpace(line[1 : len(line)-1])
-
-			if len(name) == 0 {
-				return nil, &IniError{
-					Message:    "empty section name",
-					File:       filename,
-					LineNumber: lineno,
-				}
-			}
-
-			sectionname = name
-			section = ret[name]
-
-			if section == nil {
-				section = make(iniSection, 0, 10)
-				ret[name] = section
-			}
-
-			continue
-		}
-
-		// Parse option here
-		keyval := strings.SplitN(line, "=", 2)
-
-		if len(keyval) != 2 {
-			return nil, &IniError{
-				Message:    fmt.Sprintf("malformed key=value (%s)", line),
-				File:       filename,
-				LineNumber: lineno,
-			}
-		}
-
-		name := strings.TrimSpace(keyval[0])
-		value := strings.TrimSpace(keyval[1])
-
-		section = append(section, iniValue{
-			Name:  name,
-			Value: value,
-		})
-
-		ret[sectionname] = section
-	}
-
-	return ret, nil
-}
-
-func (i *IniParser) matchingGroups(name string) []*Group {
-	if len(name) == 0 {
-		var ret []*Group
-
-		i.parser.eachGroup(func(g *Group) {
-			ret = append(ret, g)
-		})
-
-		return ret
-	}
-
-	g := i.parser.groupByName(name)
-
-	if g != nil {
-		return []*Group{g}
-	}
-
-	return nil
-}
-
-func (i *IniParser) parse(ini ini) error {
-	p := i.parser
-
-	for name, section := range ini {
-		groups := i.matchingGroups(name)
-
-		if len(groups) == 0 {
-			return newError(ErrUnknownGroup,
-				fmt.Sprintf("could not find option group `%s'", name))
-		}
-
-		for _, inival := range section {
-			var opt *Option
-
-			for _, group := range groups {
-				opt = group.optionByName(inival.Name, func(o *Option, n string) bool {
-					return strings.ToLower(o.tag.Get("ini-name")) == strings.ToLower(n)
-				})
-
-				if opt != nil && len(opt.tag.Get("no-ini")) != 0 {
-					opt = nil
-				}
-
-				if opt != nil {
-					break
-				}
-			}
-
-			if opt == nil {
-				if (p.Options & IgnoreUnknown) == None {
-					return newError(ErrUnknownFlag,
-						fmt.Sprintf("unknown option: %s", inival.Name))
-				}
-
-				continue
-			}
-
-			pval := &inival.Value
-
-			if !opt.canArgument() && len(inival.Value) == 0 {
-				pval = nil
-			}
-
-			if err := opt.set(pval); err != nil {
-				return wrapError(err)
-			}
-
-			opt.tag.Set("_read-ini-name", inival.Name)
-		}
-	}
-
-	return nil
-}

github.com/jessevdk/go-flags/ini_test.go

@@ -1,170 +0,0 @@
-package flags
-
-import (
-	"bytes"
-	"strings"
-	"testing"
-)
-
-func TestWriteIni(t *testing.T) {
-	var opts helpOptions
-
-	p := NewNamedParser("TestIni", Default)
-	p.AddGroup("Application Options", "The application options", &opts)
-
-	p.ParseArgs([]string{"-vv", "--intmap=a:2", "--intmap", "b:3"})
-
-	inip := NewIniParser(p)
-
-	var b bytes.Buffer
-	inip.Write(&b, IniDefault|IniIncludeDefaults)
-
-	got := b.String()
-	expected := `[Application Options]
-; Show verbose debug information
-verbose = true
-verbose = true
-
-; A slice of pointers to string
-; PtrSlice =
-
-; Option only available in ini
-only-ini =
-
-[Other Options]
-; A slice of strings
-; StringSlice =
-
-; A map from string to int
-int-map = a:2
-int-map = b:3
-
-`
-
-	if got != expected {
-		ret, err := helpDiff(got, expected)
-
-		if err != nil {
-			t.Errorf("Unexpected ini, expected:\n\n%s\n\nbut got\n\n%s", expected, got)
-		} else {
-			t.Errorf("Unexpected ini:\n\n%s", ret)
-		}
-	}
-}
-
-func TestReadIni(t *testing.T) {
-	var opts helpOptions
-
-	p := NewNamedParser("TestIni", Default)
-	p.AddGroup("Application Options", "The application options", &opts)
-
-	inip := NewIniParser(p)
-
-	inic := `
-; Show verbose debug information
-verbose = true
-verbose = true
-
-[Application Options]
-; A slice of pointers to string
-; PtrSlice =
-
-[Other Options]
-; A slice of strings
-; StringSlice =
-
-; A map from string to int
-int-map = a:2
-int-map = b:3
-
-`
-
-	b := strings.NewReader(inic)
-	err := inip.Parse(b)
-
-	if err != nil {
-		t.Fatalf("Unexpected error: %s", err)
-	}
-
-	assertBoolArray(t, opts.Verbose, []bool{true, true})
-
-	if v, ok := opts.Other.IntMap["a"]; !ok {
-		t.Errorf("Expected \"a\" in Other.IntMap")
-	} else if v != 2 {
-		t.Errorf("Expected Other.IntMap[\"a\"] = 2, but got %v", v)
-	}
-
-	if v, ok := opts.Other.IntMap["b"]; !ok {
-		t.Errorf("Expected \"b\" in Other.IntMap")
-	} else if v != 3 {
-		t.Errorf("Expected Other.IntMap[\"b\"] = 3, but got %v", v)
-	}
-}
-
-func TestIniCommands(t *testing.T) {
-	var opts struct {
-		Value string `short:"v" long:"value"`
-
-		Add struct {
-			Name int `short:"n" long:"name" ini-name:"AliasName"`
-
-			Other struct {
-				O string `short:"o" long:"other"`
-			} `group:"Other Options"`
-		} `command:"add"`
-	}
-
-	p := NewNamedParser("TestIni", Default)
-	p.AddGroup("Application Options", "The application options", &opts)
-
-	inip := NewIniParser(p)
-
-	inic := `[Application Options]
-value = some value
-
-[add]
-AliasName = 5
-
-[add.Other Options]
-other = subgroup
-`
-
-	b := strings.NewReader(inic)
-	err := inip.Parse(b)
-
-	if err != nil {
-		t.Fatalf("Unexpected error: %s", err)
-	}
-
-	assertString(t, opts.Value, "some value")
-
-	if opts.Add.Name != 5 {
-		t.Errorf("Expected opts.Add.Name to be 5, but got %v", opts.Add.Name)
-	}
-
-	assertString(t, opts.Add.Other.O, "subgroup")
-}
-
-func TestIniNoIni(t *testing.T) {
-	var opts struct {
-		Value string `short:"v" long:"value" no-ini:"yes"`
-	}
-
-	p := NewNamedParser("TestIni", Default)
-	p.AddGroup("Application Options", "The application options", &opts)
-
-	inip := NewIniParser(p)
-
-	inic := `[Application Options]
-value = some value
-`
-
-	b := strings.NewReader(inic)
-	err := inip.Parse(b)
-
-	if err == nil {
-		t.Fatalf("Expected error")
-	}
-
-	assertError(t, err, ErrUnknownFlag, "unknown option: value")
-}

github.com/jessevdk/go-flags/long_test.go

@@ -1,85 +0,0 @@
-package flags
-
-import (
-	"testing"
-)
-
-func TestLong(t *testing.T) {
-	var opts = struct {
-		Value bool `long:"value"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "--value")
-
-	assertStringArray(t, ret, []string{})
-
-	if !opts.Value {
-		t.Errorf("Expected Value to be true")
-	}
-}
-
-func TestLongArg(t *testing.T) {
-	var opts = struct {
-		Value string `long:"value"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "--value", "value")
-
-	assertStringArray(t, ret, []string{})
-	assertString(t, opts.Value, "value")
-}
-
-func TestLongArgEqual(t *testing.T) {
-	var opts = struct {
-		Value string `long:"value"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "--value=value")
-
-	assertStringArray(t, ret, []string{})
-	assertString(t, opts.Value, "value")
-}
-
-func TestLongDefault(t *testing.T) {
-	var opts = struct {
-		Value string `long:"value" default:"value"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts)
-
-	assertStringArray(t, ret, []string{})
-	assertString(t, opts.Value, "value")
-}
-
-func TestLongOptional(t *testing.T) {
-	var opts = struct {
-		Value string `long:"value" optional:"yes" optional-value:"value"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "--value")
-
-	assertStringArray(t, ret, []string{})
-	assertString(t, opts.Value, "value")
-}
-
-func TestLongOptionalArg(t *testing.T) {
-	var opts = struct {
-		Value string `long:"value" optional:"yes" optional-value:"value"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "--value", "no")
-
-	assertStringArray(t, ret, []string{"no"})
-	assertString(t, opts.Value, "value")
-}
-
-func TestLongOptionalArgEqual(t *testing.T) {
-	var opts = struct {
-		Value string `long:"value" optional:"yes" optional-value:"value"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "--value=value", "no")
-
-	assertStringArray(t, ret, []string{"no"})
-	assertString(t, opts.Value, "value")
-}

github.com/jessevdk/go-flags/man.go

@@ -1,134 +0,0 @@
-package flags
-
-import (
-	"fmt"
-	"io"
-	"strings"
-	"time"
-)
-
-func formatForMan(wr io.Writer, s string) {
-	for {
-		idx := strings.IndexRune(s, '`')
-
-		if idx < 0 {
-			fmt.Fprintf(wr, "%s", s)
-			break
-		}
-
-		fmt.Fprintf(wr, "%s", s[:idx])
-
-		s = s[idx+1:]
-		idx = strings.IndexRune(s, '\'')
-
-		if idx < 0 {
-			fmt.Fprintf(wr, "%s", s)
-			break
-		}
-
-		fmt.Fprintf(wr, "\\fB%s\\fP", s[:idx])
-		s = s[idx+1:]
-	}
-}
-
-func writeManPageOptions(wr io.Writer, grp *Group) {
-	grp.eachGroup(func(group *Group) {
-		for _, opt := range group.options {
-			if !opt.canCli() {
-				continue
-			}
-
-			fmt.Fprintln(wr, ".TP")
-			fmt.Fprintf(wr, "\\fB")
-
-			if opt.ShortName != 0 {
-				fmt.Fprintf(wr, "-%c", opt.ShortName)
-			}
-
-			if len(opt.LongName) != 0 {
-				if opt.ShortName != 0 {
-					fmt.Fprintf(wr, ", ")
-				}
-
-				fmt.Fprintf(wr, "--%s", opt.LongName)
-			}
-
-			fmt.Fprintln(wr, "\\fP")
-			formatForMan(wr, opt.Description)
-			fmt.Fprintln(wr, "")
-		}
-	})
-}
-
-func writeManPageSubCommands(wr io.Writer, name string, root *Command) {
-	commands := root.sortedCommands()
-
-	for _, c := range commands {
-		var nn string
-
-		if len(name) != 0 {
-			nn = name + " " + c.Name
-		} else {
-			nn = c.Name
-		}
-
-		writeManPageCommand(wr, nn, c)
-	}
-}
-
-func writeManPageCommand(wr io.Writer, name string, command *Command) {
-	fmt.Fprintf(wr, ".SS %s\n", name)
-	fmt.Fprintln(wr, command.ShortDescription)
-
-	if len(command.LongDescription) > 0 {
-		fmt.Fprintln(wr, "")
-
-		cmdstart := fmt.Sprintf("The %s command", command.Name)
-
-		if strings.HasPrefix(command.LongDescription, cmdstart) {
-			fmt.Fprintf(wr, "The \\fI%s\\fP command", command.Name)
-
-			formatForMan(wr, command.LongDescription[len(cmdstart):])
-			fmt.Fprintln(wr, "")
-		} else {
-			formatForMan(wr, command.LongDescription)
-			fmt.Fprintln(wr, "")
-		}
-	}
-
-	writeManPageOptions(wr, command.Group)
-	writeManPageSubCommands(wr, name, command)
-}
-
-// WriteManPage writes a basic man page in groff format to the specified
-// writer.
-func (p *Parser) WriteManPage(wr io.Writer) {
-	t := time.Now()
-
-	fmt.Fprintf(wr, ".TH %s 1 \"%s\"\n", p.Name, t.Format("2 January 2006"))
-	fmt.Fprintln(wr, ".SH NAME")
-	fmt.Fprintf(wr, "%s \\- %s\n", p.Name, p.ShortDescription)
-	fmt.Fprintln(wr, ".SH SYNOPSIS")
-
-	usage := p.Usage
-
-	if len(usage) == 0 {
-		usage = "[OPTIONS]"
-	}
-
-	fmt.Fprintf(wr, "\\fB%s\\fP %s\n", p.Name, usage)
-	fmt.Fprintln(wr, ".SH DESCRIPTION")
-
-	formatForMan(wr, p.LongDescription)
-	fmt.Fprintln(wr, "")
-
-	fmt.Fprintln(wr, ".SH OPTIONS")
-
-	writeManPageOptions(wr, p.Command.Group)
-
-	if len(p.commands) > 0 {
-		fmt.Fprintln(wr, ".SH COMMANDS")
-
-		writeManPageSubCommands(wr, "", p.Command)
-	}
-}

github.com/jessevdk/go-flags/marshal_test.go

@@ -1,78 +0,0 @@
-package flags
-
-import (
-	"fmt"
-	"testing"
-)
-
-type marshalled bool
-
-func (m *marshalled) UnmarshalFlag(value string) error {
-	if value == "yes" {
-		*m = true
-	} else if value == "no" {
-		*m = false
-	} else {
-		return fmt.Errorf("`%s' is not a valid value, please specify `yes' or `no'", value)
-	}
-
-	return nil
-}
-
-func (m marshalled) MarshalFlag() string {
-	if m {
-		return "yes"
-	}
-
-	return "no"
-}
-
-func TestMarshal(t *testing.T) {
-	var opts = struct {
-		Value marshalled `short:"v"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "-v=yes")
-
-	assertStringArray(t, ret, []string{})
-
-	if !opts.Value {
-		t.Errorf("Expected Value to be true")
-	}
-}
-
-func TestMarshalDefault(t *testing.T) {
-	var opts = struct {
-		Value marshalled `short:"v" default:"yes"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts)
-
-	assertStringArray(t, ret, []string{})
-
-	if !opts.Value {
-		t.Errorf("Expected Value to be true")
-	}
-}
-
-func TestMarshalOptional(t *testing.T) {
-	var opts = struct {
-		Value marshalled `short:"v" optional:"yes" optional-value:"yes"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "-v")
-
-	assertStringArray(t, ret, []string{})
-
-	if !opts.Value {
-		t.Errorf("Expected Value to be true")
-	}
-}
-
-func TestMarshalError(t *testing.T) {
-	var opts = struct {
-		Value marshalled `short:"v"`
-	}{}
-
-	assertParseFail(t, ErrMarshal, "invalid argument for flag `-v' (expected flags.marshalled): `invalid' is not a valid value, please specify `yes' or `no'", &opts, "-vinvalid")
-}

github.com/jessevdk/go-flags/multitag.go

@@ -1,140 +0,0 @@
-package flags
-
-import (
-	"strconv"
-)
-
-type multiTag struct {
-	value string
-	cache map[string][]string
-}
-
-func newMultiTag(v string) multiTag {
-	return multiTag{
-		value: v,
-	}
-}
-
-func (x *multiTag) scan() (map[string][]string, error) {
-	v := x.value
-
-	ret := make(map[string][]string)
-
-	// This is mostly copied from reflect.StructTag.Get
-	for v != "" {
-		i := 0
-
-		// Skip whitespace
-		for i < len(v) && v[i] == ' ' {
-			i++
-		}
-
-		v = v[i:]
-
-		if v == "" {
-			break
-		}
-
-		// Scan to colon to find key
-		i = 0
-
-		for i < len(v) && v[i] != ' ' && v[i] != ':' && v[i] != '"' {
-			i++
-		}
-
-		if i >= len(v) {
-			return nil, newErrorf(ErrTag, "expected `:' after key name, but got end of tag (in `%v`)", x.value)
-		}
-
-		if v[i] != ':' {
-			return nil, newErrorf(ErrTag, "expected `:' after key name, but got `%v' (in `%v`)", v[i], x.value)
-		}
-
-		if i+1 >= len(v) {
-			return nil, newErrorf(ErrTag, "expected `\"' to start tag value at end of tag (in `%v`)", x.value)
-		}
-
-		if v[i+1] != '"' {
-			return nil, newErrorf(ErrTag, "expected `\"' to start tag value, but got `%v' (in `%v`)", v[i+1], x.value)
-		}
-
-		name := v[:i]
-		v = v[i+1:]
-
-		// Scan quoted string to find value
-		i = 1
-
-		for i < len(v) && v[i] != '"' {
-			if v[i] == '\n' {
-				return nil, newErrorf(ErrTag, "unexpected newline in tag value `%v' (in `%v`)", name, x.value)
-			}
-
-			if v[i] == '\\' {
-				i++
-			}
-			i++
-		}
-
-		if i >= len(v) {
-			return nil, newErrorf(ErrTag, "expected end of tag value `\"' at end of tag (in `%v`)", x.value)
-		}
-
-		val, err := strconv.Unquote(v[:i+1])
-
-		if err != nil {
-			return nil, newErrorf(ErrTag, "Malformed value of tag `%v:%v` => %v (in `%v`)", name, v[:i+1], err, x.value)
-		}
-
-		v = v[i+1:]
-
-		ret[name] = append(ret[name], val)
-	}
-
-	return ret, nil
-}
-
-func (x *multiTag) Parse() error {
-	vals, err := x.scan()
-	x.cache = vals
-
-	return err
-}
-
-func (x *multiTag) cached() map[string][]string {
-	if x.cache == nil {
-		cache, _ := x.scan()
-
-		if cache == nil {
-			cache = make(map[string][]string)
-		}
-
-		x.cache = cache
-	}
-
-	return x.cache
-}
-
-func (x *multiTag) Get(key string) string {
-	c := x.cached()
-
-	if v, ok := c[key]; ok {
-		return v[len(v)-1]
-	}
-
-	return ""
-}
-
-func (x *multiTag) GetMany(key string) []string {
-	c := x.cached()
-	return c[key]
-}
-
-func (x *multiTag) Set(key string, value string) {
-	c := x.cached()
-	c[key] = []string{value}
-}
-
-func (x *multiTag) SetMany(key string, value []string) {
-	c := x.cached()
-	c[key] = value
-}

github.com/jessevdk/go-flags/option.go

@@ -1,95 +0,0 @@
-package flags
-
-import (
-	"fmt"
-	"reflect"
-	"unicode/utf8"
-)
-
-// Option flag information. Contains a description of the option, short and
-// long name as well as a default value and whether an argument for this
-// flag is optional.
-type Option struct {
-	// The description of the option flag. This description is shown
-	// automatically in the builtin help.
-	Description string
-
-	// The short name of the option (a single character). If not 0, the
-	// option flag can be 'activated' using -<ShortName>. Either ShortName
-	// or LongName needs to be non-empty.
-	ShortName rune
-
-	// The long name of the option. If not "", the option flag can be
-	// activated using --<LongName>. Either ShortName or LongName needs
-	// to be non-empty.
-	LongName string
-
-	// The default value of the option.
-	Default []string
-
-	// If true, specifies that the argument to an option flag is optional.
-	// When no argument to the flag is specified on the command line, the
-	// value of Default will be set in the field this option represents.
-	// This is only valid for non-boolean options.
-	OptionalArgument bool
-
-	// The optional value of the option. The optional value is used when
-	// the option flag is marked as having an OptionalArgument. This means
-	// that when the flag is specified, but no option argument is given,
-	// the value of the field this option represents will be set to
-	// OptionalValue. This is only valid for non-boolean options.
-	OptionalValue []string
-
-	// If true, the option _must_ be specified on the command line. If the
-	// option is not specified, the parser will generate an ErrRequired type
-	// error.
-	Required bool
-
-	// A name for the value of an option shown in the Help as --flag [ValueName]
-	ValueName string
-
-	// A mask value to show in the help instead of the default value. This
-	// is useful for hiding sensitive information in the help, such as
-	// passwords.
-	DefaultMask string
-
-	// The struct field which the option represents.
-	field reflect.StructField
-
-	// The struct field value which the option represents.
-	value reflect.Value
-
-	defaultValue reflect.Value
-	iniUsedName  string
-	tag          multiTag
-}
-
-// String converts an option to a human friendly readable string describing the
-// option.
-func (option *Option) String() string {
-	var s string
-	var short string
-
-	if option.ShortName != 0 {
-		data := make([]byte, utf8.RuneLen(option.ShortName))
-		utf8.EncodeRune(data, option.ShortName)
-		short = string(data)
-
-		if len(option.LongName) != 0 {
-			s = fmt.Sprintf("%s%s, %s%s",
-				string(defaultShortOptDelimiter), short,
-				defaultLongOptDelimiter, option.LongName)
-		} else {
-			s = fmt.Sprintf("%s%s", string(defaultShortOptDelimiter), short)
-		}
-	} else if len(option.LongName) != 0 {
-		s = fmt.Sprintf("%s%s", defaultLongOptDelimiter, option.LongName)
-	}
-
-	return s
-}
-
-// Value returns the option value as an interface{}.
-func (option *Option) Value() interface{} {
-	return option.value.Interface()
-}

github.com/jessevdk/go-flags/option_private.go

@@ -1,125 +0,0 @@
-package flags
-
-import (
-	"reflect"
-)
-
-// Set the value of an option to the specified value. An error will be returned
-// if the specified value could not be converted to the corresponding option
-// value type.
-func (option *Option) set(value *string) error {
-	if option.isFunc() {
-		return option.call(value)
-	} else if value != nil {
-		return convert(*value, option.value, option.tag)
-	} else {
-		return convert("", option.value, option.tag)
-	}
-
-	return nil
-}
-
-func (option *Option) canCli() bool {
-	return option.ShortName != 0 || len(option.LongName) != 0
-}
-
-func (option *Option) canArgument() bool {
-	if u := option.isUnmarshaler(); u != nil {
-		return true
-	}
-
-	return !option.isBool()
-}
-
-func (option *Option) clear() {
-	tp := option.value.Type()
-
-	switch tp.Kind() {
-	case reflect.Func:
-		// Skip
-	case reflect.Map:
-		// Empty the map
-		option.value.Set(reflect.MakeMap(tp))
-	default:
-		zeroval := reflect.Zero(tp)
-		option.value.Set(zeroval)
-	}
-}
-
-func (option *Option) isUnmarshaler() Unmarshaler {
-	v := option.value
-
-	for {
-		if !v.CanInterface() {
-			return nil
-		}
-
-		i := v.Interface()
-
-		if u, ok := i.(Unmarshaler); ok {
-			return u
-		}
-
-		if !v.CanAddr() {
-			return nil
-		}
-
-		v = v.Addr()
-	}
-
-	return nil
-}
-
-func (option *Option) isBool() bool {
-	tp := option.value.Type()
-
-	for {
-		switch tp.Kind() {
-		case reflect.Bool:
-			return true
-		case reflect.Slice:
-			return (tp.Elem().Kind() == reflect.Bool)
-		case reflect.Func:
-			return tp.NumIn() == 0
-		case reflect.Ptr:
-			tp = tp.Elem()
-		default:
-			return false
-		}
-	}
-
-	return false
-}
-
-func (option *Option) isFunc() bool {
-	return option.value.Type().Kind() == reflect.Func
-}
-
-func (option *Option) call(value *string) error {
-	var retval []reflect.Value
-
-	if value == nil {
-		retval = option.value.Call(nil)
-	} else {
-		tp := option.value.Type().In(0)
-
-		val := reflect.New(tp)
-		val = reflect.Indirect(val)
-
-		if err := convert(*value, val, option.tag); err != nil {
-			return err
-		}
-
-		retval = option.value.Call([]reflect.Value{val})
-	}
-
-	if len(retval) == 1 && retval[0].Type() == reflect.TypeOf((*error)(nil)).Elem() {
-		if retval[0].Interface() == nil {
-			return nil
-		}
-
-		return retval[0].Interface().(error)
-	}
-
-	return nil
-}

github.com/jessevdk/go-flags/options_test.go

@@ -1,45 +0,0 @@
-package flags
-
-import (
-	"testing"
-)
-
-func TestPassDoubleDash(t *testing.T) {
-	var opts = struct {
-		Value bool `short:"v"`
-	}{}
-
-	p := NewParser(&opts, PassDoubleDash)
-	ret, err := p.ParseArgs([]string{"-v", "--", "-v", "-g"})
-
-	if err != nil {
-		t.Fatalf("Unexpected error: %v", err)
-		return
-	}
-
-	if !opts.Value {
-		t.Errorf("Expected Value to be true")
-	}
-
-	assertStringArray(t, ret, []string{"-v", "-g"})
-}
-
-func TestPassAfterNonOption(t *testing.T) {
-	var opts = struct {
-		Value bool `short:"v"`
-	}{}
-
-	p := NewParser(&opts, PassAfterNonOption)
-	ret, err := p.ParseArgs([]string{"-v", "arg", "-v", "-g"})
-
-	if err != nil {
-		t.Fatalf("Unexpected error: %v", err)
-		return
-	}
-
-	if !opts.Value {
-		t.Errorf("Expected Value to be true")
-	}
-
-	assertStringArray(t, ret, []string{"arg", "-v", "-g"})
-}

github.com/jessevdk/go-flags/optstyle_other.go

@@ -1,54 +0,0 @@
-// +build !windows
-
-package flags
-
-import (
-	"strings"
-)
-
-const (
-	defaultShortOptDelimiter = '-'
-	defaultLongOptDelimiter  = "--"
-	defaultNameArgDelimiter  = '='
-)
-
-func argumentIsOption(arg string) bool {
-	return len(arg) > 0 && arg[0] == '-'
-}
-
-// stripOptionPrefix returns the option without the prefix and whether or
-// not the option is a long option or not.
-func stripOptionPrefix(optname string) (prefix string, name string, islong bool) {
-	if strings.HasPrefix(optname, "--") {
-		return "--", optname[2:], true
-	} else if strings.HasPrefix(optname, "-") {
-		return "-", optname[1:], false
-	}
-
-	return "", optname, false
-}
-
-// splitOption attempts to split the passed option into a name and an argument.
-// When there is no argument specified, nil will be returned for it.
-func splitOption(prefix string, option string, islong bool) (string, *string) {
-	pos := strings.Index(option, "=")
-
-	if (islong && pos >= 0) || (!islong && pos == 1) {
-		rest := option[pos+1:]
-		return option[:pos], &rest
-	}
-
-	return option, nil
-}
-
-// addHelpGroup adds a new group that contains default help parameters.
-func (c *Command) addHelpGroup(showHelp func() error) *Group {
-	var help struct {
-		ShowHelp func() error `short:"h" long:"help" description:"Show this help message"`
-	}
-
-	help.ShowHelp = showHelp
-	ret, _ := c.AddGroup("Help Options", "", &help)
-
-	return ret
-}

github.com/jessevdk/go-flags/optstyle_windows.go

@@ -1,85 +0,0 @@
-package flags
-
-import (
-	"strings"
-)
-
-// Windows uses a front slash for both short and long options.  Also it uses
-// a colon for name/argument delimter.
-const (
-	defaultShortOptDelimiter = '/'
-	defaultLongOptDelimiter  = "/"
-	defaultNameArgDelimiter  = ':'
-)
-
-func argumentIsOption(arg string) bool {
-	// Windows-style options allow front slash for the option
-	// delimiter.
-	return len(arg) > 0 && (arg[0] == '-' || arg[0] == '/')
-}
-
-// stripOptionPrefix returns the option without the prefix and whether or
-// not the option is a long option or not.
-func stripOptionPrefix(optname string) (prefix string, name string, islong bool) {
-	// Determine if the argument is a long option or not.  Windows
-	// typically supports both long and short options with a single
-	// front slash as the option delimiter, so handle this situation
-	// nicely.
-	possplit := 0
-
-	if strings.HasPrefix(optname, "--") {
-		possplit = 2
-		islong = true
-	} else if strings.HasPrefix(optname, "-") {
-		possplit = 1
-		islong = false
-	} else if strings.HasPrefix(optname, "/") {
-		possplit = 1
-		islong = len(optname) > 2
-	}
-
-	return optname[:possplit], optname[possplit:], islong
-}
-
-// splitOption attempts to split the passed option into a name and an argument.
-// When there is no argument specified, nil will be returned for it.
-func splitOption(prefix string, option string, islong bool) (string, *string) {
-	if len(option) == 0 {
-		return option, nil
-	}
-
-	// Windows typically uses a colon for the option name and argument
-	// delimiter while POSIX typically uses an equals.  Support both styles,
-	// but don't allow the two to be mixed.  That is to say /foo:bar and
-	// --foo=bar are acceptable, but /foo=bar and --foo:bar are not.
-	var pos int
-
-	if prefix == "/" {
-		pos = strings.Index(option, ":")
-	} else if len(prefix) > 0 {
-		pos = strings.Index(option, "=")
-	}
-
-	if (islong && pos >= 0) || (!islong && pos == 1) {
-		rest := option[pos+1:]
-		return option[:pos], &rest
-	}
-
-	return option, nil
-}
-
-// addHelpGroup adds a new group that contains default help parameters.
-func (c *Command) addHelpGroup(showHelp func() error) *Group {
-	// Windows CLI applications typically use /? for help, so make both
-	// that available as well as the POSIX style h and help.
-	var help struct {
-		ShowHelpWindows func() error `short:"?" description:"Show this help message"`
-		ShowHelpPosix   func() error `short:"h" long:"help" description:"Show this help message"`
-	}
-
-	help.ShowHelpWindows = showHelp
-	help.ShowHelpPosix = showHelp
-
-	ret, _ := c.AddGroup("Help Options", "", &help)
-	return ret
-}

github.com/jessevdk/go-flags/parser.go

@@ -1,212 +0,0 @@
-// Copyright 2012 Jesse van den Kieboom. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package flags
-
-import (
-	"os"
-	"path"
-)
-
-// A Parser provides command line option parsing. It can contain several
-// option groups each with their own set of options.
-type Parser struct {
-	// Embedded, see Command for more information
-	*Command
-
-	// A usage string to be displayed in the help message.
-	Usage   string
-
-	// Option flags changing the behavior of the parser.
-	Options Options
-
-	internalError error
-}
-
-// Options provides parser options that change the behavior of the option
-// parser.
-type Options uint
-
-const (
-	// None indicates no options.
-	None Options = 0
-
-	// HelpFlag adds a default Help Options group to the parser containing
-	// -h and --help options. When either -h or --help is specified on the
-	// command line, the parser will return the special error of type
-	// ErrHelp. When PrintErrors is also specified, then the help message
-	// will also be automatically printed to os.Stderr.
-	HelpFlag = 1 << iota
-
-	// PassDoubleDash passes all arguments after a double dash, --, as
-	// remaining command line arguments (i.e. they will not be parsed for
-	// flags).
-	PassDoubleDash
-
-	// IgnoreUnknown ignores any unknown options and passes them as
-	// remaining command line arguments instead of generating an error.
-	IgnoreUnknown
-
-	// PrintErrors prints any errors which occurred during parsing to
-	// os.Stderr.
-	PrintErrors
-
-	// PassAfterNonOption passes all arguments after the first non option
-	// as remaining command line arguments. This is equivalent to strict
-	// POSIX processing.
-	PassAfterNonOption
-
-	// Default is a convenient default set of options which should cover
-	// most of the uses of the flags package.
-	Default = HelpFlag | PrintErrors | PassDoubleDash
-)
-
-// Parse is a convenience function to parse command line options with default
-// settings. The provided data is a pointer to a struct representing the
-// default option group (named "Application Options"). For more control, use
-// flags.NewParser.
-func Parse(data interface{}) ([]string, error) {
-	return NewParser(data, Default).Parse()
-}
-
-// ParseArgs is a convenience function to parse command line options with default
-// settings. The provided data is a pointer to a struct representing the
-// default option group (named "Application Options"). The args argument is
-// the list of command line arguments to parse. If you just want to parse the
-// default program command line arguments (i.e. os.Args), then use flags.Parse
-// instead. For more control, use flags.NewParser.
-func ParseArgs(data interface{}, args []string) ([]string, error) {
-	return NewParser(data, Default).ParseArgs(args)
-}
-
-// NewParser creates a new parser. It uses os.Args[0] as the application
-// name and then calls Parser.NewNamedParser (see Parser.NewNamedParser for
-// more details). The provided data is a pointer to a struct representing the
-// default option group (named "Application Options"), or nil if the default
-// group should not be added. The options parameter specifies a set of options
-// for the parser.
-func NewParser(data interface{}, options Options) *Parser {
-	ret := NewNamedParser(path.Base(os.Args[0]), options)
-
-	if data != nil {
-		_, ret.internalError = ret.AddGroup("Application Options", "", data)
-	}
-
-	return ret
-}
-
-// NewNamedParser creates a new parser. The appname is used to display the
-// executable name in the builtin help message. Option groups and commands can
-// be added to this parser by using AddGroup and AddCommand.
-func NewNamedParser(appname string, options Options) *Parser {
-	return &Parser{
-		Command: newCommand(appname, "", "", nil),
-		Options: options,
-	}
-}
-
-// Parse parses the command line arguments from os.Args using Parser.ParseArgs.
-// For more detailed information see ParseArgs.
-func (p *Parser) Parse() ([]string, error) {
-	return p.ParseArgs(os.Args[1:])
-}
-
-// ParseArgs parses the command line arguments according to the option groups that
-// were added to the parser. On successful parsing of the arguments, the
-// remaining, non-option, arguments (if any) are returned. The returned error
-// indicates a parsing error and can be used with PrintError to display
-// contextual information on where the error occurred exactly.
-//
-// When the common help group has been added (AddHelp) and either -h or --help
-// was specified in the command line arguments, a help message will be
-// automatically printed. Furthermore, the special error type ErrHelp is returned.
-// It is up to the caller to exit the program if so desired.
-func (p *Parser) ParseArgs(args []string) ([]string, error) {
-	if p.internalError != nil {
-		return nil, p.internalError
-	}
-
-	p.eachCommand(func(c *Command) {
-		p.eachGroup(func(g *Group) {
-			g.storeDefaults()
-		})
-	}, true)
-
-	// Add builtin help group to all commands if necessary
-	if (p.Options & HelpFlag) != None {
-		p.addHelpGroups(p.showBuiltinHelp)
-	}
-
-	s := &parseState{
-		args:    args,
-		retargs: make([]string, 0, len(args)),
-		command: p.Command,
-		lookup:  p.makeLookup(),
-	}
-
-	for !s.eof() {
-		arg := s.pop()
-
-		// When PassDoubleDash is set and we encounter a --, then
-		// simply append all the rest as arguments and break out
-		if (p.Options&PassDoubleDash) != None && arg == "--" {
-			s.retargs = append(s.retargs, s.args...)
-			break
-		}
-
-		if !argumentIsOption(arg) {
-			// Note: this also sets s.err, so we can just check for
-			// nil here and use s.err later
-			if p.parseNonOption(s) != nil {
-				break
-			}
-
-			continue
-		}
-
-		var err error
-		var option *Option
-
-		prefix, optname, islong := stripOptionPrefix(arg)
-		optname, argument := splitOption(prefix, optname, islong)
-
-		if islong {
-			option, err = p.parseLong(s, optname, argument)
-		} else {
-			option, err = p.parseShort(s, optname, argument)
-		}
-
-		if err != nil {
-			ignoreUnknown := (p.Options & IgnoreUnknown) != None
-			parseErr := wrapError(err)
-
-			if !(parseErr.Type == ErrUnknownFlag && ignoreUnknown) {
-				s.err = parseErr
-				break
-			}
-
-			if ignoreUnknown {
-				s.retargs = append(s.retargs, arg)
-			}
-		} else {
-			delete(s.lookup.required, option)
-		}
-	}
-
-	if s.err == nil {
-		s.checkRequired()
-	}
-
-	if s.err != nil {
-		return nil, p.printError(s.err)
-	}
-
-	if len(s.command.commands) != 0 {
-		return nil, p.printError(s.estimateCommand())
-	} else if cmd, ok := s.command.data.(Commander); ok {
-		return nil, p.printError(cmd.Execute(s.retargs))
-	}
-
-	return s.retargs, nil
-}

github.com/jessevdk/go-flags/parser_private.go

@@ -1,243 +0,0 @@
-package flags
-
-import (
-	"bytes"
-	"fmt"
-	"os"
-	"strings"
-	"unicode/utf8"
-)
-
-type parseState struct {
-	arg     string
-	args    []string
-	retargs []string
-	err     error
-
-	command *Command
-	lookup  lookup
-}
-
-func (p *parseState) eof() bool {
-	return len(p.args) == 0
-}
-
-func (p *parseState) pop() string {
-	if p.eof() {
-		return ""
-	}
-
-	p.arg = p.args[0]
-	p.args = p.args[1:]
-
-	return p.arg
-}
-
-func (p *parseState) peek() string {
-	if p.eof() {
-		return ""
-	}
-
-	return p.args[0]
-}
-
-func (p *parseState) checkRequired() error {
-	required := p.lookup.required
-
-	if len(required) == 0 {
-		return nil
-	}
-
-	names := make([]string, 0, len(required))
-
-	for k := range required {
-		names = append(names, "`"+k.String()+"'")
-	}
-
-	var msg string
-
-	if len(names) == 1 {
-		msg = fmt.Sprintf("the required flag %s was not specified", names[0])
-	} else {
-		msg = fmt.Sprintf("the required flags %s and %s were not specified",
-			strings.Join(names[:len(names)-1], ", "), names[len(names)-1])
-	}
-
-	p.err = newError(ErrRequired, msg)
-	return p.err
-}
-
-func (p *parseState) estimateCommand() error {
-	commands := p.command.sortedCommands()
-	cmdnames := make([]string, len(commands))
-
-	for i, v := range commands {
-		cmdnames[i] = v.Name
-	}
-
-	var msg string
-
-	if len(p.retargs) != 0 {
-		c, l := closestChoice(p.retargs[0], cmdnames)
-		msg = fmt.Sprintf("Unknown command `%s'", p.retargs[0])
-
-		if float32(l)/float32(len(c)) < 0.5 {
-			msg = fmt.Sprintf("%s, did you mean `%s'?", msg, c)
-		} else if len(cmdnames) == 1 {
-			msg = fmt.Sprintf("%s. You should use the %s command",
-				msg,
-				cmdnames[0])
-		} else {
-			msg = fmt.Sprintf("%s. Please specify one command of: %s or %s",
-				msg,
-				strings.Join(cmdnames[:len(cmdnames)-1], ", "),
-				cmdnames[len(cmdnames)-1])
-		}
-	} else {
-		if len(cmdnames) == 1 {
-			msg = fmt.Sprintf("Please specify the %s command", cmdnames[0])
-		} else {
-			msg = fmt.Sprintf("Please specify one command of: %s or %s",
-				strings.Join(cmdnames[:len(cmdnames)-1], ", "),
-				cmdnames[len(cmdnames)-1])
-		}
-	}
-
-	return newError(ErrRequired, msg)
-}
-
-func (p *Parser) parseOption(s *parseState, name string, option *Option, canarg bool, argument *string) (retoption *Option, err error) {
-	if !option.canArgument() {
-		if argument != nil {
-			msg := fmt.Sprintf("bool flag `%s' cannot have an argument", option)
-			return option, newError(ErrNoArgumentForBool, msg)
-		}
-
-		err = option.set(nil)
-	} else if argument != nil {
-		err = option.set(argument)
-	} else if canarg && !s.eof() {
-		arg := s.pop()
-		err = option.set(&arg)
-	} else if option.OptionalArgument {
-		option.clear()
-
-		for _, v := range option.OptionalValue {
-			err = option.set(&v)
-
-			if err != nil {
-				break
-			}
-		}
-	} else {
-		msg := fmt.Sprintf("expected argument for flag `%s'", option)
-		err = newError(ErrExpectedArgument, msg)
-	}
-
-	if err != nil {
-		if _, ok := err.(*Error); !ok {
-			msg := fmt.Sprintf("invalid argument for flag `%s' (expected %s): %s",
-				option,
-				option.value.Type(),
-				err.Error())
-
-			err = newError(ErrMarshal, msg)
-		}
-	}
-
-	return option, err
-}
-
-func (p *Parser) parseLong(s *parseState, name string, argument *string) (option *Option, err error) {
-	if option := s.lookup.longNames[name]; option != nil {
-		// Only long options that are required can consume an argument
-		// from the argument list
-		canarg := !option.OptionalArgument
-
-		return p.parseOption(s, name, option, canarg, argument)
-	}
-
-	return nil, newError(ErrUnknownFlag, fmt.Sprintf("unknown flag `%s'", name))
-}
-
-func (p *Parser) splitShortConcatArg(s *parseState, optname string) (string, *string) {
-	c, n := utf8.DecodeRuneInString(optname)
-
-	if n == len(optname) {
-		return optname, nil
-	}
-
-	first := string(c)
-
-	if option := s.lookup.shortNames[first]; option != nil && option.canArgument() {
-		arg := optname[n:]
-		return first, &arg
-	}
-
-	return optname, nil
-}
-
-func (p *Parser) parseShort(s *parseState, optname string, argument *string) (option *Option, err error) {
-	if argument == nil {
-		optname, argument = p.splitShortConcatArg(s, optname)
-	}
-
-	for i, c := range optname {
-		shortname := string(c)
-
-		if option = s.lookup.shortNames[shortname]; option != nil {
-			// Only the last short argument can consume an argument from
-			// the arguments list, and only if it's non optional
-			canarg := (i+utf8.RuneLen(c) == len(optname)) && !option.OptionalArgument
-
-			if _, err := p.parseOption(s, shortname, option, canarg, argument); err != nil {
-				return option, err
-			}
-		} else {
-			return nil, newError(ErrUnknownFlag, fmt.Sprintf("unknown flag `%s'", shortname))
-		}
-
-		// Only the first option can have a concatted argument, so just
-		// clear argument here
-		argument = nil
-	}
-
-	return option, nil
-}
-
-func (p *Parser) parseNonOption(s *parseState) error {
-	if cmd := s.lookup.commands[s.arg]; cmd != nil {
-		if err := s.checkRequired(); err != nil {
-			return err
-		}
-
-		s.command.Active = cmd
-
-		s.command = cmd
-		s.lookup = cmd.makeLookup()
-	} else if (p.Options & PassAfterNonOption) != None {
-		// If PassAfterNonOption is set then all remaining arguments
-		// are considered positional
-		s.retargs = append(append(s.retargs, s.arg), s.args...)
-		s.args = []string{}
-	} else {
-		s.retargs = append(s.retargs, s.arg)
-	}
-
-	return nil
-}
-
-func (p *Parser) showBuiltinHelp() error {
-	var b bytes.Buffer
-
-	p.WriteHelp(&b)
-	return newError(ErrHelp, b.String())
-}
-
-func (p *Parser) printError(err error) error {
-	if err != nil && (p.Options&PrintErrors) != None {
-		fmt.Fprintln(os.Stderr, err)
-	}
-
-	return err
-}

github.com/jessevdk/go-flags/pointer_test.go

@@ -1,81 +0,0 @@
-package flags
-
-import (
-	"testing"
-)
-
-func TestPointerBool(t *testing.T) {
-	var opts = struct {
-		Value *bool `short:"v"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "-v")
-
-	assertStringArray(t, ret, []string{})
-
-	if !*opts.Value {
-		t.Errorf("Expected Value to be true")
-	}
-}
-
-func TestPointerString(t *testing.T) {
-	var opts = struct {
-		Value *string `short:"v"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "-v", "value")
-
-	assertStringArray(t, ret, []string{})
-	assertString(t, *opts.Value, "value")
-}
-
-func TestPointerSlice(t *testing.T) {
-	var opts = struct {
-		Value *[]string `short:"v"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "-v", "value1", "-v", "value2")
-
-	assertStringArray(t, ret, []string{})
-	assertStringArray(t, *opts.Value, []string{"value1", "value2"})
-}
-
-func TestPointerMap(t *testing.T) {
-	var opts = struct {
-		Value *map[string]int `short:"v"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "-v", "k1:2", "-v", "k2:-5")
-
-	assertStringArray(t, ret, []string{})
-
-	if v, ok := (*opts.Value)["k1"]; !ok {
-		t.Errorf("Expected key \"k1\" to exist")
-	} else if v != 2 {
-		t.Errorf("Expected \"k1\" to be 2, but got %#v", v)
-	}
-
-	if v, ok := (*opts.Value)["k2"]; !ok {
-		t.Errorf("Expected key \"k2\" to exist")
-	} else if v != -5 {
-		t.Errorf("Expected \"k2\" to be -5, but got %#v", v)
-	}
-}
-
-type PointerGroup struct {
-	Value bool `short:"v"`
-}
-
-func TestPointerGroup(t *testing.T) {
-	var opts = struct {
-		Group *PointerGroup `group:"Group Options"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "-v")
-
-	assertStringArray(t, ret, []string{})
-
-	if !opts.Group.Value {
-		t.Errorf("Expected Group.Value to be true")
-	}
-}

github.com/jessevdk/go-flags/short_test.go

@@ -1,169 +0,0 @@
-package flags
-
-import (
-	"testing"
-)
-
-func TestShort(t *testing.T) {
-	var opts = struct {
-		Value bool `short:"v"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "-v")
-
-	assertStringArray(t, ret, []string{})
-
-	if !opts.Value {
-		t.Errorf("Expected Value to be true")
-	}
-}
-
-func TestShortTooLong(t *testing.T) {
-	var opts = struct {
-		Value bool `short:"vv"`
-	}{}
-
-	assertParseFail(t, ErrShortNameTooLong, "short names can only be 1 character long, not `vv'", &opts)
-}
-
-func TestShortRequired(t *testing.T) {
-	var opts = struct {
-		Value bool `short:"v" required:"true"`
-	}{}
-
-	assertParseFail(t, ErrRequired, "the required flag `-v' was not specified", &opts)
-}
-
-func TestShortMultiConcat(t *testing.T) {
-	var opts = struct {
-		V bool `short:"v"`
-		O bool `short:"o"`
-		F bool `short:"f"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "-vo", "-f")
-
-	assertStringArray(t, ret, []string{})
-
-	if !opts.V {
-		t.Errorf("Expected V to be true")
-	}
-
-	if !opts.O {
-		t.Errorf("Expected O to be true")
-	}
-
-	if !opts.F {
-		t.Errorf("Expected F to be true")
-	}
-}
-
-func TestShortMultiSlice(t *testing.T) {
-	var opts = struct {
-		Values []bool `short:"v"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "-v", "-v")
-
-	assertStringArray(t, ret, []string{})
-	assertBoolArray(t, opts.Values, []bool{true, true})
-}
-
-func TestShortMultiSliceConcat(t *testing.T) {
-	var opts = struct {
-		Values []bool `short:"v"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "-vvv")
-
-	assertStringArray(t, ret, []string{})
-	assertBoolArray(t, opts.Values, []bool{true, true, true})
-}
-
-func TestShortWithEqualArg(t *testing.T) {
-	var opts = struct {
-		Value string `short:"v"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "-v=value")
-
-	assertStringArray(t, ret, []string{})
-	assertString(t, opts.Value, "value")
-}
-
-func TestShortWithArg(t *testing.T) {
-	var opts = struct {
-		Value string `short:"v"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "-vvalue")
-
-	assertStringArray(t, ret, []string{})
-	assertString(t, opts.Value, "value")
-}
-
-func TestShortArg(t *testing.T) {
-	var opts = struct {
-		Value string `short:"v"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "-v", "value")
-
-	assertStringArray(t, ret, []string{})
-	assertString(t, opts.Value, "value")
-}
-
-func TestShortMultiWithEqualArg(t *testing.T) {
-	var opts = struct {
-		F     []bool `short:"f"`
-		Value string `short:"v"`
-	}{}
-
-	assertParseFail(t, ErrExpectedArgument, "expected argument for flag `-v'", &opts, "-ffv=value")
-}
-
-func TestShortMultiArg(t *testing.T) {
-	var opts = struct {
-		F     []bool `short:"f"`
-		Value string `short:"v"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "-ffv", "value")
-
-	assertStringArray(t, ret, []string{})
-	assertBoolArray(t, opts.F, []bool{true, true})
-	assertString(t, opts.Value, "value")
-}
-
-func TestShortMultiArgConcatFail(t *testing.T) {
-	var opts = struct {
-		F     []bool `short:"f"`
-		Value string `short:"v"`
-	}{}
-
-	assertParseFail(t, ErrExpectedArgument, "expected argument for flag `-v'", &opts, "-ffvvalue")
-}
-
-func TestShortMultiArgConcat(t *testing.T) {
-	var opts = struct {
-		F     []bool `short:"f"`
-		Value string `short:"v"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "-vff")
-
-	assertStringArray(t, ret, []string{})
-	assertString(t, opts.Value, "ff")
-}
-
-func TestShortOptional(t *testing.T) {
-	var opts = struct {
-		F     []bool `short:"f"`
-		Value string `short:"v" optional:"yes" optional-value:"value"`
-	}{}
-
-	ret := assertParseSuccess(t, &opts, "-fv", "f")
-
-	assertStringArray(t, ret, []string{"f"})
-	assertString(t, opts.Value, "value")
-}

github.com/jessevdk/go-flags/tag_test.go

@@ -1,39 +0,0 @@
-package flags
-
-import (
-	"testing"
-)
-
-func TestTagMissingColon(t *testing.T) {
-	var opts = struct {
-		Value bool `short`
-	}{}
-
-	assertParseFail(t, ErrTag, "expected `:' after key name, but got end of tag (in `short`)", &opts, "")
-}
-
-func TestTagMissingValue(t *testing.T) {
-	var opts = struct {
-		Value bool `short:`
-	}{}
-
-	assertParseFail(t, ErrTag, "expected `\"' to start tag value at end of tag (in `short:`)", &opts, "")
-}
-
-func TestTagMissingQuote(t *testing.T) {
-	var opts = struct {
-		Value bool `short:"v`
-	}{}
-
-	assertParseFail(t, ErrTag, "expected end of tag value `\"' at end of tag (in `short:\"v`)", &opts, "")
-}
-
-func TestTagNewline(t *testing.T) {
-	var opts = struct {
-		Value bool `long:"verbose" description:"verbose
-something"`
-	}{}
-
-	assertParseFail(t, ErrTag, "unexpected newline in tag value `description' (in `long:\"verbose\" description:\"verbose\nsomething\"`)", &opts, "")
-}
-

github.com/jessevdk/go-flags/termsize.go

@@ -1,5 +0,0 @@
-package flags
-
-func getTerminalColumns() int {
-	return 80
-}

github.com/jessevdk/go-flags/unknown_test.go

@@ -1,66 +0,0 @@
-package flags
-
-import (
-	"testing"
-)
-
-func TestUnknownFlags(t *testing.T) {
-	var opts = struct {
-		Verbose []bool `short:"v" long:"verbose" description:"Verbose output"`
-	}{}
-
-	args := []string{
-		"-f",
-	}
-
-	p := NewParser(&opts, 0)
-	args, err := p.ParseArgs(args)
-
-	if err == nil {
-		t.Fatal("Expected error for unknown argument")
-	}
-}
-
-func TestIgnoreUnknownFlags(t *testing.T) {
-	var opts = struct {
-		Verbose []bool `short:"v" long:"verbose" description:"Verbose output"`
-	}{}
-
-	args := []string{
-		"hello",
-		"world",
-		"-v",
-		"--foo=bar",
-		"--verbose",
-		"-f",
-	}
-
-	p := NewParser(&opts, IgnoreUnknown)
-	args, err := p.ParseArgs(args)
-
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	exargs := []string{
-		"hello",
-		"world",
-		"--foo=bar",
-		"-f",
-	}
-
-	issame := (len(args) == len(exargs))
-
-	if issame {
-		for i := 0; i < len(args); i++ {
-			if args[i] != exargs[i] {
-				issame = false
-				break
-			}
-		}
-	}
-
-	if !issame {
-		t.Fatalf("Expected %v but got %v", exargs, args)
-	}
-}

gui.go

@@ -1,132 +0,0 @@
-package main
-
-import (
-	"encoding/json"
-	"fmt"
-	"io"
-	"log"
-	"mime"
-	"net/http"
-	"path/filepath"
-
-	"bitbucket.org/tebeka/nrsc"
-	"github.com/calmh/syncthing/model"
-	"github.com/codegangsta/martini"
-)
-
-func startGUI(addr string, m *model.Model) {
-	router := martini.NewRouter()
-	router.Get("/", getRoot)
-	router.Get("/rest/version", restGetVersion)
-	router.Get("/rest/model", restGetModel)
-	router.Get("/rest/connections", restGetConnections)
-	router.Get("/rest/config", restGetConfig)
-	router.Get("/rest/need", restGetNeed)
-
-	go func() {
-		mr := martini.New()
-		mr.Use(nrscStatic("gui"))
-		mr.Use(martini.Recovery())
-		mr.Action(router.Handle)
-		mr.Map(m)
-		http.ListenAndServe(addr, mr)
-	}()
-}
-
-func getRoot(w http.ResponseWriter, r *http.Request) {
-	http.Redirect(w, r, "/index.html", 302)
-}
-
-func restGetVersion() string {
-	return Version
-}
-
-func restGetModel(m *model.Model, w http.ResponseWriter) {
-	var res = make(map[string]interface{})
-
-	globalFiles, globalDeleted, globalBytes := m.GlobalSize()
-	res["globalFiles"], res["globalDeleted"], res["globalBytes"] = globalFiles, globalDeleted, globalBytes
-
-	localFiles, localDeleted, localBytes := m.LocalSize()
-	res["localFiles"], res["localDeleted"], res["localBytes"] = localFiles, localDeleted, localBytes
-
-	inSyncFiles, inSyncBytes := m.InSyncSize()
-	res["inSyncFiles"], res["inSyncBytes"] = inSyncFiles, inSyncBytes
-
-	files, total := m.NeedFiles()
-	res["needFiles"], res["needBytes"] = len(files), total
-
-	w.Header().Set("Content-Type", "application/json")
-	json.NewEncoder(w).Encode(res)
-}
-
-func restGetConnections(m *model.Model, w http.ResponseWriter) {
-	var res = m.ConnectionStats()
-	w.Header().Set("Content-Type", "application/json")
-	json.NewEncoder(w).Encode(res)
-}
-
-func restGetConfig(w http.ResponseWriter) {
-	var res = make(map[string]interface{})
-	res["repository"] = config.OptionMap("repository")
-	res["nodes"] = config.OptionMap("nodes")
-	w.Header().Set("Content-Type", "application/json")
-	json.NewEncoder(w).Encode(res)
-}
-
-type guiFile model.File
-
-func (f guiFile) MarshalJSON() ([]byte, error) {
-	type t struct {
-		Name string
-		Size int
-	}
-	return json.Marshal(t{
-		Name: f.Name,
-		Size: model.File(f).Size(),
-	})
-}
-
-func restGetNeed(m *model.Model, w http.ResponseWriter) {
-	files, _ := m.NeedFiles()
-	gfs := make([]guiFile, len(files))
-	for i, f := range files {
-		gfs[i] = guiFile(f)
-	}
-	w.Header().Set("Content-Type", "application/json")
-	json.NewEncoder(w).Encode(gfs)
-}
-
-func nrscStatic(path string) interface{} {
-	if err := nrsc.Initialize(); err != nil {
-		panic("Unable to initialize nrsc: " + err.Error())
-	}
-	return func(res http.ResponseWriter, req *http.Request, log *log.Logger) {
-		file := req.URL.Path
-
-		// nrsc expects there not to be a leading slash
-		if file[0] == '/' {
-			file = file[1:]
-		}
-
-		f := nrsc.Get(file)
-		if f == nil {
-			return
-		}
-
-		rdr, err := f.Open()
-		if err != nil {
-			http.Error(res, "Internal Server Error", http.StatusInternalServerError)
-		}
-		defer rdr.Close()
-
-		mtype := mime.TypeByExtension(filepath.Ext(req.URL.Path))
-		if len(mtype) != 0 {
-			res.Header().Set("Content-Type", mtype)
-		}
-		res.Header().Set("Content-Size", fmt.Sprintf("%d", f.Size()))
-		res.Header().Set("Last-Modified", f.ModTime().UTC().Format(http.TimeFormat))
-
-		io.Copy(res, rdr)
-	}
-}

gui/app.js

@@ -1,40 +1,126 @@
+/*jslint browser: true, continue: true, plusplus: true */
+/*global $: false, angular: false */
+
+'use strict';
+
 var syncthing = angular.module('syncthing', []);
 
 syncthing.controller('SyncthingCtrl', function ($scope, $http) {
-    $http.get("/rest/version").success(function (data) {
+    var prevDate = 0,
+        modelGetOK = true;
+
+    $scope.connections = {};
+    $scope.config = {};
+    $scope.myID = '';
+    $scope.nodes = [];
+    $scope.configInSync = true;
+    $scope.errors = [];
+    $scope.seenError = '';
+
+    // Strings before bools look better
+    $scope.settings = [
+        {id: 'ListenStr', descr: 'Sync Protocol Listen Addresses', type: 'text', restart: true},
+        {id: 'GUIAddress', descr: 'GUI Listen Address', type: 'text', restart: true},
+        {id: 'MaxSendKbps', descr: 'Outgoing Rate Limit (KBps)', type: 'number', restart: true},
+        {id: 'RescanIntervalS', descr: 'Rescan Interval (s)', type: 'number', restart: true},
+        {id: 'ReconnectIntervalS', descr: 'Reconnect Interval (s)', type: 'number', restart: true},
+        {id: 'ParallelRequests', descr: 'Max Outstanding Requests', type: 'number', restart: true},
+        {id: 'MaxChangeKbps', descr: 'Max File Change Rate (KBps)', type: 'number', restart: true},
+
+        {id: 'ReadOnly', descr: 'Read Only', type: 'bool', restart: true},
+        {id: 'AllowDelete', descr: 'Allow Delete', type: 'bool', restart: true},
+        {id: 'FollowSymlinks', descr: 'Follow Symlinks', type: 'bool', restart: true},
+        {id: 'GlobalAnnEnabled', descr: 'Global Announce', type: 'bool', restart: true},
+        {id: 'LocalAnnEnabled', descr: 'Local Announce', type: 'bool', restart: true},
+    ];
+
+    function modelGetSucceeded() {
+        if (!modelGetOK) {
+            $('#networkError').modal('hide');
+            modelGetOK = true;
+        }
+    }
+
+    function modelGetFailed() {
+        if (modelGetOK) {
+            $('#networkError').modal({backdrop: 'static', keyboard: false});
+            modelGetOK = false;
+        }
+    }
+
+    function nodeCompare(a, b) {
+        if (a.NodeID === $scope.myID) {
+            return -1;
+        }
+        if (b.NodeID === $scope.myID) {
+            return 1;
+        }
+        if (a.NodeID < b.NodeID) {
+            return -1;
+        }
+        return a.NodeID > b.NodeID;
+    }
+
+    $http.get('/rest/version').success(function (data) {
         $scope.version = data;
     });
-    $http.get("/rest/config").success(function (data) {
-        $scope.config = data;
+    $http.get('/rest/system').success(function (data) {
+        $scope.system = data;
+        $scope.myID = data.myID;
+
+        $http.get('/rest/config').success(function (data) {
+            $scope.config = data;
+            $scope.config.Options.ListenStr = $scope.config.Options.ListenAddress.join(', ');
+
+            var nodes = $scope.config.Repositories[0].Nodes;
+            nodes.sort(nodeCompare);
+            $scope.nodes = nodes;
+        });
+        $http.get('/rest/config/sync').success(function (data) {
+            $scope.configInSync = data.configInSync;
+        });
     });
-    
-    var prevDate = 0;
 
     $scope.refresh = function () {
-        $http.get("/rest/model").success(function (data) {
-            $scope.model = data;
+        $http.get('/rest/system').success(function (data) {
+            $scope.system = data;
         });
-        $http.get("/rest/connections").success(function (data) {
-            var now = Date.now();
-            var td = (now - prevDate) / 1000;
-            prevDate = now;
+        $http.get('/rest/model').success(function (data) {
+            $scope.model = data;
+            modelGetSucceeded();
+        }).error(function () {
+            modelGetFailed();
+        });
+        $http.get('/rest/connections').success(function (data) {
+            var now = Date.now(),
+                td = (now - prevDate) / 1000,
+                id;
 
-            for (var id in data) {
+            prevDate = now;
+            $scope.inbps = 0;
+            $scope.outbps = 0;
+
+            for (id in data) {
+                if (!data.hasOwnProperty(id)) {
+                    continue;
+                }
                 try {
-                    data[id].inbps = 8 * (data[id].InBytesTotal - $scope.connections[id].InBytesTotal) / td;
-                    data[id].outbps = 8 * (data[id].OutBytesTotal - $scope.connections[id].OutBytesTotal) / td;
+                    data[id].inbps = Math.max(0, 8 * (data[id].InBytesTotal - $scope.connections[id].InBytesTotal) / td);
+                    data[id].outbps = Math.max(0, 8 * (data[id].OutBytesTotal - $scope.connections[id].OutBytesTotal) / td);
                 } catch (e) {
                     data[id].inbps = 0;
                     data[id].outbps = 0;
                 }
+                $scope.inbps += data[id].inbps;
+                $scope.outbps += data[id].outbps;
             }
             $scope.connections = data;
         });
-        $http.get("/rest/need").success(function (data) {
+        $http.get('/rest/need').success(function (data) {
             var i, name;
             for (i = 0; i < data.length; i++) {
-                name = data[i].Name.split("/");
-                data[i].ShortName = name[name.length-1];
+                name = data[i].Name.split('/');
+                data[i].ShortName = name[name.length - 1];
             }
             data.sort(function (a, b) {
                 if (a.ShortName < b.ShortName) {
@@ -47,75 +133,290 @@ syncthing.controller('SyncthingCtrl', function ($scope, $http) {
             });
             $scope.need = data;
         });
+        $http.get('/rest/errors').success(function (data) {
+            $scope.errors = data;
+        });
+    };
+
+    $scope.nodeStatus = function (nodeCfg) {
+        var conn = $scope.connections[nodeCfg.NodeID];
+        if (conn) {
+            if (conn.Completion === 100) {
+                return 'In Sync';
+            } else {
+                return 'Syncing (' + conn.Completion + '%)';
+            }
+        }
+
+        return 'Disconnected';
+    };
+
+    $scope.nodeIcon = function (nodeCfg) {
+        var conn = $scope.connections[nodeCfg.NodeID];
+        if (conn) {
+            if (conn.Completion === 100) {
+                return 'ok';
+            } else {
+                return 'refresh';
+            }
+        }
+
+        return 'minus';
+    };
+
+    $scope.nodeClass = function (nodeCfg) {
+        var conn = $scope.connections[nodeCfg.NodeID];
+        if (conn) {
+            if (conn.Completion === 100) {
+                return 'success';
+            } else {
+                return 'primary';
+            }
+        }
+
+        return 'info';
+    };
+
+    $scope.nodeAddr = function (nodeCfg) {
+        var conn = $scope.connections[nodeCfg.NodeID];
+        if (conn) {
+            return conn.Address;
+        }
+        return '(unknown address)';
+    };
+
+    $scope.nodeCompletion = function (nodeCfg) {
+        var conn = $scope.connections[nodeCfg.NodeID];
+        if (conn) {
+            return conn.Completion + '%';
+        }
+        return '';
+    };
+
+    $scope.nodeVer = function (nodeCfg) {
+        if (nodeCfg.NodeID === $scope.myID) {
+            return $scope.version;
+        }
+        var conn = $scope.connections[nodeCfg.NodeID];
+        if (conn) {
+            return conn.ClientVersion;
+        }
+        return '(unknown version)';
+    };
+
+    $scope.nodeName = function (nodeCfg) {
+        if (nodeCfg.Name) {
+            return nodeCfg.Name;
+        }
+        return nodeCfg.NodeID.substr(0, 6);
+    };
+
+    $scope.saveSettings = function () {
+        $scope.configInSync = false;
+        $scope.config.Options.ListenAddress = $scope.config.Options.ListenStr.split(',').map(function (x) { return x.trim(); });
+        $http.post('/rest/config', JSON.stringify($scope.config), {headers: {'Content-Type': 'application/json'}});
+        $('#settingsTable').collapse('hide');
+    };
+
+    $scope.restart = function () {
+        $http.post('/rest/restart');
+        $scope.configInSync = true;
+    };
+
+    $scope.editNode = function (nodeCfg) {
+        $scope.currentNode = nodeCfg;
+        $scope.editingExisting = true;
+        $scope.currentNode.AddressesStr = nodeCfg.Addresses.join(', ');
+        $('#editNode').modal({backdrop: 'static', keyboard: false});
+    };
+
+    $scope.addNode = function () {
+        $scope.currentNode = {NodeID: '', AddressesStr: 'dynamic'};
+        $scope.editingExisting = false;
+        $('#editNode').modal({backdrop: 'static', keyboard: false});
+    };
+
+    $scope.deleteNode = function () {
+        var newNodes = [], i;
+
+        $('#editNode').modal('hide');
+        if (!$scope.editingExisting) {
+            return;
+        }
+
+        for (i = 0; i < $scope.nodes.length; i++) {
+            if ($scope.nodes[i].NodeID !== $scope.currentNode.NodeID) {
+                newNodes.push($scope.nodes[i]);
+            }
+        }
+
+        $scope.nodes = newNodes;
+        $scope.config.Repositories[0].Nodes = newNodes;
+
+        $scope.configInSync = false;
+        $http.post('/rest/config', JSON.stringify($scope.config), {headers: {'Content-Type': 'application/json'}});
+    };
+
+    $scope.saveNode = function () {
+        var nodeCfg, done, i;
+
+        $scope.configInSync = false;
+        $('#editNode').modal('hide');
+        nodeCfg = $scope.currentNode;
+        nodeCfg.Addresses = nodeCfg.AddressesStr.split(',').map(function (x) { return x.trim(); });
+
+        done = false;
+        for (i = 0; i < $scope.nodes.length; i++) {
+            if ($scope.nodes[i].NodeID === nodeCfg.NodeID) {
+                $scope.nodes[i] = nodeCfg;
+                done = true;
+                break;
+            }
+        }
+
+        if (!done) {
+            $scope.nodes.push(nodeCfg);
+        }
+
+        $scope.nodes.sort(nodeCompare);
+        $scope.config.Repositories[0].Nodes = $scope.nodes;
+
+        $http.post('/rest/config', JSON.stringify($scope.config), {headers: {'Content-Type': 'application/json'}});
+    };
+
+    $scope.otherNodes = function () {
+        var nodes = [], i, n;
+
+        for (i = 0; i < $scope.nodes.length; i++) {
+            n = $scope.nodes[i];
+            if (n.NodeID !== $scope.myID) {
+                nodes.push(n);
+            }
+        }
+        return nodes;
+    };
+
+    $scope.thisNode = function () {
+        var i, n;
+
+        for (i = 0; i < $scope.nodes.length; i++) {
+            n = $scope.nodes[i];
+            if (n.NodeID === $scope.myID) {
+                return [n];
+            }
+        }
+    };
+
+    $scope.errorList = function () {
+        var errors = [];
+        for (var i = 0; i < $scope.errors.length; i++) {
+            var e = $scope.errors[i];
+            if (e.Time > $scope.seenError) {
+                errors.push(e);
+            }
+        }
+        return errors;
+    };
+    
+    $scope.clearErrors = function () {
+        $scope.seenError = $scope.errors[$scope.errors.length - 1].Time;
+    };
+
+    $scope.friendlyNodes = function (str) {
+        for (var i = 0; i < $scope.nodes.length; i++) {
+            var cfg = $scope.nodes[i];
+            str = str.replace(cfg.NodeID, $scope.nodeName(cfg));
+        }
+        return str;
     };
 
     $scope.refresh();
     setInterval($scope.refresh, 10000);
 });
 
-function decimals(num) {
-    if (num > 100) {
+function decimals(val, num) {
+    var digits, decs;
+
+    if (val === 0) {
         return 0;
     }
-    if (num > 10) {
-        return 1;
-    }
-    return 2;
+
+    digits = Math.floor(Math.log(Math.abs(val)) / Math.log(10));
+    decs = Math.max(0, num - digits);
+    return decs;
 }
 
-syncthing.filter('binary', function() {
-    return function(input) {
+syncthing.filter('natural', function () {
+    return function (input, valid) {
+        return input.toFixed(decimals(input, valid));
+    };
+});
+
+syncthing.filter('binary', function () {
+    return function (input) {
         if (input === undefined) {
-            return '- '
+            return '0 ';
         }
         if (input > 1024 * 1024 * 1024) {
             input /= 1024 * 1024 * 1024;
-            return input.toFixed(decimals(input)) + ' Gi';
+            return input.toFixed(decimals(input, 2)) + ' Gi';
         }
         if (input > 1024 * 1024) {
             input /= 1024 * 1024;
-            return input.toFixed(decimals(input)) + ' Mi';
+            return input.toFixed(decimals(input, 2)) + ' Mi';
         }
         if (input > 1024) {
             input /= 1024;
-            return input.toFixed(decimals(input)) + ' Ki';
+            return input.toFixed(decimals(input, 2)) + ' Ki';
         }
         return Math.round(input) + ' ';
-    }
+    };
 });
 
-syncthing.filter('metric', function() {
-    return function(input) {
+syncthing.filter('metric', function () {
+    return function (input) {
         if (input === undefined) {
-            return '- '
+            return '0 ';
         }
         if (input > 1000 * 1000 * 1000) {
             input /= 1000 * 1000 * 1000;
-            return input.toFixed(decimals(input)) + ' G';
+            return input.toFixed(decimals(input, 2)) + ' G';
         }
         if (input > 1000 * 1000) {
             input /= 1000 * 1000;
-            return input.toFixed(decimals(input)) + ' M';
+            return input.toFixed(decimals(input, 2)) + ' M';
         }
         if (input > 1000) {
             input /= 1000;
-            return input.toFixed(decimals(input)) + ' k';
+            return input.toFixed(decimals(input, 2)) + ' k';
         }
         return Math.round(input) + ' ';
-    }
+    };
 });
 
-syncthing.filter('short', function() {
-    return function(input) {
+syncthing.filter('short', function () {
+    return function (input) {
         return input.substr(0, 6);
-    }
+    };
 });
 
-syncthing.filter('alwaysNumber', function() {
-    return function(input) {
+syncthing.filter('alwaysNumber', function () {
+    return function (input) {
         if (input === undefined) {
             return 0;
         }
         return input;
-    }
+    };
+});
+
+syncthing.directive('optionEditor', function () {
+    return {
+        restrict: 'C',
+        replace: true,
+        transclude: true,
+        scope: {
+            setting: '=setting',
+        },
+        template: '<input type="text" ng-model="config.Options[setting.id]"></input>',
+    };
 });

gui/bootstrap/css/bootstrap-theme.css

@@ -1,397 +0,0 @@
-/*!
- * Bootstrap v3.0.3 (http://getbootstrap.com)
- * Copyright 2013 Twitter, Inc.
- * Licensed under http://www.apache.org/licenses/LICENSE-2.0
- */
-
-.btn-default,
-.btn-primary,
-.btn-success,
-.btn-info,
-.btn-warning,
-.btn-danger {
-  text-shadow: 0 -1px 0 rgba(0, 0, 0, 0.2);
-  -webkit-box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.15), 0 1px 1px rgba(0, 0, 0, 0.075);
-          box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.15), 0 1px 1px rgba(0, 0, 0, 0.075);
-}
-
-.btn-default:active,
-.btn-primary:active,
-.btn-success:active,
-.btn-info:active,
-.btn-warning:active,
-.btn-danger:active,
-.btn-default.active,
-.btn-primary.active,
-.btn-success.active,
-.btn-info.active,
-.btn-warning.active,
-.btn-danger.active {
-  -webkit-box-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);
-          box-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);
-}
-
-.btn:active,
-.btn.active {
-  background-image: none;
-}
-
-.btn-default {
-  text-shadow: 0 1px 0 #fff;
-  background-image: -webkit-linear-gradient(top, #ffffff 0%, #e0e0e0 100%);
-  background-image: linear-gradient(to bottom, #ffffff 0%, #e0e0e0 100%);
-  background-repeat: repeat-x;
-  border-color: #dbdbdb;
-  border-color: #ccc;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffffffff', endColorstr='#ffe0e0e0', GradientType=0);
-  filter: progid:DXImageTransform.Microsoft.gradient(enabled=false);
-}
-
-.btn-default:hover,
-.btn-default:focus {
-  background-color: #e0e0e0;
-  background-position: 0 -15px;
-}
-
-.btn-default:active,
-.btn-default.active {
-  background-color: #e0e0e0;
-  border-color: #dbdbdb;
-}
-
-.btn-primary {
-  background-image: -webkit-linear-gradient(top, #428bca 0%, #2d6ca2 100%);
-  background-image: linear-gradient(to bottom, #428bca 0%, #2d6ca2 100%);
-  background-repeat: repeat-x;
-  border-color: #2b669a;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff428bca', endColorstr='#ff2d6ca2', GradientType=0);
-  filter: progid:DXImageTransform.Microsoft.gradient(enabled=false);
-}
-
-.btn-primary:hover,
-.btn-primary:focus {
-  background-color: #2d6ca2;
-  background-position: 0 -15px;
-}
-
-.btn-primary:active,
-.btn-primary.active {
-  background-color: #2d6ca2;
-  border-color: #2b669a;
-}
-
-.btn-success {
-  background-image: -webkit-linear-gradient(top, #5cb85c 0%, #419641 100%);
-  background-image: linear-gradient(to bottom, #5cb85c 0%, #419641 100%);
-  background-repeat: repeat-x;
-  border-color: #3e8f3e;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff5cb85c', endColorstr='#ff419641', GradientType=0);
-  filter: progid:DXImageTransform.Microsoft.gradient(enabled=false);
-}
-
-.btn-success:hover,
-.btn-success:focus {
-  background-color: #419641;
-  background-position: 0 -15px;
-}
-
-.btn-success:active,
-.btn-success.active {
-  background-color: #419641;
-  border-color: #3e8f3e;
-}
-
-.btn-warning {
-  background-image: -webkit-linear-gradient(top, #f0ad4e 0%, #eb9316 100%);
-  background-image: linear-gradient(to bottom, #f0ad4e 0%, #eb9316 100%);
-  background-repeat: repeat-x;
-  border-color: #e38d13;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#fff0ad4e', endColorstr='#ffeb9316', GradientType=0);
-  filter: progid:DXImageTransform.Microsoft.gradient(enabled=false);
-}
-
-.btn-warning:hover,
-.btn-warning:focus {
-  background-color: #eb9316;
-  background-position: 0 -15px;
-}
-
-.btn-warning:active,
-.btn-warning.active {
-  background-color: #eb9316;
-  border-color: #e38d13;
-}
-
-.btn-danger {
-  background-image: -webkit-linear-gradient(top, #d9534f 0%, #c12e2a 100%);
-  background-image: linear-gradient(to bottom, #d9534f 0%, #c12e2a 100%);
-  background-repeat: repeat-x;
-  border-color: #b92c28;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffd9534f', endColorstr='#ffc12e2a', GradientType=0);
-  filter: progid:DXImageTransform.Microsoft.gradient(enabled=false);
-}
-
-.btn-danger:hover,
-.btn-danger:focus {
-  background-color: #c12e2a;
-  background-position: 0 -15px;
-}
-
-.btn-danger:active,
-.btn-danger.active {
-  background-color: #c12e2a;
-  border-color: #b92c28;
-}
-
-.btn-info {
-  background-image: -webkit-linear-gradient(top, #5bc0de 0%, #2aabd2 100%);
-  background-image: linear-gradient(to bottom, #5bc0de 0%, #2aabd2 100%);
-  background-repeat: repeat-x;
-  border-color: #28a4c9;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff5bc0de', endColorstr='#ff2aabd2', GradientType=0);
-  filter: progid:DXImageTransform.Microsoft.gradient(enabled=false);
-}
-
-.btn-info:hover,
-.btn-info:focus {
-  background-color: #2aabd2;
-  background-position: 0 -15px;
-}
-
-.btn-info:active,
-.btn-info.active {
-  background-color: #2aabd2;
-  border-color: #28a4c9;
-}
-
-.thumbnail,
-.img-thumbnail {
-  -webkit-box-shadow: 0 1px 2px rgba(0, 0, 0, 0.075);
-          box-shadow: 0 1px 2px rgba(0, 0, 0, 0.075);
-}
-
-.dropdown-menu > li > a:hover,
-.dropdown-menu > li > a:focus {
-  background-color: #e8e8e8;
-  background-image: -webkit-linear-gradient(top, #f5f5f5 0%, #e8e8e8 100%);
-  background-image: linear-gradient(to bottom, #f5f5f5 0%, #e8e8e8 100%);
-  background-repeat: repeat-x;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#fff5f5f5', endColorstr='#ffe8e8e8', GradientType=0);
-}
-
-.dropdown-menu > .active > a,
-.dropdown-menu > .active > a:hover,
-.dropdown-menu > .active > a:focus {
-  background-color: #357ebd;
-  background-image: -webkit-linear-gradient(top, #428bca 0%, #357ebd 100%);
-  background-image: linear-gradient(to bottom, #428bca 0%, #357ebd 100%);
-  background-repeat: repeat-x;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff428bca', endColorstr='#ff357ebd', GradientType=0);
-}
-
-.navbar-default {
-  background-image: -webkit-linear-gradient(top, #ffffff 0%, #f8f8f8 100%);
-  background-image: linear-gradient(to bottom, #ffffff 0%, #f8f8f8 100%);
-  background-repeat: repeat-x;
-  border-radius: 4px;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffffffff', endColorstr='#fff8f8f8', GradientType=0);
-  filter: progid:DXImageTransform.Microsoft.gradient(enabled=false);
-  -webkit-box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.15), 0 1px 5px rgba(0, 0, 0, 0.075);
-          box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.15), 0 1px 5px rgba(0, 0, 0, 0.075);
-}
-
-.navbar-default .navbar-nav > .active > a {
-  background-image: -webkit-linear-gradient(top, #ebebeb 0%, #f3f3f3 100%);
-  background-image: linear-gradient(to bottom, #ebebeb 0%, #f3f3f3 100%);
-  background-repeat: repeat-x;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffebebeb', endColorstr='#fff3f3f3', GradientType=0);
-  -webkit-box-shadow: inset 0 3px 9px rgba(0, 0, 0, 0.075);
-          box-shadow: inset 0 3px 9px rgba(0, 0, 0, 0.075);
-}
-
-.navbar-brand,
-.navbar-nav > li > a {
-  text-shadow: 0 1px 0 rgba(255, 255, 255, 0.25);
-}
-
-.navbar-inverse {
-  background-image: -webkit-linear-gradient(top, #3c3c3c 0%, #222222 100%);
-  background-image: linear-gradient(to bottom, #3c3c3c 0%, #222222 100%);
-  background-repeat: repeat-x;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff3c3c3c', endColorstr='#ff222222', GradientType=0);
-  filter: progid:DXImageTransform.Microsoft.gradient(enabled=false);
-}
-
-.navbar-inverse .navbar-nav > .active > a {
-  background-image: -webkit-linear-gradient(top, #222222 0%, #282828 100%);
-  background-image: linear-gradient(to bottom, #222222 0%, #282828 100%);
-  background-repeat: repeat-x;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff222222', endColorstr='#ff282828', GradientType=0);
-  -webkit-box-shadow: inset 0 3px 9px rgba(0, 0, 0, 0.25);
-          box-shadow: inset 0 3px 9px rgba(0, 0, 0, 0.25);
-}
-
-.navbar-inverse .navbar-brand,
-.navbar-inverse .navbar-nav > li > a {
-  text-shadow: 0 -1px 0 rgba(0, 0, 0, 0.25);
-}
-
-.navbar-static-top,
-.navbar-fixed-top,
-.navbar-fixed-bottom {
-  border-radius: 0;
-}
-
-.alert {
-  text-shadow: 0 1px 0 rgba(255, 255, 255, 0.2);
-  -webkit-box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.25), 0 1px 2px rgba(0, 0, 0, 0.05);
-          box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.25), 0 1px 2px rgba(0, 0, 0, 0.05);
-}
-
-.alert-success {
-  background-image: -webkit-linear-gradient(top, #dff0d8 0%, #c8e5bc 100%);
-  background-image: linear-gradient(to bottom, #dff0d8 0%, #c8e5bc 100%);
-  background-repeat: repeat-x;
-  border-color: #b2dba1;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffdff0d8', endColorstr='#ffc8e5bc', GradientType=0);
-}
-
-.alert-info {
-  background-image: -webkit-linear-gradient(top, #d9edf7 0%, #b9def0 100%);
-  background-image: linear-gradient(to bottom, #d9edf7 0%, #b9def0 100%);
-  background-repeat: repeat-x;
-  border-color: #9acfea;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffd9edf7', endColorstr='#ffb9def0', GradientType=0);
-}
-
-.alert-warning {
-  background-image: -webkit-linear-gradient(top, #fcf8e3 0%, #f8efc0 100%);
-  background-image: linear-gradient(to bottom, #fcf8e3 0%, #f8efc0 100%);
-  background-repeat: repeat-x;
-  border-color: #f5e79e;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#fffcf8e3', endColorstr='#fff8efc0', GradientType=0);
-}
-
-.alert-danger {
-  background-image: -webkit-linear-gradient(top, #f2dede 0%, #e7c3c3 100%);
-  background-image: linear-gradient(to bottom, #f2dede 0%, #e7c3c3 100%);
-  background-repeat: repeat-x;
-  border-color: #dca7a7;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#fff2dede', endColorstr='#ffe7c3c3', GradientType=0);
-}
-
-.progress {
-  background-image: -webkit-linear-gradient(top, #ebebeb 0%, #f5f5f5 100%);
-  background-image: linear-gradient(to bottom, #ebebeb 0%, #f5f5f5 100%);
-  background-repeat: repeat-x;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffebebeb', endColorstr='#fff5f5f5', GradientType=0);
-}
-
-.progress-bar {
-  background-image: -webkit-linear-gradient(top, #428bca 0%, #3071a9 100%);
-  background-image: linear-gradient(to bottom, #428bca 0%, #3071a9 100%);
-  background-repeat: repeat-x;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff428bca', endColorstr='#ff3071a9', GradientType=0);
-}
-
-.progress-bar-success {
-  background-image: -webkit-linear-gradient(top, #5cb85c 0%, #449d44 100%);
-  background-image: linear-gradient(to bottom, #5cb85c 0%, #449d44 100%);
-  background-repeat: repeat-x;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff5cb85c', endColorstr='#ff449d44', GradientType=0);
-}
-
-.progress-bar-info {
-  background-image: -webkit-linear-gradient(top, #5bc0de 0%, #31b0d5 100%);
-  background-image: linear-gradient(to bottom, #5bc0de 0%, #31b0d5 100%);
-  background-repeat: repeat-x;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff5bc0de', endColorstr='#ff31b0d5', GradientType=0);
-}
-
-.progress-bar-warning {
-  background-image: -webkit-linear-gradient(top, #f0ad4e 0%, #ec971f 100%);
-  background-image: linear-gradient(to bottom, #f0ad4e 0%, #ec971f 100%);
-  background-repeat: repeat-x;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#fff0ad4e', endColorstr='#ffec971f', GradientType=0);
-}
-
-.progress-bar-danger {
-  background-image: -webkit-linear-gradient(top, #d9534f 0%, #c9302c 100%);
-  background-image: linear-gradient(to bottom, #d9534f 0%, #c9302c 100%);
-  background-repeat: repeat-x;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffd9534f', endColorstr='#ffc9302c', GradientType=0);
-}
-
-.list-group {
-  border-radius: 4px;
-  -webkit-box-shadow: 0 1px 2px rgba(0, 0, 0, 0.075);
-          box-shadow: 0 1px 2px rgba(0, 0, 0, 0.075);
-}
-
-.list-group-item.active,
-.list-group-item.active:hover,
-.list-group-item.active:focus {
-  text-shadow: 0 -1px 0 #3071a9;
-  background-image: -webkit-linear-gradient(top, #428bca 0%, #3278b3 100%);
-  background-image: linear-gradient(to bottom, #428bca 0%, #3278b3 100%);
-  background-repeat: repeat-x;
-  border-color: #3278b3;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff428bca', endColorstr='#ff3278b3', GradientType=0);
-}
-
-.panel {
-  -webkit-box-shadow: 0 1px 2px rgba(0, 0, 0, 0.05);
-          box-shadow: 0 1px 2px rgba(0, 0, 0, 0.05);
-}
-
-.panel-default > .panel-heading {
-  background-image: -webkit-linear-gradient(top, #f5f5f5 0%, #e8e8e8 100%);
-  background-image: linear-gradient(to bottom, #f5f5f5 0%, #e8e8e8 100%);
-  background-repeat: repeat-x;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#fff5f5f5', endColorstr='#ffe8e8e8', GradientType=0);
-}
-
-.panel-primary > .panel-heading {
-  background-image: -webkit-linear-gradient(top, #428bca 0%, #357ebd 100%);
-  background-image: linear-gradient(to bottom, #428bca 0%, #357ebd 100%);
-  background-repeat: repeat-x;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff428bca', endColorstr='#ff357ebd', GradientType=0);
-}
-
-.panel-success > .panel-heading {
-  background-image: -webkit-linear-gradient(top, #dff0d8 0%, #d0e9c6 100%);
-  background-image: linear-gradient(to bottom, #dff0d8 0%, #d0e9c6 100%);
-  background-repeat: repeat-x;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffdff0d8', endColorstr='#ffd0e9c6', GradientType=0);
-}
-
-.panel-info > .panel-heading {
-  background-image: -webkit-linear-gradient(top, #d9edf7 0%, #c4e3f3 100%);
-  background-image: linear-gradient(to bottom, #d9edf7 0%, #c4e3f3 100%);
-  background-repeat: repeat-x;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffd9edf7', endColorstr='#ffc4e3f3', GradientType=0);
-}
-
-.panel-warning > .panel-heading {
-  background-image: -webkit-linear-gradient(top, #fcf8e3 0%, #faf2cc 100%);
-  background-image: linear-gradient(to bottom, #fcf8e3 0%, #faf2cc 100%);
-  background-repeat: repeat-x;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#fffcf8e3', endColorstr='#fffaf2cc', GradientType=0);
-}
-
-.panel-danger > .panel-heading {
-  background-image: -webkit-linear-gradient(top, #f2dede 0%, #ebcccc 100%);
-  background-image: linear-gradient(to bottom, #f2dede 0%, #ebcccc 100%);
-  background-repeat: repeat-x;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#fff2dede', endColorstr='#ffebcccc', GradientType=0);
-}
-
-.well {
-  background-image: -webkit-linear-gradient(top, #e8e8e8 0%, #f5f5f5 100%);
-  background-image: linear-gradient(to bottom, #e8e8e8 0%, #f5f5f5 100%);
-  background-repeat: repeat-x;
-  border-color: #dcdcdc;
-  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffe8e8e8', endColorstr='#fff5f5f5', GradientType=0);
-  -webkit-box-shadow: inset 0 1px 3px rgba(0, 0, 0, 0.05), 0 1px 0 rgba(255, 255, 255, 0.1);
-          box-shadow: inset 0 1px 3px rgba(0, 0, 0, 0.05), 0 1px 0 rgba(255, 255, 255, 0.1);
-}

gui/index.html

@@ -6,97 +6,270 @@
 <meta name="viewport" content="width=device-width, initial-scale=1.0">
 <meta name="description" content="">
 <meta name="author" content="">
-<link rel="shortcut icon" href="../../docs-assets/ico/favicon.png">
+<link rel="shortcut icon" href="favicon.png">
 
 <title>syncthing</title>
-<link href="bootstrap/css/bootstrap.css" rel="stylesheet">
+<link href="bootstrap/css/bootstrap.min.css" rel="stylesheet">
 <style type="text/css">
-body {
-    padding-top: 20px;
-    padding-bottom: 20px;
-}
 
-.header {
-    border-bottom: 1px solid #e5e5e5;
-    padding-bottom: 10px;
+body {
+    padding-top: 70px;
+    padding-bottom: 70px;
 }
 
 .text-monospace {
     font-family: monospace;
 }
+
+.table-condensed>thead>tr>th, .table-condensed>tbody>tr>th, .table-condensed>tfoot>tr>th, .table-condensed>thead>tr>td, .table-condensed>tbody>tr>td, .table-condensed>tfoot>tr>td {
+    border-top: none;
+}
+
+thead tr th {
+    text-align: center;
+}
+
+.logo {
+    margin: 0;
+    padding: 0;
+    top: -5px;
+    position: relative;
+}
+
 </style>
 </head>
 
 <body ng-controller="SyncthingCtrl">
-<div class="container">
-    <div class="header">
-        <h3 class="text-muted">syncthing&emsp;<small>|</small>&emsp;<small>{{version}}</small></h3>
-    </div>
-
-    <div class="row">
-        <div class="col-md-12">
-            <h2>Synchronization</h2>
-            <div class="progress">
-                <div class="progress-bar" role="progressbar" aria-valuenow="60" aria-valuemin="0" aria-valuemax="100"
-                    ng-class="{'progress-bar-success': model.needBytes === 0, 'progress-bar-info': model.needBytes !== 0}"
-                    style="width: {{100 * model.inSyncBytes / model.globalBytes | number:2}}%;">
-                        {{100 * model.inSyncBytes / model.globalBytes | number:0}}%
-                </div>
-            </div>
-            <p ng-show="model.needBytes > 0">Need {{model.needFiles | alwaysNumber}} files, {{model.needBytes | binary}}B</p>
+<div class="navbar navbar-fixed-top navbar-default">
+    <div class="container">
+        <a class="navbar-brand"><img class="logo" src="st-logo-128.png" width="32" height="32"> Syncthing</a>
+        <div ng-if="!configInSync">
+            <form class="navbar-form navbar-right">
+                <button type="button" class="btn btn-primary" ng-click="restart()">Restart Now</button>
+            </form>
+            <p class="navbar-text navbar-right">The configuration has been changed but not activated. Syncthing must restart to activate the new configuration.</p>
         </div>
     </div>
+</div>
 
+<div class="container">
     <div class="row">
-        <div class="col-md-6">
-            <h1>Repository Status</h1>
+        <div class="col-md-12">
+            <div ng-if="errorList().length > 0" class="alert alert-warning">
+                <p ng-repeat="err in errorList()"><small>{{err.Time | date:"hh:mm:ss.sss"}}:</small> {{friendlyNodes(err.Error)}}</p>
+                    <button type="button" class="pull-right btn btn-warning" ng-click="clearErrors()">OK</button>
+            <div class="clearfix"></div>
+            </div>
 
-            <p>Cluster contains {{model.globalFiles | alwaysNumber}} files, {{model.globalBytes | binary}}B
-            <span class="text-muted">(+{{model.globalDeleted | alwaysNumber}} delete records)</span></p>
-
-            <p>Local repository has {{model.localFiles | alwaysNumber}} files, {{model.localBytes | binary}}B
-            <span class="text-muted">(+{{model.localDeleted | alwaysNumber}} delete records)</span></p>
-
-            <div ng-show="model.needFiles > 0">
-                <h2>Files to Synchronize</h2>
-                <table class="table table-condensed table-striped">
-                    <tr ng-repeat="file in need track by $index">
-                        <td><abbr title="{{file.Name}}">{{file.ShortName}}</abbr></td>
-                        <td class="text-right">{{file.Size | binary}}B</td>
+            <div class="panel panel-info">
+                <div class="panel-heading"><h3 class="panel-title">Cluster</h3></div>
+                <table class="table table-condensed">
+                    <tbody>
+                    <!-- myself -->
+                    <tr class="text-muted" ng-repeat="nodeCfg in thisNode()">
+                        <td style="width:12%">
+                            <span class="label label-default">
+                                <span class="glyphicon glyphicon-ok"></span> This node
+                            </span>
+                        </td>
+                        <td style="width:10%">
+                            <span class="text-monospace">{{nodeName(nodeCfg)}}</span>
+                        </td>
+                        <td style="width:20%">{{version}}</td>
+                        <td style="width:25%">(this node)</td>
+                        <td style="width:9%" class="text-right">
+                            {{inbps | metric}}bps
+                            <span class="text-muted glyphicon glyphicon-chevron-down"></span>
+                        </td>
+                        <td style="width:9%" class="text-right">
+                            {{outbps | metric}}bps
+                            <span class="text-muted glyphicon glyphicon-chevron-up"></span>
+                        </td>
+                        <td style="width:7%" class="text-right">
+                            <button type="button" ng-click="editNode(nodeCfg)" class="btn btn-default btn-xs"><span class="glyphicon glyphicon-pencil"></span> Edit</button>
+                        </td>
                     </tr>
+                    <!-- all other nodes -->
+                    <tr ng-repeat="nodeCfg in otherNodes()">
+                        <td>
+                            <span class="label label-{{nodeClass(nodeCfg)}}">
+                                <span class="glyphicon glyphicon-{{nodeIcon(nodeCfg)}}"></span> {{nodeStatus(nodeCfg)}}
+                            </span>
+                        </td>
+                        <td>
+                            <span class="text-monospace">{{nodeName(nodeCfg)}}</span>
+                        </td>
+                        <td>{{nodeVer(nodeCfg)}}</td>
+                        <td>{{nodeAddr(nodeCfg)}}</td>
+                        <td class="text-right">
+                            <abbr title="{{connections[nodeCfg.NodeID].InBytesTotal | binary}}B">{{connections[nodeCfg.NodeID].inbps | metric}}bps</abbr>
+                            <span class="text-muted glyphicon glyphicon-chevron-down"></span>
+                        </td>
+                        <td class="text-right">
+                            <abbr title="{{connections[nodeCfg.NodeID].OutBytesTotal | binary}}B">{{connections[nodeCfg.NodeID].outbps | metric}}bps</abbr>
+                            <span class="text-muted glyphicon glyphicon-chevron-up"></span>
+                        </td>
+                        <td class="text-right">
+                            <button type="button" ng-click="editNode(nodeCfg)" class="btn btn-default btn-xs"><span class="glyphicon glyphicon-pencil"></span> Edit</button>
+                        </td>
+                    </tr>
+                    <tr>
+                        <td></td>
+                        <td></td>
+                        <td></td>
+                        <td></td>
+                        <td></td>
+                        <td></td>
+                        <td class="text-right">
+                            <button type="button" class="btn btn-default btn-xs" ng-click="addNode()"><span class="glyphicon glyphicon-plus"></span> Add</button>
+                        </td>
+                    </tr>
+                    </tbody>
                 </table>
             </div>
         </div>
+    </div>
+
+    <div class="row">
         <div class="col-md-6">
-            <h1>Cluster Status</h1>
-            <table class="table table-condensed">
-                <tbody>
-                <tr ng-repeat="(node, address) in config.nodes" ng-class="{'text-primary': !!connections[node]}">
-                    <td><abbr class="text-monospace" title="{{node}}">{{node | short}}</abbr></td>
-                    <td>
-                        <span ng-show="!!connections[node]">
-                            <span class="glyphicon glyphicon-link"></span>
-                            {{connections[node].Address}}
-                        </span>
-                        <span ng-hide="!!connections[node]">
-                            <span class="glyphicon glyphicon-cog"></span>
-                            {{address}}
-                        </span>
-                    </td>
-                    <td class="text-right">
-                        <abbr title="{{connections[node].InBytesTotal | binary}}B">{{connections[node].inbps | metric}}b/s</abbr>
-                        <span class="text-muted glyphicon glyphicon-cloud-download"></span>
-                    </td>
-                    <td class="text-right">
-                        <abbr title="{{connections[node].OutBytesTotal | binary}}B">{{connections[node].outbps | metric}}b/s</abbr>
-                        <span class="text-muted glyphicon glyphicon-cloud-upload"></span>
-                    </td>
-                </tr>
-                </tbody>
-            </table>
+            <div class="panel panel-info">
+                <div class="panel-heading"><h3 class="panel-title">Repository</h3></div>
+                <div class="panel-body">
+                    <p>Cluster contains {{model.globalFiles | alwaysNumber}} files, {{model.globalBytes | binary}}B
+                    <span class="text-muted">(+{{model.globalDeleted | alwaysNumber}} delete records)</span></p>
+
+                    <p>Local repository has {{model.localFiles | alwaysNumber}} files, {{model.localBytes | binary}}B
+                    <span class="text-muted">(+{{model.localDeleted | alwaysNumber}} delete records)</span></p>
+                </div>
+            </div>
+        </div>
+        <div class="col-md-6">
+            <div class="panel" ng-class="{'panel-success': model.needBytes === 0, 'panel-primary': model.needBytes !== 0}">
+                <div class="panel-heading"><h3 class="panel-title">Synchronization</h3></div>
+                <div class="panel-body">
+                    <div class="progress">
+                        <div class="progress-bar" role="progressbar" aria-valuenow="60" aria-valuemin="0" aria-valuemax="100"
+                            ng-class="{'progress-bar-success': model.needBytes === 0, 'progress-bar-info': model.needBytes !== 0}"
+                            style="width: {{100 * model.inSyncBytes / model.globalBytes | number:2}}%;">
+                            {{100 * model.inSyncBytes / model.globalBytes | alwaysNumber | number:0}}%
+                        </div>
+                    </div>
+                    <p ng-show="model.needBytes > 0">Need {{model.needFiles | alwaysNumber}} files, {{model.needBytes | binary}}B</p>
+                </div>
+            </div>
         </div>
     </div>
 
+    <div class="row">
+        <div class="col-md-6">
+            <div class="panel panel-info">
+                <div class="panel-heading"><h3 class="panel-title"><a href="" data-toggle="collapse" data-target="#system">System</a></h3></div>
+                <div id="system" class="panel-collapse collapse">
+                    <div class="panel-body">
+                        <p>{{system.sys | binary}}B RAM allocated, {{system.alloc | binary}}B in use</p>
+                        <p>{{system.cpuPercent | alwaysNumber | natural:1}}% CPU, {{system.goroutines | alwaysNumber}} goroutines</p>
+                    </div>
+                </div>
+            </div>
+        </div>
+        <div class="col-md-6">
+            <div class="panel panel-info">
+                <div class="panel-heading"><h3 class="panel-title"><a href="" data-toggle="collapse" data-target="#settingsTable">Settings</a></h3></div>
+                <div id="settingsTable" class="panel-collapse collapse">
+                    <div class="panel-body">
+                        <form role="form">
+                            <div class="form-group" ng-repeat="setting in settings">
+                                <div ng-if="setting.type == 'text' || setting.type == 'number'">
+                                    <label for="{{setting.id}}">{{setting.descr}}</label>
+                                    <input id="{{setting.id}}" class="form-control" type="{{setting.type}}" ng-model="config.Options[setting.id]"></input>
+                                </div>
+                                <div class="checkbox" ng-if="setting.type == 'bool'">
+                                    <label>
+                                        {{setting.descr}} <input id="{{setting.id}}" type="checkbox" ng-model="config.Options[setting.id]"></input>
+                                    </label>
+                                </div>
+                            </div>
+                        </form>
+                    </div>
+                    <div class="panel-footer">
+                        <button type="button" class="btn btn-sm btn-default" ng-click="saveSettings()">Save</button>
+                        <small><span class="text-muted">Changes take effect when restarting syncthing.</span></small>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+</div>
+
+<div class="navbar navbar-default navbar-fixed-bottom">
+    <div class="container">
+        <p class="navbar-text">{{version}}</p>
+        <ul class="nav navbar-nav navbar-right">
+            <li><a class="navbar-link" href="https://github.com/calmh/syncthing/releases">Latest Release</a></li>
+            <li><a class="navbar-link" href="https://github.com/calmh/syncthing/wiki">Documentation</a></li>
+            <li><a class="navbar-link" href="https://github.com/calmh/syncthing/issues">Bugs</a></li>
+            <li><a class="navbar-link" href="https://github.com/calmh/syncthing">Source Code</a></li>
+        </ul>
+        </p>
+    </div>
+</div>
+
+<div id="networkError" class="modal fade">
+    <div class="modal-dialog">
+        <div class="modal-content">
+            <div class="modal-header alert alert-danger">
+                <h4 class="modal-title">
+                    <span class="glyphicon glyphicon-exclamation-sign"></span>
+                    Connection Error
+                </h4>
+            </div>
+            <div class="modal-body">
+                <p>
+                Syncthing seems to be down, or there is a problem with your Internet connection.
+                Retrying&hellip;
+                </p>
+            </div>
+        </div>
+    </div>
+</div>
+
+<div id="editNode" class="modal fade">
+    <div class="modal-dialog modal-lg">
+        <div class="modal-content">
+            <div class="modal-header">
+                <button type="button" class="close" data-dismiss="modal" aria-hidden="true">&times;</button>
+                <h4 class="modal-title">Edit Node</h4>
+            </div>
+            <div class="modal-body">
+                <form role="form">
+                    <div class="form-group">
+                        <label for="nodeID">Node ID</label>
+                        <input placeholder="YUFJOUDPORCMA..." ng-disabled="editingExisting" id="nodeID" class="form-control" type="text" ng-model="currentNode.NodeID"></input>
+                        <p class="help-block">The node ID can be found in the logs or in the "Add Node" dialog on the other node.</p>
+                    </div>
+                    <div class="form-group">
+                        <label for="name">Name</label>
+                        <input placeholder="Home Server" id="name" class="form-control" type="text" ng-model="currentNode.Name"></input>
+                        <p class="help-block">Shown instead of Node ID in the cluster status.</p>
+                    </div>
+                    <div class="form-group">
+                        <label for="addresses">Addresses</label>
+                        <input placeholder="dynamic" ng-disabled="currentNode.NodeID == myID" id="addresses" class="form-control" type="text" ng-model="currentNode.AddressesStr"></input>
+                        <p class="help-block">Enter comma separated <span class="text-monospace">ip:port</span> addresses or <span class="text-monospace">dynamic</span> to perform automatic discovery of the address.</p>
+                    </div>
+                </form>
+                <div ng-show="!editingExisting">
+                    When adding a new node, keep in mind that <em>this node</em> must be added on the other side too. The Node ID of this node is:
+                    <pre>{{myID}}</pre>
+                </div>
+            </div>
+            <div class="modal-footer">
+                <button type="button" class="btn btn-primary" ng-click="saveNode()">Save</button>
+                <button type="button" class="btn btn-default" data-dismiss="modal">Close</button>
+                <button ng-if="editingExisting" type="button" class="btn btn-danger pull-left" ng-click="deleteNode()">Delete</button>
+            </div>
+        </div>
+    </div>
 </div>
 
 <script src="angular.min.js"></script>