Compare commits

...

1087 Commits

Author SHA1 Message Date
Jakob Borg
ee36e2d46d lib/weakhash: Limit number of hits for any given weakhash
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3925
2017-01-24 08:26:45 +00:00
Jakob Borg
de49ea594a lib/upnp: Wrong order of internal/external port after OnlyPermanentLeasesSupported (fixes #3924) 2017-01-23 23:17:02 +01:00
Jakob Borg
6d4fa27ea7 cmd/syncthing: Report real hashing performance, including weakhash
Instead of

    [I6KAH] 19:05:56 INFO: Single thread hash performance is 359 MB/s using minio/sha256-simd (354 MB/s using crypto/sha256).

it now says

    [I6KAH] 19:06:16 INFO: Single thread SHA256 performance is 359 MB/s using minio/sha256-simd (354 MB/s using crypto/sha256).
    [I6KAH] 19:06:17 INFO: Actual hashing performance is 299.01 MB/s

which is more informative. This is also the number it reports in usage
reporting.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3918
2017-01-23 21:56:43 +00:00
Jakob Borg
9587b89d9d gui, man: Update docs & translations 2017-01-23 22:03:08 +01:00
Jakob Borg
79c7f7193b lib/upnp: Remove unnecessary error allocation
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3923
2017-01-23 21:02:55 +00:00
Audrius Butkevicius
2c4b92d410 lib/connections: Fix rate limiting on arm64 (fixes #3921)
Skip-check: metalint

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3922
2017-01-23 20:55:00 +00:00
Audrius Butkevicius
dd78177ae0 scanner: Allow disabling weak hash in scanning (fixes #3891)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3905
2017-01-23 13:50:32 +00:00
Jakob Borg
bd55ec79d2 goals, readme: Add updated project goals 2017-01-19 18:12:40 +01:00
Jakob Borg
1313ba8c0a gui, vendor: Update license, copyright for github.com/chmduquesne/rollinghash 2017-01-19 15:59:39 +01:00
Jakob Borg
842e873a94 cmd/syncthing: Fix -logfile/-no-restart test on non-Windows. 2017-01-18 18:59:48 +01:00
Jakob Borg
d4c4b1fb4c vendor: Temporarily patch github.com/chmduquesne/rollinghash
To avoid allocations in the hasher. PR files, should be available for
update soon.
2017-01-18 18:45:29 +01:00
Jakob Borg
68f1c6ccab lib/scanner: Avoid per iteration allocations in Blocks()
Resetting the io.LimitReader is better than creating a new one on every
iteration.
2017-01-18 18:43:00 +01:00
Antony Male
4c8aa14e07 cmd/syncthing: Fail if -logfile and -no-restart passed together (fixes #3912)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3915
2017-01-18 12:19:22 +00:00
Jakob Borg
bd1c29ee32 lib/scanner, vendor: Fix previous commit
Can't do what I did, as the rolling function is not the same as the
non-rolling one. Instead this uses an improved version of the rolling
adler32 to accomplish the same thing. (PR filed on upstream, so should
be able to use that directly in the future.)
2017-01-18 11:57:01 +01:00
Jakob Borg
9b1c592fb7 lib/scanner: Speed up weak hash
The rolling version of adler32 is just a wrapper around the standard
hash/adler32 when used in a non-rolling fashion, but it's inefficient as
it allocates a new hash instance for every Write(). This uses the
default version instead in the block hasher, and adds a test to verify
the result is the same as they were before. It reduces allocations by
88% and increases speed about 5%.

	benchmark               old ns/op     new ns/op     delta
	BenchmarkHashFile-8     64434698      61303647      -4.86%

	benchmark               old MB/s     new MB/s     speedup
	BenchmarkHashFile-8     276.65       290.78       1.05x

	benchmark               old allocs     new allocs     delta
	BenchmarkHashFile-8     1238           150            -87.88%

	benchmark               old bytes     new bytes     delta
	BenchmarkHashFile-8     17877363      49292         -99.72%
2017-01-18 10:33:17 +01:00
Jakob Borg
f36f00e87b gui: Update translation base (lang-en) (fixes #3909) 2017-01-17 15:58:35 +01:00
Simon Frei
dbb3a34887 lib/ignore: Centralize handling of temporary filenames (fixes #3899)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3901
LGTM: calmh, AudriusButkevicius
2017-01-17 07:33:48 +00:00
KAMADA Ken'ichi
929a4d0c0c gui: Improve warnings when creating folder in a subdirectory (fixes #3197, fixes #3902)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3904
2017-01-14 12:18:48 +00:00
ProactiveServices
c953cdc375 gui: Package attribution and copyright bumps (fixes #3861)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3863
2017-01-10 07:50:11 +00:00
Jakob Borg
c20c17e3c6 gui, man: Update docs & translations 2017-01-10 08:41:14 +01:00
Audrius Butkevicius
1a1e35d998 lib/osutil: Replace IsDir with TraversesSymlink (fixes #3839)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3883
LGTM: calmh
2017-01-10 07:09:31 +00:00
Adel Qalieh
8d2a31e38e lib/model: Remove syncthing-specific files (fixes #3819)
Syncthing adds some hidden files when a folder is added, but there is currently
no equivalent cleanup procedure. This change is conservative as not to
accidentally cause data loss.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3874
2017-01-07 17:05:30 +00:00
Jakob Borg
fe377a166a authors: Update adelq 2017-01-07 17:56:46 +01:00
Jakob Borg
5770d80bf9 authors: Add adelq 2017-01-07 17:54:41 +01:00
Jakob Borg
7a16dbd31d lib/scanner: Fix previous commit: don't stop scan completely 2017-01-05 15:05:49 +01:00
Jakob Borg
926c88cfc4 lib/scanner: Never, ever descend into symlinks (ref #3857)
On Windows we would descend into SYMLINKD type links when we scanned
them successfully, as we would return nil from the walk function and the
filepath.Walk iterator apparently thought it OK to descend into the
symlinked directory.

With this change we always return filepath.SkipDir no matter what.

Tested on Windows 10 as admin, does what it should.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3875
2017-01-05 13:26:29 +00:00
Audrius Butkevicius
29d010ec0e lib/model, lib/weakhash: Hash using adler32, add heuristic in puller
Adler32 is much faster, and the heuristic avoid the obvious cases where it
will not help.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3872
2017-01-04 21:04:13 +00:00
Jakob Borg
920274bce4 lib/db: Don't panic on unknown folder in ListFolders (fixes #3584)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3869
2017-01-04 10:34:52 +00:00
Jakob Borg
2ebd6ad77f lib/scanner: Don't stop byte counter ticks before scan is done 2017-01-03 15:03:32 +01:00
Kudalufi
79dd6918f2 cmd/syncthing: Add support for -auditfile= (fixes #3859)
Adds support for -auditfile= where is "-" for stdout, "--" for stderr, or a
filename. It can be left blank (or left out entirely) for the original
behaviour of creating a timestamped filename.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3860
2017-01-03 08:54:28 +00:00
Jakob Borg
79f7f50c4d authors: Add Kudalufi 2017-01-03 09:24:29 +01:00
Jakob Borg
987718baf8 vendor: Update github.com/gogo/protobuf
Also tweaks the proto definitions:

 - [packed=false] on the block_indexes field to retain compat with
   v0.14.16 and earlier.

 - Uses the vendored protobuf package in include paths.

And, "build.go setup" will install the vendored protoc-gen-gogofast.
This should ensure that a proto rebuild isn't so dependent on whatever
version of the compiler and package the developer has installed...

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3864
2017-01-03 00:16:21 +00:00
Jakob Borg
4fb9c143ac authors: Update ProactiveServices 2017-01-02 15:12:57 +01:00
Jakob Borg
ec62888539 lib/connections: Allow on the fly changes to rate limits (fixes #3846)
Also replaces github.com/juju/ratelimit with golang.org/x/time/rate as
the latter supports changing the rate on the fly.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3862
2017-01-02 11:29:20 +00:00
Jakob Borg
8c34a76f7a man: refresh.sh requires bash 2017-01-01 20:45:52 +01:00
Jakob Borg
6809d38cde lib/protocol: Revert protobuf encoder changes in v0.14.17 (fixes #3855)
The protobuf encoder now produces packed arrays for things like []int32,
which is actually correct according to the proto3 spec. However
Syncthing v0.14.16 and earlier doesn't support this. This reverts the
encoding change, but keeps the updated decoder so that we are both more
compatible with other proto3 implementations and can move to the updated
encoder in the future.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3856
2017-01-01 17:19:00 +00:00
Mark Pulford
69ae4aa024 cmd/syncthing: Avoid Keepalive/GUI refresh race
This avoids unnecessary browser request failures and retries. Eg:
- Browser reuses existing HTTP connection for GUI refresh request
- Server closes connection with request in flight
- Browser retries GET request.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3854
2017-01-01 12:38:31 +00:00
Jakob Borg
8e8b867fba authors: Add mpx 2017-01-01 13:28:33 +01:00
Jakob Borg
0a118d2979 lib/config, lib/model: Temporarily disable bad tests (ref #3834, #3843) 2017-01-01 13:27:18 +01:00
Nathan Morrison
8daaa5d0d2 gui: Populate global changes on load
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3848
2016-12-30 01:33:27 +00:00
Jakob Borg
eb14f85a57 vendor: Update github.com/syndtr/goleveldb 2016-12-28 12:19:14 +01:00
Jakob Borg
c69c3c7c36 lib/sha256: Smoke test the implementation on startup (hello OpenSUSE!) 2016-12-28 12:15:51 +01:00
Jakob Borg
54911d44c5 gui: s/foldersendonly.html/foldertypes.html 2016-12-27 11:29:12 +01:00
Jakob Borg
c765f7be8d gui, man: Update docs and translations 2016-12-26 14:23:55 +01:00
Jakob Borg
44bdaf3ac2 cmd/syncthing: Add -reset-deltas option to reset delta index IDs
Also rename and clarify the description of -reset-database (formerly
-reset).
2016-12-26 13:49:51 +01:00
Jakob Borg
fc16e49cb0 Merge branch 'v0.14.16-hotfix'
* v0.14.16-hotfix:
  gui, man: Update docs & translations
  lib/model: Allow empty subdirs in scan request (fixes #3829)
  lib/model: Don't send symlinks to old devices that can't handle them (fixes #3802)
  lib/model: Accept scan requests of paths ending in slash (fixes #3804)
  gui: Avoid pause between event polls (ref #3527)
2016-12-24 20:12:53 +01:00
Jakob Borg
f5a310ad64 Revert "lib/model: Handle filename conflicts on Windows."
This reverts commit 01e50eb3fa.
2016-12-23 11:10:58 +01:00
Unrud
01e50eb3fa lib/model: Handle filename conflicts on Windows.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3810
LGTM: calmh
2016-12-22 23:04:53 +00:00
Jakob Borg
722b81c6f0 gui, man: Update docs & translations 2016-12-21 19:46:28 +01:00
Jakob Borg
f0efa2b974 lib/model: Allow empty subdirs in scan request (fixes #3829)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3833
2016-12-21 19:45:38 +01:00
Audrius Butkevicius
bab7c8ebbf all: Add folder pause, make pauses permanent (fixes #3407, fixes #215, fixes #3001)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3520
2016-12-21 18:41:25 +00:00
Nathan Morrison
0725e3af38 all: Add a global change list
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3694
2016-12-21 16:35:20 +00:00
Jakob Borg
dd7bb6c4b8 lib/model: Fix tests, clean up pool usage in protocol 2016-12-21 14:53:45 +01:00
Jakob Borg
d41c131364 build: Enable gometalinter "gosimple" check, improve build.go 2016-12-21 14:53:45 +01:00
Jakob Borg
47f22ff3e5 build: Enable gometalinter "unconvert" check 2016-12-21 14:53:45 +01:00
Jakob Borg
744c2e82b5 build: Enable gometalinter "staticcheck" check 2016-12-21 14:53:45 +01:00
Jakob Borg
ead7281c20 build: Enable gometalinter "unused" check 2016-12-21 14:53:45 +01:00
AudriusButkevicius
aa3ef49dd7 lib/model: Fix lock order
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3823
2016-12-21 12:22:18 +00:00
Jakob Borg
5c067661f4 lib/model: Consistently show folder description in startup messages
Since we anyway need the folderConfig for this I'm skipping the copying
of all it's attributes that rwfolder did and just keeping the original
around instead.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3825
2016-12-21 11:23:20 +00:00
Jakob Borg
226da976dc lib/model: Allow empty subdirs in scan request (fixes #3829)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3833
2016-12-21 10:33:07 +00:00
AudriusButkevicius
ba17cc0a11 gui: Show introducedBy (fixes #3809) 2016-12-21 11:01:15 +01:00
AudriusButkevicius
9e0afb7d8a lib/connections: Support setting traffic class (fixes #3790) 2016-12-21 11:01:15 +01:00
AudriusButkevicius
9e7d50bc76 cmd/syncthing: Explain corruption panics (fixes #3689) 2016-12-21 11:01:15 +01:00
Jakob Borg
d7d5687faa lib/protocol: Warnln should have been Warnf 2016-12-21 10:43:12 +01:00
Jakob Borg
21eb098dd2 vendor: Update github.com/minio/sha256-simd, CPU detection (Linux) 2016-12-20 09:20:28 +01:00
Jakob Borg
2f770f8bfb lib/config, lib/protocol: Improve folder description with empty label 2016-12-19 10:12:06 +01:00
Jakob Borg
f09698d845 gui: Add missing strings to lang-en.json 2016-12-18 22:29:43 +01:00
Jakob Borg
3fbcb024a7 gui: Update link to send only documentation page 2016-12-18 22:29:43 +01:00
Jakob Borg
b8c1c0e048 cmd/syncthing: Enable better crypto, print negotiated cipher suite
This adds support for AES_256_GCM_SHA384 (in there since Go 1.5, a bit
of a shame we missed it) and ChaCha20-Poly1305 (if built with Go 1.8;
ignored on older Gos).

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3822
2016-12-18 21:07:44 +00:00
Jakob Borg
2d47242d54 jenkins: Don't clean out when building with old Go (nukes coverage data) 2016-12-18 19:58:46 +01:00
Jakob Borg
66a7829eee jenkins: Also try build with old Go version, if available 2016-12-18 19:25:27 +01:00
Jakob Borg
9c67bd2550 lib/connections: Fix port fixup in Go 1.8 (fixes #3817)
The test for the error string is fragile, and the error string changed
in Go 1.8 so the relevant part is no longer a prefix. This covers it
with a test though, so it should be fine in the future as well.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3818
2016-12-18 11:28:18 +00:00
Jakob Borg
f67c5a2fd6 lib/model: Don't send symlinks to old devices that can't handle them (fixes #3802)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3811
2016-12-17 19:48:33 +00:00
Jakob Borg
0437f6dd66 lib/model: Don't send symlinks to old devices that can't handle them (fixes #3802) 2016-12-17 12:39:22 +01:00
Jakob Borg
11b35d650d lib/model: Accept scan requests of paths ending in slash (fixes #3804)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3805
2016-12-17 12:39:22 +01:00
Jakob Borg
b279e261a1 gui: Avoid pause between event polls (ref #3527)
We lose important events due to the frequency of ItemStarted /
ItemFinished events.
2016-12-17 12:39:22 +01:00
Jakob Borg
263402f80a cmd/stcli: Add copyright headers to satisfy check 2016-12-17 12:28:59 +01:00
Jakob Borg
920a83ec7a cmd/stcli: Fix metalint ineffasign complaint 2016-12-17 10:51:48 +01:00
Jakob Borg
3c2ac3522c cmd/stcli: Update for new folder type 2016-12-17 01:44:18 +01:00
Jakob Borg
9fdaa637a8 vendor: Add github.com/AudriusButkevicius/cli 2016-12-17 01:33:17 +01:00
Jakob Borg
81a9d7f2b9 cmd/stcli: Make it build again 2016-12-17 01:33:17 +01:00
Audrius Butkevičius
d8d3f05164 cmd/stcli: Import from syncthing-cli repository 2016-12-17 01:33:17 +01:00
Jakob Borg
653be136ee gui: Update lang-en.json 2016-12-17 00:59:25 +01:00
Heiko Zuerker
398c356f22 lib/model: Clarify master terminology (fixes #2679)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3793
2016-12-16 22:23:35 +00:00
Audrius Butkevicius
542b76f687 lib/model: Moar sleep
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3807
2016-12-16 12:05:27 +00:00
Jakob Borg
abb8a1914a lib/model: Accept scan requests of paths ending in slash (fixes #3804)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3805
2016-12-16 11:21:22 +00:00
Jakob Borg
163d335078 gui: Update lang-en.json 2016-12-16 09:42:13 +01:00
Audrius Butkevicius
0582836820 lib/model, lib/scanner: Efficient inserts/deletes in the middle of the file
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3527
2016-12-14 23:30:29 +00:00
Jakob Borg
bb15776ae6 gui: Avoid pause between event polls (ref #3527)
We lose important events due to the frequency of ItemStarted /
ItemFinished events.
2016-12-14 10:31:16 +01:00
Jakob Borg
dde9d4c9eb lib/rc: Remove pause to aggregate events (ref #3527) 2016-12-14 10:24:36 +01:00
Jakob Borg
1ef75be1c6 gui, man: Update docs & translations 2016-12-13 11:29:40 +01:00
Jakob Borg
3582783972 lib/model, lib/osutil: Verify target directory before pulling / requesting
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3798
2016-12-13 10:24:10 +00:00
Jakob Borg
5070d52f2f lib/model: Add benchmark for model.Request() 2016-12-09 23:14:56 +01:00
Jakob Borg
7b07ed6580 lib/model, lib/protocol, lib/scanner: Include symlink target in index, pull symlinks synchronously
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3792
2016-12-09 18:02:18 +00:00
Han Boetes
f6a2b6252a gui: Tweak wording (fixes #3769)
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3796
2016-12-09 17:16:29 +00:00
Jakob Borg
a9b03de99a gui, lib/db: Correct space accounting of symlinks, for "out of sync" status 2016-12-09 10:38:36 +01:00
Jakob Borg
a7f7058636 vendor: Update github.com/gobwas/glob (bugfix) 2016-12-07 09:25:58 +01:00
Lars K.W. Gohlke
8ce9b026e9 lib/model: Minor cleanups
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3765
2016-12-06 08:54:04 +00:00
Audrius Butkevicius
0dcf2f1bc8 cmd/strelaysrv: Use legacy dial (fixes #3753)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3784
2016-12-02 22:45:08 +00:00
Audrius Butkevicius
99922feb3b gui: Disable device removal when we know it will be reintroduced
Skip-check: pr-build-windows

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3762
2016-12-02 21:07:02 +00:00
Jakob Borg
2fd1dca905 lib/connections: Fix odd logging, forgot to call function 2016-12-02 12:56:14 +01:00
Jakob Borg
e3cf718998 lib/ignore: Add central check for internal files, used in scanning, pulling and requests
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3779
2016-12-01 14:00:11 +00:00
Stefan Kuntz
7157917a16 etc: Updated ufw firewall application preset with default GUI port
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3774
2016-12-01 12:36:15 +00:00
Jakob Borg
3266aae1c3 lib/protocol: Apply input filtering on file names
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3775
2016-12-01 12:35:32 +00:00
Jakob Borg
63194a37f6 lib/model: Double check results in filepath.Join where needed
Wherever we have untrusted relative paths, make sure they are not
escaping their folder root.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3776
2016-12-01 12:35:11 +00:00
Unrud
cabe94552a lib/model: Prevent collisions in the progressemitter registry
Using filepath.Join can cause collisions. The folder ID could be something
like ".." or "../..".

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3778
2016-12-01 12:34:20 +00:00
Jakob Borg
48a229a0cd lib/model: Temp names from all platforms should be recognized as such
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3777
LGTM: AudriusButkevicius
2016-11-30 21:23:24 +00:00
Jakob Borg
e4db86836b lib/model: Locking in the request test 2016-11-30 13:11:06 +01:00
Jakob Borg
913a85c571 lib/model: Add simple file syncing test
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3772
2016-11-30 09:32:28 +00:00
Jakob Borg
ed4f6fc4b3 lib/connections, lib/model: Connection service should expose a single interface
Makes testing easier, which we'll need

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3771
2016-11-30 07:54:20 +00:00
Jakob Borg
9da422f1c5 gui, man: Update docs & translations 2016-11-29 11:56:02 +01:00
Stefan Tatschner
ab1739ba34 cmd/syncthing: Trigger usage message on extra CLI parameters
fixes: #3690

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3763
2016-11-27 11:21:05 +00:00
Jakob Borg
fc1430aa92 lib/fs: The interface and basicfs
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3748
2016-11-24 12:07:14 +00:00
Jakob Borg
3cde608eda lib/db: Fix ineffassign lint issue 2016-11-24 12:08:44 +01:00
Jakob Borg
2898552f4b build: Setup should insteall deadcode metalinter 2016-11-24 12:05:04 +01:00
Jakob Borg
911c148c71 build: Improve setup, add metalint ineffasign 2016-11-24 11:33:43 +01:00
Jakob Borg
91568a173a lib/model: Remove ineffectual assignment in test 2016-11-24 11:33:27 +01:00
Jakob Borg
e57f5499a1 lib/sync: Remove unused struct field 2016-11-24 11:30:55 +01:00
Jakob Borg
9abb7b71a9 lib/osutil: Fix lint warning on error formatting (fixes #3760) 2016-11-24 11:20:51 +01:00
Jakob Borg
724c354d62 cmd/stdiscosrv: Fix lint warning on Context keys (fixes #3760) 2016-11-24 11:20:51 +01:00
Jakob Borg
c44779094d build: Setup should download latest version of linters etc 2016-11-24 11:20:51 +01:00
Wulf Weich
eeedab4091 gui: bottom nav always behind dropdown (fixes #3758)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3759
2016-11-23 17:03:43 +00:00
Jakob Borg
8559e20237 lib/osutil: Don't chmod in atomic file creation (fixes #2472)
Instead, trust (and test) that the temp file has appropriate permissions
from the start. The only place where this changes our behavior is for
ignores which go from 0644 to 0600. I'm OK with that.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3756
2016-11-23 14:06:08 +00:00
Jakob Borg
26730eb083 lib/model: Fix test that relies on ignore reloading 2016-11-23 14:42:29 +01:00
Simon Frei
4160ce674d model: consistently use cfg when referring to config instance and not package
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3755
2016-11-22 23:14:20 +00:00
Jakob Borg
2dbeea21c4 lib/ignore: Don't slow down tests by sleeping 2016-11-22 22:44:04 +01:00
Jakob Borg
a2b8485a89 lib/ignore: Fast reload of unchanged ignores (fixes #3394)
This changes the "seen" map that we're anyway keeping around to track
the modtimes of loaded files instead. When doing a Load() we check that
1) the file we are loading is in the modtime set, and 2) that none of
the files in the modtime set have changed modtimes. If that's the case
we do a quick return without parsing anything or clearing the cache.

This required adding two one seconds sleeps in the tests to make sure
the modtimes were updated when we expect cache reloads, because I'm on a
crappy filesystem with one second timestamp granularity. That also
proves it works...

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3754
2016-11-22 21:30:45 +00:00
Jakob Borg
5bb74ee61c gui, man: Update docs & translations 2016-11-22 09:32:57 +01:00
Jakob Borg
462fde5e7d cmd/syncthing: Make the default folder default again
The current way is quite confusing for new users - we create a default
folder, but it's not usable with the default folder created somewhere
else. Instead, when setting up for the first time with two devices, the
default folder must be removed and recreated on one of them. This comes
up on IRC and the forum now and then.

I think this matches expectactions better.

Another alternative would be to remove it entirely (not create a default
folder), but then we should also add some guidance in the UI on how to
proceed.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3751
2016-11-22 08:18:43 +00:00
Unrud
f1e83a57cd lib/osutil: Remove unnecessary fsync in Copy()
Fsyncing the file has a small performance penalty and seems unnecessary. The
file will be fsynced anyway, when the changes are commited to the database.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3749
2016-11-22 07:59:54 +00:00
Jakob Borg
cc9a9fb390 lib/model, lib/protocol: Add Folder.Description() for logging (ref #3741) 2016-11-22 08:36:14 +01:00
Jakob Borg
8fbcceb742 authors: Add further Unrud address 2016-11-22 08:14:22 +01:00
Roman Zaynetdinov
d3a251e6d9 lib/model: Log folder IDs and labels (fixes #3724)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3741
2016-11-21 20:09:18 +01:00
Jakob Borg
be80b26c18 authors: Add zaynetro 2016-11-21 20:08:31 +01:00
Unrud
1574b7d834 lib/model: Add fsync of files and directories, option to disable (fixes #3711) 2016-11-21 18:09:51 +01:00
Jakob Borg
51e10e344d Add Unrud 2016-11-21 17:59:44 +01:00
kwhite17
0d55d8c5b0 gui: Convert URLs in warning messages to HTML links (fixes #3241)
Skip-check: metalint (annoying timeout)

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3747
2016-11-21 08:27:44 +00:00
Jakob Borg
1392589d36 authors: Add kwhite17 2016-11-21 09:12:03 +01:00
Jakob Borg
548a324256 lib/connections: Slow down failing listeners
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3745
2016-11-19 12:37:14 +00:00
Jakob Borg
a8a0bc356a lib/model: Minor cleanup to not fondle cfg.Raw things in handleDeintroductions
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3739
2016-11-17 08:56:55 +00:00
Jakob Borg
faee1d5a8d lib/model: Fix locking around introduction handling (fixes #3737)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3738
2016-11-17 08:50:24 +00:00
Jakob Borg
3088dac33b lib/model: Clean up generateClusterConfig, fix spurious test failure by sorting 2016-11-17 07:45:45 +01:00
Jakob Borg
2641062c17 gui, man: Update docs & translations 2016-11-15 07:23:48 +01:00
Jakob Borg
95c738ea28 lib/protocol: Serialize the all zeroes device ID to the empty string
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3734
2016-11-15 06:22:36 +00:00
Jakob Borg
562d2f67a6 snapcraft: Point home and config dir towards non-versioned snap home (fixes #3730) 2016-11-14 19:06:05 +01:00
Ben Schulz
ba6aff4a1b gui: Use icons and tooltips for folder size info (fixes #3710)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3731
2016-11-13 13:56:07 +00:00
Audrius Butkevicius
bb23e3940e cmd/strelaysrv: Use listen address for outgoing HTTP requests (fixes #3682) 2016-11-13 09:32:05 +01:00
Audrius Butkevicius
94e4370c7e cmd/strelaysrv: Outbox will get GCed (fixes #3718) 2016-11-13 09:32:05 +01:00
Audrius Butkevicius
38d28c3f4a lib/relay: Close invitation channel in all error cases (fixes #3726) 2016-11-13 09:32:05 +01:00
Audrius Butkevicius
f60b424d70 lib/config: Raw() -> RawCopy() 2016-11-13 09:29:35 +01:00
Audrius Butkevicius
a1a91d5ef4 lib/model: Introducer can remove stuff it introduced (fixes #1015)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3522
2016-11-13 09:29:33 +01:00
Jakob Borg
bfb48b5dde jenkins: Clean should remove old snaps 2016-11-12 10:08:13 +01:00
Jakob Borg
2860813a8e build: Set snap grade to "stable" for releases 2016-11-12 09:47:57 +01:00
Jakob Borg
72538e350d build: Snap versions should not have initial "v"
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3728
2016-11-12 08:36:19 +00:00
Jakob Borg
59f3d1445f Revert "lib/model: Introducer can remove stuff it introduced (fixes #1015)"
This reverts commit 0b88cf1d03.
2016-11-12 08:38:29 +01:00
Audrius Butkevicius
0b88cf1d03 lib/model: Introducer can remove stuff it introduced (fixes #1015)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3522
2016-11-11 15:54:25 +01:00
Audrius Butkevicius
56e2ba29d0 lib/config: Subscribers get a copy of the config
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3722
2016-11-11 14:52:23 +00:00
Jakob Borg
6ec9b84674 test: Fix test config 2016-11-09 09:02:55 +08:00
Leo Arias
afd15392b1 build: Build snaps for ARM
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3717
2016-11-09 00:52:33 +00:00
Jakob Borg
ae4cc94a9d lib/model: Fix locking order in Availability() (fixes #3634)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3714
2016-11-08 06:38:50 +00:00
Jakob Borg
3f9b75b7b3 Revert "lib/model: Introducer can remove stuff it introduced (fixes #1015)"
This reverts commit ec2b097313.
2016-11-08 14:27:32 +08:00
Audrius Butkevicius
ec2b097313 lib/model: Introducer can remove stuff it introduced (fixes #1015)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3522
2016-11-08 00:40:48 +08:00
Audrius Butkevicius
caaab462bc lib/sync: Fix broken build 2016-11-05 02:31:52 +00:00
Audrius Butkevicius
da413b823b lib/sync: Add option for sasha-s/go-deadlock 2016-11-05 02:24:53 +00:00
Audrius Butkevicius
14937e7dd2 build: Fix proto builder on Windows 2016-11-03 22:06:51 +00:00
Audrius Butkevicius
3418497f3d lib/sync: Log everything... 2016-11-03 21:33:33 +00:00
Stefan Kuntz
e408f1061a etc: Added ufw firewall application preset
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3703
2016-11-03 15:46:25 +00:00
佛跳墙
c08fe4e2c5 gui: Remove erroneous right parenthesis
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3699
2016-11-02 13:03:57 +00:00
Jakob Borg
b7e21984a1 gui, man: Update docs & translations 2016-11-01 11:26:25 +01:00
Audrius Butkevicius
7fba8cf759 lib/sync: Print all lockers, add holder to RWMutex 2016-10-30 00:17:25 +01:00
Jakob Borg
0296c23685 lib/protocol: Use DeviceID in protocol messages, with custom marshalling
This makes the device ID a real type that can be used in the protobuf
schema. That avoids the juggling back and forth from []byte in a bunch
of places and simplifies the code.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3695
2016-10-29 21:56:24 +00:00
Jakob Borg
1cdfef4d6a script: Missed a newline in the commit-msg hook output 2016-10-27 21:47:14 +02:00
Jakob Borg
cead20ec91 script: Add commit message check hook 2016-10-27 21:42:05 +02:00
MikeLund
74dd051d51 all: Update docs.s.n links to use https
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3691
2016-10-27 17:02:19 +00:00
Jakob Borg
5473285010 gui: Add help link for sync protocol listen addresses 2016-10-26 21:16:53 +02:00
Jakob Borg
4b3adfa21c vendor: Update gobwas/glob to fix question mark handling 2016-10-23 15:47:31 +02:00
Simon Frei
7c37301c91 lib/ignore: Add directory separator to glob.Compile call
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3674
LGTM: calmh
2016-10-21 07:33:40 +00:00
Jakob Borg
d9040f8038 authors: Add imsodin 2016-10-21 15:23:17 +08:00
Jakob Borg
f41606c0b0 jenkins: Build snap
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3678
2016-10-20 09:31:07 +00:00
Leo Arias
31d9750579 build: Add build method for snapcraft
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3636
2016-10-20 09:16:30 +00:00
Jakob Borg
173fb97832 authors: Add elopio 2016-10-20 16:38:14 +08:00
Wulf Weich
81248c3f56 gui: resurrect old dark theme as black theme
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3676
2016-10-19 09:06:27 +00:00
Frank Isemann
2914a0a0a5 gui: Slightly lighten Text for Disconnected/Scanning in Dark Theme 2016-10-19 08:27:17 +08:00
Audrius Butkevicius
815588daba lib/sync, lib/model: Capture locker routine ID, print locker details on deadlock 2016-10-18 21:00:01 +01:00
Jakob Borg
6152eb6d6d gui: Hide "Failed Items" unless there is an actual failure (fixes #3647)
Since delta indexes it's perfectly normal for us to need files that are
currently unavailable due to devices being disconnected. This doesn't
imply a failure, so we should not show the "Failed Items" line and
corresponding eternal spinner (since it would never be filled in, since
there is no failure).

We still show state "Out of Sync" (correct) and the list of files we
need (correct).
2016-10-17 23:57:58 +02:00
Jakob Borg
ff0ebc196c gui: Improve display of local size, ignore pattern status (fixes #3623)
The discrepancy between global and local sizes is fine and expected in
the presence of ignores. This just moves the "we have ignore patterns"
indication to the actual local size metric, as an explanation of why it
may differ from the global size...
2016-10-17 23:57:58 +02:00
Jakob Borg
4e8c8d7e2c cmd/syncthing, lib/db, lib/model: Track more detailed file/dirs/links/deleted counts 2016-10-17 23:57:43 +02:00
Jakob Borg
b8a90b7eaa lib/model: Remove chatty delta index confirmation
This had its moment when making sure it worked initially, but nowadays
there is not point discussing the obvious on every folder on every
connect.
2016-10-17 09:21:58 +02:00
Jakob Borg
60e7ca4a4c gui, man: Update docs & translations 2016-10-17 09:12:16 +02:00
Dale Visser
3a3c8ec6b8 readme: Add Core Infrastructure Initiative Badge
Skip-check: authors, pr-build

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3515
2016-10-17 07:03:52 +00:00
Benny Ng
05c37e58c1 lib/osutil: Prevent infinite Glob recursion (fixes #3577)
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3665
2016-10-12 20:55:38 +00:00
Jakob Borg
d203dd4770 cmd/syncthing: go fmt traceback.go 2016-10-12 20:37:26 +02:00
Jakob Borg
29ccf10d0b cmd/syncthing: Only SetTraceback on Go 1.7+ (fixes #3664) 2016-10-10 17:16:18 +02:00
Jakob Borg
b49df09fec build: Trivial perf improvement of shouldRebuildAssets 2016-10-09 14:28:20 +02:00
Jakob Borg
ce3e117976 cmd/stdiscosrv: rm 'cmd/stdiscosrv/stdiscosrv' (fixes #3663) 2016-10-09 14:26:43 +02:00
Audrius Butkevicius
309795198d cmd/strelaypoolsrv: Remove hostnames from statusAddr 2016-10-08 10:03:53 +01:00
Audrius Butkevicius
7db00132b2 cmd/strelaysrv: Fix sorting zeros versus undefined 2016-10-07 21:24:47 +01:00
Audrius Butkevicius
76a2862b7a authors: Add Xavier O. 2016-10-07 21:24:47 +01:00
Jakob Borg
215503b4f7 cmd/syncthing: Delay browser start until the GUI is ready (fixes #3619) 2016-10-07 12:10:26 +09:00
Jakob Borg
54d4010f1a gui, man: Update docs and translations 2016-10-07 11:09:19 +09:00
Jakob Borg
cb1b53cfb5 gui: Update English base strings (fixes #3638) 2016-10-07 11:07:12 +09:00
Xav
96e8f94833 skip-check: authors
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3650
2016-10-05 19:13:47 +00:00
MikeLund
1e54a3e801 jenkins: use https when downloading docs (fixes #3651)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3652
2016-10-05 12:17:35 +00:00
Tim Howes
fe9c2b9857 lib/ignore: Match directory contents for patterns ending in / (fixes #3639)
Appends "**" to patterns with a terminal slash, so that directory
contents are ignored, but not the directory itself.
2016-10-04 08:12:55 +09:00
Jakob Borg
2a2177e7fa authors: Add timhowes 2016-10-04 08:11:57 +09:00
Jakob Borg
d1d565e58b cmd/syncthing: Localhost header comparison should be case insensitive 2016-10-03 17:34:13 +09:00
Peter Hoeg
891ff383ec etc/linux-systemd: Remove bogus dependency on networking for user unit
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3627
2016-10-03 03:49:00 +00:00
Nathan Morrison
d322ebd0b9 Add API service for local disk changes
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3626
LGTM: calmh, AudriusButkevicius
2016-09-28 15:54:13 +00:00
Peter Hoeg
50190236bb Ignore pkill error on resume
The ```syncthing-resume.service``` will show as a failed service in case
there are no syncthing processes running after resume but it can be
safely ignored because it makes no difference.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3630
2016-09-28 10:21:15 +00:00
Jakob Borg
d5a0f91cb4 cmd/syncthing: Restore useful levels of traceback on panic 2016-09-26 21:14:17 +02:00
Jakob Borg
467c1b26fb cmd/syncthing, lib/config: Log errors replacing or saving config (ref #3567) 2016-09-24 09:59:09 +02:00
Jakob Borg
3cabecda04 lib/upnp: Correct the result deduplication mechanism (fixes #3578)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3618
2016-09-24 07:33:56 +00:00
Jakob Borg
6d3160b0ab gui: Folder is out of sync when it needs deletes, too (fixes #3588) 2016-09-24 09:11:38 +02:00
Jakob Borg
d328e0fb75 cmd/syncthing: Add selectable sha256 package (fixes #3613, fixes #3614)
This adds autodetection of the fastest hashing library on startup, thus
handling the performance regression. It also adds an environment
variable to control the selection, STHASHING=standard (Go standard
library version, avoids SIGILL crash when the minio library has bugs on
odd CPUs), STHASHING=minio (to force using the minio version) or unset
for the default autodetection.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3617
2016-09-23 19:33:54 +00:00
Jakob Borg
5f01afb7ea build: No need for outdated go2xunit 2016-09-18 21:02:42 +02:00
fti7
6fe2fa5ff0 gui: Slightly lighten the dark theme
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3605
2016-09-18 13:47:36 +00:00
Jakob Borg
b371b1fe34 lib/versioner: Test both spaces and parens in ext versioner paths
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3610
2016-09-18 12:24:55 +00:00
Jakob Borg
90c0a39df8 lib/versioner: Test for external versioner
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3609
2016-09-17 20:34:50 +00:00
Lars K.W. Gohlke
70c5a5dff1 lib/versioner: Rename versioner_test to simple_test
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3603
2016-09-16 11:01:43 +00:00
Jakob Borg
da0b7cc7f2 lib/model: Correct lock taking order in ConnectionStats (fixes #3596)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3597
2016-09-14 19:38:55 +00:00
Jakob Borg
139e9b144e lib/config: Fix tests for changes in previous commit 2016-09-13 22:20:22 +02:00
Jakob Borg
77c0a19451 vendor: Update github.com/d4l3k/messagediff 2016-09-13 22:20:22 +02:00
Jakob Borg
58cbd19742 vendor: Update golang.org/cznic/... 2016-09-13 22:20:22 +02:00
Jakob Borg
9bf6917ae8 vendor: Update golang.org/x/crypto/... 2016-09-13 22:20:22 +02:00
Jakob Borg
897cca0a82 vendor: Update golang.org/x/net/... 2016-09-13 22:20:22 +02:00
Jakob Borg
6af09c61be vendor: Update github.com/thejerf/suture 2016-09-13 22:20:22 +02:00
Jakob Borg
c3c7798446 vendor: Update github.com/gobwas/glob 2016-09-13 22:20:22 +02:00
Jakob Borg
06dc91fadf vendor: Update github.com/syndtr/goleveldb 2016-09-13 22:20:22 +02:00
Jakob Borg
526cab538a jenkins: Don't fetch --prune unnecessarily, print build version on Windows 2016-09-13 22:18:55 +02:00
Jakob Borg
81d19a00aa vendor: Add github.com/cznic/lldb and friends (new recursive dependency) 2016-09-13 21:57:19 +02:00
Jakob Borg
ca755ec9e0 vendor: Add golang.org/x/net/bpf 2016-09-13 21:56:33 +02:00
Jakob Borg
4f6206cb2d build: Simpler creation of Debian packages
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3591
2016-09-12 12:21:07 +00:00
Jakob Borg
7fb53ec954 lib/config: Correct name of discovery-v6-4 server 2016-09-12 11:30:06 +02:00
Jakob Borg
d8b5070ca8 lib/config: Update default set of discovery servers 2016-09-12 09:55:45 +02:00
Jakob Borg
5e99d38412 all: Use github.com/minio/sha256-simd
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3581
2016-09-09 09:57:51 +00:00
Laurent Etiemble
3990014073 cmd/syncthing: Conditionally enable CORS
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3541
LGTM: AudriusButkevicius
2016-09-06 22:16:50 +00:00
Jakob Borg
3e51206a6b build, jenkins: Jenkins version tag should be same as when building manually 2016-09-06 13:02:17 +02:00
Aranjedeath
7569b75d61 cmd/strelaysrv: Correct go get command in README
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3564
2016-09-04 21:06:30 +00:00
Jakob Borg
8fcabac518 jenkins: Add batch file for Windows 2016-09-04 16:43:56 +02:00
Jakob Borg
abb0cfde72 jenkins: Add scripts for automated builds (Linux & Mac) 2016-09-04 15:30:16 +02:00
Jakob Borg
7990ffcc60 cmd/syncthing: Copy config on upgrade, instead of renaming (fixes #3525)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3560
2016-09-03 21:29:32 +00:00
Jakob Borg
49910a1d85 lib/config, cmd/syncthing: Enforce localhost only connections
When the GUI/API is bound to localhost, we enforce that the Host header
looks like localhost. This can be disabled by setting
insecureSkipHostCheck in the GUI config.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3558
2016-09-03 08:33:34 +00:00
Jakob Borg
46a143e80e lib/model: Handle deleted-then-ignored files (fixes #3502)
When files that were previously marked as deleted became ignored, we
used to do nothing at all. This changes that behavior to set the Invalid
bit (that we should rename to Ignored). This then becomes an update to
other devices that they should not trust our knowledge about the file in
question.

Read this diff without whitespace...

Tested by
- creating a bunch of files on s1
- letting them sync to s2
- shutting down s2
- deleting the files on s1 and rescanning
- adding the files to .stignore on s1 and rescanning
- starting up s2 and letting it sync
- observing the files are not deleted on s2, and it considers itself up
  to date.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3557
2016-09-02 13:23:24 +00:00
Jakob Borg
69b7f26e4c lib/model, cmd/syncthing: Also account for deleted files in folder summary events (ref #3496)
This should probably be reflected in the GUI somewhere as well...
2016-09-02 10:45:39 +02:00
Jakob Borg
5b37d0356c lib/model, gui: Correct completion percentages when there are lots of deletes (fixes #3496)
We used to consider deleted files & directories 128 bytes large. After
the delta indexes change a bug slipped in where deleted files would be
weighted according to their old non-deleted size. Both ways are
incorrect (but the latest change made it worse), as if there are more
files deleted than remaining data in the repo the needSize can be
greater than the globalSize, resulting in a negative completion
percentage.

This change makes it so that deleted items are zero bytes large, which
makes more sense. Instead we expose the number of files that we need to
delete as a separate field in the Completion() result, and hack the
percentage down to 95% complete if it was 100% complete but we need to
delete files. This latter part is sort of ugly, but necessary to give
the user some sort of feedback.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3556
2016-09-02 06:45:46 +00:00
Jakob Borg
1188ebbb7b gui, man: Update docs & translations 2016-08-23 10:42:09 +02:00
Audrius Butkevicius
76b903b2e0 lib/upgrade: Cleanup failed upgrades (fixes #3500, fixes #3530)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3535
2016-08-23 06:53:39 +00:00
Audrius Butkevicius
be38c2111f cmd/strelaysrv: Add uPNP support, ability to set listen protocol (fixes #3503, fixes #3505, fixes #3506) 2016-08-23 08:43:27 +02:00
Audrius Butkevicius
1de787fab8 cmd/strelaypoolsrv: Ability to select listen protocol 2016-08-23 08:42:57 +02:00
Audrius Butkevicius
81f683a61c cmd/stdiscosrv: Generate keys if missing on startup (fixes #3511) 2016-08-23 08:41:49 +02:00
Audrius Butkevicius
db6f68d031 cmd/stdiscosrv: Use UTC in database timestamps (fixes #3509) 2016-08-23 08:41:15 +02:00
Jakob Borg
d0a1c805e9 cmd/syncthing: uintptr may not be stored in a variable
Must do the whole uintptr(unsafe.Pointer(&whatever)) directly in the
call to sycall.Call, as per https://golang.org/pkg/unsafe/.
2016-08-22 18:24:26 +02:00
Jakob Borg
00a654845f cmd/syncthing: Remove old temp index dbs on startup (fixes #3529)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3532
2016-08-22 12:19:19 +00:00
Jakob Borg
04dad8485a authors: Add calmh email 2016-08-18 19:38:15 +02:00
Jakob Borg
0b1475169f lib/model: Correct virtual mtime handling (fixes #3516)
We previously set the mtime on the temp file, and then renamed it to the
real path. Unfortunately that means we'd save the real timestamp under
the under the temp name ".syncthing.foo.tmp" when the actual file that
we will look up on the next scan is "foo". This moves the Chtimes later,
ensuring that it gets recorded correctly under the right name.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3519
2016-08-16 18:22:19 +00:00
Audrius Butkevicius
6ec4fbc82b lib/model: Add minumum interval for progress emitter (fixes #3517)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3518
2016-08-16 18:22:01 +00:00
Jakob Borg
18cc7a663b lib: Remove osutil.Remove & osutil.RemoveAll (fixes #3513)
These are no longer required with Go 1.7. Change made by removing the
functions, doing a global s/osutil.Remove/os.Remove/.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3514
2016-08-16 10:01:58 +00:00
Jakob Borg
cf5febad47 build, cmd, lib: Minimum supported compiler version is Go 1.5 2016-08-15 08:37:32 +02:00
Jakob Borg
42849af5a8 cmd/syncthing: Default folder should also have lower case ID 2016-08-13 23:23:18 +02:00
Jakob Borg
e6364407a9 cmd/stdiscosrv: Fix index creation checks on startup 2016-08-12 11:39:10 +02:00
Jakob Borg
480b78f2c8 cmd/stdiscosrv: Longer address in schema 2016-08-12 11:38:37 +02:00
Jakob Borg
fa8f339478 gui: Fix division by zero in completion calc (ref #3493) 2016-08-12 08:49:16 +02:00
Jakob Borg
7776839c82 cmd/syncthing, gui: Improve completion calculation (fixes #3492)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3493
2016-08-12 06:41:43 +00:00
Jakob Borg
7114cacb85 gui, man: Update docs & translations 2016-08-10 11:41:46 +02:00
Jakob Borg
e52be3d83e lib/connections, lib/model: Refactor connection close handling (fixes #3466)
So there were some issues here. The main problem was that
model.Close(deviceID) was overloaded to mean "the connection was closed
by the protocol layer" and "i want to close this connection". That meant
it could get called twice - once *to* close the connection and then once
more when the connection *was* closed.

After this refactor there is instead a Closed(conn) method that is the
callback. I didn't need to change the parameter in the end, but I think
it's clearer what it means when it takes the connection that was closed
instead of a device ID. To close a connection, the new close(deviceID)
method is used instead, which only closes the underlying connection and
leaves the cleanup to the Closed() callback.

I also changed how we do connection switching. Instead of the connection
service calling close and then adding the connection, it just adds the
new connection. The model knows that it already has a connection and
makes sure to close and clean out that one before adding the new
connection.

To make sure to sequence this properly I added a new map of channels
that get created on connection add and closed by Closed(), so that
AddConnection() can do the close and wait for the cleanup to happen
before proceeding.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3490
2016-08-10 09:37:32 +00:00
Antoine Lamielle
c9cf01e0b6 gui: weighting % of devices according to folder size (fixes #1300)
The completion of remote devices was based only on the average of the percentages of all folders, which is irrelevant in case of two folders with very different sizes.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3481
LGTM: calmh, AudriusButkevicius
2016-08-09 19:58:44 +00:00
Jakob Borg
dcbf68e104 lib/versioner: Hack to make test coverage stable from run to run
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3485
2016-08-08 18:27:55 +00:00
Jakob Borg
c2d8c07137 lib/events: Hack to make test coverage stable from run to run
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3484
2016-08-08 18:09:40 +00:00
Jakob Borg
a4ed50ca85 build, lib: Correct total test coverage calculation
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3483
2016-08-08 16:29:32 +00:00
Jakob Borg
b3788c8ea0 authors: Fixup 0x010C 2016-08-08 08:34:36 +02:00
Jakob Borg
946c074a41 authors: Add 0x010C 2016-08-08 08:19:02 +02:00
Jakob Borg
19f79afb0f build: Setup should install golint 2016-08-07 21:58:27 +02:00
Audrius Butkevicius
af3b6f9c83 lib/model, lib/config: Support "live" device removal, folder unsharing and folder configuration changes
Furthermore:
1. Cleans configs received, migrates them as we receive them.
2. Clears indexes of devices we no longer share the folder with

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3478
2016-08-07 16:21:59 +00:00
Jakob Borg
fbe42c156d gui: Move "ignore patterns" away from "remove" in folder edit dialog 2016-08-07 14:26:32 +02:00
Jakob Borg
a1f6cbd354 lib/protocol: Clean away outdated files 2016-08-07 14:24:25 +02:00
Audrius Butkevicius
a4f052ad31 lib/connections: Fix connection switching
It seems that it would be impossible to drop down to relay after establishing a direct connection
Also, we should not drop the existing connection until after we've passed the validation steps,
and it seems it's being dropped in two places unnecesserily at the moment.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3480
2016-08-07 12:20:37 +00:00
Jakob Borg
ea87bcefd6 lib/protocol, lib/model: Implement high precision time stamps (fixes #3305)
This adds a new nanoseconds field to the FileInfo, populates it during
scans and sets the non-truncated time in Chtimes calls.

The actual file modification time is defined as modified_s seconds +
modified_ns nanoseconds. It's expected that the modified_ns field is <=
1e9 (that is, all whole seconds should go in the modified_s field) but
not really enforced. Given that it's an int32 the timestamp can be
adjusted += ~2.9 seconds by the modified_ns field...

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3431
2016-08-06 13:05:59 +00:00
Jakob Borg
0655991a19 lib/db, lib/fs, lib/model: Introduce fs.MtimeFS, remove VirtualMtimeRepo
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3479
2016-08-05 17:45:45 +00:00
Jakob Borg
f368d2278f lib/config, lib/connections: Refactor handling of ignored devices (fixes #3470)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3471
2016-08-05 09:29:49 +00:00
Jakob Borg
1eb6db6ca8 cmd/syncthing, lib/...: Correctly handle ignores & invalid file names (fixes #3012, fixes #3457, fixes #3458)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3464
2016-08-05 07:13:52 +00:00
Jakob Borg
a25b63e2df cmd/syncthing: Delete old format indexes after a while (fixes #3468)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3469
2016-08-02 15:44:09 +00:00
Jakob Borg
ffe7a2fcd7 cmd/syncthing, lib/config: Enable HTTP CPU/heap profile collection for users
This adds a config to enable debug functions on the API server, which is
by default disabled. When enabled, the /rest/debug things become
available and become available without requiring a CSRF token (although
authentication is required if configured).

We also add a new endpoint /rest/debug/cpuprof?duration=15s (with the
duration being configurable, defaulting to 30s). This runs a CPU profile
for the duration and returns it as a file. It sets headers so that a
browser will save the file with an informative name.

The same is done for heap profiles, /rest/debug/heapprof, which does not
take any parameters.

The purpose of this is that any user can enable debugging under
advanced, then point their browser to the endpoint above and get a file
that contains a CPU or heap profile we can use, with the filename
telling us what version and architecture the profile is from.

On the command line, this becomes

    curl -O -J http://localhost:8082/rest/debug/cpuprof?duration=5s
    curl: Saved to filename
    'syncthing-cpu-darwin-amd64-v0.14.3+4-g935bcc0-110307.pprof'

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3467
2016-08-02 11:06:45 +00:00
Audrius Butkevicius
08b5a7908f gui: Add one-off notifications that need to be acked
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3254
2016-08-02 08:07:30 +00:00
derekriemer
a8cd9d0154 gui: Improve accessibility (fixes #3297)
skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3463
2016-07-31 22:59:44 +00:00
Jakob Borg
297240facf all: Rename LocalVersion to Sequence (fixes #3461)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3462
2016-07-29 19:54:24 +00:00
Jakob Borg
a022b0cfff gui, man: Update docs & translations 2016-07-28 13:15:14 +02:00
Jakob Borg
72026db599 lib/db, lib/model: Create temp sorting database in config dir (fixes #3449)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3454
2016-07-27 21:38:43 +00:00
Jakob Borg
aafc96f58f lib/model, lib/protocol: Sequence ClusterConfig properly (fixes #3448)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3452
2016-07-27 21:36:25 +00:00
Jakob Borg
7c7e8648ff lib/model: Trigger a puller iteration on connection (fixes #3451)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3453
2016-07-27 21:35:41 +00:00
Jakob Borg
24e2ce0764 build: Allow easy influencing build user and build host
To facilitate reproducible builds.
2016-07-27 23:27:47 +02:00
aviau
d7cb4d407b man: Include stdiscosrv and strelaysrv manpages
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3450
2016-07-27 15:00:10 +00:00
Jakob Borg
66a506e72b lib/scanner: Correctly scan symlinks (fixes #3445)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3446
2016-07-26 11:55:25 +00:00
Jakob Borg
25a7b0a6f8 gui, man: Update docs & translations 2016-07-26 10:53:00 +02:00
Jakob Borg
7aaa1dd8a3 lib/scanner: Recheck file size and modification time after hashing (ref #3440)
To catch the case where the file changed. Also make sure we never let a
size-vs-blocklist mismatch slip through.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3443
2016-07-26 08:51:39 +00:00
Jakob Borg
2a6f164923 lib/scanner: When scanning a file, stick to the size given by Lstat (fixes #3440)
Otherwise if the file grows during scanning the block list will be out
of sync with the stated size and things get confused. We could fixup the
size afterwards based on the block list, but then we might see other
inconsistencies as the mtime should have changed to reflect the new size
etc. Better stick to the original state and let the next scan pick up
the change.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3442
2016-07-25 19:16:49 +00:00
Jakob Borg
0f28626bb4 cmd/syncthing: Generate FolderCompletion events for folders shared with a connecting device (fixes #3436)
This used to happen by itself as the connecting device always sent an
Index message and we triggered on that. Nowadays there's no guarantee
for that, but we anyway need to send out one event to let listeners know
the state of folders shared with the device.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3438
2016-07-25 10:42:17 +00:00
Jakob Borg
6ed22d0885 lib/model: Stricter temporary file permissions
We could have a file to sync with permissions rw------- but we'd create
the temp file with rw-rw-rw- minus umask, usually rw-r--r--. This
potentially exposes private data while the file is being synced.

Similarly, when ignorePerms was set and we were reusing a temp files we
would set the permissions to rw-r--r-- explicitly, potentially
overriding a strict umask that would otherwise have had the file be
rw-------.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3437
2016-07-25 10:18:05 +00:00
Jakob Borg
6715b91a6c build: Remove unused docker-* commands 2016-07-25 08:10:46 +02:00
Jakob Borg
694da60659 lib/db: Reinstate database update locking
The previous commit loosened the locking around database updates.
Apparently that was not fine - what happens is that parallell updates
to the same file for different devices stomp on each others updates to
the global index, leaving it missing one of the two devices.
2016-07-23 20:32:15 +02:00
Jakob Borg
47fa4b0a2c cmd/syncthing, lib/db, lib/model, lib/protocol: Implement delta indexes (fixes #438)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3427
2016-07-23 12:46:31 +00:00
Jakob Borg
8ab6b60778 lib/model: Sort outgoing index updates by LocalVersion
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3411
2016-07-21 17:21:15 +00:00
Jakob Borg
e1a4f81e50 gui, man: Update docs & translations 2016-07-17 23:45:22 +02:00
Jakob Borg
7b7e35d339 lib/protocol: Hello message length is an int16
It used to be an int32, but that's unnecessary and the spec now says
int16. Also relaxes the size requirement to that which fits in a signed
int16 instead of limiting to 1024 bytes, to allow for future growth.

As reported in
https://forum.syncthing.net/t/difference-between-documented-and-implemented-protocol/7798

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3406
2016-07-17 21:41:20 +00:00
Jakob Borg
3176629410 cmd, lib: Fix ineffectual assignments (ineffasign) and comment spelling
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3405
2016-07-15 14:23:20 +00:00
Cedric Staniewski
e3ccc45d19 gui: Fix usage statistics URL in report usage preview (fixes #3397)
This applies the fix from 9d75652 to the usage report preview.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3402
2016-07-12 22:31:11 +00:00
Jakob Borg
beec9e834e gui, man: Update docs & translations 2016-07-10 09:23:58 +02:00
Audrius Butkevicius
f6f0486ff9 repo: Add message about voting
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3398
2016-07-09 15:58:55 +00:00
Jakob Borg
518f446d31 cmd/strelaypoolsrv: Fix vet warnings about type inference
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3393
2016-07-08 06:40:46 +00:00
Jakob Borg
fbbd510088 vendor: Update to latest github.com/syndtr/goleveldb 2016-07-06 09:57:15 +02:00
Jakob Borg
e440d30028 lib/protocol: Allow unknown message types
This lets us add message types in the future, for authentication or
other purposes, without completely breaking old clients. I see this as
similar behavior to adding fields to messages - newer clients must
simple be aware that older ones may ignore the message and act
accordingly.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3390
2016-07-05 09:29:28 +00:00
Jakob Borg
44d30c83bf lib/config, cmd/syncthing: Handle committing configuration better (fixes #3077)
This slightly changes the interface used for committing configuration
changes. The two parts are now:

 - VerifyConfiguration, which runs synchronously and locked, and can
   abort the config change. These callbacks shouldn't *do* anything
   apart from looking at the config changes and saying yes or no. No
   change from previously.

 - CommitConfiguration, which runs asynchronously (one goroutine per
   call) *after* replacing the config and releasing any locks. Returning
   false from these methods sets the "requires restart" flag, which now
   lives in the config.Wrapper.

This should be deadlock free as the CommitConfiguration calls can take
as long as they like and can wait for locks to be released when they
need to tweak things. I think this should be safe compared to before as
the CommitConfiguration calls were always made from a random background
goroutine (typically one from the HTTP server), so it was always
concurrent with everything else anyway.

Hence the CommitResponse type is gone, instead you get an error back on
verification failure only, and need to explicitly check
w.RequiresRestart() afterwards if you care.

As an added bonus this fixes a bug where we would reset the "requires
restart" indicator if a config that did not require restart was saved,
even if we already were in the requires-restart state.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3386
2016-07-04 20:32:34 +00:00
Jakob Borg
7ff7b55732 cmd/strelaypoolsrv: Remove unused var (metalint) 2016-07-04 21:22:53 +02:00
Jakob Borg
44346b3a5a cmd/strelaypoolsrv: Fixup import in main 2016-07-04 14:58:29 +02:00
Jakob Borg
23a538d61a script: Copyright in protofmt.go 2016-07-04 14:55:17 +02:00
Jakob Borg
dcb5026f33 script, lib/discover: Fixup copyright checks 2016-07-04 14:53:11 +02:00
Jakob Borg
778ff9daa9 script: Fixup check-authors after strelaypoolsrv merge 2016-07-04 14:46:24 +02:00
Jakob Borg
ce9dc809bc build, cmd/strelaypoolsrv: Build assets using standard script 2016-07-04 13:34:44 +02:00
Jakob Borg
59370588dd vendor: Add dependencies for strelaypoolsrv 2016-07-04 13:34:34 +02:00
Jakob Borg
7d434aa9c4 build: Add strelaypoolsrv target 2016-07-04 13:34:28 +02:00
Jakob Borg
59ce7c0424 cmd/strelaypoolsrv: Merge relaypoolsrv repo into main
* relaypoolsrv/master: (32 commits)
  Fetch deps of deps X_x
  Here we go with gvt bugs
  Screw godep
  Add solaris support back in
  Add font awesome
  No value is less than zero
  Screw solaris
  Godeps
  Refactor javascript, always show table, add sorting
  Add local geoip
  Update dependencies
  Hey look, had to check all code out on linux to fix the deps
  Update godeps, reduce amount of time spent testing a relay. Goddamit godeps.
  Add timeouts, deal with overlapping markers, add a table, increase circle radiuses
  Fix a couple of issues with the relays map (geoip, 'data unavailable')
  Rate infos are in kbps, not kBps
  Add support for header holding IP address
  Update relay parameters even if it already exists (fixes #3)
  Add missing space
  Add homepage
  ...
2016-07-04 13:33:57 +02:00
Jakob Borg
9a0e5a7c18 lib/discover: Add instance ID to local discovery (fixes #3278)
A random "instance ID" is generated on each start of the local discovery
service. The instance ID is included in the announcement. When we see a
new instance ID we treat is a new device and respond with an
announcement of our own. Hence devices get to know each other quickly on
restart.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3385
2016-07-04 11:16:48 +00:00
Jakob Borg
8d0019595f cmd/syncthing: Update code name for v0.14
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3384
2016-07-04 10:58:45 +00:00
aviau
6ff74cfcab build, cmd/stdiscosrv, cmd/strelaysrv: Rename binaries to add "st" prefix
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3371
2016-07-04 10:51:22 +00:00
Jakob Borg
aa50ef4069 lib/model: Invalidate files with trailing white space on Windows (fixes #3227)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3383
2016-07-04 10:44:30 +00:00
Jakob Borg
fa0101bd60 lib/protocol, lib/discover, lib/db: Use protocol buffer serialization (fixes #3080)
This changes the BEP protocol to use protocol buffer serialization
instead of XDR, and therefore also the database format. The local
discovery protocol is also updated to be protocol buffer format.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3276
LGTM: AudriusButkevicius
2016-07-04 10:40:29 +00:00
Cedric Staniewski
21f5b16e47 gui: Sort device folder lists by label
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3381
2016-07-03 21:11:39 +00:00
Cedric Staniewski
223a835f33 lib/discover: Respect the listen address scheme (fixes #3346)
This is a supplement patch to commit a58f69b which only fixed global
discovery. This patch adds the missing parts for the local discovery.

If the listen address scheme is set to tcp4:// or tcp6:// and no
explicit host is specified, an address should not be considered if the
source address does not match this scheme.

This prevents invalid URIs like tcp4://<IPv6 address>:<port> or tcp6://<IPv4
address>:<port> for local discovery.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3380
2016-07-03 20:43:26 +00:00
Jakob Borg
223e14b0d0 gui, man: Update docs & translations 2016-07-03 13:29:32 +02:00
Cedric Staniewski
a58f69be04 cmd/discosrv: Respect the listen address scheme (fixes #3346)
If the listen address scheme is set to tcp4:// or tcp6://, it needs to be
made sure that the remote address matches this scheme before it is added to
the database.

This prevents invalid URIs like tcp4://<IPv6 address>:<port> or tcp6://<IPv4
address>:<port>.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3378
2016-07-03 11:19:12 +00:00
Phil Davis
e194eb1f69 cmd/relaysrv: Typos in options
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3377
2016-07-03 08:44:41 +00:00
Jakob Borg
672824641b lib/connections: TLS handshake must complete in a timely fashion (fixes #3375)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3376
2016-07-02 20:33:31 +00:00
Jakob Borg
6d357211b2 lib/config: Remove "Invalid" attribute (fixes #2471)
This contains the following behavioral changes:

 - Duplicate folder IDs is now fatal during startup
 - Invalid folder flags in the ClusterConfig is fatal for the connection
   (this will go away soon with the proto changes, as we won't have any
   unknown flags any more then)
 - Empty path is a folder error reported at runtime

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3370
2016-07-02 19:38:39 +00:00
Nicolas Braud-Santoni
8e39e2889d lib: Fix typos in connections/service.go and model/model.go
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3366
LGTM: aviau
2016-06-30 13:42:53 +00:00
Nicolas Braud-Santoni
a9ee4bb9f1 lib/upgrade: Remove TestGithubRelease (fixes #3362)
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3365
2016-06-29 19:06:09 +00:00
Jakob Borg
80fd6c2400 build: Use SOURCE_DATE_EPOCH for build time stamp when available
Apparently common practice for reproducible builds:

   https://reproducible-builds.org/specs/source-date-epoch/

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3364
2016-06-29 18:52:49 +00:00
aviau
3cbe7d40d1 script: Remove build date in genassets.go
The build date prevented the builds from being reproducible.

Debian bug: https://bugs.debian.org/828994

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3363
2016-06-29 18:41:33 +00:00
Lars K.W. Gohlke
af0bc95de5 lib/model: Refactor encapsulation of the folder scanning
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3017
2016-06-29 06:37:34 +00:00
Jakob Borg
4bf3e7485b lib/events: Make events test less likely to fail 2016-06-28 08:29:49 +02:00
Peter Dave Hello
b701de60ce gui, assets: Compress PNGs using ZopfliPNG
Skip-check: authors pr-build

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3358
2016-06-28 06:19:12 +00:00
Antony Male
7ef2743964 lib/events: Introduce per-subscription event IDs (fixes #3335)
Events API consumers rely on being able to detect that events were skipped
by the fact that the event ID has increased by more than 1. This is
documented, and is absolutely necessary when trying to maintain a local
model of Syncthing's state.

With the introduction of LocalChangeDetected, which is not exposed to the
Events API, this contract was broken.

This commit introduces separate concepts of a "Global ID" and a
"Subscription ID". The Global ID of an event is unique across all
subscriptions. The Subscription ID is local to a particular subscription,
and always increments by 1. They are both exposed over the Events API, but
the Subscription ID uses the key "id" for backwards compatibility, and
the "?since=xx" parameter refers to the Subscription ID (making the Global
ID for information only).

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3351
LGTM: calmh
2016-06-27 21:18:58 +00:00
Jakob Borg
a165838cbd lib/model: Decrease max temp filename length (fixes #3338, fixes #3355)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3356
2016-06-27 11:47:40 +00:00
Jakob Borg
3c77b8388c gui: Suggest lower case only folder ID (fixes #3128)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3353
2016-06-27 09:39:25 +00:00
Jakob Borg
9d16f4545d lib/events: Add logging/receiving benchmark 2016-06-27 10:26:59 +02:00
Jakob Borg
d57e6808cc lib/db: Fix alignment crash on 32 bit platforms
Fixes #3347
Fixes #3348
Fixes #3349

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3350
2016-06-26 13:40:51 +00:00
Jakob Borg
b71cc8a580 gui, man: Update docs & translations 2016-06-26 12:48:18 +02:00
Jakob Borg
ac3b03881a gui: Add addresses for disconnected devices (fixes #3340)
Also fixes an issue where the discovery cache call would only return the
newest cache entry for a given device instead of the merged addresses
from all cache entries (which is more useful).

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3344
2016-06-26 10:47:23 +00:00
Jakob Borg
b0d03d1f1c lib/config: Retain slash at end of path after expanding ~ (fixes #2782)
The various path cleaning operations done in in cleanedPath() removes
it, so we make sure it's added again at the end. This makes adding the
slash in prepare() unnecessary, but keep it anyway for display purposes
(people looking at the config).

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3342
2016-06-26 10:17:20 +00:00
Jakob Borg
a2dcffcca2 lib/nat: Avoid concurrent reset of NAT timer (fixes #3337)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3341
2016-06-26 10:17:12 +00:00
Jakob Borg
9323f0faf8 lib/model: Refactor CheckFolderHealth into separate methods
While attempting to fix #2782 I thought the problem was the
CheckFolderHealth method, so I cleaned it up. That turned out not to be
the case, but I think this is better anyhow.

It also moves the "create folder and marker if the folder was empty in
the index" code to StartFolder where I think it makes better sense.

This is covered by a number of existing tests.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3343
2016-06-26 10:07:27 +00:00
Jakob Borg
f343c8ba36 lib/model, lib/scanner: Silence vet warnings
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3333
2016-06-20 21:00:39 +00:00
Audrius Butkevicius
502bee9a09 lib/osutil: Return "/" as filesystem root on non-windows (fixes #3321)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3332
2016-06-20 20:25:00 +00:00
Jakob Borg
379e2119a8 build: Use forward slashes in Zip and Tar files (fixes #3330)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3331
2016-06-20 09:49:19 +00:00
Cedric Staniewski
89a29946f9 gui: Sort folders by label, fall back to ids if required (fixes #3310)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3326
2016-06-18 19:16:53 +00:00
Jakob Borg
20a94fafa7 authors: Add xduugu 2016-06-18 21:04:32 +02:00
Daniel Harte
99ddf1e4ab gui: Adjust border-radius on accordion title buttons (fixes #3299)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3320
2016-06-17 06:59:07 +00:00
Daniel Harte
fb778218f5 gui: Improve layout of "out of sync" modal (fixes #3306)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3296
2016-06-17 06:54:33 +00:00
Daniel Harte
55fc3cb2c5 gui: Load modals before calling initController() (fixes #3301)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3319
2016-06-17 06:44:55 +00:00
Jakob Borg
b779e22205 lib/model: Don't set ignore bit when it's already set
This adds a metric for "committed items" to the database instance that I
use in the test code, and a couple of tests that ensure that scans that
don't change anything also don't commit anything.

There was a case in the scanner where we set the invalid bit on files
that are ignored, even though they were already ignored and had the
invalid bit set. I had assumed this would result in an extra database
commit, but it was in fact filtered out by the Set... Anyway, I think we
can save some work on not pushing that change to the Set at all.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3298
2016-06-13 17:44:03 +00:00
Jakob Borg
bb5b1f8f01 gui: Temporarily disable the usage reporting prompt (ref #3301) 2016-06-13 18:06:12 +02:00
Jakob Borg
c1a96d4900 gui, man: Update docs & translations 2016-06-12 16:21:34 +02:00
Daniel Harte
de298da532 gui: Modal tweaks
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3292
LGTM: AudriusButkevicius, calmh
2016-06-12 14:06:48 +00:00
Jakob Borg
6f5ca53f99 lib/connections: Limit rate at which we print warnings about version mismatch
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3291
2016-06-09 12:30:35 +00:00
Jakob Borg
d507126101 lib/protocol: Understand older/newer Hello messages (fixes #3287)
This is in preparation for future changes, but also improves the
handling when talking to pre-v0.13 clients. It breaks out the Hello
message and magic from the rest of the protocol implementation, with the
intention that this small part of the protocol will survive future
changes.

To enable this, and future testing, the new ExchangeHello function takes
an interface that can be implemented by future Hello versions and
returns a version indendent result type. It correctly detects pre-v0.13
protocols and returns a "too old" error message which gets logged to the
user at warning level:

   [I6KAH] 09:21:36 WARNING: Connecting to [...]:
     the remote device speaks an older version of the protocol (v0.12) not
     compatible with this version

Conversely, something entirely unknown will generate:

   [I6KAH] 09:40:27 WARNING: Connecting to [...]:
     the remote device speaks an unknown (newer?) version of the protocol

The intention is that in future iterations the Hello exchange will
succeed on at least one side and ExchangeHello will return the actual
data from the Hello together with ErrTooOld and an even more precise
message can be generated.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3289
2016-06-09 10:50:14 +00:00
Daniel Harte
9a25df01fe gui: Add support for multiple stacked modals
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3288
2016-06-09 10:45:43 +00:00
perewa
11b9212948 cmd/syncthing: Increase timeout in hello message exchange
Required to establish connections on high latency links

Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3286
2016-06-08 19:46:54 +00:00
Jakob Borg
b4e2914b70 build: Move metalint to a separate build step (and add build step timings)
I run a lot of builds. They're quite slow now:

    jb@syno:~/s/g/s/syncthing $ BUILDDEBUG=1 ./build.sh
        ... snipped commands ...
    runError: gometalinter --disable-all --deadline=60s --enable=varcheck . ./cmd/... ./lib/...
    ... in 13.00592726s
    ... build completed in 15.392265235s

That's 15 s total build time, 13 s of which is the varcheck call. The
build server is welcome to run it, but I don't want to on each build. :)

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3285
2016-06-08 16:15:45 +00:00
Daniel Harte
09b7348595 gui: Accordion titles as buttons
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3284
LGTM: calmh, AudriusButkevicius
2016-06-08 15:55:44 +00:00
Daniel Harte
d2bb6e0c0a gui: Bootstrap tooltips (in modals)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3280
2016-06-08 14:55:30 +00:00
Daniel Harte
8632a03662 gui: Remove tooltip shown over whole modal
Changed the attribute 'title' (reserved) to 'heading' for the modal
template

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3281
2016-06-08 13:07:53 +00:00
Daniel Harte
e71c78ae84 cmd/syncthing: Remove folder limit on /rest/system/browse
Previously limited to 10 results, now unlimited.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3279
LGTM: calmh, AudriusButkevicius
2016-06-08 07:09:50 +00:00
Jakob Borg
03a8027efc cmd/syncthing: Refactor out staticsServer (prev. embeddedStatic) a bit
The purpose of this operation is to separate the serving of GUI assets a
bit from the serving of the REST API. It's by no means complete. The end
goal is something like a combined server type that embeds a statics
server and an API server and wraps it in authentication and HTTPS and
stuff, plus possibly a named pipe server that only provides the API and
does not wrap in the same authentication etc.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3273
2016-06-07 07:46:45 +00:00
Jakob Borg
b7e186b370 cmd/discosrv: Fix lint warnings
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3275
2016-06-07 07:33:11 +00:00
Jakob Borg
4a69f3987f cmd/relaysrv: Fix lint warnings
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3274
2016-06-07 07:31:43 +00:00
Lars K.W. Gohlke
343dc486e0 build: Extract runCommand from main
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3160
2016-06-07 07:12:10 +00:00
Jakob Borg
5aacfd1639 cmd/syncthing: Make API serve loop more robust (fixes #3136)
This sacrifices the ability to return an error when creating the service
for being more persistent in keeping it running.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3270
LGTM: AudriusButkevicius, canton7
2016-06-06 22:12:23 +00:00
Jakob Borg
06e63aedea gui: "Syncing" favicon is no longer animated (fixes #3267)
Resaved with just first frame in Photoshop, ran pngcrush on it.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3269
2016-06-06 13:01:40 +00:00
Daniel Harte
0320194757 gui: Swap edit / pause buttons on devices
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3266
2016-06-06 12:39:47 +00:00
Jakob Borg
1753771356 build: Tags must be joined by space, not comma (fixes #3262)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3268
2016-06-06 11:39:08 +00:00
scienmind
bc794e7c15 lib/connections: Relay failures should be informative, not warning
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3263
2016-06-04 10:41:36 +00:00
Jakob Borg
eefcecc7ce gui, man: Update docs & translations 2016-06-03 13:03:24 +02:00
Daniel Harte
3795a786c9 gui: CSS tweaks for mobile views
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3257
2016-06-02 23:21:19 +00:00
Daniel Harte
855a1bef89 gui: Vertically center identicons in panel titles
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3256
2016-06-02 20:52:10 +00:00
Jakob Borg
6a67921e40 vendor: Revert to github.com/jackpal/gateway instead of fork
It now includes all the fixes

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3246
2016-06-02 20:40:30 +00:00
Daniel Harte
8709fec517 gui: Make warning titles more readable in Dark Theme
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3253
2016-06-02 20:03:21 +00:00
Majed Abdulaziz
48245effdf lib/model, lib/stats: Keep track of folder's last scan time (ref #3143)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3250
LGTM: calmh, AudriusButkevicius
2016-06-02 19:26:52 +00:00
Daniel Harte
16063933d1 gui: Vertically center identicons in accordion titles
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3252
2016-06-02 19:03:18 +00:00
Daniel Harte
d317f197be gui: Early return 'danger' over 'warning'
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3251
2016-06-02 18:34:25 +00:00
Jakob Borg
8ac862f50a build: Use purego build tag on tests 2016-06-02 16:39:17 +02:00
Jakob Borg
0e996c4664 build: Use purego tags on 'all' target 2016-06-02 16:32:23 +02:00
Jakob Borg
287cfee73c cmd/syncthing: Re-enable auto upgrade for dev builds (fixes #901)
As noted in the ticket I no longer agree that dev builds should not auto
upgrade. The main reason is that we give dev builds to users to test
specific fixes, and noone is happier by them being inadvertently stuck
on that version when a newer version including the fix is released.

For developers, it's first of all probably unlikely that development is
happening on a build that's older than release, and secondly STNOUPGRADE
can be set in the environment once and for all if it an issue.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3244
2016-06-02 13:01:59 +00:00
Jakob Borg
a6c465e929 cmd/relaysrv: Go 1.3 fix (we should probably drop that compatibility soon) 2016-06-02 14:48:25 +02:00
Audrius Butkevicius
becb5ab1dc cmd/relaysrv: Missed changes in the repo merge (README, systemd) 2016-06-02 14:42:57 +02:00
Audrius Butkevicius
49170bf2d8 cmd/relaysrv: Add number of routines 2016-06-02 14:39:19 +02:00
Majed Abdulaziz
b1205db7ac cmd/discosrv: Accept host names in announced addresses
GitHub-Pull-Request: https://github.com/syncthing/discosrv/pull/48
2016-06-02 14:34:04 +02:00
Jakob Borg
ff0cd413e6 build: Add default purego tag to discosrv build 2016-06-02 14:22:40 +02:00
Jakob Borg
7a56e4a0e5 cmd/relaysrv: Copyright headers 2016-06-02 14:16:02 +02:00
Jakob Borg
d17608d0a0 cmd/relaysrv: vet: composite literal uses unkeyed fields 2016-06-02 14:10:55 +02:00
Jakob Borg
0af216fea0 cmd/relaysrv: Add build stamped version, print at startup 2016-06-02 14:09:36 +02:00
Jakob Borg
1287433a99 build: Add build steps for relaysrv 2016-06-02 14:07:29 +02:00
Jakob Borg
56a9964101 cmd/relaysrv: Merge relaysrv repo
* relaysrv/master: (60 commits)
  Add new dependencies
  Add more logging in the case of relaypoolsrv internal server error
  Dependency update
  Update deps
  Update packages, fix testutil. Goddamit godep.
  Typo
  Add signal handlers (fixes #15)
  Update readme (fixes #16)
  Limit number of connections (fixes #23)
  Enable extra logging in pool.go even when -debug not specified
  Add Antony Male to CONTRIBUTORS
  Allow extAddress to be set from the command line
  URLs should have Go units
  Add CORS headers
  Fix units
  Expose provided by in status endpoint
  Add ability to advertise provider
  Change the URL
  Rename relaysrv binary, see #11
  Jail the whole thing a bit more
  ...
2016-06-02 14:04:22 +02:00
Jakob Borg
532b4383bf cmd/discosrv: Add build stamped version, print at startup 2016-06-02 13:58:39 +02:00
Jakob Borg
f9e2623fdc vendor: Add dependencies for discosrv 2016-06-02 13:53:30 +02:00
Jakob Borg
eacae83886 authors: Add majedev 2016-06-02 13:52:18 +02:00
Jakob Borg
5fc53f59c7 build: Add build steps for discosrv 2016-06-02 13:51:43 +02:00
Jakob Borg
7035ea3ab7 cmd/discosrv: Merge discosrv repo
* discosrv/master: (64 commits)
  Use atomics for statistics handling (fixes #45)
  Lower case JSON fields are nicer
  Change v13 to v2
  Remove explicit relay handling
  Update vendored github.com/cznic/ql (fixes #34)
  Defer fd.Close() (fixes #37)
  There is no "get dependencies" step
  Add vendor/golang.org/x/net/context
  Use Go 1.5 vendoring instead of Godeps
  Add debug performance logging per request
  Must close result sets
  Set Retry-After header
  Ignores
  lru.Cache is not concurrency safe
  We need a limit on the number of PostgreSQL connections
  Correct example DSN (fixes #29)
  Allow plain HTTP serving behind a proxy
  Fix Query/Answer stats
  Reduce our patience with slow clients somewhat
  Discovery server should print device ID of certificate at startup
  ...
2016-06-02 13:51:17 +02:00
Jakob Borg
d67c0a1eda authors: Clean up AUTHORS and NICKS files
Git didn't really understand the multiple email addresses in the NICKS
file the same way I expected it to, and this fixes that. It also makes
AUTHORS the "master" file that everything else depends on, so it
now includes all of name, nickname and email addresses.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3243
2016-06-02 08:19:12 +00:00
Daniel Harte
36c6a1955f gui: Improve navigation header layout on mobile
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3240
2016-06-02 06:08:18 +00:00
Daniel Harte
f792989d9b gui: Show 'scanning' on unshared folders (fixes #3068)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3239
2016-06-02 00:17:48 +00:00
Daniel Harte
ee398f17e1 gui: Restore broken logo on mobile
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3238
2016-06-01 23:40:11 +00:00
Audrius Butkevicius
8c4723ff43 gui: Fix editing devices (fixes #3236)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3237
2016-06-01 20:24:43 +00:00
Daniel Harte
01ae866d58 gui: Use favicon as indication for status (fixes #1018)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3217
2016-06-01 19:06:36 +00:00
Jakob Borg
3b8ae33fe3 contributing: Clarify license situation for parts of the project
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3234
2016-06-01 13:47:25 +00:00
Audrius Butkevicius
6f63909c65 lib/db,cmd/stindex: Expose VersionList and use it in stindex
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3231
2016-05-31 19:29:26 +00:00
Audrius Butkevicius
1612baca92 gui: /rest/system/browse with no arguments returns drives on Windows (ref #3201)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3203
2016-05-31 19:27:07 +00:00
Jakob Borg
4970bd7f65 lib/relay: Correctly get IP from remote addr via proxy (fixes #3223)
Correctly handles addresses, and fixes one more panicing place.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3230
2016-05-31 14:42:10 +00:00
Jakob Borg
a775dd2b79 script: Improve changelog layout
Pull issue information from Github to show both the resolved issue
subject and the commit subject. Also show reviewer, when different from
author.

    * #3201: api: /rest/system/browse behaves strangely on Windows

      lib/osutil: Fix globbing at root (by @AudriusButkevicius, reviewed by
      @calmh)

    * #3174: Ignore patterns with non-ASCII characters causes out of memory
      crash

      vendor: Update github.com/gobwas/glob (by @calmh)

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3228
2016-05-31 12:40:30 +00:00
Jakob Borg
137894348b test: Update test configs to latest format 2016-05-31 10:36:33 +02:00
Jakob Borg
ac40b27c79 lib/connections: Handle wrapped connection in SetTCPOptions (fixes #3223)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3225
2016-05-31 08:11:57 +00:00
Jakob Borg
9d756525ce gui: Extract URL from translated string (fixes #3204)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3224
2016-05-31 07:24:42 +00:00
Antony Male
6361172bea cmd/syncthing: Be more explicit about how assets should be cached (fixes #3182)
With the previous setup, browsers were free to use a local cache for any
length of time they pleased: we didn't set an 'Expires' header (or max-age
directive), and Cache-Control just said "you're free to cache this".

Therefore be more explicit: we don't mind if browsers cache things, but they
MUST revalidate everything on every request.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3221
2016-05-30 13:54:55 +00:00
Antony Male
56b6383407 gui: Prevent log bar from flashing up while page is loading
The log bar is hidden by CSS, but will appear briefly while the page is
loading (after the html is fetched, but before dev.css is fetched).

Hide it by using an inline style instead, so this does not happen.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3220
2016-05-30 13:16:15 +00:00
Daniel Harte
46fa5a374b gui: Improve layout of accordion titles
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3172
2016-05-30 08:18:09 +00:00
Alexander Graf
7373d2eb3c cmd/syncthing: Fix upgrade of running syncthing from CLI (fixes #3193)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3198
2016-05-28 14:08:26 +00:00
Jakob Borg
4453236949 vendor: Update github.com/gobwas/glob (fixes #3174)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3207
2016-05-28 04:43:54 +00:00
Audrius Butkevicius
c2dc4a8e06 lib/db: Have prefix should be normalized
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3206
2016-05-28 04:18:31 +00:00
Audrius Butkevicius
92a23da3ec lib/model: Make the (?d) prefix actually work
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3205
2016-05-28 04:17:34 +00:00
Audrius Butkevicius
242db26343 lib/osutil: Fix globbing at root (fixes #3201)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3202
2016-05-28 04:13:34 +00:00
Audrius Butkevicius
87701339fe lib/nat, lib/connections: Fix a few issues with NAT traversal
1. For the same internal port we ask for the same external port on all devices. This can be a problem if one device speaks over two protocols.
2. Always add a nil address even if we managed to get external address of the gateway, just because the gateway might be in DMZ behind another gateway.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3196
2016-05-27 06:28:46 +00:00
Jakob Borg
4669ce0766 debian: Rename debian directory to debtpl (fixes #3099)
To keep it out of the way for actual, real, Debian packagers

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3195
2016-05-26 16:37:09 +00:00
Jakob Borg
9bb5988b4e lib/model: Don't deadlock when returning temp index block counts
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3194
2016-05-26 09:16:08 +00:00
Jakob Borg
c513171014 gui: Update translations 2016-05-26 09:49:07 +02:00
Jakob Borg
da5010d37a cmd/syncthing: Use API to generate API Key and folder ID (fixes #3179)
Expose a random string generator in the API and use it when the GUI
needs random strings for API key and folder ID.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3192
2016-05-26 07:25:34 +00:00
Jakob Borg
e6b78e5d56 lib/rand: Break out random functions into separate package
The intention for this package is to provide a combination of the
security of crypto/rand and the convenience of math/rand. It should be
the first choice of random data unless ultimate performance is required
and the usage is provably irrelevant from a security standpoint.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3186
2016-05-26 07:02:56 +00:00
Audrius Butkevicius
410d700ae3 cmd/syncthing: Do not modify events (fixes #3002)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3190
2016-05-26 06:54:44 +00:00
Audrius Butkevicius
fc173bf679 lib/model: Fix wild completion percentages
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3188
2016-05-26 06:53:27 +00:00
Jakob Borg
72154aa668 lib/upgrade: Prefer a minor upgrade over a major (fixes #3163)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3184
2016-05-25 14:01:52 +00:00
Jakob Borg
31b5156191 lib/util: Add secure random numbers source (fixes #3178)
The math/rand package contains lots of convenient functions, for example
to get an integer in a specified range without running into issues
caused by just truncating a number from a different distribution and so
on. But it's insecure, and we use if for things that benefit from being
more secure like session IDs, CSRF tokens and API keys.

This implements a math/rand.Source that reads from crypto/rand.Reader,
this bridging the gap between them. It also updates our RandomString to
use the new source, thus giving us secure session IDs and CSRF tokens.

Some future work remains:

 - Fix API keys by making the generation in the UI use this code as well

 - Refactor out these things into an actual random package, and audit
   our use of randomness everywhere

I'll leave both of those for the future in order to not muddy the waters
on this diff...

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3180
2016-05-25 06:38:38 +00:00
Lars K.W. Gohlke
ebce5d07ac lib/connections: Shorten connection limiting lines
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3177
2016-05-24 21:57:56 +00:00
Audrius Butkevicius
915e1ac7de lib/model: Handle (?d) deletes of directories (fixes #3164)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3170
2016-05-23 23:32:08 +00:00
Lars K.W. Gohlke
b78bfc0a43 build.go: add gometalinter to lint runs
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3085
2016-05-23 21:19:08 +00:00
Lars K.W. Gohlke
30436741a7 build: Also vet and lint build script
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3159
2016-05-23 12:23:55 +00:00
Jakob Borg
98734375f2 cmd/syncthing: Correctly set, parse and compare modified time HTTP headers (fixes #3165)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3167
2016-05-23 12:16:14 +00:00
norgeous
37816e3818 gui: Remove extra href on folder panel titles
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3139
2016-05-22 16:17:33 +00:00
Jakob Borg
4bc2b3f369 gui: Set CSRF stuff earlier (fixes #3138)
We need to set these properties *before* Angular starts making requests,
and doing that from the response to a request is too late. The obvious
choice (to me) would be to use the angular $cookies service, but that
service isn't available until after initialization so we can't use it.
Instead, add a special file that is loaded by index.html and includes
the info we need before the JS app even starts running.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3152
2016-05-22 10:26:09 +00:00
Audrius Butkevicius
00be2bf18d lib/model: Track puller creation times (fixes #3145)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3150
2016-05-22 10:16:09 +00:00
Jakob Borg
44290a66b7 lib/model: Leave temp file in place when final rename fails (fixes #3146)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3148
2016-05-22 09:06:07 +00:00
Jakob Borg
f6cc344623 vendor: Replace github.com/jackpal/gateway with github.com/calmh/gateway (fixes #3142)
Switch to my forked version which contains a fix for this issue. I'll
track upstream in the future if things update there, and attempt to
contribute back fixes...

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3149
2016-05-22 09:04:27 +00:00
Jakob Borg
a89d487510 vendor: Bump github.com/AudriusButkevicius/go-nat-pmp 2016-05-22 17:46:36 +09:00
Jakob Borg
a0ec4467fd cmd/syncthing: Emit new RemoteDownloadProgress event to track remote download progress
Without this the summary service doesn't know to recalculate completion
percentage for remote devices when DownloadProgress messages come in.
That means that completion percentage isn't updated in the GUI while
transfers of large files are ongoing. With this change, it updates
correctly.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3144
2016-05-22 07:52:08 +00:00
Jakob Borg
7dddc0de9e Use atomics for statistics handling (fixes #45)
This is one of those rare cases where that's actually cleaner, I
think...
2016-05-22 09:24:11 +09:00
Jakob Borg
e7280f1eb5 issue_template: Add note about security issues 2016-05-21 22:49:37 +09:00
Jakob Borg
bf7fcc612d cmd/syncthing: Enforce stricter CSRF policy on /rest GET requests (fixes #3134)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3137
2016-05-21 13:48:55 +00:00
Jakob Borg
cff9bbc9c5 gui, man: Update docs & translations 2016-05-21 22:44:55 +09:00
Audrius Butkevicius
fddca3d2d6 lib/connections: Do not resolve addresses (fixes #3129)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3133
2016-05-21 01:31:23 +00:00
norgeous
9db49fb45e gui: Fix dark theme help button
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3130
2016-05-20 16:50:11 +00:00
Lars K.W. Gohlke
891409aedf cmd/syncthing: Extract flag parsing.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3126
2016-05-19 21:47:53 +00:00
Lars K.W. Gohlke
77e47066ed build: Extract setGoPath
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3117
2016-05-19 21:01:23 +00:00
Audrius Butkevicius
852759f904 gui: Update translations (fixes #3125) 2016-05-19 19:44:52 +01:00
Jakob Borg
1dbc310c9b cmd/syncthing: Rename event LocalDiskUpdated -> LocalChangeDetected
I think this better reflects what it means. Also tweaks the verbose
format to be more like our other things and lightly refactors the code
to not have the boolean and include the folder in the event.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3121
2016-05-19 07:01:43 +00:00
Nate Morrison
86ca58e2a9 lib/model: Emit LocalDiskUpdated events on detecting local changes
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3055
2016-05-19 00:19:26 +00:00
Lars K.W. Gohlke
22280db5db lib: simplify code
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3119
2016-05-18 22:47:11 +00:00
Jakob Borg
8e060e23e3 lib/connections: Correctly add port to portless tcp:// URLs (fixes #3115)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3116
2016-05-18 14:27:17 +00:00
aviau
6e07742fe9 gui, lib: Add missing licenses (fixes #3100)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3108
2016-05-18 00:10:50 +00:00
Jakob Borg
04d5032055 gui: Fixup authors in about modal 2016-05-18 09:07:47 +09:00
aviau
73ae87fad1 etc: Add documentation key to syncthing-resume.service
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3109
2016-05-17 20:19:35 +00:00
Lars K.W. Gohlke
cd05282369 lib/connection: Remove unused functions
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3110
2016-05-17 20:07:18 +00:00
aviau
ee94d53bda all: Remove execute bit for non-executable files
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3105
2016-05-17 14:39:50 +00:00
Jakob Borg
922e1407c2 lib/config: Don't migrate non-HTTPS-URL discovery servers to new path (fixes #3103)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3104
2016-05-17 13:43:35 +00:00
Jakob Borg
2ea22b1850 gui, man: Update docs & translations
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3101
2016-05-17 12:02:44 +00:00
Jakob Borg
2c1323ece6 lib/connections: Un-deprecate relaysEnabled (fixes #3074)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3098
2016-05-17 00:05:38 +00:00
Audrius Butkevicius
adb7fb43cb vendor: Update go-nat-pmp 2016-05-16 20:46:03 +01:00
Alex
d59fd9c22d lib/config: use correct ReleasesURL when upgrading from v0.13-beta
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3096
2016-05-14 22:03:07 +00:00
Jakob Borg
6f743f3138 Revert "lib/model: Emit LocalDiskUpdated events on detecting local changes"
This reverts commit 5a7fad0bcd.
2016-05-14 10:55:24 +02:00
Nate Morrison
5a7fad0bcd lib/model: Emit LocalDiskUpdated events on detecting local changes
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3055
2016-05-14 08:37:07 +00:00
Jakob Borg
5d2414dfa9 lib/config: Bump config version to 14
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3092
2016-05-13 14:13:24 +00:00
Jakob Borg
bef2425025 cmd/syncthing: Set User-Agent on upgrade checks
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3093
2016-05-13 14:11:59 +00:00
Jakob Borg
e8b4286c93 lib/config: Change upgrade check URL (fixes #3086)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3089
2016-05-13 09:17:10 +00:00
Jakob Borg
2e9bf0b67c lib/upgrade: Increase size limits, send version header
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3088
2016-05-13 09:01:31 +00:00
Lars K.W. Gohlke
935c273c8f cleanup: removed deadcode in connection/tcp_listen.go
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3084
2016-05-12 20:43:11 +00:00
Jakob Borg
b993b41847 lib/config: Minor attribute updates
As discussed in
https://github.com/syncthing/docs/pull/169

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3082
2016-05-12 08:23:18 +00:00
Jakob Borg
1be40cc4fa lib/ignore: Revert comma handling, upgrade globbing package
This was fixed upstream due to our ticket, so we no longer need the
manual handling of commas. Keep the tests and better debug output around
though.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3081
2016-05-12 07:11:16 +00:00
Lars K.W. Gohlke
d628b731d1 build: Remove unused code
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3079
2016-05-11 06:21:30 +00:00
Jakob Borg
21e116aa45 lib/scanner: Refactor scanner.Walk API
The old usage pattern was to create a Walker with a bunch of attributes,
then call Walk() on it and nothing else. This extracts the attributes
into a Config struct and exposes a Walk(cfg Config) method instead, as
there was no reason to expose the state-holding walker type.

Also creates a few no-op implementations of the necessary interfaces
so that we can skip nil checks and simiplify things here and there.

Definitely look at this diff without whitespace.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3060
2016-05-09 18:25:39 +00:00
Jakob Borg
d77d8ff803 lib/connections: Don't look at devices that are already optimally connected
Just an optimization. Required exposing the priority from the factory,
so made that an interface with an extra method instead of just a func
type.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3071
2016-05-09 15:33:25 +00:00
Jakob Borg
31f64186ae lib/connections: More fine grained locking (fixes #3066)
This fixes the deadlock by reducing where we hold the various locks. To
start with it splits up the existing "mut" into a "listenersMut" and a
"curConMut" as these are the two things being protected and I can see no
relation between them that requires a shared lock. It also moves all
model calls outside of the lock, as I see no reason to hold the lock
while calling the model (and it's risky, as proven).

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3069
2016-05-09 15:03:12 +00:00
Jakob Borg
1a703efa78 lib/model: Fix accounting error in rescan with multiple subs (fixes #3028)
When doing prefix scans in the database, "foo" should not be considered
a prefix of "foo2". Instead, it should match "foo" exactly and also
strings with the prefix "foo/". This is more restrictive than what the
standard leveldb prefix scan does so we add some code to enforce it.

Also exposes the initialScanCompleted on the rwfolder for testing, and
change it to be a channel (so we can wait for it from another
goroutine). Otherwise we can't be sure when the initial scan has
completed, and we need to wait for that or it might pick up changes
we're doing at an unexpected time.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3067
2016-05-09 12:56:21 +00:00
Jakob Borg
8b7b0a03eb lib/config: Don't require restart when adding folders/devices or changing listen address
The VersioningConfig change is because it defaults to nil but gets
deserialized to map[string]string{}. Now prepare() enforces a single
representation of the empty map.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3065
2016-05-09 11:30:19 +00:00
Jakob Borg
0761d804a4 cmd/syncthing: Use random folder ID for default folder, limit random charset
This uses the same charset as the Javascript code, excluding confusing
characters like 0, O, I, 1, l etc.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3064
2016-05-09 09:43:40 +00:00
Jakob Borg
3ad42d9279 lib/util: Should seed random number generator on startup
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3063
2016-05-09 09:36:42 +00:00
klemens
bd41e21c26 all: Correct spelling in comments
Skip-check: authors pr-build-mac

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3056
2016-05-08 10:54:22 +00:00
Jakob Borg
10fe23b8f2 script: Don't verify authors on commits tagged 'Skip-check: authors'
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3057
2016-05-08 10:47:57 +00:00
Jakob Borg
39899e40bf cmd/syncthing: Use ReadAll + json.Unmarshal in places were we care about consuming the reader
Because json.NewDecoder(r).Decode(&v) doesn't necessarily consume all
data on the reader, that means an HTTP connection can't be reused. We
don't do a lot of HTTP traffic where we read JSON responses, but the
discovery is one such place. The other two are for POSTs from the GUI,
where it's not exactly critical but still nice if the connection still
can be keep-alive'd after the request as well.

Also ensure that we call req.Body.Close() for clarity, even though this
should by all accounts not really be necessary.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3050
2016-05-06 22:01:56 +00:00
Jakob Borg
5d337bb24f lib/ignore: Handle bare commas in ignore patterns (fixes #3042)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3048
2016-05-06 15:45:11 +00:00
Jakob Borg
dd5909568f lib/upgrade: Don't attempt processing files larger than expected max binary size (ref #3045)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3047
2016-05-06 14:14:19 +00:00
Jakob Borg
38166e976f lib/upgrade: Enforce limits on download archives (fixes #3045)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3046
2016-05-06 13:58:34 +00:00
Jakob Borg
d6a7ffe0d4 lib/upgrade: Auto upgrade signature should cover version & arch (fixes #3044)
New signature is the HMAC of archive name (which includes the release
version and architecture) plus the contents of the binary. This is
expected in a new file "release.sig" which may be present in a
subdirectory. The new release tools put this in [.]metadata/release.sig.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3043
2016-05-06 13:30:35 +00:00
Jakob Borg
2ebc6996a2 cmd/stsigtool: Sign stdin when not given a file to sign, or when given "-"
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3041
2016-05-05 19:05:45 +00:00
Jakob Borg
2e840134d2 lib/protocol: Add Request benchmarks over raw and TLS encrypted TCP channels
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3040
2016-05-04 23:07:07 +00:00
Jakob Borg
66e1be33cf lib/protocol: Delete erroneously checked in test binary
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3039
2016-05-04 22:09:07 +00:00
Jakob Borg
591959261c gui: Fix comparison operator in expression (ref #3035)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3036
2016-05-04 20:30:18 +00:00
Jakob Borg
459930df09 gui, man: Update docs & translations
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3034
2016-05-04 19:47:08 +00:00
Audrius Butkevicius
674fc566bb lib/connections: Refactor
1. Removes separate relay lists and relay clients/services, just makes it a listen address
2. Easier plugging-in of other transports
3. Allows "hot" disabling and enabling NAT services
4. Allows "hot" listen address changes
5. Changes listen address list with a preferable "default" value just like for discovery
6. Debounces global discovery announcements as external addresses change (which it might alot upon starting)
7. Stops this whole "pick other peers relay by latency". This information is no longer available,
   but I don't think it matters as most of the time other peer only has one relay.
8. Rename ListenAddress to ListenAddresses, as well as in javascript land.
9. Stop serializing deprecated values to JSON

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2982
2016-05-04 19:38:12 +00:00
Jakob Borg
09832abe50 lib/config: Change folder type attribute to a FolderType type
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3032
2016-05-04 11:26:36 +00:00
Audrius Butkevicius
eabd2fc936 lib/model: Use factories for creating folders
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3029
2016-05-04 10:47:33 +00:00
Jakob Borg
6720906ee5 lib/ignore: Refactor: notMatched should be one of the constants
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3031
2016-05-04 07:15:56 +00:00
Audrius Butkevicius
abb96802cb lib/ignores: Use bitmask for result
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3030
2016-05-01 15:58:23 +00:00
Audrius Butkevicius
29fa05ae05 lib/model: Discard download progress upon receiving an index update (fixes #2993)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3023
2016-05-01 06:49:29 +00:00
Jakob Borg
7b43ba809b Lower case JSON fields are nicer 2016-04-30 10:53:42 +02:00
Audrius Butkevicius
49387f9494 lib/protocol: Clean up error values, unused flags
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3025
2016-04-30 04:35:38 +00:00
Alex
953482de53 cmd/stvanity: x509.GenerateCertificate requires pointer for public key
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3022
2016-04-28 22:22:33 +00:00
Audrius Butkevicius
8cf3a7aeda lib/model: Prettify tests (fixes #3014)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3016
2016-04-26 20:19:30 +00:00
Audrius Butkevicius
175f65aabc Change v13 to v2
GitHub-Pull-Request: https://github.com/syncthing/discosrv/pull/41
2016-04-26 20:18:37 +00:00
Jakob Borg
b8c5cf1142 lib/model: Refactor: complete renaming of p -> f
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3015
2016-04-26 15:11:19 +00:00
Lars K.W. Gohlke
236f121c4e lib/model: Refactor out folder and folderscan types, simplify somewhat
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3007
2016-04-26 14:01:46 +00:00
Audrius Butkevicius
94a392144b Remove explicit relay handling
GitHub-Pull-Request: https://github.com/syncthing/discosrv/pull/40
2016-04-26 07:46:43 +00:00
Audrius Butkevicius
2467678bd4 lib/dialer: Add env var to disable proxy fallback (fixes #3006)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3009
2016-04-24 16:30:20 +00:00
Lars K.W. Gohlke
e87c1abd4e all: Clean up dead code
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3005
2016-04-22 21:15:36 +00:00
Lars K.W. Gohlke
dffc34559b lib/config: Remove dead code
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3004
2016-04-22 20:30:58 +00:00
Lars K.W. Gohlke
80f2a9a6bf readme: Remove Appveyor icon
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3003
2016-04-22 20:30:37 +00:00
Audrius Butkevicius
4aa6ecb122 lib/model: Do not use WRONLY (ref #2584)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2994
2016-04-22 08:12:10 +00:00
Jakob Borg
ccfcdf7f48 cmd/syncthing: Don't compact database at startup
This happens automatically in the background anyway, and it can take a
long time on low powered devices at an inconvenient time. We just want
to get up and running as quickly as possible.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3000
2016-04-22 07:34:11 +00:00
Jakob Borg
4eb23a38b1 cmd/stvanity: Use Go 1.3 compatible interface
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2998
2016-04-20 08:16:42 +00:00
Jakob Borg
cb38213444 build: Remove Appveyor and old CircleCI stuff
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2997
2016-04-20 07:12:01 +00:00
Jakob Borg
842b6111db vendor: Update github.com/gobwas/glob to solve 1.3 build issue
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2996
2016-04-20 07:05:08 +00:00
Audrius Butkevicius
ea54525a33 lib/connections: Try not to deadlock (fixes #2987)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2991
2016-04-18 20:25:31 +00:00
Jakob Borg
893cc025f9 cmd/syncthing: Accept ISO-8859-1 and UTF-8 in HTTP BasicAuth header (fixes #2779)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2989
2016-04-18 20:24:38 +00:00
Jakob Borg
b81c8d2e1b lib/model: Drop incoming updates for ignored items (fixes #1701)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2975
2016-04-18 18:35:31 +00:00
Jakob Borg
4b07535e86 gui: Downgrade Angular 1.5.3 -> 1.2.9 (fixes #2961)
I haven't been able to figure out the problem, despite a lot of
experimenting with this...

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2988
2016-04-18 12:34:29 +00:00
Jakob Borg
0d2fe320a7 cmd/stvanity: New utility to create vanity device IDs
A potential practical use is to encode a short version of the hostname
at the beginning of the device ID.

For example:

	jb@syno:~/s/g/s/s/c/stvanity $ stvanity abc
	Want 15 bits for prefix "ABC", about 3.3e+04 certs to test (statistically speaking)
	Found ABCFPWS-JKDIFV3-E5IUAQW-DK53WVR-HY7XWBS-56H33GR-CJQI67Q-VGXRMAW
	Saved to cert.pem, key.pem

	jb@syno:~/s/g/s/s/c/stvanity $ stvanity $(hostname)
	Want 20 bits for prefix "SYNO", about 1e+06 certs to test (statistically speaking)
	Trying 554 certs/s, tested 8307 so far in 15s, expect ~32m total time to complete
	Trying 543 certs/s, tested 16277 so far in 30s, expect ~32m total time to complete
	...

The rest is just a matter of patience.

	jb@syno:~/s/g/s/s/c/stvanity $ stvanity syncthing
	Want 50 bits for prefix "SYNCTHI-NG", about 1.1e+15 certs to test (statistically speaking)
	Trying 529 certs/s, tested 7941 so far in 15s, expect ~67443 years total time to complete
	...

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2986
2016-04-17 20:42:26 +00:00
Jakob Borg
f294113d01 cmd/stdisco: New utility to debug local discovery
When run without parameters, attempts to listen for local discovery
announcements just like Syncthing, and prints them.

With -send, it also sends fake discovery packets. This can be used on
two or more computers simultaneously to verify that they can see each
other.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2985
2016-04-17 18:47:38 +00:00
Audrius Butkevicius
e9063c639a Fetch deps of deps X_x 2016-04-17 15:03:02 +01:00
Audrius Butkevicius
8d6dedc15b Here we go with gvt bugs 2016-04-17 14:57:31 +01:00
Audrius Butkevicius
1bc4c1a8ac Screw godep 2016-04-17 14:49:00 +01:00
Audrius Butkevicius
1c7af1a72e lib/upnp: Fix port order
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2980
2016-04-16 22:44:07 +00:00
Audrius Butkevicius
e61f424ade lib/{nat,pmp}: Fix shadowing and nil IPs
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2979
2016-04-16 16:48:07 +00:00
Jakob Borg
6d3aae32bc Update vendored github.com/cznic/ql (fixes #34) 2016-04-16 12:59:53 +02:00
Jakob Borg
fa1cfd94d0 lib/versioner: Refactor for testing, speed up test
Test now takes <1 second instead of 100 seconds

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2978
2016-04-15 14:26:39 +00:00
Jakob Borg
0155b6f841 Update docs & translations
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2976
2016-04-15 11:47:33 +00:00
Jakob Borg
f6953624dd lib/model: Test should pass go vet inspections
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2977
2016-04-15 11:41:18 +00:00
AudriusButkevicius
1a5f524ae4 lib/model, lib/protocol: Implement temporary indexes (fixes #950)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2252
2016-04-15 10:59:41 +00:00
Jakob Borg
a4cd4cc253 build: Clean up "go vet" and "go lint" steps
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2972
2016-04-15 07:26:25 +00:00
AudriusButkevicius
1a35c440e8 Add solaris support back in 2016-04-14 19:28:06 -04:00
Audrius Butkevicius
2c6c84ac61 Add font awesome 2016-04-14 22:31:56 +01:00
Audrius Butkevicius
bd666daf82 No value is less than zero 2016-04-14 22:26:31 +01:00
AudriusButkevicius
ca3831c4f5 Screw solaris 2016-04-14 17:21:44 -04:00
AudriusButkevicius
bbe0d34f43 Godeps 2016-04-14 17:19:56 -04:00
Audrius Butkevicius
dd364c962f Refactor javascript, always show table, add sorting 2016-04-14 22:01:25 +01:00
Audrius Butkevicius
50068b0b0f Add local geoip 2016-04-13 21:34:11 +01:00
Audrius Butkevicius
c49453c519 lib/pmp: Add NAT-PMP support (ref #698)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2968
2016-04-13 18:50:40 +00:00
Jakob Borg
52c7804f32 lib/connections: Silence vet and lint warnings
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2971
2016-04-13 11:50:51 +00:00
Audrius Butkevicius
19b4f3bfb4 lib/nat: Add a nat package and service to track mappings on multiple IGDs 2016-04-10 19:36:38 +00:00
Jakob Borg
f3ac421266 lib/protocol: Comment the bit numbers for flags in IndexMessage 2016-04-10 10:47:30 +00:00
Michael Ploujnikov
7533a61203 unifySubs: add two trivial test cases 2016-04-10 02:41:28 +00:00
Jakob Borg
6355a7019b gui: Update translations for previous commit 2016-04-09 12:23:13 +00:00
Audrius Butkevicius
490464e170 gui: Some browsers force lowercase attributes 2016-04-09 12:20:56 +00:00
Michael Ploujnikov
467d338fe4 lib/model: Scanning unknown items is OK as long as the parent is known (fixes #2915) 2016-04-09 11:25:06 +00:00
Audrius Butkevicius
6130578d18 lib/db: Empty slice is not nil (fixes #2872) 2016-04-09 07:46:19 +00:00
Audrius Butkevicius
4389bb037d lib/model: Add option for overwriting names on connect (fixes #2912) 2016-04-09 07:43:47 +00:00
Audrius Butkevicius
2eb8a9ef56 all: Dead code cleanup 2016-04-09 01:10:31 +00:00
Audrius Butkevicius
393798098c cmd/syncthing: Listening on a 0 port is not valid (fixes #2926) 2016-04-09 01:06:55 +00:00
Jakob Borg
668eb7c398 build: Archives should have release name as first file name component 2016-04-08 10:53:29 +00:00
Jakob Borg
0937f85534 gui, man: Update docs & translations 2016-04-08 10:25:40 +00:00
Jakob Borg
cf64376dca build: go vet is now included in the distribution 2016-04-08 10:19:11 +00:00
Audrius Butkevicius
5a98af622d lib/ignore: Implement deletable ignores using (?d) prefix (fixes #1362) 2016-04-07 09:34:07 +00:00
Jakob Borg
4f5d0b46f7 build: Parameterize build targets 2016-04-06 22:18:30 +01:00
Lars K.W. Gohlke
492e92d65d gui: Show Javascript error indicator on dev builds, remove logging of missing translations 2016-04-05 06:36:53 +00:00
Jakob Borg
181939c841 lib/ignore: Correct case insensitive matching on Mac/Windows
There was a bug in that we only did the lowercase folding when the
pattern had an explicit (?i), which is not the case on Windows/Mac
necessarily.
2016-04-05 06:35:51 +00:00
Audrius Butkevicius
b678b4e048 cmd/syncthing: Skip a calculation if timediff is zero (fixes #2854) 2016-04-05 07:12:17 +02:00
Jakob Borg
1934b3a5b6 lib/ignore: Remove pattern for foo/** which is already covered by foo/
Actual speed difference according to benchmarks is hidden in the noise



Also make the "pattern" field for each entry match what is actually

evaluated.
2016-04-04 13:22:25 +01:00
Jakob Borg
cc1d122352 lib/model: Correctly detect deleted but previously ignored files as deleted 2016-04-04 11:53:55 +01:00
Jakob Borg
a4f0b85462 lib/config: Disable cacheIgnoredFiles, new default is disabled 2016-04-03 20:18:16 +01:00
Laurent Etiemble
7b4e1e9055 cmd/syncthing: Fix handler ordering so CORS middleware wraps all the others but the debug one 2016-04-03 13:24:55 +02:00
Jakob Borg
4c3cd4c9e3 lib/ignore: Replace lib/fnmatch with github.com/gobwas/glob
Because it's literally ten times faster:



	benchmark                  old ns/op     new ns/op     delta

	BenchmarkMatch-8           13842         1200          -91.33%

	BenchmarkMatchCached-8     139           147           +5.76%



	benchmark                  old allocs     new allocs     delta

	BenchmarkMatch-8           0              0              +0.00%

	BenchmarkMatchCached-8     0              0              +0.00%



	benchmark                  old bytes     new bytes     delta

	BenchmarkMatch-8           12            0             -100.00%

	BenchmarkMatchCached-8     0             0             +0.00%
2016-04-02 20:03:24 +01:00
Lars K.W. Gohlke
46e913dc23 gui: Improve header and footer layout on small screens 2016-04-02 10:31:16 +02:00
Jakob Borg
8f580b13df gui: Update translations and documentation 2016-04-01 07:33:42 +00:00
Jakob Borg
a551686d37 lib/discovery: Receiving a new announcement should be non-blocking
Pretty sure the intention of the select was for it to be non-blocking.
Not that it will matter almost ever.
2016-04-01 07:24:04 +00:00
Audrius Butkevicius
432c78079b lib/connections: Increase lock periods to prevent races (fixes #2899) 2016-04-01 07:23:11 +00:00
Jakob Borg
f5f0e46016 lib: Use bytes.Equal instead of bytes.Compare where possible 2016-03-31 15:12:46 +00:00
Jakob Borg
b6f32b6e45 build: Clean up environment handling
Don't set variables for cross compilation when building assets, cleaner
check for Go version.
2016-03-31 07:33:09 +00:00
Michael Ploujnikov
66f480519b lib/model: Refactor out scanning method from rwfolder.Serve loop 2016-03-30 06:53:47 +00:00
Jakob Borg
8044522691 vendor: Update calmh/xdr to avoid unexpected string behavior (fixes #2882) 2016-03-29 19:55:43 +00:00
Jakob Borg
c6a67bd203 gui: Update lang-en with new strings from GUI 2016-03-28 10:55:35 +00:00
Jakob Borg
c6881b6d02 gui: Update Bootstrap (v3.3.6), non-minified 2016-03-28 10:51:12 +00:00
Jakob Borg
4489bec6ef gui: Upgrade jQuery (v2.2.2), non-minified 2016-03-28 10:50:19 +00:00
Jakob Borg
3d71e68696 gui: Upgrade Angular (v1.5.3 plus various), non-minified 2016-03-28 10:46:51 +00:00
Jakob Borg
783d2da4a8 gui: Upgrade Font Awesome (v4.5.0), non-minified 2016-03-28 10:46:05 +00:00
Jakob Borg
6be4b49999 build: Generate gui.files.go on the fly, remove from repo 2016-03-28 10:03:13 +00:00
Audrius Butkevicius
68185dd93c gui: Remove bootswatch 2016-03-27 14:08:17 +00:00
Jakob Borg
d01ea9d6fb lib/discovery: Handle nil relayService (fixes #2890) 2016-03-27 11:37:43 +00:00
Jakob Borg
d91e6023eb lib/sync: Skip the timing tests if the host timer is flaky 2016-03-27 10:41:38 +00:00
Jakob Borg
17ed01a0c9 lib/connections: Rename makeTcp -> makeTCP according to go vet's wishes 2016-03-27 07:18:33 +00:00
Audrius Butkevicius
4b6c2d0d3d gui: Pretty theme names 2016-03-27 06:40:50 +00:00
norgeous
46c07bb207 gui: Clean up CSS 2016-03-27 00:43:07 +00:00
norgeous
eaa805b9f0 gui: add Bootswatch themes 2016-03-26 21:57:05 +00:00
Lars K.W. Gohlke
436fd0b88e pull_request_template: Add note about docs needing update 2016-03-26 07:00:12 +00:00
Jakob Borg
f706d3c393 cmd/stbench: Add utility to run benchmark tests 2016-03-25 20:52:20 +00:00
Jakob Borg
c58eb1d47a cmd/stgenfiles: Add utility for generating test data
I use this to generate ~40 gigs of random test data in 200k files:

    ~/stgenfiles -dir /data/benchdata -files 200000 -maxexp 22 -src /dev/urandom
2016-03-25 20:50:48 +00:00
Audrius Butkevicius
b4f9a55e6e protocol: Add "Hello" message at connection start, also for unauthed peers 2016-03-25 20:29:07 +00:00
Audrius Butkevicius
1d17891286 lib/upnp: Refactor out methods to util with tests, refactor IGD 2016-03-25 20:22:29 +00:00
Audrius Butkevicius
6a3f3f5577 gui: Add theme.css, move dark theme, adjust popover advanced folder settings colors (fixes #2878) 2016-03-25 16:55:53 +00:00
Audrius Butkevicius
29913dd1e4 lib/connections: Refactor address listing into connection service 2016-03-25 07:35:18 +00:00
Audrius Butkevicius
690837dbe5 lib/connections: Allow "tcp4" and "tcp6" addresses 2016-03-25 07:15:32 +00:00
norgeous
82e80a479a gui: Add bootstrap tooltip to existing tooltip on folders 2016-03-25 07:09:55 +00:00
norgeous
bc508aee7b gui: Docs links should be HTTPS 2016-03-25 07:02:29 +00:00
Jakob Borg
95247f7740 cmd/syncthing: Basic smoke test of all API endpoints
... except /rest/system/upgrade that requires a correct response from
Github, which we shouldn't depend on.
2016-03-24 10:17:04 +00:00
Jakob Borg
e5731229c7 cmd/syncthing: Add test for starting API service and requesting some URLs 2016-03-24 08:55:33 +00:00
Jakob Borg
52c74ad866 cmd/syncthing: Add mock types for API service testing 2016-03-24 08:09:13 +00:00
Jakob Borg
a28f890e83 issue_template: Add note about using forum for support 2016-03-23 20:50:52 +00:00
Wulf Weich
31362dfc17 gui: Better accessibility for folder & device panels (fixes #2288) 2016-03-22 20:53:56 +00:00
Jakob Borg
a492cfba13 cmd/syncthing: Extract interfaces for things the API depends on
Enables testing of the API service, in the long run.
2016-03-21 19:36:08 +00:00
Jakob Borg
96afcd90e3 Merge pull request #38 from kc1212/issue-37
Defer fd.Close() (fixes #37)
2016-03-21 14:29:47 +01:00
kc1212
ea61f8f597 Defer fd.Close() (fixes #37) 2016-03-21 01:07:51 +01:00
Audrius Butkevicius
894ccd18ff Merge pull request #2855 from calmh/marshalfail
cmd/syncthing: Return 500 with an error object instead of empty 200 on marshalling error in REST response
2016-03-20 11:09:59 +00:00
Jakob Borg
9dec6f1324 cmd/syncthing: Return 500 with an error object instead of empty 200 on marshalling failure in REST response 2016-03-20 11:54:53 +01:00
Jakob Borg
2e44473ce4 There is no "get dependencies" step 2016-03-18 14:42:12 +01:00
Jakob Borg
26d6969384 Add vendor/golang.org/x/net/context 2016-03-18 14:41:00 +01:00
Jakob Borg
2dbde224d9 Use Go 1.5 vendoring instead of Godeps 2016-03-18 14:38:08 +01:00
Jakob Borg
8d7ed9f8bf Add debug performance logging per request 2016-03-18 14:34:55 +01:00
Jakob Borg
aba2cc4db2 lib/model: Properly handle deleting multiple files when doing scans with subs (fixes #2851) 2016-03-18 12:16:33 +00:00
Jakob Borg
2df001fe5c lib/model: Correct handling of multiple subs when scanning (fixes #2851)
Previously the code failed in that it would return top-level plus a sub,
i.e. ["", "foo"], and it would consider "usr/lib" a prefix of
"usr/libexec" which it is not.
2016-03-18 08:28:44 +00:00
Audrius Butkevicius
a49b8a2608 lib/relay/client: Log relay client messages (fixes #2624) 2016-03-18 07:25:37 +00:00
kc1212
bea272c40b Confirmation box for when adding multiple folders on the same path (#1960) 2016-03-17 23:05:37 +00:00
Jakob Borg
a455e32adf meta: Amend wweich in NICKS 2016-03-17 21:23:47 +01:00
Jakob Borg
9d522bd626 gui: Update translation files and assets 2016-03-17 21:05:55 +01:00
Wulf Weich
0427396f50 gui: Differentiate local and remote devices more clearly 2016-03-17 20:05:09 +00:00
Jakob Borg
c952468e13 gui: Improve layout of footer on narrow screens (fixes #2663) 2016-03-17 16:39:50 +00:00
Jakob Borg
94b3ce44e6 connections: The Max{Send,Recv}Kbps variables are supposed to be in KiB/s 2016-03-17 08:18:23 +01:00
Jakob Borg
c439c543d0 tests: messagediff argument order should be expected, actual
So that the diff describes the changes that happened in actual as
compared to expected. The opposite is confusing.
2016-03-17 08:03:29 +01:00
norgeous
78120bd989 Use Bootstrap tooltips instead of plain title attributes
By using data-original-title the tooltips live update without reapplying the
js code, such as .tooltip('fixTitle') each time the content changes. This
method also works well with angular expressions:

    data-original-title="{{'Download Rate' | translate}}"

This example provides a bootstrap tooltip saying 'Download Rate' that changes
automatically when the language is updated.
2016-03-16 14:55:29 +00:00
Jakob Borg
f66c1c3c9c Amend norgeous 2016-03-16 15:37:03 +01:00
dinosore
6f82d83bd6 Let "systemctl help" command work
Before this change, issuing either
    systemctl --user help syncthing[.service]
or
    systemctl help syncthing@user[.service]
gave the message
    Can't show: http://docs.syncthing.net/

Following this change the syncthing man page is displayed
2016-03-16 13:09:01 +00:00
Jakob Borg
3e218b146e Add dinosore 2016-03-16 14:08:24 +01:00
Jakob Borg
17517bcc3d Don't show restart prompt when changing folder label (fixes #2840) 2016-03-16 12:18:21 +01:00
Jakob Borg
d8fba47870 Amend wweich 2016-03-14 08:37:43 +01:00
Jakob Borg
e9c5261a49 Mend GUI tests 2016-03-13 17:24:49 +01:00
Jakob Borg
8d53175c20 Compact and slightly reorder author list
More prominent positions are given to authors with more commits, in
steps of magnitude. Authors with 100-999 commits are listed before
authors with 10-99 commits. Yes, this puts me at the head of the list
and is a slight ego trip, but I still think it's the right thing to do.
2016-03-13 15:38:13 +01:00
Jakob Borg
ba5231dc89 apiService should not reference global variable 'locations' (hinders testing) 2016-03-13 11:03:00 +01:00
Jakob Borg
032365d57c Fix STGUIASSETS search paths & order (fixes #2827) 2016-03-12 12:17:25 +00:00
Jakob Borg
e9aed494f8 Add wweich (noreply-address) 2016-03-11 16:42:06 +01:00
Lars K.W. Gohlke
16c3d39fd2 Add folder label in addition to ID (fixes #966)
An auto generated ID is suggested on folder creation to reduce conflicts with
folders created on other devices.
2016-03-11 09:48:46 +00:00
wweich
1875f7287e Increase contrast for readonly form controls in dark theme (fixes #2820)
Increase the dark theme color value for text in readonly form controls for better contrast between text and background.
2016-03-11 09:08:12 +00:00
Audrius Butkevicius
d619031f68 Merge pull request #2832 from calmh/dont-edit-authors
Update pull request template with authorship info
2016-03-11 09:02:26 +00:00
Jakob Borg
4ef759dba8 Update pull request template with authorship info
People want to add themselves to AUTHORS. That's fine, but it's not
enough as it also needs to be added to NICKS and script/authors.go needs
to be run. I'd rather have us do this and do it correctly so lets
document that people should not worry about it.
2016-03-11 09:36:06 +01:00
Jakob Borg
0d16c8eab4 Add norgeous 2016-03-11 09:25:38 +01:00
Jakob Borg
de7d176edf Update goleveldb dependency 2016-03-11 09:25:38 +01:00
Jakob Borg
d37ed65f42 Include syncthing-resume systemd service in Debian package 2016-03-11 08:05:46 +00:00
Jakob Borg
710ddf7906 Rebuild assets 2016-03-10 16:56:24 +01:00
Lars K.W. Gohlke
3abb80885e Collapse advanced settings in folder editor modal 2016-03-10 15:54:33 +00:00
Jakob Borg
fd962c5e99 Also update allowed version tests 2016-03-10 13:24:36 +01:00
Jakob Borg
07f944bf48 More lenient expression for allowed version tags 2016-03-10 13:19:00 +01:00
Jakob Borg
012423338e Not to mention regexps, and testing. 2016-03-10 10:49:11 +01:00
Jakob Borg
64cfebc63c Branch names are hard 2016-03-10 10:47:15 +01:00
Jakob Borg
28d74f5d9b Correct the branch finding logic 2016-03-10 10:37:24 +01:00
Jakob Borg
8418fae82b Add branch name to build version when appropriate 2016-03-10 10:24:11 +01:00
Lars K.W. Gohlke
9b1bebc9b2 Correct path to genxdr after the change to Go1.5+ vendoring 2016-03-09 12:43:16 +00:00
Jakob Borg
8d888bb756 Add lkwg82 2016-03-09 13:33:18 +01:00
Jakob Borg
83c29e1945 Fix tests on Go 1.3 2016-03-08 09:07:18 +01:00
Jakob Borg
09ebc33b30 Fix tests on 32 bit 2016-03-08 09:06:59 +01:00
Jakob Borg
ff9bfae722 Remove one apostrophe in and reformat ISSUE_TEMPLATE 2016-03-08 08:23:00 +01:00
Audrius Butkevicius
3b146eda0d Clarify GUI stuff (fixes #2819) 2016-03-06 22:07:15 +00:00
Audrius Butkevicius
a8ffde6f21 Add deps 2016-03-06 20:32:10 +00:00
Richard Hartmann
dd9a4e044a README.md: Spelling 2016-03-06 21:12:01 +01:00
Jakob Borg
b8c72ade4c Default to modern -ldflags syntax for unknown Go version 2016-03-05 22:25:28 +01:00
Audrius Butkevicius
5dd55d3811 Merge pull request #2817 from calmh/vendoring
Use Go 1.5 vendoring instead of Godeps
2016-03-05 20:56:37 +00:00
Audrius Butkevicius
f00b133eee Merge pull request #2818 from calmh/prtemplate
Add a pull request template
2016-03-05 20:52:14 +00:00
Jakob Borg
a117b0c723 Add a pull request template 2016-03-05 21:50:51 +01:00
Jakob Borg
65aaa607ab Use Go 1.5 vendoring instead of Godeps
Change made by:

- running "gvt fetch" on each of the packages mentioned in
  Godeps/Godeps.json
- `rm -rf Godeps`
- tweaking the build scripts to not mention Godeps
- tweaking the build scripts to test `./lib/...`, `./cmd/...` explicitly
  (to avoid testing vendor)
- tweaking the build scripts to not juggle GOPATH for Godeps and instead
  set GO15VENDOREXPERIMENT.

This also results in some updated packages at the same time I bet.

Building with Go 1.3 and 1.4 still *works* but won't use our vendored
dependencies - the user needs to have the actual packages in their
GOPATH then, which they'll get with a normal "go get". Building with Go
1.6+ will get our vendored dependencies by default even when not using
our build script, which is nice.

By doing this we gain some freedom in that we can pick and choose
manually what to include in vendor, as it's not based on just dependency
analysis of our own code. This is also a risk as we might pick up
dependencies we are unaware of, as the build may work locally with those
packages present in GOPATH. On the other hand the build server will
detect this as it has no packages in it's GOPATH beyond what is included
in the repo.

Recommended tool to manage dependencies is github.com/FiloSottile/gvt.
2016-03-05 21:21:24 +01:00
Laurent Arnoud
9259425a9a Add priority,section and homepage to debian/control 2016-03-04 16:26:56 +01:00
Laurent Arnoud
6816e2436b Fix description-contains-tabs and improve description 2016-03-04 16:26:56 +01:00
Laurent Etiemble
c8b6e6fd9b Increase maximum allowed file size to 10 Mblocks
Upgrade FileInfo up to 10000000 blocks. 1310 GB files can be shared.
Increase limit when unmarshaling XDR.
Increase the size of message.
2016-03-04 16:24:54 +01:00
Jakob Borg
ac2343ea57 Only check specified paths in check-authors.go 2016-03-04 16:20:36 +01:00
Jakob Borg
a6a9af4f02 Fix marshalling tests for Go 1.6 2016-03-04 14:16:42 +01:00
Jakob Borg
35dc173c80 Minor tweaks to README.md 2016-03-02 07:50:15 +01:00
Audrius Butkevicius
a686be8ba4 Add kralo 2016-02-28 13:26:03 +00:00
Audrius Butkevicius
d61b03701c Merge pull request #2806 from kralo/master
gui: add a lock icon to the folder title for easy overview (fixes #2703)
2016-02-28 13:25:04 +00:00
Max Schulze
81d9857888 gui: add a lock icon to the folder title for easy overview (fixes #2703)
(to indicate it is a master directory)
2016-02-27 21:53:39 +01:00
Audrius Butkevicius
cd9e142db3 Merge pull request #2805 from kralo/master
gui: add tooltips (title) to the folder path and syncthing version
2016-02-27 18:19:26 +00:00
Max Schulze
8682a33ab1 gui: add html tooltips (title) to the folder path and syncthing version elements (fixes #2758) 2016-02-27 19:00:11 +01:00
Audrius Butkevicius
0631e4395a Merge pull request #2794 from rumpelsepp/master
systemd: Add syncthing-resume.service
2016-02-22 09:13:10 +00:00
Stefan Tatschner
d78425eab4 systemd: Add syncthing-resume.service
This systemd service restarts Syncthing after resume from suspend
via sending SIGHUP. By default Syncthing detects resume from sleep
on its own by looking for jumps in the system clock. Since systemd
knows exactly when the system resumes from sleep let's trigger
the Syncthing restart from there. Doing this in systemd eliminates
some annoying delay, as the service is restarted immediately after
resume. Also, using the systemd dependency mechanism syncthing-inotify
is restarted as well.

$ journalctl -e --identifier syncthing --identifier syncthing-inotify --identifier systemd
Feb 22 09:44:27 kronos systemd[1]: Reached target Sleep.
Feb 22 09:44:27 kronos systemd[1]: Starting Suspend...
Feb 22 09:44:33 kronos systemd[1]: Time has been changed
Feb 22 09:44:33 kronos systemd[963]: Time has been changed
Feb 22 09:44:33 kronos systemd[1]: Started Suspend.
Feb 22 09:44:33 kronos systemd[1]: sleep.target: Unit not needed anymore. Stopping.
Feb 22 09:44:33 kronos systemd[1]: Stopped target Sleep.
Feb 22 09:44:33 kronos systemd[1]: Reached target Suspend.
Feb 22 09:44:33 kronos systemd[1]: suspend.target: Unit is bound to inactive unit systemd-suspend.service. Stopping, too.
Feb 22 09:44:33 kronos systemd[1]: Stopped target Suspend.
Feb 22 09:44:33 kronos systemd[1]: Starting Restart Syncthing after resume...
Feb 22 09:44:33 kronos syncthing[2561]: [35K66] OK: Exiting
Feb 22 09:44:33 kronos systemd[1]: Started Restart Syncthing after resume.
Feb 22 09:44:34 kronos systemd[963]: syncthing.service: Service hold-off time over, scheduling restart.
Feb 22 09:44:34 kronos systemd[963]: Stopping Syncthing Inotify File Watcher...
Feb 22 09:44:34 kronos systemd[963]: Stopped Syncthing Inotify File Watcher.
Feb 22 09:44:34 kronos systemd[963]: Stopped Syncthing - Open Source Continuous File Synchronization.
Feb 22 09:44:34 kronos systemd[963]: Started Syncthing - Open Source Continuous File Synchronization.
Feb 22 09:44:34 kronos systemd[963]: Started Syncthing Inotify File Watcher.
Feb 22 09:44:34 kronos syncthing[2836]: [35K66] INFO: syncthing v0.12.19 "Beryllium Bedbug" (go1.5.3 linux-amd64) builduser@svetlemodry 2016-02-14 19:26:33 UTC

This system service has to be located in "/etc/systemd/system/syncthing-resume.service",
and for packages in "/usr/lib/systemd/system/syncthing-resume.service". It can be
enabled using "systemctl enable syncthing-resume.service".
2016-02-22 09:51:14 +01:00
Audrius Butkevicius
0b03a640fb Merge pull request #2790 from syncthing/issuetpl
Add ISSUE_TEMPLATE
2016-02-21 11:32:03 +00:00
Jakob Borg
9d277ac2ac Add ISSUE_TEMPLATE
Template for newly created issues. We want this text to be short and clear and request the required information from the user, and also be clear that it is a template and should be removed/replaced in the actual issue before saving...
2016-02-21 11:57:17 +07:00
Jakob Borg
54c1ffe5f3 Only test with -race on supported platforms (fixes #2765) 2016-02-15 11:33:24 +01:00
Jakob Borg
e11302172e Report versioning usage in usage report
I consider it a bug that we didn't already and that this is covered
already under the agreement that we report which features are in use.
2016-02-13 08:19:30 +01:00
Audrius Butkevicius
bf353a42cd Merge pull request #2780 from letiemble/CORS_Support2
Move CORS middleware to process un-authenticated OPTIONS requests
2016-02-12 21:29:45 +00:00
Laurent Etiemble
d8e19b776e Swap the corsMiddleware and the csrfMiddleware to the unauthenticated OPTIONS requests are first processed. 2016-02-12 22:10:08 +01:00
Audrius Butkevicius
cf96bb464f Merge pull request #2777 from calmh/dbfile404
Return "No such object in the index" when /rest/db/file gets called on something that doesn't exist
2016-02-12 20:12:34 +00:00
Jakob Borg
3c7164846d Return "No such object in the index" when /rest/db/file gets called on something that doesn't exist
Better than the confusing result of getting a blank fileinfo that looks
valid apart from being all crap.
2016-02-12 14:55:16 +01:00
Jakob Borg
4fa4668ed6 Revert "Add .arcconfig to project root"
This reverts commit 0ce21aea08.
2016-02-11 21:17:01 +01:00
Jakob Borg
0ce21aea08 Add .arcconfig to project root 2016-02-09 16:50:57 +01:00
Audrius Butkevicius
6f2de31146 Merge pull request #2757 from calmh/newxdr
Use v2 of XDR package
2016-02-02 14:44:06 +00:00
Jakob Borg
e1ac740ac4 Use v2 of XDR package (actual changes) 2016-02-02 15:33:46 +01:00
Jakob Borg
4feeaf1641 Use v2 of XDR package (auto generated) 2016-02-02 12:44:33 +01:00
Jakob Borg
a08bbabd4d Use v2 of XDR package (deps) 2016-02-02 12:43:33 +01:00
Jakob Borg
a7a9d7d85c Return correct content type for /rest/events 2016-02-02 12:40:42 +01:00
Jakob Borg
e93c766c42 Rename RawAPIKey -> APIKey in GUIConfiguration 2016-02-02 11:12:25 +01:00
Audrius Butkevicius
5d4bfdabd6 Merge pull request #2755 from calmh/dashconfig
Add -paths option to print config, key, database paths
2016-02-02 09:53:28 +00:00
Jakob Borg
39c16d1cc4 Add -paths option to print config, key, database paths 2016-02-02 10:41:49 +01:00
Jakob Borg
eb55d19786 Clean up error handling a bit in protocol.readMessage 2016-02-02 10:18:19 +01:00
Jakob Borg
ae36fada6b Remove old reference to moved protocol 2016-02-02 10:18:18 +01:00
Audrius Butkevicius
60ca7784ba Merge pull request #2748 from canton7/feature/multiple-api-keys
Support multiple API keys (command-line and config) (fixed #2747)
2016-02-01 09:20:51 +00:00
Jakob Borg
7e8db13854 Update docs & translations 2016-01-31 10:38:05 +01:00
Jakob Borg
3e7d0ec14f build.sh prerelease should rebuild author credits in about dialog 2016-01-30 22:49:14 +01:00
Antony Male
5971c00a4f Support multiple API keys (command-line and config) (fixes #2747) 2016-01-30 15:18:09 +00:00
Audrius Butkevicius
8ff7531f89 Merge pull request #2749 from AudriusButkevicius/relayprx
Use dialer in relay checks (fixes #2732)
2016-01-30 12:39:13 +00:00
Audrius Butkevicius
f59e1ad854 Use dialer in relay checks (fixes #2732) 2016-01-30 12:33:42 +00:00
Audrius Butkevicius
1a0a8a1655 Merge pull request #2738 from tpng/patch-1
Handle null case for invalid ng-model value (fixes #2392)
2016-01-30 03:25:31 +00:00
Benny Ng
24023ff9e8 Handle null case for invalid ng-model value (fixes #2392)
Invalid ng-model value is assigned `null` by angular.js which is being matched as `object`, thus disappear in the UI when a minus sign is entered.
2016-01-30 10:40:57 +08:00
Audrius Butkevicius
016f799983 Merge pull request #2745 from calmh/redirect307
Return status code 307 instead of 302 when redirecting from HTTP to HTTPS
2016-01-29 15:06:01 +00:00
Jakob Borg
fae68a5396 Return status code 307 instead of 302 when redirecting from HTTP to HTTPS 2016-01-29 11:07:51 +01:00
Jakob Borg
79680b1d5e Benchmark for single database update 2016-01-28 09:12:01 +01:00
Jakob Borg
0ce45c20b8 Merge branch 'pr/2735'
* pr/2735:
  Add a CORS handler to deal with preflight OPTIONS requests
2016-01-26 21:56:06 +01:00
Laurent Etiemble
fed374fcb6 Add a CORS handler to deal with preflight OPTIONS requests 2016-01-26 21:55:51 +01:00
Jakob Borg
374202ac45 Add letiemble 2016-01-26 21:51:20 +01:00
Jakob Borg
56db1d3dfa Update docs and translations 2016-01-24 17:35:33 +01:00
Audrius Butkevicius
d4796261d7 Merge pull request #2729 from syncthing/rumpelsepp-patch-1
Correct order of pkill(1) arguments in debian script (fixes #2728)
2016-01-24 12:31:47 +00:00
Stefan Tatschner
452c5d5e91 Correct order of pkill(1) arguments in debian script (fixes #2728)
According to http://linux.die.net/man/1/pkill the signal name must be **before** anything else.
2016-01-24 13:10:30 +01:00
Jakob Borg
cc5f93e717 Merge pull request #2724 from plouj/master
rest/db/scan: Only scan the requested subdirectories/files.
2016-01-22 21:03:23 -08:00
Michael Ploujnikov
49601a63c8 Model.internalScanFolder: Don't ignore special .stfolder and .stignore files.
Fixes #2151.

Since Walk.walkAndHashFiles ignores .stfolder and .stignore, they will
never be found by fs.Get(protocol.LocalDeviceID, sub) in
Model.internalScanFolder. As a result, when asked to scan those subs
we end up scanning the whole folder.
2016-01-22 23:27:47 -05:00
Michael Ploujnikov
6c33188af3 Model.internalScanFolderSubs: Scan only requested subs.
This reverts the change introduced in 9b9fe0d Reduce scanning effort.
That commit caused us to automatically ignore the basename of the
specified subs and instead scan closest known root folder. For
example, in a folder that looks like:

Sync/
├── 00
│   ├── one
│   ├── three
│   └── two
├── 01
│   ├── one
│   ├── three
│   └── two
├── 02
│   ├── one
│   ├── three
│   └── two
└── one

calling '/rest/db/scan?folder=default&sub=01' called filepath.Walk on
the whole Sync/ folder instead of just the desired subfolder. This
contradicts the scan behavior promised by the documentation.

This is related to #2151.
2016-01-22 23:27:38 -05:00
Jakob Borg
1353e3916f A couple of protocol tests 2016-01-20 11:37:48 -08:00
Jakob Borg
11d4986517 Humanize serialization of version vectors (again) 2016-01-20 11:14:08 -08:00
Audrius Butkevicius
e267bf3e09 Merge pull request #2715 from plouj/master
FetchLatestReleases: fix the error log message
2016-01-20 08:41:52 +00:00
Michael Ploujnikov
39c5c8c1d1 FetchLatestReleases: fix the error log message 2016-01-19 21:32:33 -05:00
Jakob Borg
6cce073da5 Merge pull request #2713 from calmh/debrestart
Restart on Debian package upgrade
2016-01-19 10:20:47 -08:00
Jakob Borg
99372c69e5 Add postinst script to restart after upgrade 2016-01-19 10:13:45 -08:00
Jakob Borg
042b703fe4 Templatize Debian files 2016-01-19 10:06:16 -08:00
Audrius Butkevicius
1880284bde Merge pull request #2711 from calmh/fix2704
Don't require restart for usage reporting changes (fixes #2704)
2016-01-18 18:12:58 +00:00
Jakob Borg
6c1faa4bdb Don't require restart for usage reporting changes (fixes #2704) 2016-01-18 10:06:31 -08:00
Audrius Butkevicius
33f97d7d8f Merge pull request #2708 from Zillode/fix-typo
RLimit comment typo
2016-01-17 12:06:47 +00:00
Lode Hoste
82e033942e RLimit comment typo
Thanks @plouj
2016-01-17 12:54:44 +01:00
Audrius Butkevicius
693e1c93f1 Merge pull request #2707 from calmh/notok
The "OK" log level is silly and should not exist
2016-01-16 22:21:06 +00:00
Jakob Borg
2919b76947 The "OK" log level is silly and should not exist 2016-01-16 23:04:41 +01:00
Audrius Butkevicius
42b94561a2 Merge pull request #2706 from calmh/fix2705
Don't crash on folder remove while pulling (fixes #2705)
2016-01-16 20:54:43 +00:00
Jakob Borg
acaf134dfe Don't crash on folder remove while pulling (fixes #2705) 2016-01-16 21:42:32 +01:00
Jakob Borg
ab9109e0dc Only print codesign success if we tried to codesign 2016-01-16 19:59:01 +01:00
Audrius Butkevicius
f88b2c11fe Merge pull request #2702 from calmh/codesign
Codesign binaries in Mac OS X distribution packages
2016-01-16 18:57:46 +00:00
Jakob Borg
d5d330413b Codesign binaries in Mac OS X distribution packages 2016-01-16 19:50:04 +01:00
Jakob Borg
61d7e11001 Merge pull request #2701 from AudriusButkevicius/raceee
Handle race while in the job queue (fixes #1263)
2016-01-16 18:59:50 +01:00
Audrius Butkevicius
c4c6df179b Handle race within the job queue (fixes #1263) 2016-01-16 17:20:21 +00:00
Audrius Butkevicius
b04b7bf357 Merge pull request #2696 from calmh/fix2694
Improve API/GUI shutdown handling (fixes #2694)
2016-01-14 10:38:06 +00:00
Jakob Borg
97b1c66d4a Improve API/GUI shutdown handling (fixes #2694)
This fixes both a race condition where we could assign s.stop from one
goroutine and then read it from another without locking, and handles the
fact that listener may be nil at shutdown if we've had a bad
CommitConfiguration call in the meantime.
2016-01-14 11:06:36 +01:00
Jakob Borg
74a210f198 Fix asset locations in one more place 2016-01-13 21:11:46 +01:00
Audrius Butkevicius
31861052e5 Merge pull request #2687 from kluppy/master
Fix #2662 update Edit menu to Action
2016-01-13 11:02:40 +00:00
kluppy
7c42b5cb17 Update 'Edit' menu to 'Action' menu (fixes #2662) 2016-01-13 20:09:32 +10:00
kluppy
df7fea4412 Fix location of build translation scripts. 2016-01-13 20:09:32 +10:00
Audrius Butkevicius
5fa8b42fac Merge pull request #2690 from calmh/fix2665
Always run relaying when enabled (fixes #2665)
2016-01-12 13:48:22 +00:00
Audrius Butkevicius
d3fa67fe2e Merge pull request #2689 from calmh/nohashalgo
Undo the hash algorithm additions, retain flag checks
2016-01-12 13:46:34 +00:00
Jakob Borg
357089a438 Mend protocol tests, for sure 2016-01-12 14:35:00 +01:00
Jakob Borg
8b3d75b339 Undo the hash algorithm additions; retain flag checks 2016-01-12 14:35:00 +01:00
Jakob Borg
f741066466 Always run relaying when enabled (fixes #2665) 2016-01-12 14:15:47 +01:00
Audrius Butkevicius
1e45111bde Merge pull request #2688 from calmh/prototests
Improve protocol tests, close handling
2016-01-12 08:44:02 +00:00
Jakob Borg
9595687bce Improve protocol tests, close handling 2016-01-12 09:30:02 +01:00
Audrius Butkevicius
7427b9de35 Merge pull request #2684 from calmh/fix2589
Don't leak sendIndexes on disconnect (fixes #2589)
2016-01-11 17:34:53 +00:00
Jakob Borg
acdddc0b79 Don't leak sendIndexes on disconnect (fixes #2589)
Adds a Closed() method on protocol.Connection and clears up
wireformatConnection a little too.
2016-01-11 17:57:25 +01:00
Jakob Borg
01c70caa8f Merge pull request #2682 from AudriusButkevicius/themes
Add theme support
2016-01-10 19:04:57 +01:00
Audrius Butkevicius
ff4bab4c07 Silence the linter 2016-01-10 18:00:52 +00:00
alessandro.g89
5c36029274 Add dark theme by alessandro.g89
Source: https://userstyles.org/styles/122502/syncthing-dark
2016-01-10 18:00:44 +00:00
Audrius Butkevicius
cd54186113 Add support for themes (fixes #1925) 2016-01-10 17:57:27 +00:00
Jakob Borg
353689857e Fix version string check to allow properly tagged betas 2016-01-10 18:41:15 +01:00
Jakob Borg
d74377350b v0.13.0 is the Copper Cockroach 2016-01-10 10:12:29 +01:00
Audrius Butkevicius
6c0a973ac3 Merge pull request #2480 from calmh/shortdblabels
Change database folder label format
2016-01-10 01:33:56 +00:00
Jakob Borg
3ca46c29c3 Merge pull request #2680 from calmh/xunit
Add XUnit compatible test results
2016-01-10 01:00:07 +01:00
Jakob Borg
837fde70ae Add XUnit compatible test output 2016-01-10 00:50:43 +01:00
Jakob Borg
1e52cc474f Merge pull request #2677 from nrm21/default_sync
Added STNODEFAULTFOLDER envvar to skip default folder creation
2016-01-09 13:24:53 +01:00
Nate Morrison
76807006be Added STNODEFAULTFOLDER envvar to skip default folder creation on new
install.
2016-01-08 20:11:06 -05:00
Audrius Butkevicius
0d35fe0f21 Merge pull request #2676 from calmh/fix2667
More fine grained locking in discovery cache (fixes #2667)
2016-01-09 00:58:14 +00:00
Jakob Borg
370b0fc5da More fine grained locking in discovery cache (fixes #2667)
We only need to protect the integrity of the "finders" and "caches"
slices, and for that we only need an RLock except while actually
appending to them. The actual finders and caches are concurrency safe on
their own.
2016-01-09 00:56:03 +01:00
Audrius Butkevicius
25b3c09f6a Merge pull request #2670 from calmh/noresolve
Don't unnecessary resolve destination address
2016-01-07 13:27:31 +00:00
Jakob Borg
576c365753 Don't resolve destination address until we need to (fixes #2671) 2016-01-07 12:32:10 +01:00
Jakob Borg
bc2ed60b92 Translations for relaying stuff 2016-01-07 10:28:21 +01:00
Jakob Borg
a4385100c4 HTML attribute typo 2016-01-05 11:19:27 +01:00
Audrius Butkevicius
4c01709cdf Merge pull request #2664 from calmh/fix2433
Add relaying to main settings dialog (fixes #2433)
2016-01-05 09:25:11 +00:00
Jakob Borg
1df924f4f8 Add relaying to main settings dialog (fixes #2433) 2016-01-05 10:08:56 +01:00
Audrius Butkevicius
400bfe9251 Merge pull request #2656 from calmh/csrf
Don't allow in use CSRF tokens to expire (fixes #1008)
2016-01-03 21:24:37 +00:00
Jakob Borg
6e1d364d60 Don't allow in use CSRF tokens to expire (fixes #1008) 2016-01-03 22:03:02 +01:00
Audrius Butkevicius
504ad86648 Merge pull request #2655 from calmh/fix2605
Don't conflict copy conflict copies (fixes #2605)
2016-01-03 20:44:05 +00:00
Jakob Borg
096b2d73cd Don't conflict copy conflict copies (fixes #2605) 2016-01-03 21:16:31 +01:00
Audrius Butkevicius
5754d31d0f Merge pull request #2653 from calmh/fix2604
Change default max conflicts to 10 (fixes #2604)
2016-01-03 20:13:51 +00:00
Jakob Borg
ed3ed1f90a Change default max conflicts to 10 (fixes #2604) 2016-01-03 21:10:25 +01:00
Jakob Borg
adb1227b2e Update kardianos/osext (fixes #2650) 2016-01-03 19:59:56 +01:00
Jakob Borg
ac190b2e39 Change DB label format (index folders, devices) 2016-01-03 19:32:40 +01:00
Jakob Borg
18ae87962d Clarify points of contact 2016-01-03 13:07:22 +01:00
Jakob Borg
4673862981 Update docs & translations 2016-01-03 09:56:33 +01:00
Jakob Borg
212d7257e9 Pretty print the new architectures 2016-01-01 21:19:15 +01:00
Audrius Butkevicius
9df8d0848a Merge pull request #2641 from calmh/archs
Also build linux-arm64, linux-ppc64, linux-ppc64le
2016-01-01 20:08:08 +00:00
Jakob Borg
ded7abb1f6 Also build linux-arm64, linux-ppc64, linux-ppc64le 2016-01-01 20:56:31 +01:00
Audrius Butkevicius
44d5a61cfe Merge pull request #2639 from calmh/minihashalgo
Detect nonstandard hash algo and stop folder (ref #2314)
2016-01-01 19:42:08 +00:00
Jakob Borg
0db80710aa Detect nonstandard hash algo and stop folder (ref #2314) 2016-01-01 20:14:31 +01:00
Jakob Borg
7c47eff112 Update lang-en.json for translations 2016-01-01 20:13:44 +01:00
Jakob Borg
d81849ab3b Merge pull request #2631 from AudriusButkevicius/igwarn
Don't warn about failed ignores if folder unhealthy (fixes #2630)
2016-01-01 16:48:31 +01:00
Audrius Butkevicius
dbb3f80995 Merge pull request #2193 from AudriusButkevicius/lans
Remove windows specialisation from osutil.GetLans (fixes #2192)
2016-01-01 15:01:23 +00:00
Audrius Butkevicius
80b9a3e00b Don't warn about failed ignores if folder unhealthy (fixes #2630) 2016-01-01 12:59:13 +00:00
Jakob Borg
be5e5d837b Merge branch 'pr/2625'
* pr/2625:
  Show device ID QR code from edit dialog (fixes #1494)
2016-01-01 11:24:49 +01:00
Kevin Allen
ffc4a60bc6 Show device ID QR code from edit dialog (fixes #1494) 2016-01-01 11:24:09 +01:00
Audrius Butkevicius
1edfa4474f Merge pull request #2632 from calmh/fix2627
Ensure loaded config is free of duplicate devices (fixes #2627)
2015-12-31 08:37:52 +00:00
Jakob Borg
18e70f4e79 Ensure loaded config is free of duplicate devices (fixes #2627) 2015-12-31 09:17:17 +01:00
Jakob Borg
62a81cfdd1 Update lang-en.json and assets 2015-12-30 08:54:09 +01:00
Jakob Borg
6365a026c1 Merge branch 'pr/2628'
* pr/2628:
  Disallow adding duplicate device ID in GUI
2015-12-30 08:53:29 +01:00
Jakob Borg
7c89193398 Add ironmig 2015-12-30 08:53:19 +01:00
Kevin Allen
4a6f1718b8 Disallow adding duplicate device ID in GUI
Adds check in valid device id to check for uniqueness
2015-12-30 08:52:11 +01:00
Jakob Borg
7b3fa8da60 Update translations & docs 2015-12-27 13:26:14 +01:00
Jakob Borg
2fae7ccf5c Ignore error on os.Chtimes in config archiving 2015-12-26 18:25:36 +01:00
Audrius Butkevicius
f36f48c2cf Merge pull request #2596 from andersonvom/load_config
Centralize config loading logic (also fixes #2509)
2015-12-26 16:12:16 +00:00
Jakob Borg
8b726c7e8b Correct GUI asset dir handling (fixes #2621) 2015-12-26 13:32:46 +01:00
Anderson Mesquita
8eb0687407 Update mtime of config file before upgrading (fixes #2509)
This updates the modified time of the config file before archiving it
during an update so that the clean up routine doesn't delete it if it's
too old, preventing the user from being able to rollback after an
upgrade.
2015-12-24 09:27:31 -05:00
Anderson Mesquita
9b9912ba9e Centralize config loading logic
This gets rid of redundant checks and centralizes the logic of loading
the config files so that we don't have to keep doing the same thing in
multiple places.
2015-12-24 09:27:31 -05:00
Anderson Mesquita
22d0ed8225 Fix typo 2015-12-24 09:27:31 -05:00
Jakob Borg
b7a58d2f87 Merge pull request #2620 from andersonvom/issue-2454
Remove fixed footer at first media break (fixes #2454)
2015-12-24 11:26:40 +01:00
Anderson Mesquita
219ece22fc Remove fixed footer at first media break (fixes #2454)
After the first media break (under 1200px), the footer is too long to
fit in a single line, taking up too much space in small screen devices.
This makes it so that it will stop being fixed at the bottom, freeing up
valuable screen real estate.
2015-12-23 22:19:41 -05:00
Jakob Borg
4874301615 Merge pull request #2618 from AudriusButkevicius/svc
Svc -> Service
2015-12-23 21:33:42 +01:00
Audrius Butkevicius
1827dda0c6 Svc -> Service 2015-12-23 15:31:12 +00:00
Jakob Borg
d088b01f75 Rebuild assets 2015-12-23 16:09:42 +01:00
Jakob Borg
00c5062eab Merge branch 'pr/2613'
* pr/2613:
  Add an icon for Safari's pinned tabs
2015-12-23 16:09:31 +01:00
Jakob Borg
84eacde63a Add jgke 2015-12-23 16:09:11 +01:00
Jakob Borg
d98290c17f Merge pull request #2617 from andersonvom/issue-2598
Refactor rwfolder tests
2015-12-23 07:55:47 +01:00
Anderson Mesquita
6d94a3be05 Refactor rwfolder tests
This creates a few utility functions to avoid repetition and removes
some redundant checks.
2015-12-22 23:43:07 -05:00
Audrius Butkevicius
a2833d18ed Merge pull request #2615 from calmh/jsonvv
Humanize serialization of version vectors
2015-12-22 21:12:23 +00:00
Jakob Borg
6f95afdc59 Humanize serialization of version vectors 2015-12-22 21:53:25 +01:00
Jakob Borg
aaa75a32a5 Temporarily patch kardianos/osext to build on FreeBSD 2015-12-22 13:41:04 +01:00
Jaakko Hannikainen
c300015ac2 Add an icon for Safari's pinned tabs
Safari has its own standard for handling icons for pinned tabs,
which requires a black-and-white .svg and a special tag.
Without using this, pinning a tab to localhost will show just
a blank square, instead of a pre-generated letter.
2015-12-22 12:29:59 +02:00
Jakob Borg
eb4f5e9faa Update deps, step two (because I suck) 2015-12-22 09:43:47 +01:00
Jakob Borg
c7aec839ae Merge pull request #2614 from andersonvom/issue-2598
WIP: Consider tempfile when checking for free space (fixes #2598)
2015-12-22 09:42:34 +01:00
Jakob Borg
37aa89b19d Update kardianos/osext (ref #1272) 2015-12-22 09:39:47 +01:00
Anderson Mesquita
3f94e70488 WIP: Consider tempfile when checking for free space (fixes #2598)
Checks the existing blocks that can be reused when downloading a file so
that it only requires the space corresponding to the missing blocks.
This will prevent syncthing from claiming the folder doesn't have enough
space when resuming download of large files after they have been
partially downloaded.
2015-12-21 13:36:08 -05:00
Jakob Borg
435afa0eea Merge pull request #2610 from calmh/fix2608-2
Correctly set default logfile location on Windows (fixes #2608)
2015-12-21 12:26:49 +01:00
Jakob Borg
fb82a5e086 Correctly set default logfile location on Windows (fixes #2608) 2015-12-21 12:19:28 +01:00
Audrius Butkevicius
71d98c2f26 Merge pull request #2609 from calmh/fix2608
Don't crash on stat error in ensureDir (fixes #2608)
2015-12-21 10:35:55 +00:00
Jakob Borg
4a97aa12d6 Don't crash on stat error in ensureDir (fixes #2608)
I'm not really sure under what circumstances MkdirAll returns a nil
error but a subsequent stat fails, but apparently it can happen and we
need to handle it. The "mode >= 0" was a no-op, and we never call
ensureDir anyway without the intention of ensuring the mode, so removed
that.
2015-12-21 09:35:43 +01:00
Audrius Butkevicius
c9e67fb460 Log when we fail to connect to relay 2015-12-20 22:14:13 +00:00
Audrius Butkevicius
cec87be4e3 Remove windows specialisation from osutil.GetLans (fixes #2192) 2015-12-20 18:10:02 +00:00
Jakob Borg
eb1a234a77 Translation & docs update 2015-12-20 10:05:17 +01:00
Jakob Borg
394c2b67d6 Merge pull request #2595 from AudriusButkevicius/leaks
Always exit via error select, making sure reader routine is exits (fi…
2015-12-20 10:03:34 +01:00
Jakob Borg
aeb3af7105 Merge branch 'pr/2594'
* pr/2594:
  Move event timestamp to header for share folder prompt
  Edit device after accepting new connection (fixes #1929)
  Move new device timestamp to box header
  Fix unmatching closing span tag
2015-12-20 09:53:26 +01:00
Jakob Borg
e5cf99e31c Move event timestamp to header for share folder prompt 2015-12-20 09:52:59 +01:00
Anderson Mesquita
8120535a35 Edit device after accepting new connection (fixes #1929)
This will open the "edit device" dialogue after accepting a new device
connection. This will allow the user to specify the name of the device
or leave it blank in case they want to accept whatever the device
advertises once it connects.
2015-12-19 09:48:27 -05:00
Anderson Mesquita
ba01433381 Move new device timestamp to box header
The device ID is already long enough to make it a bit hard to read the
body message, so moving the timestamp to the header unclutters the body
a bit.
2015-12-19 09:42:33 -05:00
Anderson Mesquita
6b99cdb83a Fix unmatching closing span tag 2015-12-19 09:42:33 -05:00
Audrius Butkevicius
e7b71f8743 Merge pull request #2601 from calmh/fix2600
Don't verify free space for files when folder MinDiskFreePct==0 (fixes #2600)
2015-12-19 13:02:57 +00:00
Jakob Borg
03935b2d64 Don't verify free space for files when folder MinDiskFreePct==0 (fixes #2600) 2015-12-19 13:59:08 +01:00
Jakob Borg
2b89f33765 Merge pull request #2592 from andersonvom/remove_globals
Remove some global variables from main.go
2015-12-19 09:05:05 +01:00
Jakob Borg
9c82a4ca60 Merge pull request #2593 from AudriusButkevicius/httpfoo
Only override TLS settings if scheme is specified
2015-12-19 08:47:46 +01:00
Anderson Mesquita
7eb9b9b1c6 Merge pull request #2597 from andersonvom/html
Fix alignment of settings fields
2015-12-18 22:48:00 -05:00
Anderson Mesquita
69b35b2ede Fix alignment of settings fields
When upgrade info is not available and the "Automatic Upgrades" options
is hidden, then "Global Discovery Server" wraps around and gets
misaligned. This fixes all that.
2015-12-18 22:40:56 -05:00
AudriusButkevicius
e8016abd97 Always exit via error select, making sure reader routine is exits (fixes #2547) 2015-12-18 22:29:04 +00:00
Anderson Mesquita
e54036be25 Reuse existing ensureDir function 2015-12-18 16:24:48 -05:00
Anderson Mesquita
1dc894087c Rename noConsole to hideConsole
This avoids the double negative of having noConsole = false to represent
not hiding the console. It is also consistent with the action performed
by osutils.
2015-12-18 16:24:48 -05:00
Anderson Mesquita
b014967550 Rename guiAssets to assetDir for consistency 2015-12-18 16:24:48 -05:00
Anderson Mesquita
490962ccdb Move logFlags into RuntimeOptions 2015-12-18 16:24:48 -05:00
Anderson Mesquita
d10e81fb3d Remove most global variables from main.go
This takes advantage of the newly created parseCommandLineOptions()
function and makes it work so that it now returns a nice struct of
options rather than relying on global variables.

There are a few global variables left, but they will take a bit more
refactoring in order to be removed, so it'll happen in later commits.
2015-12-18 16:24:44 -05:00
AudriusButkevicius
33a87f54bb Only override TLS settings if scheme is specified 2015-12-18 19:42:39 +00:00
Jakob Borg
2cdfa59fbe Merge pull request #2588 from andersonvom/browser-only
Add command line option to open GUI (fixes #2210)
2015-12-17 08:41:31 +01:00
Jakob Borg
bebe74fa4a Merge pull request #2590 from andersonvom/refactor
Refactor main.go
2015-12-17 08:34:21 +01:00
Anderson Mesquita
4098f97735 Extract cmdline option parsing into a new function
Another step towards reducing the general size of the main() function
in favor of shorter, more focused functions.
2015-12-16 17:19:32 -05:00
Anderson Mesquita
a0b7ac402d Refactor main.ensureDir()
ensureDir() did not handle one last error case and there was some logic
in the main() function that belonged to ensureDir() as well. It was also
creating a directory with a hardcoded 0700 mode, regardless of what mode
was passed to it.

This refactors it a little to fix the broken behavior, avoid redundant
checks by taking advantage of the behavior of MkdirAll, and move the
extra logic from main() into ensureDir().
2015-12-16 17:19:26 -05:00
Anderson Mesquita
0ca4482977 Refactor upgrade and check upgrade cmdline options
The main() function is growing too big (142 lines as of the date of this
commit), so this attempts to extract some functionality out of there and
into their own functions to make it easier to reason about them and keep
functions short and concise.
2015-12-16 14:29:37 -05:00
Anderson Mesquita
5cf15db6e4 Add command line option to open GUI (fixes #2210)
--browser-only assumes syncthing is already running and will open the
user's browser in the URL:port currently used in the configuration if
the GUI is enabled.
2015-12-16 13:24:21 -05:00
Jakob Borg
6379d50f07 Sort and retabify AUTHORS & NICKS 2015-12-16 09:16:16 +01:00
Jakob Borg
5a2792ae24 Merge branch 'pr/2587'
* pr/2587:
  Add helper function to format JSON responses
2015-12-16 09:08:16 +01:00
Jakob Borg
9b26f3cd84 Add andersonvom 2015-12-16 09:08:03 +01:00
Anderson Mesquita
316be5ee34 Add helper function to format JSON responses
Every time a JSON object is returned in an HTTP response, the
appropriate header needs to be set and the object itself needs to be
encoded. Doing this in every function is repetitive and error prone
(getDBFile and postDBScan, for instance, never set any headers).

This adds a helper function to centralize the appropriate JSON response
handling.
2015-12-16 09:06:17 +01:00
Audrius Butkevicius
f208e6f0b6 Fix typos 2015-12-15 20:11:07 +00:00
Jakob Borg
7c8c131e1a Merge pull request #2582 from Zillode/debug-2580
Additional output on insufficient error (ref #2580)
2015-12-15 10:19:59 +01:00
Jakob Borg
81e71d7275 Merge pull request #2576 from AudriusButkevicius/reconn
More debug to reconnect loop
2015-12-15 10:18:47 +01:00
Audrius Butkevicius
fc6c4b8e16 Merge pull request #2583 from calmh/globerrors
Better error messages for globbing errors
2015-12-15 09:13:49 +00:00
Jakob Borg
2280566bca Better error messages for globbing errors 2015-12-15 10:04:13 +01:00
Jakob Borg
33173e76a0 Merge pull request #2581 from rumpelsepp/master
systemd: Remove exit code 2, due to #2578
2015-12-15 09:10:36 +01:00
Lode Hoste
cc0b9e5088 Additional output on insufficient error (ref #2580) 2015-12-15 08:45:33 +01:00
Stefan Tatschner
ecc72d7693 systemd: Remove exit code 2, due to #2578
PR #2578 enables us to remove the exit code 2 from the list of
success status codes, because SIGINT will be handled properly.
I have also converted STNORESTART to --no-restart for the sake
of consistency.
2015-12-15 08:38:08 +01:00
Jakob Borg
4ab4aeacb0 Example GUI override address (fixes #2530) 2015-12-14 14:39:07 +01:00
Audrius Butkevicius
348d12bd60 Merge pull request #2578 from calmh/sighup
Implement signal handling
2015-12-14 12:18:51 +00:00
Jakob Borg
7b686c1103 Implement signal handling
With this change, the behavior is as follows:

 - SIGTERM / SIGINT: Exit cleanly (exit code 0)
 - SIGHUP: Restart

This is the case both when hitting the monitor process or the inner
process, or if running with NORESTART (but then we won't restart,
obviously, just exit with code exitRestarting).

It enables "pkill -HUP syncthing" to do the right thing to restart all
Syncthings on package upgrade, for example.
2015-12-14 13:07:42 +01:00
Audrius Butkevicius
403583cfbb More debug to reconnect loop 2015-12-13 12:20:25 +00:00
Jakob Borg
29bff06cd6 Translation update 2015-12-13 10:39:09 +01:00
Jakob Borg
2b80057ac9 Minimal error handling refactor 2015-12-13 10:38:21 +01:00
Jakob Borg
c3625e16d7 Add wkennington 2015-12-13 10:35:34 +01:00
Audrius Butkevicius
f4642e9e66 Merge pull request #2574 from wkennington/master.upnp
upnp: Use a separate error for the error unmarshalling
2015-12-12 18:25:27 +00:00
William A. Kennington III
5bdf4c6143 upnp: Use a separate error for the error unmarshalling
Previously, when unmarshing the SOAP error code data we would overwrite
the original err, typically with null since the parsing of the error
code information succeeds. If we don't have a upnp 725 error, we would fall
back to returning null or no error. This broke our upnp error handling
logic for AddPortMappings as it would think it succeeds if it gets a 718
permission error.
2015-12-12 09:58:23 -08:00
Jakob Borg
1250850492 Must close result sets 2015-12-09 09:55:49 +01:00
Jakob Borg
80aaf6a065 Merge pull request #2513 from calmh/noandroidhax
Remove Android hacks (fixes #2505)
2015-12-08 12:53:10 +01:00
Jakob Borg
3025caf932 Also remove IP per disco server 2015-12-08 10:10:34 +01:00
Jakob Borg
95cfc50fbd Remove Android hacks (fixes #2505)
Also fixes what I think migh thave been a bug where we did not use the
proxy for usage reports. And removes the BuildEnv field that we don't
need any more.
2015-12-08 10:10:06 +01:00
Jakob Borg
ded0925155 Neater build.sh all 2015-12-08 09:33:34 +01:00
Jakob Borg
ebfef15fb0 Add new dependencies 2015-12-08 09:19:16 +01:00
Jakob Borg
931408037c Build should use -i (keep built pkg:s) and -v (verbose) 2015-12-08 09:14:52 +01:00
Jakob Borg
ad418abf91 Merge pull request #25 from syncthing/fixes
Fix ALL THE BUGS
2015-12-07 13:36:49 +01:00
Audrius Butkevicius
c7d51a26f6 Merge pull request #26 from canton7/feature/better-logging
Add more logging in the case of relaypoolsrv internal server error
2015-12-07 11:18:34 +00:00
Antony Male
2c01cc000e Add more logging in the case of relaypoolsrv internal server error
It's useful to know *why* relaypoolsrv returns an internal server error
2015-12-07 11:11:01 +00:00
Jakob Borg
3318651565 Correct type assertion in verbose logger, restart (fixes #2561) 2015-12-07 08:04:09 +01:00
Audrius Butkevicius
2ab07f3aac Merge pull request #2558 from canton7/feature/logging-debug
Only print 'Enabled/Disabled debug data for ...' if it was enabled/disabled
2015-12-06 17:57:17 +00:00
Antony Male
1340e54327 Only print 'Enabled/Disabled debug data for ...' if it was enabled/disabled 2015-12-06 17:15:55 +00:00
Jakob Borg
1b6e4645b1 Update docs & translations 2015-12-06 13:09:22 +01:00
Jakob Borg
a6a573f5dc Fix STTRACE=http (it should use the http debug logger) 2015-12-05 15:59:35 +01:00
Jakob Borg
415415b5b2 Response with Bad Request to bad config posts, not Internal Server Errors
The error is the clients fault, or so we think, so we should not accept
the blame.
2015-12-05 15:56:10 +01:00
Jakob Borg
175769b53e Update dependencies 2015-12-04 15:27:55 +01:00
Jakob Borg
22f193f042 Dependency update 2015-12-04 15:20:01 +01:00
Audrius Butkevicius
38e9b92c42 Merge pull request #2550 from canton7/feature/deadline
staticClient.connect(): don't handshake twice (fixes #2547, #2548)
2015-12-04 12:05:28 +00:00
Antony Male
345d727936 staticClient.connect(): don't handshake twice (fixes #2547, #2548)
The first handshake occurred before setting the Deadline, which could
cause an unintended hang.
2015-12-04 12:01:01 +00:00
Nate Morrison
86e8e5199e Filenames added to audit log in the LocalIndexUpdated event (fixes #2549) 2015-12-04 08:42:00 +01:00
Jakob Borg
739979a116 Add nrm21 2015-12-04 08:38:45 +01:00
Jakob Borg
7cbc81adca Merge pull request #2545 from rumpelsepp/master
Add syncthing-relay(7)
2015-12-01 15:54:10 +01:00
Stefan Tatschner
737d0fa23c Add syncthing-relay(7) 2015-12-01 15:34:58 +01:00
Jakob Borg
ab1962934d Docs & translations update 2015-12-01 11:24:30 +01:00
Jakob Borg
192455702d Repair AUTHORS 2015-12-01 11:23:35 +01:00
Jakob Borg
cb0d739daf New key for discovery-*-3 2015-12-01 11:20:16 +01:00
Jakob Borg
a937fcc477 Lol shadowing :( 2015-12-01 11:20:16 +01:00
Audrius Butkevicius
9503e60444 Merge pull request #2544 from calmh/negcache2
Accept Retry-After header on discovery lookup failures
2015-12-01 10:14:32 +00:00
Jakob Borg
9f2dc4554d Accept Retry-After header on discovery lookup failures 2015-12-01 11:10:03 +01:00
Audrius Butkevicius
55da600433 Merge pull request #33 from syncthing/negcache
Set Retry-After header
2015-12-01 09:58:20 +00:00
Jakob Borg
96b5c2ae00 Set Retry-After header 2015-12-01 10:49:16 +01:00
Jakob Borg
3008dc76d3 Add buinsky 2015-11-30 18:30:36 +01:00
Audrius Butkevicius
596c4b77e8 Merge pull request #2539 from buinsky/master
Fix deleting folders on WinXP (fixes #2522)
2015-11-30 16:21:46 +00:00
buinsky
05a31c2686 Fix deleting folders on WinXP (fixes #2522) 2015-11-30 19:09:36 +03:00
Audrius Butkevicius
45535c0f5a Merge pull request #2538 from canton7/feature/issue-2537
Allow #urPreview to scroll in the browser (fixes #2537)
2015-11-30 13:36:28 +00:00
Antony Male
4bd0dd2123 Allow #urPreview to scroll in the browser (fixes #2537)
This is the same issue as #2014/#2062. Bootstrap doesn't like having two dialogs
open at once: it marks the body has having no dialogs open when the first dialog
is closed, regardless of whether the second dialog is still open.

This means that scrolling doesn't happen properly, and the user cannot
scroll to the dialog's 'close' button.

Work around this by making sure the first dialog (the settings page) is fully closed
before the second dialog (usage preview) is opened.
2015-11-30 13:27:07 +00:00
Jakob Borg
9b2a643626 Update docs and translations 2015-11-29 08:55:38 +01:00
Jakob Borg
1ebc9a9a88 Merge pull request #2525 from kluppy/master
Don't chmod in Atomic on android (fixes  #2472)
2015-11-28 22:57:00 +01:00
Jakob Borg
f0c8b7ce40 Add kluppy 2015-11-28 22:17:05 +01:00
Jakob Borg
ec54550f21 Merge pull request #2529 from AudriusButkevicius/fixlinks
Fix symlinks (fixes #2524)
2015-11-28 21:29:33 +01:00
Audrius Butkevicius
4474d200b0 Fix symlinks (fixes #2524) 2015-11-28 20:23:08 +00:00
Audrius Butkevicius
b24a9e57fd Update deps 2015-11-27 21:04:40 +00:00
Audrius Butkevicius
07722dc33d Hey look, had to check all code out on linux to fix the deps 2015-11-27 21:02:19 +00:00
kluppy
f062e35641 Don't chmod in Atomic on android (fixes #2472) 2015-11-28 02:46:06 +10:00
Jakob Borg
321ef9816c Merge pull request #2521 from AudriusButkevicius/dialtimeout
Add dialer.DialTimeout, use that when connecting to relays
2015-11-27 10:01:21 +01:00
Audrius Butkevicius
be01e925c7 Merge pull request #2523 from calmh/ecdsa
Generate ECDSA keys instead of RSA
2015-11-27 08:28:51 +00:00
Jakob Borg
6d11006b54 Generate ECDSA keys instead of RSA
This replaces the current 3072 bit RSA certificates with 384 bit ECDSA
certificates. The advantage is these certificates are smaller and
essentially instantaneous to generate. According to RFC4492 (ECC Cipher
Suites for TLS), Table 1: Comparable Key Sizes, ECC has comparable
strength to 3072 bit RSA at 283 bits - so we exceed that.

There is no compatibility issue with existing Syncthing code - this is
verified by the integration test ("h2" instance has the new
certificate).

There are browsers out there that don't understand ECC certificates yet,
although I think they're dying out. In the meantime, I've retained the
RSA code for the HTTPS certificate, but pulled it down to 2048 bits. I
don't think a higher security level there is motivated, is this matches
current industry standard for HTTPS certificates.
2015-11-27 09:15:12 +01:00
Audrius Butkevicius
ed792b97c0 Take timeout into account when dialing 2015-11-26 23:41:11 +00:00
Audrius Butkevicius
a4b8c2298a Add dialer.DialTimeout support 2015-11-26 23:31:37 +00:00
Jakob Borg
e5b33ce9f6 Regenerate XDR for empty struct types 2015-11-24 20:54:49 +01:00
Jakob Borg
30374d46c9 Merge pull request #2510 from plouj/master
Log problems fetching releases
2015-11-24 20:39:29 +01:00
Michael Ploujnikov
9edf8233f7 Improve upgrade error messages 2015-11-24 08:29:42 -05:00
Michael Ploujnikov
bd4a14519c FetchLatestReleases: just log the error here
Since the return value is being ignored by the caller.
2015-11-24 08:29:42 -05:00
Michael Ploujnikov
f12bf8c09a Rename LatestGithubReleases -> FetchLatestReleases 2015-11-24 08:29:42 -05:00
Audrius Butkevicius
53cc45a0d5 Merge pull request #2512 from calmh/compact
Compact database on startup (ref #2400)
2015-11-24 12:43:38 +00:00
Jakob Borg
fa4b4dece1 Compact database on startup (ref #2400) 2015-11-24 13:17:30 +01:00
Jakob Borg
02f044a2a1 Add plouj 2015-11-24 08:35:25 +01:00
Audrius Butkevicius
f39f816a98 Update godeps, reduce amount of time spent testing a relay. Goddamit godeps. 2015-11-23 21:33:22 +00:00
Audrius Butkevicius
bc5b95be8a Update packages, fix testutil. Goddamit godep. 2015-11-23 21:29:23 +00:00
Audrius Butkevicius
431d51f5c4 Add timeouts to relay methods 2015-11-23 21:14:46 +00:00
Jakob Borg
45c1357bab Update osext dependency (fixes #1272) 2015-11-23 13:09:42 +01:00
Audrius Butkevicius
845f31b98f Add timeouts, deal with overlapping markers, add a table, increase circle radiuses 2015-11-22 22:47:48 +00:00
Audrius Butkevicius
89b6c32cee Merge pull request #6 from canton7/feature/fix-map
Fix a couple of issues with the relays map (geoip, 'data unavailable')
2015-11-22 14:27:58 +00:00
Antony Male
6ee36fe361 Fix a couple of issues with the relays map (geoip, 'data unavailable')
- Move to ipinfo.io for geoip, rather than Telize. Telize has been closed
   down. ipinfo.io has apparently got decent availability, and allows
   1,000 requests per day on the free tier. Since requests are made by the
   client, this should be more than enough (and the total across all clients
   should still be less than this).

 - Fix issue where one nonresponsive relay would cause 'data unavailable'
   to be shown for many relays. This was caused by the relay status
   promise not being correctly added to the list of things being waited
   for before the map was rendered. Any delayed relay status requests
   would therefore occur after the map was rendered, which was too late.
2015-11-22 14:10:29 +00:00
AudriusButkevicius
77572d0aee Typo 2015-11-21 18:58:52 +00:00
Audrius Butkevicius
37b79735bf Add signal handlers (fixes #15) 2015-11-21 00:35:38 +00:00
Audrius Butkevicius
9d9ad6de88 Update readme (fixes #16) 2015-11-21 00:35:38 +00:00
Audrius Butkevicius
20b925abec Limit number of connections (fixes #23) 2015-11-21 00:35:31 +00:00
Jakob Borg
7d00722bbf Ignores 2015-11-13 10:14:10 +01:00
Jakob Borg
4ea600d34e lru.Cache is not concurrency safe 2015-11-13 09:13:53 +01:00
Audrius Butkevicius
b61d7c2428 Merge pull request #5 from andyleap/patch-1
Rate infos are in kbps, not kBps
2015-11-10 10:05:54 -05:00
andyleap
bcc5d7c00f Rate infos are in kbps, not kBps 2015-11-10 09:52:07 -05:00
Jakob Borg
4a36cca703 We need a limit on the number of PostgreSQL connections 2015-11-09 15:11:21 +01:00
Audrius Butkevicius
f83ae630c1 Merge pull request #31 from syncthing/http
Allow plain HTTP serving behind a proxy
2015-11-08 12:26:05 -05:00
Jakob Borg
5894f35364 Correct example DSN (fixes #29) 2015-11-08 14:53:39 +01:00
Jakob Borg
c5acbf7e22 Allow plain HTTP serving behind a proxy 2015-11-07 16:01:31 +01:00
Audrius Butkevicius
567aaf87c6 Merge pull request #19 from canton7/feature/pool-logging
Enable extra logging in pool.go even when -debug not specified
2015-11-06 13:51:46 +00:00
Antony Male
e660d683a0 Enable extra logging in pool.go even when -debug not specified
Knowing why a relay server failed to join the pool can be important. This
is typically an issue which must be investigated after it occurred, so
having logs available is useful.

Running with -debug permanently enabled is impractical, due to the amount
of traffic that is generated, particularly when data is being transferred.

Logging is limited to at most one message per minute, although one message
per hour is more likely.
2015-11-06 12:58:44 +00:00
Jakob Borg
685306c386 Fix Query/Answer stats 2015-11-06 11:21:28 +01:00
Jakob Borg
5e04274d84 Reduce our patience with slow clients somewhat 2015-11-06 11:20:28 +01:00
Audrius Butkevicius
3357fded14 Merge pull request #18 from canton7/feature/contributer
Add Antony Male to CONTRIBUTORS
2015-11-05 23:24:04 +00:00
Antony Male
618fc54ac2 Add Antony Male to CONTRIBUTORS
A 1-line hack makes me a contributer, apparently :)
2015-11-05 23:10:14 +00:00
Audrius Butkevicius
339e058b64 Merge pull request #17 from canton7/feature/ext-address
Allow extAddress to be set from the command line
2015-11-05 21:43:12 +00:00
Antony Male
102027a343 Allow extAddress to be set from the command line
This allows relaysrv to listen on an unprivileged port, with port
forwarding directing traffic from 443, thus providing an alternative
to using setcap cap_net_bind_service=+ep
2015-11-05 21:26:58 +00:00
Jakob Borg
0d1df6bec3 Discovery server should print device ID of certificate at startup 2015-11-04 16:55:21 +00:00
Audrius Butkevicius
925f60d9c3 Add support for header holding IP address 2015-11-03 21:23:35 +00:00
Audrius Butkevicius
8b3f5fda07 Update relay parameters even if it already exists (fixes #3) 2015-10-31 17:27:43 +00:00
Audrius Butkevicius
ac17b2c584 Add missing space 2015-10-29 19:42:42 +00:00
Jakob Borg
c67c861dc6 Merge pull request #2 from syncthing/homepage
Add homepage
2015-10-29 17:45:21 +01:00
Audrius Butkevicius
09ba9e6259 Add homepage 2015-10-24 00:06:02 +01:00
Audrius Butkevicius
7775166477 URLs should have Go units 2015-10-23 22:24:53 +01:00
Audrius Butkevicius
0e167f5c24 Add CORS headers 2015-10-22 21:44:50 +01:00
Audrius Butkevicius
a310a32371 Add CORS headers 2015-10-22 21:44:29 +01:00
Audrius Butkevicius
c00e26be81 Fix units 2015-10-22 21:40:36 +01:00
Audrius Butkevicius
ce1a5cd2ce Expose provided by in status endpoint 2015-10-18 23:15:01 +01:00
Audrius Butkevicius
5c8a28d717 Add ability to advertise provider 2015-10-18 16:57:13 +01:00
Audrius Butkevicius
59c5d984af Change the URL 2015-10-17 00:07:01 +01:00
Audrius Butkevicius
c885903ff2 Change endpoint URL, as we might want to run some stats pages 2015-10-17 00:05:44 +01:00
Audrius Butkevicius
e4403ca396 Merge pull request #12 from rumpelsepp/systemd
Rename relaysrv binary, see #11
2015-10-10 14:26:12 +01:00
Stefan Tatschner
04912ea888 Rename relaysrv binary, see #11 2015-10-10 15:24:20 +02:00
Audrius Butkevicius
103238066d Merge pull request #11 from rumpelsepp/systemd
Jail the whole thing a bit more
2015-10-10 13:59:40 +01:00
Stefan Tatschner
7e4f08c033 Jail the whole thing a bit more
Add WorkingDirectory to create and use the certificates within
/var/lib/syncthing-relaysrv. Add RootDirectory to chroot(2) the whole
thing into that directory.
2015-10-10 14:56:47 +02:00
Jakob Borg
d47d82d8e1 Merge pull request #10 from syncthing/stuff
Add more info to status
2015-10-10 20:14:05 +09:00
Audrius Butkevicius
9b9b44dd65 Merge pull request #4 from rumpelsepp/systemd
Add systemd service file
2015-10-10 11:51:31 +01:00
Stefan Tatschner
dc5627a2ef Add systemd service file 2015-10-10 12:50:21 +02:00
Audrius Butkevicius
c1dfae1a6e Add options to status 2015-10-10 11:49:34 +01:00
Audrius Butkevicius
7b5e4ab426 Add uptime 2015-10-10 11:43:07 +01:00
Audrius Butkevicius
26a44068d8 Merge pull request #9 from syncthing/deps
Use vendored dependencies, new protocol location
2015-09-22 19:22:40 +01:00
Audrius Butkevicius
602b12dcf5 Merge pull request #23 from syncthing/deps
Use vendored dependencies, new relay/client location
2015-09-22 19:22:29 +01:00
Audrius Butkevicius
a91a836224 Merge pull request #1 from syncthing/deps
Use vendored dependencies, new relay/client location
2015-09-22 19:18:21 +01:00
Jakob Borg
969d7c802d Use vendored dependencies, new relay/client location 2015-09-22 19:55:12 +02:00
Jakob Borg
4e196d408a Use vendored dependencies, new protocol location 2015-09-22 19:54:20 +02:00
Jakob Borg
8450ab8dab Use vendored dependencies, new relay/client location 2015-09-22 19:51:40 +02:00
Jakob Borg
168889d999 Option for perm relay file, keep test cert in temp dir 2015-09-22 09:02:18 +02:00
Jakob Borg
e1339628d9 Default values tweak 2015-09-22 08:55:06 +02:00
Audrius Butkevicius
1ee190e844 Update README.md 2015-09-21 23:07:39 +01:00
Audrius Butkevicius
aadcfed17d Update README.md 2015-09-21 23:06:37 +01:00
Audrius Butkevicius
8f99f6eb66 Update README.md 2015-09-21 22:55:13 +01:00
Audrius Butkevicius
a51b948f45 Update README.md 2015-09-21 22:53:29 +01:00
Audrius Butkevicius
425f61cf34 Division by zero not good 2015-09-21 21:51:12 +00:00
Audrius Butkevicius
87cc2d2313 A bit more verbose 2015-09-21 22:33:29 +01:00
Audrius Butkevicius
0e2132ad3e Always print URI 2015-09-21 22:15:29 +01:00
Audrius Butkevicius
7d9df5abc6 Update README.md 2015-09-21 22:06:12 +01:00
Audrius Butkevicius
118cba4d9b Add build file 2015-09-21 20:53:01 +00:00
Jakob Borg
3b2adc9a3e /ping with empty response 2015-09-21 12:49:17 +02:00
Audrius Butkevicius
009b5bc72b Merge pull request #22 from syncthing/tls
New discovery protocol over HTTPS
2015-09-21 08:56:52 +01:00
Jakob Borg
9b541a28e6 New discovery protocol over HTTPS 2015-09-20 22:00:19 +02:00
Audrius Butkevicius
3533429563 Merge pull request #8 from syncthing/latency
Connected clients should know their own latency
2015-09-20 12:47:37 +01:00
Jakob Borg
500230af51 Connected clients should know their own latency 2015-09-20 13:40:24 +02:00
Jakob Borg
4a2cbc1715 Merge pull request #5 from syncthing/info
Tweaks
2015-09-14 16:20:08 +02:00
Audrius Butkevicius
61f8fdd9e8 Merge pull request #7 from syncthing/ping
Server should respond to ping
2015-09-14 12:49:12 +01:00
Jakob Borg
cfdca9f702 Server should respond to ping 2015-09-14 13:46:20 +02:00
Jakob Borg
cbe24d0c61 Errors should not increment for ever 2015-09-12 22:44:59 +02:00
Audrius Butkevicius
50f0da6793 Drop all sessions when we realize a node has gone away 2015-09-11 22:29:50 +01:00
Audrius Butkevicius
0b7ab0a095 Tweaks
1. Advertise relay server paramters so that clients could make a decision wether or not to connect
2. Generate certificate if it's not there.
2015-09-11 20:06:14 +01:00
AudriusButkevicius
3cacb48f3c Add IP based rate limiting, check if client IP matches advertised relay, reorder stuff 2015-09-07 18:13:50 +01:00
AudriusButkevicius
f6a58151cb Handle 403 2015-09-07 18:12:18 +01:00
AudriusButkevicius
3404393974 Join relay pool by default 2015-09-07 09:21:23 +01:00
AudriusButkevicius
6965812d79 Relays are matched by ip:port pairs 2015-09-07 09:14:14 +01:00
AudriusButkevicius
78fb7fe9f9 Implementation 2015-09-06 20:52:31 +01:00
AudriusButkevicius
24bcf6a088 Receive the invite, otherwise stop blocks, add extra arguments 2015-09-06 20:25:53 +01:00
AudriusButkevicius
25d0a363a8 Add a test method, fix nil pointer panic 2015-09-06 18:35:38 +01:00
Audrius Butkevicius
d7c8075862 Initial commit 2015-09-06 17:29:14 +01:00
AudriusButkevicius
041b97dd25 Use new method name 2015-09-02 22:02:17 +01:00
AudriusButkevicius
9b85a6fb7c Use a single socket for relaying 2015-09-02 21:35:52 +01:00
Jakob Borg
f407ff8861 Improve status reporter 2015-08-20 14:29:57 +02:00
Jakob Borg
a413b83c01 Fix broken connection close 2015-08-20 13:58:07 +02:00
Jakob Borg
81f4de965f Very basic status service 2015-08-20 12:59:44 +02:00
Jakob Borg
030b1f3467 I contribute stuff 2015-08-20 12:33:52 +02:00
Jakob Borg
b7a180114e Cleaner build 2015-08-20 12:33:11 +02:00
Jakob Borg
4c9a26dbca Cleaner build 2015-08-20 12:28:26 +02:00
Jakob Borg
e611828249 Merge branch 'v0.12'
* v0.12:
  Add relay support, add ql support
  Stats files
  Rewrite for a PostgreSQL backend
2015-08-20 12:20:09 +02:00
Audrius Butkevicius
e80a9b0075 Fix after package move 2015-08-19 20:49:34 +01:00
Jakob Borg
9370f9cae4 s/internal/lib/ 2015-08-09 09:39:28 +02:00
Audrius Butkevicius
604f2c9161 Connection errors are debug errors 2015-07-23 20:53:16 +01:00
Audrius Butkevicius
4d9ca822a7 Add relay support, add ql support 2015-07-23 19:12:40 +01:00
Audrius Butkevicius
d1f3d95c96 Add ability to lookup relay status 2015-07-22 22:34:05 +01:00
Audrius Butkevicius
efa0a06947 Merge pull request #1 from syncthing/review
Code review
2015-07-20 18:43:31 +01:00
Jakob Borg
11eb241c8f Style and minor fixes, client package 2015-07-20 14:04:34 +02:00
Jakob Borg
ebef239a06 Style and minor fixes, main package 2015-07-20 14:04:34 +02:00
Audrius Butkevicius
3d5507451b Merge pull request #2 from syncthing/ratelimit
Implement global and per session rate limiting
2015-07-20 12:57:55 +01:00
Jakob Borg
98a13204b2 Implement global and per session rate limiting 2015-07-20 13:37:11 +02:00
Jakob Borg
c318fdc94b Build script from discosrv 2015-07-20 12:11:06 +02:00
Audrius Butkevicius
d0229b62da Fix bugs 2015-07-17 22:04:02 +01:00
Audrius Butkevicius
37ad20a71b Change receiver type, add GoStringer 2015-07-17 21:49:45 +01:00
Audrius Butkevicius
fcd6ebb06e General cleanup 2015-07-17 20:17:49 +01:00
Audrius Butkevicius
dc9c86e3a1 Change EOL 2015-06-28 21:18:38 +01:00
Audrius Butkevicius
6bc6ae2d28 Do scheme validation in the client 2015-06-28 20:34:28 +01:00
Audrius Butkevicius
f8bedc55e5 Progress 2015-06-28 19:57:13 +01:00
Audrius Butkevicius
f376c79f7f Add initial code 2015-06-24 15:02:23 +01:00
Audrius Butkevicius
a98824b4cf Initial commit 2015-06-24 00:34:16 +01:00
Jakob Borg
860fbe48dd Stats files 2015-05-31 13:31:28 +02:00
Jakob Borg
9d06132743 Rewrite for a PostgreSQL backend 2015-03-25 15:37:00 +01:00
Jakob Borg
51eea3f90b GPL->MIT 2015-03-25 08:07:33 +01:00
Jakob Borg
27c70bdf07 Build moar platforms 2015-03-21 20:22:19 +01:00
Jakob Borg
f8abb8e541 Also build for solaris and freebsd 2015-02-11 13:40:58 +01:00
Jakob Borg
c8346d0581 Add a simple build script 2015-02-11 12:57:58 +01:00
Jakob Borg
7aaea6d005 Protocol has moved 2015-01-13 23:15:55 +01:00
Jakob Borg
e3911bacde Fix goleveldb API change 2014-12-11 12:53:00 +01:00
Jakob Borg
962eaa8a4b Handle error from XDR marshalling 2014-10-21 08:48:51 +02:00
Jakob Borg
bfba18fdcb Use WriteToUDP rather than WriteMsgUDP (fixes #4) 2014-10-07 10:50:09 +02:00
Jakob Borg
175669c61e GPL 2014-09-30 16:43:54 +02:00
Jakob Borg
3599b98dca Node -> Device here too 2014-09-28 22:39:38 +02:00
Jakob Borg
d1c3be3251 Use syncthing internal packages 2014-09-27 15:44:40 +02:00
Jakob Borg
b9f83c7780 Optionally log unknown packet data (for debugging) 2014-09-21 10:57:19 +02:00
Jakob Borg
cbf73ef29e Align cleaning routine in time 2014-09-08 12:43:30 +02:00
Jakob Borg
db6d3b495b Use persistent (leveldb) storage 2014-09-08 11:48:26 +02:00
Jakob Borg
6ea8e2525a Latest build badge should link to latest build 2014-08-20 12:22:23 +02:00
Jakob Borg
29296ec998 Link to build.syncthing.net instead 2014-08-13 13:52:53 +02:00
Jakob Borg
bdd265a1b1 Link to linux binary 2014-08-02 09:03:38 +02:00
Jakob Borg
2c9df7aad1 Update import paths, calmh -> syncthing 2014-08-02 08:25:17 +02:00
Jakob Borg
1fca248d4c Build status from drone.io 2014-07-31 12:37:18 +02:00
Jakob Borg
99081ea2a0 LICENSE & README 2014-07-30 22:15:16 +02:00
Jakob Borg
1f62247c7e New port number for new format global discovery 2014-07-13 09:36:22 +02:00
Jakob Borg
6415d1a6a5 Copyright wording 2014-07-13 01:07:49 +02:00
Jakob Borg
926b08c197 Refactor node ID handling, use check digits (fixes #269)
New node ID:s contain four Luhn check digits and are grouped
differently. Code uses NodeID type instead of string, so it's formatted
homogenously everywhere.
2014-06-30 01:42:03 +02:00
Jakob Borg
aff41d0b08 discosrv: Tunable limiter settings 2014-06-27 22:39:03 +02:00
Jakob Borg
5d9c968614 Add license header 2014-06-01 22:50:14 +02:00
Jakob Borg
c020cf05e1 Fix discosrv build, build as part of all (fixes #257) 2014-05-22 08:46:19 +02:00
Jakob Borg
09e8d85b1e discosrv: Better statistics 2014-04-19 23:14:56 +02:00
Jakob Borg
4d3eb134a2 discosrv: Remove deprecated v1 support 2014-04-19 23:02:14 +02:00
Jakob Borg
b92df85893 discosrv: Clean up debug logging 2014-04-16 15:06:54 +02:00
Jakob Borg
545025ed2b discosrv: Remove duplicate logging of limiter cache entries 2014-04-04 12:00:52 +02:00
Jakob Borg
3158962506 discosrv: Source based rate limiting 2014-04-03 23:40:10 +02:00
Jakob Borg
c314f74de6 discosrv: Refactor handler loop 2014-04-03 23:40:03 +02:00
Jakob Borg
65615385e7 Rework XDR encoding 2014-02-20 17:42:17 +01:00
Jakob Borg
727f35b35b discosrv: Expire nodes, reduce debug logging 2014-02-17 09:23:37 +01:00
Jakob Borg
07ddf7e87b External discover 2013-12-22 21:35:05 -05:00
2472 changed files with 1099266 additions and 26520 deletions

3
.gitattributes vendored
View File

@@ -2,8 +2,7 @@
* text=auto
# Except the dependencies, which we leave alone
Godeps/** -text=auto
vendor/** -text=auto
# Diffs on these files are meaningless
gui.files.go -diff
*.svg -diff

9
.gitignore vendored
View File

@@ -1,10 +1,11 @@
syncthing
!gui/syncthing
!Godeps/_workspace/src/github.com/syncthing
/syncthing
/stdiscosrv
syncthing.exe
stdiscosrv.exe
*.tar.gz
*.zip
*.asc
*.deb
.jshintrc
coverage.out
files/pidx
@@ -14,3 +15,5 @@ coverage.xml
syncthing.sig
RELEASE
deb
lib/auto/gui.files.go
snapcraft.yaml

178
AUTHORS
View File

@@ -1,72 +1,110 @@
# This is the official list of Syncthing authors for copyright purposes.
# The format is:
#
# Name Name Name (nickname) <email1@example.com> <email2@example.com>
#
# The NICKS list is auto generated from this file.
Aaron Bieber <qbit@deftly.net>
Adam Piggott <aD@simplypeachy.co.uk> <simplypeachy@users.noreply.github.com>
Alexander Graf <register-github@alex-graf.de>
Andrew Dunham <andrew@du.nham.ca>
Antony Male <antony.male@gmail.com>
Arthur Axel fREW Schmidt <frew@afoolishmanifesto.com> <frioux@gmail.com>
Audrius Butkevicius <audrius.butkevicius@gmail.com>
Bart De Vries <devriesb@gmail.com>
Ben Curthoys <ben@bencurthoys.com>
Ben Schulz <ueomkail@gmail.com> <uok@users.noreply.github.com>
Ben Sidhom <bsidhom@gmail.com>
Brandon Philips <brandon@ifup.org>
Brendan Long <self@brendanlong.com>
Brian R. Becker <brbecker@gmail.com>
Caleb Callaway <enlightened.despot@gmail.com>
Carsten Hagemann <moter8@gmail.com>
Cathryne Linenweaver <cathryne.linenweaver@gmail.com> <Cathryne@users.noreply.github.com>
Chris Howie <me@chrishowie.com>
Chris Joel <chris@scriptolo.gy>
Colin Kennedy <moshen.colin@gmail.com>
Daniel Bergmann <dan.arne.bergmann@gmail.com> <brgmnn@users.noreply.github.com>
Daniel Martí <mvdan@mvdan.cc>
Denis A. <denisva@gmail.com>
Dennis Wilson <dw@risu.io>
Dominik Heidler <dominik@heidler.eu>
Elias Jarlebring <jarlebring@gmail.com>
Emil Hessman <emil@hessman.se>
Erik Meitner <e.meitner@willystreet.coop>
Federico Castagnini <federico.castagnini@gmail.com>
Felix Ableitner <me@nutomic.com>
Felix Unterpaintner <bigbear2nd@gmail.com>
Francois-Xavier Gsell <fxgsell@gmail.com>
Frank Isemann <frank@isemann.name>
Gilli Sigurdsson <gilli@vx.is>
Jacek Szafarkiewicz <szafar@linux.pl>
Jakob Borg <jakob@nym.se>
Jake Peterson <jake@acogdev.com>
James Patterson <jamespatterson@operamail.com> <jpjp@users.noreply.github.com>
Jaroslav Malec <dzardacz@gmail.com>
Jens Diemer <github.com@jensdiemer.de> <git@jensdiemer.de>
Jochen Voss <voss@seehuhn.de>
Johan Vromans <jvromans@squirrel.nl>
Karol Różycki <rozycki.karol@gmail.com>
Ken'ichi Kamada <kamada@nanohz.org>
Lode Hoste <zillode@zillode.be>
Lord Landon Agahnim <lordlandon@gmail.com>
Marc Laporte <marc@marclaporte.com> <marc@laporte.name>
Marc Pujol <kilburn@la3.org>
Marcin Dziadus <dziadus.marcin@gmail.com>
Mateusz Naściszewski <matin1111@wp.pl>
Matt Burke <mburke@amplify.com> <burkemw3@gmail.com>
Michael Jephcote <rewt0r@gmx.com> <Rewt0r@users.noreply.github.com>
Michael Tilli <pyfisch@gmail.com>
Pascal Jungblut <github@pascalj.com> <mail@pascal-jungblut.com>
Peter Hoeg <peter@speartail.com>
Philippe Schommers <philippe@schommers.be>
Phill Luby <phill.luby@newredo.com>
Piotr Bejda <piotrb10@gmail.com>
Ryan Sullivan <kayoticsully@gmail.com>
Sergey Mishin <ralder@yandex.ru>
Stefan Tatschner <stefan@sevenbyte.org> <rumpelsepp@sevenbyte.org>
Stefan Kuntz <stefan.github@gmail.com> <Stefan.github@gmail.com>
Tim Abell <tim@timwise.co.uk>
Tobias Nygren <tnn@nygren.pp.se>
Tomas Cerveny <kozec@kozec.com>
Tully Robinson <tully@tojr.org>
Tyler Brazier <tyler@tylerbrazier.com>
Veeti Paananen <veeti.paananen@rojekti.fi>
Vil Brekin <vilbrekin@gmail.com>
Yannic A. <eipiminusone+github@gmail.com> <eipiminus1@users.noreply.github.com>
Aaron Bieber (qbit) <qbit@deftly.net>
Adam Piggott (ProactiveServices) <aD@simplypeachy.co.uk> <simplypeachy@users.noreply.github.com> <ProactiveServices@users.noreply.github.com>
Adel Qalieh (adelq) <aqalieh95@gmail.com> <adelq@users.noreply.github.com>
Alessandro G. (alessandro.g89) <alessandro.g89@gmail.com>
Alexander Graf (alex2108) <register-github@alex-graf.de>
Alexandre Viau (aviau) <alexandre@alexandreviau.net> <aviau@debian.org>
Anderson Mesquita (andersonvom) <andersonvom@gmail.com>
Andrew Dunham (andrew-d) <andrew@du.nham.ca>
Andrey D (scienmind) <scintertech@cryptolab.net>
Antoine Lamielle (0x010C) <antoine.lamielle@0x010c.fr> <gh@0x010c.fr>
Antony Male (canton7) <antony.male@gmail.com>
Arthur Axel fREW Schmidt (frioux) <frew@afoolishmanifesto.com> <frioux@gmail.com>
Audrius Butkevicius (AudriusButkevicius) <audrius.butkevicius@gmail.com>
Bart De Vries (mogwa1) <devriesb@gmail.com>
Ben Curthoys (bencurthoys) <ben@bencurthoys.com>
Ben Schulz (uok) <ueomkail@gmail.com> <uok@users.noreply.github.com>
Ben Sidhom (bsidhom) <bsidhom@gmail.com>
Benny Ng (tpng) <benny.tpng@gmail.com>
Brandon Philips (philips) <brandon@ifup.org>
Brendan Long (brendanlong) <self@brendanlong.com>
Brian R. Becker (brbecker) <brbecker@gmail.com>
Caleb Callaway (cqcallaw) <enlightened.despot@gmail.com>
Carsten Hagemann (Moter8) <moter8@gmail.com>
Cathryne Linenweaver (Cathryne) <cathryne.linenweaver@gmail.com> <Cathryne@users.noreply.github.com>
Cedric Staniewski (xduugu) <cedric@gmx.ca>
Chris Howie (cdhowie) <me@chrishowie.com>
Chris Joel (cdata) <chris@scriptolo.gy>
Colin Kennedy (moshen) <moshen.colin@gmail.com>
Daniel Bergmann (brgmnn) <dan.arne.bergmann@gmail.com> <brgmnn@users.noreply.github.com>
Daniel Harte (norgeous) <daniel@harte.me> <daniel@danielharte.co.uk> <norgeous@users.noreply.github.com>
Daniel Martí (mvdan) <mvdan@mvdan.cc>
David Rimmer (dinosore) <dinosore@dbrsoftware.co.uk>
Denis A. (dva) <denisva@gmail.com>
Dennis Wilson (snnd) <dw@risu.io>
Dominik Heidler (asdil12) <dominik@heidler.eu>
Elias Jarlebring (jarlebring) <jarlebring@gmail.com>
Emil Hessman (ceh) <emil@hessman.se>
Erik Meitner (WSGCSysadmin) <e.meitner@willystreet.coop>
Federico Castagnini (facastagnini) <federico.castagnini@gmail.com>
Felix Ableitner (Nutomic) <me@nutomic.com>
Felix Unterpaintner (bigbear2nd) <bigbear2nd@gmail.com>
Francois-Xavier Gsell (zukoo) <fxgsell@gmail.com>
Frank Isemann (fti7) <frank@isemann.name>
Gilli Sigurdsson (gillisig) <gilli@vx.is>
Heiko Zuerker (Smiley73) <heiko@zuerker.org>
Jaakko Hannikainen (jgke) <jgke@jgke.fi>
Jacek Szafarkiewicz (hadogenes) <szafar@linux.pl>
Jake Peterson (acogdev) <jake@acogdev.com>
Jakob Borg (calmh) <jakob@nym.se> <jakob@kastelo.net>
James Patterson (jpjp) <jamespatterson@operamail.com> <jpjp@users.noreply.github.com>
Jaroslav Malec (dzarda) <dzardacz@gmail.com>
Jens Diemer (jedie) <github.com@jensdiemer.de> <git@jensdiemer.de>
Jochen Voss (seehuhn) <voss@seehuhn.de>
Johan Vromans (sciurius) <jvromans@squirrel.nl>
Karol Różycki (krozycki) <rozycki.karol@gmail.com>
Kelong Cong (kc1212) <kc04bc@gmx.com> <kc1212@users.noreply.github.com>
Ken'ichi Kamada (kamadak) <kamada@nanohz.org>
Kevin Allen (ironmig) <kma1660@gmail.com>
Kevin White, Jr. (kwhite17) <kevinwhite1710@gmail.com>
Kurt Fitzner (Kudalufi) <kurt@va1der.ca>
Lars K.W. Gohlke (lkwg82) <lkwg82@gmx.de>
Laurent Etiemble (letiemble) <laurent.etiemble@gmail.com> <laurent.etiemble@monobjc.net>
Leo Arias (elopio) <yo@elopio.net>
Lode Hoste (Zillode) <zillode@zillode.be>
Lord Landon Agahnim (LordLandon) <lordlandon@gmail.com>
Majed Abdulaziz (majedev) <majed.alhajry@gmail.com>
Marc Laporte (marclaporte) <marc@marclaporte.com> <marc@laporte.name>
Marc Pujol (kilburn) <kilburn@la3.org>
Marcin Dziadus (marcindziadus) <dziadus.marcin@gmail.com>
Mark Pulford (mpx) <mark@kyne.com.au>
Mateusz Naściszewski (mateon1) <matin1111@wp.pl>
Matt Burke (burkemw3) <mburke@amplify.com> <burkemw3@gmail.com>
Max Schulze (kralo) <max.schulze@online.de> <kralo@users.noreply.github.com>
Michael Jephcote (Rewt0r) <rewt0r@gmx.com> <Rewt0r@users.noreply.github.com>
Michael Ploujnikov (plouj) <ploujj@gmail.com>
Michael Tilli (pyfisch) <pyfisch@gmail.com>
Nate Morrison (nrm21) <natemorrison@gmail.com>
Pascal Jungblut (pascalj) <github@pascalj.com> <mail@pascal-jungblut.com>
Peter Hoeg (peterhoeg) <peter@speartail.com>
Philippe Schommers (filoozoom) <philippe@schommers.be>
Phill Luby (pluby) <phill.luby@newredo.com>
Piotr Bejda (piobpl) <piotrb10@gmail.com>
Roman Zaynetdinov (zaynetro) <romanznet@gmail.com>
Ryan Sullivan (KayoticSully) <kayoticsully@gmail.com>
Scott Klupfel (kluppy) <kluppy@going2blue.com>
Sergey Mishin (ralder) <ralder@yandex.ru>
Simon Frei (imsodin) <freisim93@gmail.com>
Stefan Kuntz (Stefan-Code) <stefan.github@gmail.com> <Stefan.github@gmail.com>
Stefan Tatschner (rumpelsepp) <stefan@sevenbyte.org> <rumpelsepp@sevenbyte.org>
Tim Abell (timabell) <tim@timwise.co.uk>
Tim Howes (timhowes) <timhowes@berkeley.edu>
Tobias Nygren (tnn2) <tnn@nygren.pp.se>
Tomas Cerveny (kozec) <kozec@kozec.com>
Tully Robinson (tojrobinson) <tully@tojr.org>
Tyler Brazier (tylerbrazier) <tyler@tylerbrazier.com>
Unrud (Unrud) <unrud@openaliasbox.org> <Unrud@users.noreply.github.com>
Veeti Paananen (veeti) <veeti.paananen@rojekti.fi>
Victor Buinsky (buinsky) <vix_booja@tut.by>
Vil Brekin (Vilbrekin) <vilbrekin@gmail.com>
William A. Kennington III (wkennington) <william@wkennington.com>
Wulf Weich (wweich) <wweich@users.noreply.github.com> <wweich@gmx.de>
Xavier O. (damajor) <damajor@gmail.com>
Yannic A. (eipiminus1) <eipiminusone+github@gmail.com> <eipiminus1@users.noreply.github.com>

View File

@@ -33,20 +33,31 @@ latest info on Transifex.
Every contribution is welcome. If you want to contribute but are unsure
where to start, any open issues are fair game! See the [Contribution
Guidelines](http://docs.syncthing.net/dev/contributing.html) for the full
Guidelines](https://docs.syncthing.net/dev/contributing.html) for the full
story on committing code.
## Contributing Documentation
Updates to the [documentation site](http://docs.syncthing.net/) can be
Updates to the [documentation site](https://docs.syncthing.net/) can be
made as pull requests on the [documentation
repository](https://github.com/syncthing/docs).
## Licensing
All contributions are made under the same MPLv2 license as the rest of
the project, except documentation, user interface text and translation
strings which are licensed under the Creative Commons Attribution 4.0
International License. You retain the copyright to code you have
written.
All contributions are made available under the same license as the already
existing material being contributed to. For most of the project and unless
otherwise stated this means MPLv2, but there are exceptions:
- Certain commands (under cmd/...) may have a separate license, indicated by
the presence of a LICENSE file in the corresponding directory.
- The documentation (man/...) is licensed under the Creative Commons
Attribution 4.0 International License.
- Projects under vendor/... are copyright by and licensed from their
respective original authors. Contributions should be made to the original
project, not here.
Regardless of the license in effect, you retain the copyright to your
contribution.

83
GOALS.md Normal file
View File

@@ -0,0 +1,83 @@
# The Syncthing Goals
Syncthing is a **continous file synchronization program**. It synchronizes
files between two or more computers. We strive to fulfill the goals below.
The goals are listed in order of importance, the most important one being
the first.
> "Syncing files" here is precise. It means we specifically exclude things
> that are not files - calendar items, instant messages, and so on. If those
> are in fact stored as files on disk, they can of course be synced as
> files.
Syncthing should be:
### 1. Safe From Data Loss
Protecting the user's data is paramount. We take every reasonable precaution
to avoid corrupting the user's files.
> This is the overriding goal, without which synchronizing files becomes
> pointless. This means that we do not make unsafe trade offs for the sake
> of performance or, in some cases, even usability.
### 2. Secure Against Attackers
Again, protecting the user's data is paramount. Regardless of our other
goals we must never allow the user's data to be susceptible to eavesdropping
or modification by unauthorized parties.
> This should be understood in context. It is not necessarily reasonable to
> expect Syncthing to be resistant against well equipped state level
> attackers. We will however do our best. Note also that this is different
> from anonymity which is not, currently, a goal.
### 3. Easy to Use
Syncthing should be approachable, understandable and inclusive.
> Complex concepts and maths form the base of Synchting's functionality.
> This should nonetheless be abstracted or hidden to a degree where
> Syncthing is usable by the general public.
### 4. Automatic
User interaction should be required only when absolutely necessary.
> Specifically this means that changes to files are picked up without
> prompting, conflicts are resolved without prompting and connections are
> maintained without prompting. We only prompt the user when it is required
> to fulfill one of the (overriding) Secure, Safe or Easy goals.
### 5. Universally Available
Syncthing should run on every common computer. We are mindful that the
latest technology is not always available to any given individual.
> Computers include desktops, laptops, servers, virtual machines, small
> general purpose computers such as Raspberry Pis and, *where possible*,
> tablets and phones. NAS appliances, toasters, cars, firearms, thermostats
> and so on may include computing capabitilies but it is not our goal for
> Syncthing to run smoothly on these devices.
### 6. For Individuals
Syncthing is primarily about empowering the individual user with safe,
secure and easy to use file synchronization.
> We acknowledge that it's also useful in an enterprise setting and include
> functionality to support that. If this is in conflict with the
> requirements of the individual, those will however take priority.
### 7. Everything Else
There are many things we care about that don't make it on to the list. It is
fine to optimize for these values as well, as long as they are not in
conflict with the stated goals above.
> For example, performance is a thing we care about. We just don't care more
> about it than safety, security, etc. Maintainability of the code base and
> providing entertainment value for the maintainers are also things that
> matter. It is understood that there are aspects of Syncthing that are
> suboptimal or even in opposition with the goals above. However, we
> continously strive to align Syncthing more and more with these goals.

90
Godeps/Godeps.json generated
View File

@@ -1,90 +0,0 @@
{
"ImportPath": "github.com/syncthing/syncthing",
"GoVersion": "go1.5.1",
"Packages": [
"./cmd/..."
],
"Deps": [
{
"ImportPath": "github.com/bkaradzic/go-lz4",
"Rev": "74ddf82598bc4745b965729e9c6a463bedd33049"
},
{
"ImportPath": "github.com/calmh/du",
"Rev": "3c0690cca16228b97741327b1b6781397afbdb24"
},
{
"ImportPath": "github.com/calmh/luhn",
"Rev": "0c8388ff95fa92d4094011e5a04fc99dea3d1632"
},
{
"ImportPath": "github.com/calmh/xdr",
"Rev": "47c0042d09a827b81ee62497f99e5e0c7f0bd31c"
},
{
"ImportPath": "github.com/golang/snappy",
"Rev": "723cc1e459b8eea2dea4583200fd60757d40097a"
},
{
"ImportPath": "github.com/juju/ratelimit",
"Rev": "772f5c38e468398c4511514f4f6aa9a4185bc0a0"
},
{
"ImportPath": "github.com/kardianos/osext",
"Rev": "6e7f843663477789fac7c02def0d0909e969b4e5"
},
{
"ImportPath": "github.com/rcrowley/go-metrics",
"Rev": "1ce93efbc8f9c568886b2ef85ce305b2217b3de3"
},
{
"ImportPath": "github.com/syndtr/goleveldb/leveldb",
"Rev": "1a9d62f03ea92815b46fcaab357cfd4df264b1a0"
},
{
"ImportPath": "github.com/thejerf/suture",
"Comment": "v1.0.1",
"Rev": "99c1f2d613756768fc4299acd9dc621e11ed3fd7"
},
{
"ImportPath": "github.com/vitrun/qart/coding",
"Rev": "ccb109cf25f0cd24474da73b9fee4e7a3e8a8ce0"
},
{
"ImportPath": "github.com/vitrun/qart/gf256",
"Rev": "ccb109cf25f0cd24474da73b9fee4e7a3e8a8ce0"
},
{
"ImportPath": "github.com/vitrun/qart/qr",
"Rev": "ccb109cf25f0cd24474da73b9fee4e7a3e8a8ce0"
},
{
"ImportPath": "golang.org/x/crypto/bcrypt",
"Rev": "575fdbe86e5dd89229707ebec0575ce7d088a4a6"
},
{
"ImportPath": "golang.org/x/crypto/blowfish",
"Rev": "575fdbe86e5dd89229707ebec0575ce7d088a4a6"
},
{
"ImportPath": "golang.org/x/net/internal/iana",
"Rev": "042ba42fa6633b34205efc66ba5719cd3afd8d38"
},
{
"ImportPath": "golang.org/x/net/ipv6",
"Rev": "042ba42fa6633b34205efc66ba5719cd3afd8d38"
},
{
"ImportPath": "golang.org/x/net/proxy",
"Rev": "042ba42fa6633b34205efc66ba5719cd3afd8d38"
},
{
"ImportPath": "golang.org/x/text/transform",
"Rev": "5eb8d4684c4796dd36c74f6452f2c0fa6c79597e"
},
{
"ImportPath": "golang.org/x/text/unicode/norm",
"Rev": "5eb8d4684c4796dd36c74f6452f2c0fa6c79597e"
}
]
}

5
Godeps/Readme generated
View File

@@ -1,5 +0,0 @@
This directory tree is generated automatically by godep.
Please do not edit.
See https://github.com/tools/godep for more information.

2
Godeps/_workspace/.gitignore generated vendored
View File

@@ -1,2 +0,0 @@
/pkg
/bin

View File

@@ -1 +0,0 @@
/lz4-example/lz4-example

View File

@@ -1,9 +0,0 @@
language: go
go:
- 1.1
- 1.2
- 1.3
- 1.4
- 1.5
- tip

View File

@@ -1 +0,0 @@
coverage.out

View File

@@ -1,19 +0,0 @@
language: go
go:
- tip
install:
- export PATH=$PATH:$HOME/gopath/bin
- go get golang.org/x/tools/cover
- go get github.com/mattn/goveralls
script:
- ./generate.sh
- go test -coverprofile=coverage.out
after_success:
- goveralls -coverprofile=coverage.out -service=travis-ci -package=calmh/xdr -repotoken="$COVERALLS_TOKEN"
env:
global:
secure: SmgnrGfp2zLrA44ChRMpjPeujubt9veZ8Fx/OseMWECmacyV5N/TuDhzIbwo6QwV4xB0sBacoPzvxQbJRVjNKsPiSu72UbcQmQ7flN4Tf7nW09tSh1iW8NgrpBCq/3UYLoBu2iPBEBKm93IK0aGNAKs6oEkB0fU27iTVBwiTXOY=

View File

@@ -1,16 +0,0 @@
// Copyright (C) 2014 Jakob Borg. All rights reserved. Use of this source code
// is governed by an MIT-style license that can be found in the LICENSE file.
package xdr
import (
"log"
"os"
)
var (
debug = len(os.Getenv("XDRTRACE")) > 0
dl = log.New(os.Stdout, "xdr: ", log.Lshortfile|log.Ltime|log.Lmicroseconds)
)
const maxDebugBytes = 32

View File

@@ -1,10 +0,0 @@
// Copyright (C) 2014 Jakob Borg. All rights reserved. Use of this source code
// is governed by an MIT-style license that can be found in the LICENSE file.
// +build ipdr
package xdr
func pad(l int) int {
return 0
}

View File

@@ -1,14 +0,0 @@
// Copyright (C) 2014 Jakob Borg. All rights reserved. Use of this source code
// is governed by an MIT-style license that can be found in the LICENSE file.
// +build !ipdr
package xdr
func pad(l int) int {
d := l % 4
if d == 0 {
return 0
}
return 4 - d
}

View File

@@ -1,171 +0,0 @@
// Copyright (C) 2014 Jakob Borg and Contributors (see the CONTRIBUTORS file).
// All rights reserved. Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
package xdr
import (
"fmt"
"io"
"reflect"
"unsafe"
)
type Reader struct {
r io.Reader
err error
b [8]byte
}
func NewReader(r io.Reader) *Reader {
return &Reader{
r: r,
}
}
func (r *Reader) ReadRaw(bs []byte) (int, error) {
if r.err != nil {
return 0, r.err
}
var n int
n, r.err = io.ReadFull(r.r, bs)
return n, r.err
}
func (r *Reader) ReadString() string {
return r.ReadStringMax(0)
}
func (r *Reader) ReadStringMax(max int) string {
buf := r.ReadBytesMaxInto(max, nil)
bh := (*reflect.SliceHeader)(unsafe.Pointer(&buf))
sh := reflect.StringHeader{
Data: bh.Data,
Len: bh.Len,
}
return *((*string)(unsafe.Pointer(&sh)))
}
func (r *Reader) ReadBytes() []byte {
return r.ReadBytesInto(nil)
}
func (r *Reader) ReadBytesMax(max int) []byte {
return r.ReadBytesMaxInto(max, nil)
}
func (r *Reader) ReadBytesInto(dst []byte) []byte {
return r.ReadBytesMaxInto(0, dst)
}
func (r *Reader) ReadBytesMaxInto(max int, dst []byte) []byte {
if r.err != nil {
return nil
}
l := int(r.ReadUint32())
if r.err != nil {
return nil
}
if l < 0 || max > 0 && l > max {
// l may be negative on 32 bit builds
r.err = ElementSizeExceeded("bytes field", l, max)
return nil
}
if fullLen := l + pad(l); fullLen > len(dst) {
dst = make([]byte, fullLen)
} else {
dst = dst[:fullLen]
}
var n int
n, r.err = io.ReadFull(r.r, dst)
if r.err != nil {
if debug {
dl.Printf("rd bytes (%d): %v", len(dst), r.err)
}
return nil
}
if debug {
if n > maxDebugBytes {
dl.Printf("rd bytes (%d): %x...", len(dst), dst[:maxDebugBytes])
} else {
dl.Printf("rd bytes (%d): %x", len(dst), dst)
}
}
return dst[:l]
}
func (r *Reader) ReadBool() bool {
return r.ReadUint8() != 0
}
func (r *Reader) ReadUint32() uint32 {
if r.err != nil {
return 0
}
_, r.err = io.ReadFull(r.r, r.b[:4])
if r.err != nil {
if debug {
dl.Printf("rd uint32: %v", r.err)
}
return 0
}
v := uint32(r.b[3]) | uint32(r.b[2])<<8 | uint32(r.b[1])<<16 | uint32(r.b[0])<<24
if debug {
dl.Printf("rd uint32=%d (0x%08x)", v, v)
}
return v
}
func (r *Reader) ReadUint64() uint64 {
if r.err != nil {
return 0
}
_, r.err = io.ReadFull(r.r, r.b[:8])
if r.err != nil {
if debug {
dl.Printf("rd uint64: %v", r.err)
}
return 0
}
v := uint64(r.b[7]) | uint64(r.b[6])<<8 | uint64(r.b[5])<<16 | uint64(r.b[4])<<24 |
uint64(r.b[3])<<32 | uint64(r.b[2])<<40 | uint64(r.b[1])<<48 | uint64(r.b[0])<<56
if debug {
dl.Printf("rd uint64=%d (0x%016x)", v, v)
}
return v
}
type XDRError struct {
op string
err error
}
func (e XDRError) Error() string {
return "xdr " + e.op + ": " + e.err.Error()
}
func (e XDRError) IsEOF() bool {
return e.err == io.EOF
}
func (r *Reader) Error() error {
if r.err == nil {
return nil
}
return XDRError{"read", r.err}
}
func ElementSizeExceeded(field string, size, limit int) error {
return fmt.Errorf("%s exceeds size limit; %d > %d", field, size, limit)
}

View File

@@ -1,49 +0,0 @@
// Copyright (C) 2014 Jakob Borg and Contributors (see the CONTRIBUTORS file).
// All rights reserved. Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build ipdr
package xdr
import "io"
func (r *Reader) ReadUint8() uint8 {
if r.err != nil {
return 0
}
_, r.err = io.ReadFull(r.r, r.b[:1])
if r.err != nil {
if debug {
dl.Printf("rd uint8: %v", r.err)
}
return 0
}
if debug {
dl.Printf("rd uint8=%d (0x%02x)", r.b[0], r.b[0])
}
return r.b[0]
}
func (r *Reader) ReadUint16() uint16 {
if r.err != nil {
return 0
}
_, r.err = io.ReadFull(r.r, r.b[:2])
if r.err != nil {
if debug {
dl.Printf("rd uint16: %v", r.err)
}
return 0
}
v := uint16(r.b[1]) | uint16(r.b[0])<<8
if debug {
dl.Printf("rd uint16=%d (0x%04x)", v, v)
}
return v
}

View File

@@ -1,15 +0,0 @@
// Copyright (C) 2014 Jakob Borg and Contributors (see the CONTRIBUTORS file).
// All rights reserved. Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build !ipdr
package xdr
func (r *Reader) ReadUint8() uint8 {
return uint8(r.ReadUint32())
}
func (r *Reader) ReadUint16() uint16 {
return uint16(r.ReadUint32())
}

View File

@@ -1,146 +0,0 @@
// Copyright (C) 2014 Jakob Borg. All rights reserved. Use of this source code
// is governed by an MIT-style license that can be found in the LICENSE file.
package xdr
import (
"io"
"reflect"
"unsafe"
)
var padBytes = []byte{0, 0, 0}
type Writer struct {
w io.Writer
tot int
err error
b [8]byte
}
type AppendWriter []byte
func (w *AppendWriter) Write(bs []byte) (int, error) {
*w = append(*w, bs...)
return len(bs), nil
}
func NewWriter(w io.Writer) *Writer {
return &Writer{
w: w,
}
}
func (w *Writer) WriteRaw(bs []byte) (int, error) {
if w.err != nil {
return 0, w.err
}
var n int
n, w.err = w.w.Write(bs)
return n, w.err
}
func (w *Writer) WriteString(s string) (int, error) {
sh := *((*reflect.StringHeader)(unsafe.Pointer(&s)))
bh := reflect.SliceHeader{
Data: sh.Data,
Len: sh.Len,
Cap: sh.Len,
}
return w.WriteBytes(*(*[]byte)(unsafe.Pointer(&bh)))
}
func (w *Writer) WriteBytes(bs []byte) (int, error) {
if w.err != nil {
return 0, w.err
}
w.WriteUint32(uint32(len(bs)))
if w.err != nil {
return 0, w.err
}
if debug {
if len(bs) > maxDebugBytes {
dl.Printf("wr bytes (%d): %x...", len(bs), bs[:maxDebugBytes])
} else {
dl.Printf("wr bytes (%d): %x", len(bs), bs)
}
}
var l, n int
n, w.err = w.w.Write(bs)
l += n
if p := pad(len(bs)); w.err == nil && p > 0 {
n, w.err = w.w.Write(padBytes[:p])
l += n
}
w.tot += l
return l, w.err
}
func (w *Writer) WriteBool(v bool) (int, error) {
if v {
return w.WriteUint8(1)
} else {
return w.WriteUint8(0)
}
}
func (w *Writer) WriteUint32(v uint32) (int, error) {
if w.err != nil {
return 0, w.err
}
if debug {
dl.Printf("wr uint32=%d", v)
}
w.b[0] = byte(v >> 24)
w.b[1] = byte(v >> 16)
w.b[2] = byte(v >> 8)
w.b[3] = byte(v)
var l int
l, w.err = w.w.Write(w.b[:4])
w.tot += l
return l, w.err
}
func (w *Writer) WriteUint64(v uint64) (int, error) {
if w.err != nil {
return 0, w.err
}
if debug {
dl.Printf("wr uint64=%d", v)
}
w.b[0] = byte(v >> 56)
w.b[1] = byte(v >> 48)
w.b[2] = byte(v >> 40)
w.b[3] = byte(v >> 32)
w.b[4] = byte(v >> 24)
w.b[5] = byte(v >> 16)
w.b[6] = byte(v >> 8)
w.b[7] = byte(v)
var l int
l, w.err = w.w.Write(w.b[:8])
w.tot += l
return l, w.err
}
func (w *Writer) Tot() int {
return w.tot
}
func (w *Writer) Error() error {
if w.err == nil {
return nil
}
return XDRError{"write", w.err}
}

View File

@@ -1,41 +0,0 @@
// Copyright (C) 2014 Jakob Borg. All rights reserved. Use of this source code
// is governed by an MIT-style license that can be found in the LICENSE file.
// +build ipdr
package xdr
func (w *Writer) WriteUint8(v uint8) (int, error) {
if w.err != nil {
return 0, w.err
}
if debug {
dl.Printf("wr uint8=%d", v)
}
w.b[0] = byte(v)
var l int
l, w.err = w.w.Write(w.b[:1])
w.tot += l
return l, w.err
}
func (w *Writer) WriteUint16(v uint16) (int, error) {
if w.err != nil {
return 0, w.err
}
if debug {
dl.Printf("wr uint8=%d", v)
}
w.b[0] = byte(v >> 8)
w.b[1] = byte(v)
var l int
l, w.err = w.w.Write(w.b[:2])
w.tot += l
return l, w.err
}

View File

@@ -1,14 +0,0 @@
// Copyright (C) 2014 Jakob Borg. All rights reserved. Use of this source code
// is governed by an MIT-style license that can be found in the LICENSE file.
// +build !ipdr
package xdr
func (w *Writer) WriteUint8(v uint8) (int, error) {
return w.WriteUint32(uint32(v))
}
func (w *Writer) WriteUint16(v uint16) (int, error) {
return w.WriteUint32(uint32(v))
}

View File

@@ -1,254 +0,0 @@
// Copyright 2011 The Snappy-Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package snappy
import (
"encoding/binary"
"io"
)
// We limit how far copy back-references can go, the same as the C++ code.
const maxOffset = 1 << 15
// emitLiteral writes a literal chunk and returns the number of bytes written.
func emitLiteral(dst, lit []byte) int {
i, n := 0, uint(len(lit)-1)
switch {
case n < 60:
dst[0] = uint8(n)<<2 | tagLiteral
i = 1
case n < 1<<8:
dst[0] = 60<<2 | tagLiteral
dst[1] = uint8(n)
i = 2
case n < 1<<16:
dst[0] = 61<<2 | tagLiteral
dst[1] = uint8(n)
dst[2] = uint8(n >> 8)
i = 3
case n < 1<<24:
dst[0] = 62<<2 | tagLiteral
dst[1] = uint8(n)
dst[2] = uint8(n >> 8)
dst[3] = uint8(n >> 16)
i = 4
case int64(n) < 1<<32:
dst[0] = 63<<2 | tagLiteral
dst[1] = uint8(n)
dst[2] = uint8(n >> 8)
dst[3] = uint8(n >> 16)
dst[4] = uint8(n >> 24)
i = 5
default:
panic("snappy: source buffer is too long")
}
if copy(dst[i:], lit) != len(lit) {
panic("snappy: destination buffer is too short")
}
return i + len(lit)
}
// emitCopy writes a copy chunk and returns the number of bytes written.
func emitCopy(dst []byte, offset, length int) int {
i := 0
for length > 0 {
x := length - 4
if 0 <= x && x < 1<<3 && offset < 1<<11 {
dst[i+0] = uint8(offset>>8)&0x07<<5 | uint8(x)<<2 | tagCopy1
dst[i+1] = uint8(offset)
i += 2
break
}
x = length
if x > 1<<6 {
x = 1 << 6
}
dst[i+0] = uint8(x-1)<<2 | tagCopy2
dst[i+1] = uint8(offset)
dst[i+2] = uint8(offset >> 8)
i += 3
length -= x
}
return i
}
// Encode returns the encoded form of src. The returned slice may be a sub-
// slice of dst if dst was large enough to hold the entire encoded block.
// Otherwise, a newly allocated slice will be returned.
// It is valid to pass a nil dst.
func Encode(dst, src []byte) []byte {
if n := MaxEncodedLen(len(src)); len(dst) < n {
dst = make([]byte, n)
}
// The block starts with the varint-encoded length of the decompressed bytes.
d := binary.PutUvarint(dst, uint64(len(src)))
// Return early if src is short.
if len(src) <= 4 {
if len(src) != 0 {
d += emitLiteral(dst[d:], src)
}
return dst[:d]
}
// Initialize the hash table. Its size ranges from 1<<8 to 1<<14 inclusive.
const maxTableSize = 1 << 14
shift, tableSize := uint(32-8), 1<<8
for tableSize < maxTableSize && tableSize < len(src) {
shift--
tableSize *= 2
}
var table [maxTableSize]int
// Iterate over the source bytes.
var (
s int // The iterator position.
t int // The last position with the same hash as s.
lit int // The start position of any pending literal bytes.
)
for s+3 < len(src) {
// Update the hash table.
b0, b1, b2, b3 := src[s], src[s+1], src[s+2], src[s+3]
h := uint32(b0) | uint32(b1)<<8 | uint32(b2)<<16 | uint32(b3)<<24
p := &table[(h*0x1e35a7bd)>>shift]
// We need to to store values in [-1, inf) in table. To save
// some initialization time, (re)use the table's zero value
// and shift the values against this zero: add 1 on writes,
// subtract 1 on reads.
t, *p = *p-1, s+1
// If t is invalid or src[s:s+4] differs from src[t:t+4], accumulate a literal byte.
if t < 0 || s-t >= maxOffset || b0 != src[t] || b1 != src[t+1] || b2 != src[t+2] || b3 != src[t+3] {
s++
continue
}
// Otherwise, we have a match. First, emit any pending literal bytes.
if lit != s {
d += emitLiteral(dst[d:], src[lit:s])
}
// Extend the match to be as long as possible.
s0 := s
s, t = s+4, t+4
for s < len(src) && src[s] == src[t] {
s++
t++
}
// Emit the copied bytes.
d += emitCopy(dst[d:], s-t, s-s0)
lit = s
}
// Emit any final pending literal bytes and return.
if lit != len(src) {
d += emitLiteral(dst[d:], src[lit:])
}
return dst[:d]
}
// MaxEncodedLen returns the maximum length of a snappy block, given its
// uncompressed length.
func MaxEncodedLen(srcLen int) int {
// Compressed data can be defined as:
// compressed := item* literal*
// item := literal* copy
//
// The trailing literal sequence has a space blowup of at most 62/60
// since a literal of length 60 needs one tag byte + one extra byte
// for length information.
//
// Item blowup is trickier to measure. Suppose the "copy" op copies
// 4 bytes of data. Because of a special check in the encoding code,
// we produce a 4-byte copy only if the offset is < 65536. Therefore
// the copy op takes 3 bytes to encode, and this type of item leads
// to at most the 62/60 blowup for representing literals.
//
// Suppose the "copy" op copies 5 bytes of data. If the offset is big
// enough, it will take 5 bytes to encode the copy op. Therefore the
// worst case here is a one-byte literal followed by a five-byte copy.
// That is, 6 bytes of input turn into 7 bytes of "compressed" data.
//
// This last factor dominates the blowup, so the final estimate is:
return 32 + srcLen + srcLen/6
}
// NewWriter returns a new Writer that compresses to w, using the framing
// format described at
// https://github.com/google/snappy/blob/master/framing_format.txt
func NewWriter(w io.Writer) *Writer {
return &Writer{
w: w,
enc: make([]byte, MaxEncodedLen(maxUncompressedChunkLen)),
}
}
// Writer is an io.Writer than can write Snappy-compressed bytes.
type Writer struct {
w io.Writer
err error
enc []byte
buf [checksumSize + chunkHeaderSize]byte
wroteHeader bool
}
// Reset discards the writer's state and switches the Snappy writer to write to
// w. This permits reusing a Writer rather than allocating a new one.
func (w *Writer) Reset(writer io.Writer) {
w.w = writer
w.err = nil
w.wroteHeader = false
}
// Write satisfies the io.Writer interface.
func (w *Writer) Write(p []byte) (n int, errRet error) {
if w.err != nil {
return 0, w.err
}
if !w.wroteHeader {
copy(w.enc, magicChunk)
if _, err := w.w.Write(w.enc[:len(magicChunk)]); err != nil {
w.err = err
return n, err
}
w.wroteHeader = true
}
for len(p) > 0 {
var uncompressed []byte
if len(p) > maxUncompressedChunkLen {
uncompressed, p = p[:maxUncompressedChunkLen], p[maxUncompressedChunkLen:]
} else {
uncompressed, p = p, nil
}
checksum := crc(uncompressed)
// Compress the buffer, discarding the result if the improvement
// isn't at least 12.5%.
chunkType := uint8(chunkTypeCompressedData)
chunkBody := Encode(w.enc, uncompressed)
if len(chunkBody) >= len(uncompressed)-len(uncompressed)/8 {
chunkType, chunkBody = chunkTypeUncompressedData, uncompressed
}
chunkLen := 4 + len(chunkBody)
w.buf[0] = chunkType
w.buf[1] = uint8(chunkLen >> 0)
w.buf[2] = uint8(chunkLen >> 8)
w.buf[3] = uint8(chunkLen >> 16)
w.buf[4] = uint8(checksum >> 0)
w.buf[5] = uint8(checksum >> 8)
w.buf[6] = uint8(checksum >> 16)
w.buf[7] = uint8(checksum >> 24)
if _, err := w.w.Write(w.buf[:]); err != nil {
w.err = err
return n, err
}
if _, err := w.w.Write(chunkBody); err != nil {
w.err = err
return n, err
}
n += len(uncompressed)
}
return n, nil
}

View File

@@ -1,191 +0,0 @@
All files in this repository are licensed as follows. If you contribute
to this repository, it is assumed that you license your contribution
under the same license unless you state otherwise.
All files Copyright (C) 2015 Canonical Ltd. unless otherwise specified in the file.
This software is licensed under the LGPLv3, included below.
As a special exception to the GNU Lesser General Public License version 3
("LGPL3"), the copyright holders of this Library give you permission to
convey to a third party a Combined Work that links statically or dynamically
to this Library without providing any Minimal Corresponding Source or
Minimal Application Code as set out in 4d or providing the installation
information set out in section 4e, provided that you comply with the other
provisions of LGPL3 and provided that you meet, for the Application the
terms and conditions of the license(s) which apply to the Application.
Except as stated in this special exception, the provisions of LGPL3 will
continue to comply in full to this Library. If you modify this Library, you
may apply this exception to your version of this Library, but you are not
obliged to do so. If you do not wish to do so, delete this exception
statement from your version. This exception does not (and cannot) modify any
license terms which apply to the Application, with which you must still
comply.
GNU LESSER GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
This version of the GNU Lesser General Public License incorporates
the terms and conditions of version 3 of the GNU General Public
License, supplemented by the additional permissions listed below.
0. Additional Definitions.
As used herein, "this License" refers to version 3 of the GNU Lesser
General Public License, and the "GNU GPL" refers to version 3 of the GNU
General Public License.
"The Library" refers to a covered work governed by this License,
other than an Application or a Combined Work as defined below.
An "Application" is any work that makes use of an interface provided
by the Library, but which is not otherwise based on the Library.
Defining a subclass of a class defined by the Library is deemed a mode
of using an interface provided by the Library.
A "Combined Work" is a work produced by combining or linking an
Application with the Library. The particular version of the Library
with which the Combined Work was made is also called the "Linked
Version".
The "Minimal Corresponding Source" for a Combined Work means the
Corresponding Source for the Combined Work, excluding any source code
for portions of the Combined Work that, considered in isolation, are
based on the Application, and not on the Linked Version.
The "Corresponding Application Code" for a Combined Work means the
object code and/or source code for the Application, including any data
and utility programs needed for reproducing the Combined Work from the
Application, but excluding the System Libraries of the Combined Work.
1. Exception to Section 3 of the GNU GPL.
You may convey a covered work under sections 3 and 4 of this License
without being bound by section 3 of the GNU GPL.
2. Conveying Modified Versions.
If you modify a copy of the Library, and, in your modifications, a
facility refers to a function or data to be supplied by an Application
that uses the facility (other than as an argument passed when the
facility is invoked), then you may convey a copy of the modified
version:
a) under this License, provided that you make a good faith effort to
ensure that, in the event an Application does not supply the
function or data, the facility still operates, and performs
whatever part of its purpose remains meaningful, or
b) under the GNU GPL, with none of the additional permissions of
this License applicable to that copy.
3. Object Code Incorporating Material from Library Header Files.
The object code form of an Application may incorporate material from
a header file that is part of the Library. You may convey such object
code under terms of your choice, provided that, if the incorporated
material is not limited to numerical parameters, data structure
layouts and accessors, or small macros, inline functions and templates
(ten or fewer lines in length), you do both of the following:
a) Give prominent notice with each copy of the object code that the
Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the object code with a copy of the GNU GPL and this license
document.
4. Combined Works.
You may convey a Combined Work under terms of your choice that,
taken together, effectively do not restrict modification of the
portions of the Library contained in the Combined Work and reverse
engineering for debugging such modifications, if you also do each of
the following:
a) Give prominent notice with each copy of the Combined Work that
the Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the Combined Work with a copy of the GNU GPL and this license
document.
c) For a Combined Work that displays copyright notices during
execution, include the copyright notice for the Library among
these notices, as well as a reference directing the user to the
copies of the GNU GPL and this license document.
d) Do one of the following:
0) Convey the Minimal Corresponding Source under the terms of this
License, and the Corresponding Application Code in a form
suitable for, and under terms that permit, the user to
recombine or relink the Application with a modified version of
the Linked Version to produce a modified Combined Work, in the
manner specified by section 6 of the GNU GPL for conveying
Corresponding Source.
1) Use a suitable shared library mechanism for linking with the
Library. A suitable mechanism is one that (a) uses at run time
a copy of the Library already present on the user's computer
system, and (b) will operate properly with a modified version
of the Library that is interface-compatible with the Linked
Version.
e) Provide Installation Information, but only if you would otherwise
be required to provide such information under section 6 of the
GNU GPL, and only to the extent that such information is
necessary to install and execute a modified version of the
Combined Work produced by recombining or relinking the
Application with a modified version of the Linked Version. (If
you use option 4d0, the Installation Information must accompany
the Minimal Corresponding Source and Corresponding Application
Code. If you use option 4d1, you must provide the Installation
Information in the manner specified by section 6 of the GNU GPL
for conveying Corresponding Source.)
5. Combined Libraries.
You may place library facilities that are a work based on the
Library side by side in a single library together with other library
facilities that are not Applications and are not covered by this
License, and convey such a combined library under terms of your
choice, if you do both of the following:
a) Accompany the combined library with a copy of the same work based
on the Library, uncombined with any other library facilities,
conveyed under the terms of this License.
b) Give prominent notice with the combined library that part of it
is a work based on the Library, and explaining where to find the
accompanying uncombined form of the same work.
6. Revised Versions of the GNU Lesser General Public License.
The Free Software Foundation may publish revised and/or new versions
of the GNU Lesser General Public License from time to time. Such new
versions will be similar in spirit to the present version, but may
differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the
Library as you received it specifies that a certain numbered version
of the GNU Lesser General Public License "or any later version"
applies to it, you have the option of following the terms and
conditions either of that published version or of any later version
published by the Free Software Foundation. If the Library as you
received it does not specify a version number of the GNU Lesser
General Public License, you may choose any version of the GNU Lesser
General Public License ever published by the Free Software Foundation.
If the Library as you received it specifies that a proxy can decide
whether future versions of the GNU Lesser General Public License shall
apply, that proxy's public statement of acceptance of any version is
permanent authorization for you to choose that version for the
Library.

View File

@@ -1,117 +0,0 @@
# ratelimit
--
import "github.com/juju/ratelimit"
The ratelimit package provides an efficient token bucket implementation. See
http://en.wikipedia.org/wiki/Token_bucket.
## Usage
#### func Reader
```go
func Reader(r io.Reader, bucket *Bucket) io.Reader
```
Reader returns a reader that is rate limited by the given token bucket. Each
token in the bucket represents one byte.
#### func Writer
```go
func Writer(w io.Writer, bucket *Bucket) io.Writer
```
Writer returns a writer that is rate limited by the given token bucket. Each
token in the bucket represents one byte.
#### type Bucket
```go
type Bucket struct {
}
```
Bucket represents a token bucket that fills at a predetermined rate. Methods on
Bucket may be called concurrently.
#### func NewBucket
```go
func NewBucket(fillInterval time.Duration, capacity int64) *Bucket
```
NewBucket returns a new token bucket that fills at the rate of one token every
fillInterval, up to the given maximum capacity. Both arguments must be positive.
The bucket is initially full.
#### func NewBucketWithQuantum
```go
func NewBucketWithQuantum(fillInterval time.Duration, capacity, quantum int64) *Bucket
```
NewBucketWithQuantum is similar to NewBucket, but allows the specification of
the quantum size - quantum tokens are added every fillInterval.
#### func NewBucketWithRate
```go
func NewBucketWithRate(rate float64, capacity int64) *Bucket
```
NewBucketWithRate returns a token bucket that fills the bucket at the rate of
rate tokens per second up to the given maximum capacity. Because of limited
clock resolution, at high rates, the actual rate may be up to 1% different from
the specified rate.
#### func (*Bucket) Rate
```go
func (tb *Bucket) Rate() float64
```
Rate returns the fill rate of the bucket, in tokens per second.
#### func (*Bucket) Take
```go
func (tb *Bucket) Take(count int64) time.Duration
```
Take takes count tokens from the bucket without blocking. It returns the time
that the caller should wait until the tokens are actually available.
Note that if the request is irrevocable - there is no way to return tokens to
the bucket once this method commits us to taking them.
#### func (*Bucket) TakeAvailable
```go
func (tb *Bucket) TakeAvailable(count int64) int64
```
TakeAvailable takes up to count immediately available tokens from the bucket. It
returns the number of tokens removed, or zero if there are no available tokens.
It does not block.
#### func (*Bucket) TakeMaxDuration
```go
func (tb *Bucket) TakeMaxDuration(count int64, maxWait time.Duration) (time.Duration, bool)
```
TakeMaxDuration is like Take, except that it will only take tokens from the
bucket if the wait time for the tokens is no greater than maxWait.
If it would take longer than maxWait for the tokens to become available, it does
nothing and reports false, otherwise it returns the time that the caller should
wait until the tokens are actually available, and reports true.
#### func (*Bucket) Wait
```go
func (tb *Bucket) Wait(count int64)
```
Wait takes count tokens from the bucket, waiting until they are available.
#### func (*Bucket) WaitMaxDuration
```go
func (tb *Bucket) WaitMaxDuration(count int64, maxWait time.Duration) bool
```
WaitMaxDuration is like Wait except that it will only take tokens from the
bucket if it needs to wait for no greater than maxWait. It reports whether any
tokens have been removed from the bucket If no tokens have been removed, it
returns immediately.

View File

@@ -1,221 +0,0 @@
// Copyright 2014 Canonical Ltd.
// Licensed under the LGPLv3 with static-linking exception.
// See LICENCE file for details.
// The ratelimit package provides an efficient token bucket implementation
// that can be used to limit the rate of arbitrary things.
// See http://en.wikipedia.org/wiki/Token_bucket.
package ratelimit
import (
"strconv"
"sync"
"time"
"math"
)
// Bucket represents a token bucket that fills at a predetermined rate.
// Methods on Bucket may be called concurrently.
type Bucket struct {
startTime time.Time
capacity int64
quantum int64
fillInterval time.Duration
// The mutex guards the fields following it.
mu sync.Mutex
// avail holds the number of available tokens
// in the bucket, as of availTick ticks from startTime.
// It will be negative when there are consumers
// waiting for tokens.
avail int64
availTick int64
}
// NewBucket returns a new token bucket that fills at the
// rate of one token every fillInterval, up to the given
// maximum capacity. Both arguments must be
// positive. The bucket is initially full.
func NewBucket(fillInterval time.Duration, capacity int64) *Bucket {
return NewBucketWithQuantum(fillInterval, capacity, 1)
}
// rateMargin specifes the allowed variance of actual
// rate from specified rate. 1% seems reasonable.
const rateMargin = 0.01
// NewBucketWithRate returns a token bucket that fills the bucket
// at the rate of rate tokens per second up to the given
// maximum capacity. Because of limited clock resolution,
// at high rates, the actual rate may be up to 1% different from the
// specified rate.
func NewBucketWithRate(rate float64, capacity int64) *Bucket {
for quantum := int64(1); quantum < 1<<50; quantum = nextQuantum(quantum) {
fillInterval := time.Duration(1e9 * float64(quantum) / rate)
if fillInterval <= 0 {
continue
}
tb := NewBucketWithQuantum(fillInterval, capacity, quantum)
if diff := math.Abs(tb.Rate() - rate); diff/rate <= rateMargin {
return tb
}
}
panic("cannot find suitable quantum for " + strconv.FormatFloat(rate, 'g', -1, 64))
}
// nextQuantum returns the next quantum to try after q.
// We grow the quantum exponentially, but slowly, so we
// get a good fit in the lower numbers.
func nextQuantum(q int64) int64 {
q1 := q * 11 / 10
if q1 == q {
q1++
}
return q1
}
// NewBucketWithQuantum is similar to NewBucket, but allows
// the specification of the quantum size - quantum tokens
// are added every fillInterval.
func NewBucketWithQuantum(fillInterval time.Duration, capacity, quantum int64) *Bucket {
if fillInterval <= 0 {
panic("token bucket fill interval is not > 0")
}
if capacity <= 0 {
panic("token bucket capacity is not > 0")
}
if quantum <= 0 {
panic("token bucket quantum is not > 0")
}
return &Bucket{
startTime: time.Now(),
capacity: capacity,
quantum: quantum,
avail: capacity,
fillInterval: fillInterval,
}
}
// Wait takes count tokens from the bucket, waiting until they are
// available.
func (tb *Bucket) Wait(count int64) {
if d := tb.Take(count); d > 0 {
time.Sleep(d)
}
}
// WaitMaxDuration is like Wait except that it will
// only take tokens from the bucket if it needs to wait
// for no greater than maxWait. It reports whether
// any tokens have been removed from the bucket
// If no tokens have been removed, it returns immediately.
func (tb *Bucket) WaitMaxDuration(count int64, maxWait time.Duration) bool {
d, ok := tb.TakeMaxDuration(count, maxWait)
if d > 0 {
time.Sleep(d)
}
return ok
}
const infinityDuration time.Duration = 0x7fffffffffffffff
// Take takes count tokens from the bucket without blocking. It returns
// the time that the caller should wait until the tokens are actually
// available.
//
// Note that if the request is irrevocable - there is no way to return
// tokens to the bucket once this method commits us to taking them.
func (tb *Bucket) Take(count int64) time.Duration {
d, _ := tb.take(time.Now(), count, infinityDuration)
return d
}
// TakeMaxDuration is like Take, except that
// it will only take tokens from the bucket if the wait
// time for the tokens is no greater than maxWait.
//
// If it would take longer than maxWait for the tokens
// to become available, it does nothing and reports false,
// otherwise it returns the time that the caller should
// wait until the tokens are actually available, and reports
// true.
func (tb *Bucket) TakeMaxDuration(count int64, maxWait time.Duration) (time.Duration, bool) {
return tb.take(time.Now(), count, maxWait)
}
// TakeAvailable takes up to count immediately available tokens from the
// bucket. It returns the number of tokens removed, or zero if there are
// no available tokens. It does not block.
func (tb *Bucket) TakeAvailable(count int64) int64 {
return tb.takeAvailable(time.Now(), count)
}
// takeAvailable is the internal version of TakeAvailable - it takes the
// current time as an argument to enable easy testing.
func (tb *Bucket) takeAvailable(now time.Time, count int64) int64 {
if count <= 0 {
return 0
}
tb.mu.Lock()
defer tb.mu.Unlock()
tb.adjust(now)
if tb.avail <= 0 {
return 0
}
if count > tb.avail {
count = tb.avail
}
tb.avail -= count
return count
}
// Rate returns the fill rate of the bucket, in tokens per second.
func (tb *Bucket) Rate() float64 {
return 1e9 * float64(tb.quantum) / float64(tb.fillInterval)
}
// take is the internal version of Take - it takes the current time as
// an argument to enable easy testing.
func (tb *Bucket) take(now time.Time, count int64, maxWait time.Duration) (time.Duration, bool) {
if count <= 0 {
return 0, true
}
tb.mu.Lock()
defer tb.mu.Unlock()
currentTick := tb.adjust(now)
avail := tb.avail - count
if avail >= 0 {
tb.avail = avail
return 0, true
}
// Round up the missing tokens to the nearest multiple
// of quantum - the tokens won't be available until
// that tick.
endTick := currentTick + (-avail+tb.quantum-1)/tb.quantum
endTime := tb.startTime.Add(time.Duration(endTick) * tb.fillInterval)
waitTime := endTime.Sub(now)
if waitTime > maxWait {
return 0, false
}
tb.avail = avail
return waitTime, true
}
// adjust adjusts the current bucket capacity based on the current time.
// It returns the current tick.
func (tb *Bucket) adjust(now time.Time) (currentTick int64) {
currentTick = int64(now.Sub(tb.startTime) / tb.fillInterval)
if tb.avail >= tb.capacity {
return
}
tb.avail += (currentTick - tb.availTick) * tb.quantum
if tb.avail > tb.capacity {
tb.avail = tb.capacity
}
tb.availTick = currentTick
return
}

View File

@@ -1,51 +0,0 @@
// Copyright 2014 Canonical Ltd.
// Licensed under the LGPLv3 with static-linking exception.
// See LICENCE file for details.
package ratelimit
import "io"
type reader struct {
r io.Reader
bucket *Bucket
}
// Reader returns a reader that is rate limited by
// the given token bucket. Each token in the bucket
// represents one byte.
func Reader(r io.Reader, bucket *Bucket) io.Reader {
return &reader{
r: r,
bucket: bucket,
}
}
func (r *reader) Read(buf []byte) (int, error) {
n, err := r.r.Read(buf)
if n <= 0 {
return n, err
}
r.bucket.Wait(int64(n))
return n, err
}
type writer struct {
w io.Writer
bucket *Bucket
}
// Writer returns a reader that is rate limited by
// the given token bucket. Each token in the bucket
// represents one byte.
func Writer(w io.Writer, bucket *Bucket) io.Writer {
return &writer{
w: w,
bucket: bucket,
}
}
func (w *writer) Write(buf []byte) (int, error) {
w.bucket.Wait(int64(len(buf)))
return w.w.Write(buf)
}

View File

@@ -1,9 +0,0 @@
*.[68]
*.a
*.out
*.swp
_obj
_testmain.go
cmd/metrics-bench/metrics-bench
cmd/metrics-example/metrics-example
cmd/never-read/never-read

View File

@@ -1,13 +0,0 @@
language: go
go:
- 1.2
- 1.3
- 1.4
script:
- ./validate.sh
# this should give us faster builds according to
# http://docs.travis-ci.com/user/migrating-from-legacy/
sudo: false

View File

@@ -1,252 +0,0 @@
// Copyright (c) 2012, Suryandaru Triandana <syndtr@gmail.com>
// All rights reserved.
//
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
package leveldb
import (
"encoding/binary"
"fmt"
"github.com/syndtr/goleveldb/leveldb/errors"
"github.com/syndtr/goleveldb/leveldb/memdb"
)
type ErrBatchCorrupted struct {
Reason string
}
func (e *ErrBatchCorrupted) Error() string {
return fmt.Sprintf("leveldb: batch corrupted: %s", e.Reason)
}
func newErrBatchCorrupted(reason string) error {
return errors.NewErrCorrupted(nil, &ErrBatchCorrupted{reason})
}
const (
batchHdrLen = 8 + 4
batchGrowRec = 3000
)
type BatchReplay interface {
Put(key, value []byte)
Delete(key []byte)
}
// Batch is a write batch.
type Batch struct {
data []byte
rLen, bLen int
seq uint64
sync bool
}
func (b *Batch) grow(n int) {
off := len(b.data)
if off == 0 {
off = batchHdrLen
if b.data != nil {
b.data = b.data[:off]
}
}
if cap(b.data)-off < n {
if b.data == nil {
b.data = make([]byte, off, off+n)
} else {
odata := b.data
div := 1
if b.rLen > batchGrowRec {
div = b.rLen / batchGrowRec
}
b.data = make([]byte, off, off+n+(off-batchHdrLen)/div)
copy(b.data, odata)
}
}
}
func (b *Batch) appendRec(kt kType, key, value []byte) {
n := 1 + binary.MaxVarintLen32 + len(key)
if kt == ktVal {
n += binary.MaxVarintLen32 + len(value)
}
b.grow(n)
off := len(b.data)
data := b.data[:off+n]
data[off] = byte(kt)
off += 1
off += binary.PutUvarint(data[off:], uint64(len(key)))
copy(data[off:], key)
off += len(key)
if kt == ktVal {
off += binary.PutUvarint(data[off:], uint64(len(value)))
copy(data[off:], value)
off += len(value)
}
b.data = data[:off]
b.rLen++
// Include 8-byte ikey header
b.bLen += len(key) + len(value) + 8
}
// Put appends 'put operation' of the given key/value pair to the batch.
// It is safe to modify the contents of the argument after Put returns.
func (b *Batch) Put(key, value []byte) {
b.appendRec(ktVal, key, value)
}
// Delete appends 'delete operation' of the given key to the batch.
// It is safe to modify the contents of the argument after Delete returns.
func (b *Batch) Delete(key []byte) {
b.appendRec(ktDel, key, nil)
}
// Dump dumps batch contents. The returned slice can be loaded into the
// batch using Load method.
// The returned slice is not its own copy, so the contents should not be
// modified.
func (b *Batch) Dump() []byte {
return b.encode()
}
// Load loads given slice into the batch. Previous contents of the batch
// will be discarded.
// The given slice will not be copied and will be used as batch buffer, so
// it is not safe to modify the contents of the slice.
func (b *Batch) Load(data []byte) error {
return b.decode(0, data)
}
// Replay replays batch contents.
func (b *Batch) Replay(r BatchReplay) error {
return b.decodeRec(func(i int, kt kType, key, value []byte) {
switch kt {
case ktVal:
r.Put(key, value)
case ktDel:
r.Delete(key)
}
})
}
// Len returns number of records in the batch.
func (b *Batch) Len() int {
return b.rLen
}
// Reset resets the batch.
func (b *Batch) Reset() {
b.data = b.data[:0]
b.seq = 0
b.rLen = 0
b.bLen = 0
b.sync = false
}
func (b *Batch) init(sync bool) {
b.sync = sync
}
func (b *Batch) append(p *Batch) {
if p.rLen > 0 {
b.grow(len(p.data) - batchHdrLen)
b.data = append(b.data, p.data[batchHdrLen:]...)
b.rLen += p.rLen
}
if p.sync {
b.sync = true
}
}
// size returns sums of key/value pair length plus 8-bytes ikey.
func (b *Batch) size() int {
return b.bLen
}
func (b *Batch) encode() []byte {
b.grow(0)
binary.LittleEndian.PutUint64(b.data, b.seq)
binary.LittleEndian.PutUint32(b.data[8:], uint32(b.rLen))
return b.data
}
func (b *Batch) decode(prevSeq uint64, data []byte) error {
if len(data) < batchHdrLen {
return newErrBatchCorrupted("too short")
}
b.seq = binary.LittleEndian.Uint64(data)
if b.seq < prevSeq {
return newErrBatchCorrupted("invalid sequence number")
}
b.rLen = int(binary.LittleEndian.Uint32(data[8:]))
if b.rLen < 0 {
return newErrBatchCorrupted("invalid records length")
}
// No need to be precise at this point, it won't be used anyway
b.bLen = len(data) - batchHdrLen
b.data = data
return nil
}
func (b *Batch) decodeRec(f func(i int, kt kType, key, value []byte)) (err error) {
off := batchHdrLen
for i := 0; i < b.rLen; i++ {
if off >= len(b.data) {
return newErrBatchCorrupted("invalid records length")
}
kt := kType(b.data[off])
if kt > ktVal {
return newErrBatchCorrupted("bad record: invalid type")
}
off += 1
x, n := binary.Uvarint(b.data[off:])
off += n
if n <= 0 || off+int(x) > len(b.data) {
return newErrBatchCorrupted("bad record: invalid key length")
}
key := b.data[off : off+int(x)]
off += int(x)
var value []byte
if kt == ktVal {
x, n := binary.Uvarint(b.data[off:])
off += n
if n <= 0 || off+int(x) > len(b.data) {
return newErrBatchCorrupted("bad record: invalid value length")
}
value = b.data[off : off+int(x)]
off += int(x)
}
f(i, kt, key, value)
}
return nil
}
func (b *Batch) memReplay(to *memdb.DB) error {
return b.decodeRec(func(i int, kt kType, key, value []byte) {
ikey := newIkey(key, b.seq+uint64(i), kt)
to.Put(ikey, value)
})
}
func (b *Batch) memDecodeAndReplay(prevSeq uint64, data []byte, to *memdb.DB) error {
if err := b.decode(prevSeq, data); err != nil {
return err
}
return b.memReplay(to)
}
func (b *Batch) revertMemReplay(to *memdb.DB) error {
return b.decodeRec(func(i int, kt kType, key, value []byte) {
ikey := newIkey(key, b.seq+uint64(i), kt)
to.Delete(ikey)
})
}

View File

@@ -1,338 +0,0 @@
// Copyright (c) 2012, Suryandaru Triandana <syndtr@gmail.com>
// All rights reserved.
//
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
package leveldb
import (
"time"
"github.com/syndtr/goleveldb/leveldb/memdb"
"github.com/syndtr/goleveldb/leveldb/opt"
"github.com/syndtr/goleveldb/leveldb/util"
)
func (db *DB) writeJournal(b *Batch) error {
w, err := db.journal.Next()
if err != nil {
return err
}
if _, err := w.Write(b.encode()); err != nil {
return err
}
if err := db.journal.Flush(); err != nil {
return err
}
if b.sync {
return db.journalWriter.Sync()
}
return nil
}
func (db *DB) jWriter() {
defer db.closeW.Done()
for {
select {
case b := <-db.journalC:
if b != nil {
db.journalAckC <- db.writeJournal(b)
}
case _, _ = <-db.closeC:
return
}
}
}
func (db *DB) rotateMem(n int) (mem *memDB, err error) {
// Wait for pending memdb compaction.
err = db.compSendIdle(db.mcompCmdC)
if err != nil {
return
}
// Create new memdb and journal.
mem, err = db.newMem(n)
if err != nil {
return
}
// Schedule memdb compaction.
db.compSendTrigger(db.mcompCmdC)
return
}
func (db *DB) flush(n int) (mdb *memDB, mdbFree int, err error) {
delayed := false
flush := func() (retry bool) {
v := db.s.version()
defer v.release()
mdb = db.getEffectiveMem()
defer func() {
if retry {
mdb.decref()
mdb = nil
}
}()
mdbFree = mdb.Free()
switch {
case v.tLen(0) >= db.s.o.GetWriteL0SlowdownTrigger() && !delayed:
delayed = true
time.Sleep(time.Millisecond)
case mdbFree >= n:
return false
case v.tLen(0) >= db.s.o.GetWriteL0PauseTrigger():
delayed = true
err = db.compSendIdle(db.tcompCmdC)
if err != nil {
return false
}
default:
// Allow memdb to grow if it has no entry.
if mdb.Len() == 0 {
mdbFree = n
} else {
mdb.decref()
mdb, err = db.rotateMem(n)
if err == nil {
mdbFree = mdb.Free()
} else {
mdbFree = 0
}
}
return false
}
return true
}
start := time.Now()
for flush() {
}
if delayed {
db.writeDelay += time.Since(start)
db.writeDelayN++
} else if db.writeDelayN > 0 {
db.logf("db@write was delayed N·%d T·%v", db.writeDelayN, db.writeDelay)
db.writeDelay = 0
db.writeDelayN = 0
}
return
}
// Write apply the given batch to the DB. The batch will be applied
// sequentially.
//
// It is safe to modify the contents of the arguments after Write returns.
func (db *DB) Write(b *Batch, wo *opt.WriteOptions) (err error) {
err = db.ok()
if err != nil || b == nil || b.Len() == 0 {
return
}
b.init(wo.GetSync() && !db.s.o.GetNoSync())
// The write happen synchronously.
select {
case db.writeC <- b:
if <-db.writeMergedC {
return <-db.writeAckC
}
case db.writeLockC <- struct{}{}:
case err = <-db.compPerErrC:
return
case _, _ = <-db.closeC:
return ErrClosed
}
merged := 0
danglingMerge := false
defer func() {
if danglingMerge {
db.writeMergedC <- false
} else {
<-db.writeLockC
}
for i := 0; i < merged; i++ {
db.writeAckC <- err
}
}()
mdb, mdbFree, err := db.flush(b.size())
if err != nil {
return
}
defer mdb.decref()
// Calculate maximum size of the batch.
m := 1 << 20
if x := b.size(); x <= 128<<10 {
m = x + (128 << 10)
}
m = minInt(m, mdbFree)
// Merge with other batch.
drain:
for b.size() < m && !b.sync {
select {
case nb := <-db.writeC:
if b.size()+nb.size() <= m {
b.append(nb)
db.writeMergedC <- true
merged++
} else {
danglingMerge = true
break drain
}
default:
break drain
}
}
// Set batch first seq number relative from last seq.
b.seq = db.seq + 1
// Write journal concurrently if it is large enough.
if b.size() >= (128 << 10) {
// Push the write batch to the journal writer
select {
case db.journalC <- b:
// Write into memdb
if berr := b.memReplay(mdb.DB); berr != nil {
panic(berr)
}
case err = <-db.compPerErrC:
return
case _, _ = <-db.closeC:
err = ErrClosed
return
}
// Wait for journal writer
select {
case err = <-db.journalAckC:
if err != nil {
// Revert memdb if error detected
if berr := b.revertMemReplay(mdb.DB); berr != nil {
panic(berr)
}
return
}
case _, _ = <-db.closeC:
err = ErrClosed
return
}
} else {
err = db.writeJournal(b)
if err != nil {
return
}
if berr := b.memReplay(mdb.DB); berr != nil {
panic(berr)
}
}
// Set last seq number.
db.addSeq(uint64(b.Len()))
if b.size() >= mdbFree {
db.rotateMem(0)
}
return
}
// Put sets the value for the given key. It overwrites any previous value
// for that key; a DB is not a multi-map.
//
// It is safe to modify the contents of the arguments after Put returns.
func (db *DB) Put(key, value []byte, wo *opt.WriteOptions) error {
b := new(Batch)
b.Put(key, value)
return db.Write(b, wo)
}
// Delete deletes the value for the given key.
//
// It is safe to modify the contents of the arguments after Delete returns.
func (db *DB) Delete(key []byte, wo *opt.WriteOptions) error {
b := new(Batch)
b.Delete(key)
return db.Write(b, wo)
}
func isMemOverlaps(icmp *iComparer, mem *memdb.DB, min, max []byte) bool {
iter := mem.NewIterator(nil)
defer iter.Release()
return (max == nil || (iter.First() && icmp.uCompare(max, iKey(iter.Key()).ukey()) >= 0)) &&
(min == nil || (iter.Last() && icmp.uCompare(min, iKey(iter.Key()).ukey()) <= 0))
}
// CompactRange compacts the underlying DB for the given key range.
// In particular, deleted and overwritten versions are discarded,
// and the data is rearranged to reduce the cost of operations
// needed to access the data. This operation should typically only
// be invoked by users who understand the underlying implementation.
//
// A nil Range.Start is treated as a key before all keys in the DB.
// And a nil Range.Limit is treated as a key after all keys in the DB.
// Therefore if both is nil then it will compact entire DB.
func (db *DB) CompactRange(r util.Range) error {
if err := db.ok(); err != nil {
return err
}
// Lock writer.
select {
case db.writeLockC <- struct{}{}:
case err := <-db.compPerErrC:
return err
case _, _ = <-db.closeC:
return ErrClosed
}
// Check for overlaps in memdb.
mdb := db.getEffectiveMem()
defer mdb.decref()
if isMemOverlaps(db.s.icmp, mdb.DB, r.Start, r.Limit) {
// Memdb compaction.
if _, err := db.rotateMem(0); err != nil {
<-db.writeLockC
return err
}
<-db.writeLockC
if err := db.compSendIdle(db.mcompCmdC); err != nil {
return err
}
} else {
<-db.writeLockC
}
// Table compaction.
return db.compSendRange(db.tcompCmdC, -1, r.Start, r.Limit)
}
// SetReadOnly makes DB read-only. It will stay read-only until reopened.
func (db *DB) SetReadOnly() error {
if err := db.ok(); err != nil {
return err
}
// Lock writer.
select {
case db.writeLockC <- struct{}{}:
db.compWriteLocking = true
case err := <-db.compPerErrC:
return err
case _, _ = <-db.closeC:
return ErrClosed
}
// Set compaction read-only.
select {
case db.compErrSetC <- ErrReadOnly:
case perr := <-db.compPerErrC:
return perr
case _, _ = <-db.closeC:
return ErrClosed
}
return nil
}

View File

@@ -1,543 +0,0 @@
// Copyright (c) 2012, Suryandaru Triandana <syndtr@gmail.com>
// All rights reservefs.
//
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
package storage
import (
"errors"
"fmt"
"io/ioutil"
"os"
"path/filepath"
"runtime"
"strconv"
"strings"
"sync"
"time"
"github.com/syndtr/goleveldb/leveldb/util"
)
var errFileOpen = errors.New("leveldb/storage: file still open")
type fileLock interface {
release() error
}
type fileStorageLock struct {
fs *fileStorage
}
func (lock *fileStorageLock) Release() {
fs := lock.fs
fs.mu.Lock()
defer fs.mu.Unlock()
if fs.slock == lock {
fs.slock = nil
}
return
}
// fileStorage is a file-system backed storage.
type fileStorage struct {
path string
mu sync.Mutex
flock fileLock
slock *fileStorageLock
logw *os.File
buf []byte
// Opened file counter; if open < 0 means closed.
open int
day int
}
// OpenFile returns a new filesytem-backed storage implementation with the given
// path. This also hold a file lock, so any subsequent attempt to open the same
// path will fail.
//
// The storage must be closed after use, by calling Close method.
func OpenFile(path string) (Storage, error) {
if err := os.MkdirAll(path, 0755); err != nil {
return nil, err
}
flock, err := newFileLock(filepath.Join(path, "LOCK"))
if err != nil {
return nil, err
}
defer func() {
if err != nil {
flock.release()
}
}()
rename(filepath.Join(path, "LOG"), filepath.Join(path, "LOG.old"))
logw, err := os.OpenFile(filepath.Join(path, "LOG"), os.O_WRONLY|os.O_CREATE, 0644)
if err != nil {
return nil, err
}
fs := &fileStorage{path: path, flock: flock, logw: logw}
runtime.SetFinalizer(fs, (*fileStorage).Close)
return fs, nil
}
func (fs *fileStorage) Lock() (util.Releaser, error) {
fs.mu.Lock()
defer fs.mu.Unlock()
if fs.open < 0 {
return nil, ErrClosed
}
if fs.slock != nil {
return nil, ErrLocked
}
fs.slock = &fileStorageLock{fs: fs}
return fs.slock, nil
}
func itoa(buf []byte, i int, wid int) []byte {
var u uint = uint(i)
if u == 0 && wid <= 1 {
return append(buf, '0')
}
// Assemble decimal in reverse order.
var b [32]byte
bp := len(b)
for ; u > 0 || wid > 0; u /= 10 {
bp--
wid--
b[bp] = byte(u%10) + '0'
}
return append(buf, b[bp:]...)
}
func (fs *fileStorage) printDay(t time.Time) {
if fs.day == t.Day() {
return
}
fs.day = t.Day()
fs.logw.Write([]byte("=============== " + t.Format("Jan 2, 2006 (MST)") + " ===============\n"))
}
func (fs *fileStorage) doLog(t time.Time, str string) {
fs.printDay(t)
hour, min, sec := t.Clock()
msec := t.Nanosecond() / 1e3
// time
fs.buf = itoa(fs.buf[:0], hour, 2)
fs.buf = append(fs.buf, ':')
fs.buf = itoa(fs.buf, min, 2)
fs.buf = append(fs.buf, ':')
fs.buf = itoa(fs.buf, sec, 2)
fs.buf = append(fs.buf, '.')
fs.buf = itoa(fs.buf, msec, 6)
fs.buf = append(fs.buf, ' ')
// write
fs.buf = append(fs.buf, []byte(str)...)
fs.buf = append(fs.buf, '\n')
fs.logw.Write(fs.buf)
}
func (fs *fileStorage) Log(str string) {
t := time.Now()
fs.mu.Lock()
defer fs.mu.Unlock()
if fs.open < 0 {
return
}
fs.doLog(t, str)
}
func (fs *fileStorage) log(str string) {
fs.doLog(time.Now(), str)
}
func (fs *fileStorage) GetFile(num uint64, t FileType) File {
return &file{fs: fs, num: num, t: t}
}
func (fs *fileStorage) GetFiles(t FileType) (ff []File, err error) {
fs.mu.Lock()
defer fs.mu.Unlock()
if fs.open < 0 {
return nil, ErrClosed
}
dir, err := os.Open(fs.path)
if err != nil {
return
}
fnn, err := dir.Readdirnames(0)
// Close the dir first before checking for Readdirnames error.
if err := dir.Close(); err != nil {
fs.log(fmt.Sprintf("close dir: %v", err))
}
if err != nil {
return
}
f := &file{fs: fs}
for _, fn := range fnn {
if f.parse(fn) && (f.t&t) != 0 {
ff = append(ff, f)
f = &file{fs: fs}
}
}
return
}
func (fs *fileStorage) GetManifest() (f File, err error) {
fs.mu.Lock()
defer fs.mu.Unlock()
if fs.open < 0 {
return nil, ErrClosed
}
dir, err := os.Open(fs.path)
if err != nil {
return
}
fnn, err := dir.Readdirnames(0)
// Close the dir first before checking for Readdirnames error.
if err := dir.Close(); err != nil {
fs.log(fmt.Sprintf("close dir: %v", err))
}
if err != nil {
return
}
// Find latest CURRENT file.
var rem []string
var pend bool
var cerr error
for _, fn := range fnn {
if strings.HasPrefix(fn, "CURRENT") {
pend1 := len(fn) > 7
// Make sure it is valid name for a CURRENT file, otherwise skip it.
if pend1 {
if fn[7] != '.' || len(fn) < 9 {
fs.log(fmt.Sprintf("skipping %s: invalid file name", fn))
continue
}
if _, e1 := strconv.ParseUint(fn[8:], 10, 0); e1 != nil {
fs.log(fmt.Sprintf("skipping %s: invalid file num: %v", fn, e1))
continue
}
}
path := filepath.Join(fs.path, fn)
r, e1 := os.OpenFile(path, os.O_RDONLY, 0)
if e1 != nil {
return nil, e1
}
b, e1 := ioutil.ReadAll(r)
if e1 != nil {
r.Close()
return nil, e1
}
f1 := &file{fs: fs}
if len(b) < 1 || b[len(b)-1] != '\n' || !f1.parse(string(b[:len(b)-1])) {
fs.log(fmt.Sprintf("skipping %s: corrupted or incomplete", fn))
if pend1 {
rem = append(rem, fn)
}
if !pend1 || cerr == nil {
cerr = &ErrCorrupted{
File: fsParseName(filepath.Base(fn)),
Err: errors.New("leveldb/storage: corrupted or incomplete manifest file"),
}
}
} else if f != nil && f1.Num() < f.Num() {
fs.log(fmt.Sprintf("skipping %s: obsolete", fn))
if pend1 {
rem = append(rem, fn)
}
} else {
f = f1
pend = pend1
}
if err := r.Close(); err != nil {
fs.log(fmt.Sprintf("close %s: %v", fn, err))
}
}
}
// Don't remove any files if there is no valid CURRENT file.
if f == nil {
if cerr != nil {
err = cerr
} else {
err = os.ErrNotExist
}
return
}
// Rename pending CURRENT file to an effective CURRENT.
if pend {
path := fmt.Sprintf("%s.%d", filepath.Join(fs.path, "CURRENT"), f.Num())
if err := rename(path, filepath.Join(fs.path, "CURRENT")); err != nil {
fs.log(fmt.Sprintf("CURRENT.%d -> CURRENT: %v", f.Num(), err))
}
}
// Remove obsolete or incomplete pending CURRENT files.
for _, fn := range rem {
path := filepath.Join(fs.path, fn)
if err := os.Remove(path); err != nil {
fs.log(fmt.Sprintf("remove %s: %v", fn, err))
}
}
return
}
func (fs *fileStorage) SetManifest(f File) (err error) {
fs.mu.Lock()
defer fs.mu.Unlock()
if fs.open < 0 {
return ErrClosed
}
f2, ok := f.(*file)
if !ok || f2.t != TypeManifest {
return ErrInvalidFile
}
defer func() {
if err != nil {
fs.log(fmt.Sprintf("CURRENT: %v", err))
}
}()
path := fmt.Sprintf("%s.%d", filepath.Join(fs.path, "CURRENT"), f2.Num())
w, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0644)
if err != nil {
return err
}
_, err = fmt.Fprintln(w, f2.name())
// Close the file first.
if err := w.Close(); err != nil {
fs.log(fmt.Sprintf("close CURRENT.%d: %v", f2.num, err))
}
if err != nil {
return err
}
return rename(path, filepath.Join(fs.path, "CURRENT"))
}
func (fs *fileStorage) Close() error {
fs.mu.Lock()
defer fs.mu.Unlock()
if fs.open < 0 {
return ErrClosed
}
// Clear the finalizer.
runtime.SetFinalizer(fs, nil)
if fs.open > 0 {
fs.log(fmt.Sprintf("close: warning, %d files still open", fs.open))
}
fs.open = -1
e1 := fs.logw.Close()
err := fs.flock.release()
if err == nil {
err = e1
}
return err
}
type fileWrap struct {
*os.File
f *file
}
func (fw fileWrap) Sync() error {
if err := fw.File.Sync(); err != nil {
return err
}
if fw.f.Type() == TypeManifest {
// Also sync parent directory if file type is manifest.
// See: https://code.google.com/p/leveldb/issues/detail?id=190.
if err := syncDir(fw.f.fs.path); err != nil {
return err
}
}
return nil
}
func (fw fileWrap) Close() error {
f := fw.f
f.fs.mu.Lock()
defer f.fs.mu.Unlock()
if !f.open {
return ErrClosed
}
f.open = false
f.fs.open--
err := fw.File.Close()
if err != nil {
f.fs.log(fmt.Sprintf("close %s.%d: %v", f.Type(), f.Num(), err))
}
return err
}
type file struct {
fs *fileStorage
num uint64
t FileType
open bool
}
func (f *file) Open() (Reader, error) {
f.fs.mu.Lock()
defer f.fs.mu.Unlock()
if f.fs.open < 0 {
return nil, ErrClosed
}
if f.open {
return nil, errFileOpen
}
of, err := os.OpenFile(f.path(), os.O_RDONLY, 0)
if err != nil {
if f.hasOldName() && os.IsNotExist(err) {
of, err = os.OpenFile(f.oldPath(), os.O_RDONLY, 0)
if err == nil {
goto ok
}
}
return nil, err
}
ok:
f.open = true
f.fs.open++
return fileWrap{of, f}, nil
}
func (f *file) Create() (Writer, error) {
f.fs.mu.Lock()
defer f.fs.mu.Unlock()
if f.fs.open < 0 {
return nil, ErrClosed
}
if f.open {
return nil, errFileOpen
}
of, err := os.OpenFile(f.path(), os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0644)
if err != nil {
return nil, err
}
f.open = true
f.fs.open++
return fileWrap{of, f}, nil
}
func (f *file) Replace(newfile File) error {
f.fs.mu.Lock()
defer f.fs.mu.Unlock()
if f.fs.open < 0 {
return ErrClosed
}
newfile2, ok := newfile.(*file)
if !ok {
return ErrInvalidFile
}
if f.open || newfile2.open {
return errFileOpen
}
return rename(newfile2.path(), f.path())
}
func (f *file) Type() FileType {
return f.t
}
func (f *file) Num() uint64 {
return f.num
}
func (f *file) Remove() error {
f.fs.mu.Lock()
defer f.fs.mu.Unlock()
if f.fs.open < 0 {
return ErrClosed
}
if f.open {
return errFileOpen
}
err := os.Remove(f.path())
if err != nil {
f.fs.log(fmt.Sprintf("remove %s.%d: %v", f.Type(), f.Num(), err))
}
// Also try remove file with old name, just in case.
if f.hasOldName() {
if e1 := os.Remove(f.oldPath()); !os.IsNotExist(e1) {
f.fs.log(fmt.Sprintf("remove %s.%d: %v (old name)", f.Type(), f.Num(), err))
err = e1
}
}
return err
}
func (f *file) hasOldName() bool {
return f.t == TypeTable
}
func (f *file) oldName() string {
switch f.t {
case TypeTable:
return fmt.Sprintf("%06d.sst", f.num)
}
return f.name()
}
func (f *file) oldPath() string {
return filepath.Join(f.fs.path, f.oldName())
}
func (f *file) name() string {
switch f.t {
case TypeManifest:
return fmt.Sprintf("MANIFEST-%06d", f.num)
case TypeJournal:
return fmt.Sprintf("%06d.log", f.num)
case TypeTable:
return fmt.Sprintf("%06d.ldb", f.num)
case TypeTemp:
return fmt.Sprintf("%06d.tmp", f.num)
default:
panic("invalid file type")
}
}
func (f *file) path() string {
return filepath.Join(f.fs.path, f.name())
}
func fsParseName(name string) *FileInfo {
fi := &FileInfo{}
var tail string
_, err := fmt.Sscanf(name, "%d.%s", &fi.Num, &tail)
if err == nil {
switch tail {
case "log":
fi.Type = TypeJournal
case "ldb", "sst":
fi.Type = TypeTable
case "tmp":
fi.Type = TypeTemp
default:
return nil
}
return fi
}
n, _ := fmt.Sscanf(name, "MANIFEST-%d%s", &fi.Num, &tail)
if n == 1 {
fi.Type = TypeManifest
return fi
}
return nil
}
func (f *file) parse(name string) bool {
fi := fsParseName(name)
if fi == nil {
return false
}
f.t = fi.Type
f.num = fi.Num
return true
}

View File

@@ -1,203 +0,0 @@
// Copyright (c) 2013, Suryandaru Triandana <syndtr@gmail.com>
// All rights reserved.
//
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
package storage
import (
"bytes"
"os"
"sync"
"github.com/syndtr/goleveldb/leveldb/util"
)
const typeShift = 3
type memStorageLock struct {
ms *memStorage
}
func (lock *memStorageLock) Release() {
ms := lock.ms
ms.mu.Lock()
defer ms.mu.Unlock()
if ms.slock == lock {
ms.slock = nil
}
return
}
// memStorage is a memory-backed storage.
type memStorage struct {
mu sync.Mutex
slock *memStorageLock
files map[uint64]*memFile
manifest *memFilePtr
}
// NewMemStorage returns a new memory-backed storage implementation.
func NewMemStorage() Storage {
return &memStorage{
files: make(map[uint64]*memFile),
}
}
func (ms *memStorage) Lock() (util.Releaser, error) {
ms.mu.Lock()
defer ms.mu.Unlock()
if ms.slock != nil {
return nil, ErrLocked
}
ms.slock = &memStorageLock{ms: ms}
return ms.slock, nil
}
func (*memStorage) Log(str string) {}
func (ms *memStorage) GetFile(num uint64, t FileType) File {
return &memFilePtr{ms: ms, num: num, t: t}
}
func (ms *memStorage) GetFiles(t FileType) ([]File, error) {
ms.mu.Lock()
var ff []File
for x, _ := range ms.files {
num, mt := x>>typeShift, FileType(x)&TypeAll
if mt&t == 0 {
continue
}
ff = append(ff, &memFilePtr{ms: ms, num: num, t: mt})
}
ms.mu.Unlock()
return ff, nil
}
func (ms *memStorage) GetManifest() (File, error) {
ms.mu.Lock()
defer ms.mu.Unlock()
if ms.manifest == nil {
return nil, os.ErrNotExist
}
return ms.manifest, nil
}
func (ms *memStorage) SetManifest(f File) error {
fm, ok := f.(*memFilePtr)
if !ok || fm.t != TypeManifest {
return ErrInvalidFile
}
ms.mu.Lock()
ms.manifest = fm
ms.mu.Unlock()
return nil
}
func (*memStorage) Close() error { return nil }
type memReader struct {
*bytes.Reader
m *memFile
}
func (mr *memReader) Close() error {
return mr.m.Close()
}
type memFile struct {
bytes.Buffer
ms *memStorage
open bool
}
func (*memFile) Sync() error { return nil }
func (m *memFile) Close() error {
m.ms.mu.Lock()
m.open = false
m.ms.mu.Unlock()
return nil
}
type memFilePtr struct {
ms *memStorage
num uint64
t FileType
}
func (p *memFilePtr) x() uint64 {
return p.Num()<<typeShift | uint64(p.Type())
}
func (p *memFilePtr) Open() (Reader, error) {
ms := p.ms
ms.mu.Lock()
defer ms.mu.Unlock()
if m, exist := ms.files[p.x()]; exist {
if m.open {
return nil, errFileOpen
}
m.open = true
return &memReader{Reader: bytes.NewReader(m.Bytes()), m: m}, nil
}
return nil, os.ErrNotExist
}
func (p *memFilePtr) Create() (Writer, error) {
ms := p.ms
ms.mu.Lock()
defer ms.mu.Unlock()
m, exist := ms.files[p.x()]
if exist {
if m.open {
return nil, errFileOpen
}
m.Reset()
} else {
m = &memFile{ms: ms}
ms.files[p.x()] = m
}
m.open = true
return m, nil
}
func (p *memFilePtr) Replace(newfile File) error {
p1, ok := newfile.(*memFilePtr)
if !ok {
return ErrInvalidFile
}
ms := p.ms
ms.mu.Lock()
defer ms.mu.Unlock()
m1, exist := ms.files[p1.x()]
if !exist {
return os.ErrNotExist
}
m0, exist := ms.files[p.x()]
if (exist && m0.open) || m1.open {
return errFileOpen
}
delete(ms.files, p1.x())
ms.files[p.x()] = m1
return nil
}
func (p *memFilePtr) Type() FileType {
return p.t
}
func (p *memFilePtr) Num() uint64 {
return p.num
}
func (p *memFilePtr) Remove() error {
ms := p.ms
ms.mu.Lock()
defer ms.mu.Unlock()
if _, exist := ms.files[p.x()]; exist {
delete(ms.files, p.x())
return nil
}
return os.ErrNotExist
}

View File

@@ -1,173 +0,0 @@
// Copyright (c) 2012, Suryandaru Triandana <syndtr@gmail.com>
// All rights reserved.
//
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
// Package storage provides storage abstraction for LevelDB.
package storage
import (
"errors"
"fmt"
"io"
"github.com/syndtr/goleveldb/leveldb/util"
)
type FileType uint32
const (
TypeManifest FileType = 1 << iota
TypeJournal
TypeTable
TypeTemp
TypeAll = TypeManifest | TypeJournal | TypeTable | TypeTemp
)
func (t FileType) String() string {
switch t {
case TypeManifest:
return "manifest"
case TypeJournal:
return "journal"
case TypeTable:
return "table"
case TypeTemp:
return "temp"
}
return fmt.Sprintf("<unknown:%d>", t)
}
var (
ErrInvalidFile = errors.New("leveldb/storage: invalid file for argument")
ErrLocked = errors.New("leveldb/storage: already locked")
ErrClosed = errors.New("leveldb/storage: closed")
)
// ErrCorrupted is the type that wraps errors that indicate corruption of
// a file. Package storage has its own type instead of using
// errors.ErrCorrupted to prevent circular import.
type ErrCorrupted struct {
File *FileInfo
Err error
}
func (e *ErrCorrupted) Error() string {
if e.File != nil {
return fmt.Sprintf("%v [file=%v]", e.Err, e.File)
} else {
return e.Err.Error()
}
}
// Syncer is the interface that wraps basic Sync method.
type Syncer interface {
// Sync commits the current contents of the file to stable storage.
Sync() error
}
// Reader is the interface that groups the basic Read, Seek, ReadAt and Close
// methods.
type Reader interface {
io.ReadSeeker
io.ReaderAt
io.Closer
}
// Writer is the interface that groups the basic Write, Sync and Close
// methods.
type Writer interface {
io.WriteCloser
Syncer
}
// File is the file. A file instance must be goroutine-safe.
type File interface {
// Open opens the file for read. Returns os.ErrNotExist error
// if the file does not exist.
// Returns ErrClosed if the underlying storage is closed.
Open() (r Reader, err error)
// Create creates the file for writting. Truncate the file if
// already exist.
// Returns ErrClosed if the underlying storage is closed.
Create() (w Writer, err error)
// Replace replaces file with newfile.
// Returns ErrClosed if the underlying storage is closed.
Replace(newfile File) error
// Type returns the file type
Type() FileType
// Num returns the file number.
Num() uint64
// Remove removes the file.
// Returns ErrClosed if the underlying storage is closed.
Remove() error
}
// Storage is the storage. A storage instance must be goroutine-safe.
type Storage interface {
// Lock locks the storage. Any subsequent attempt to call Lock will fail
// until the last lock released.
// After use the caller should call the Release method.
Lock() (l util.Releaser, err error)
// Log logs a string. This is used for logging. An implementation
// may write to a file, stdout or simply do nothing.
Log(str string)
// GetFile returns a file for the given number and type. GetFile will never
// returns nil, even if the underlying storage is closed.
GetFile(num uint64, t FileType) File
// GetFiles returns a slice of files that match the given file types.
// The file types may be OR'ed together.
GetFiles(t FileType) ([]File, error)
// GetManifest returns a manifest file. Returns os.ErrNotExist if manifest
// file does not exist.
GetManifest() (File, error)
// SetManifest sets the given file as manifest file. The given file should
// be a manifest file type or error will be returned.
SetManifest(f File) error
// Close closes the storage. It is valid to call Close multiple times.
// Other methods should not be called after the storage has been closed.
Close() error
}
// FileInfo wraps basic file info.
type FileInfo struct {
Type FileType
Num uint64
}
func (fi FileInfo) String() string {
switch fi.Type {
case TypeManifest:
return fmt.Sprintf("MANIFEST-%06d", fi.Num)
case TypeJournal:
return fmt.Sprintf("%06d.log", fi.Num)
case TypeTable:
return fmt.Sprintf("%06d.ldb", fi.Num)
case TypeTemp:
return fmt.Sprintf("%06d.tmp", fi.Num)
default:
return fmt.Sprintf("%#x-%d", fi.Type, fi.Num)
}
}
// NewFileInfo creates new FileInfo from the given File. It will returns nil
// if File is nil.
func NewFileInfo(f File) *FileInfo {
if f == nil {
return nil
}
return &FileInfo{f.Type(), f.Num()}
}

View File

@@ -1,187 +0,0 @@
// Copyright (c) 2014, Suryandaru Triandana <syndtr@gmail.com>
// All rights reserved.
//
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
package testutil
import (
"fmt"
"math/rand"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
"github.com/syndtr/goleveldb/leveldb/errors"
"github.com/syndtr/goleveldb/leveldb/util"
)
func KeyValueTesting(rnd *rand.Rand, kv KeyValue, p DB, setup func(KeyValue) DB, teardown func(DB)) {
if rnd == nil {
rnd = NewRand()
}
if p == nil {
BeforeEach(func() {
p = setup(kv)
})
if teardown != nil {
AfterEach(func() {
teardown(p)
})
}
}
It("Should find all keys with Find", func() {
if db, ok := p.(Find); ok {
ShuffledIndex(nil, kv.Len(), 1, func(i int) {
key_, key, value := kv.IndexInexact(i)
// Using exact key.
rkey, rvalue, err := db.TestFind(key)
Expect(err).ShouldNot(HaveOccurred(), "Error for key %q", key)
Expect(rkey).Should(Equal(key), "Key")
Expect(rvalue).Should(Equal(value), "Value for key %q", key)
// Using inexact key.
rkey, rvalue, err = db.TestFind(key_)
Expect(err).ShouldNot(HaveOccurred(), "Error for key %q (%q)", key_, key)
Expect(rkey).Should(Equal(key))
Expect(rvalue).Should(Equal(value), "Value for key %q (%q)", key_, key)
})
}
})
It("Should return error if the key is not present", func() {
if db, ok := p.(Find); ok {
var key []byte
if kv.Len() > 0 {
key_, _ := kv.Index(kv.Len() - 1)
key = BytesAfter(key_)
}
rkey, _, err := db.TestFind(key)
Expect(err).Should(HaveOccurred(), "Find for key %q yield key %q", key, rkey)
Expect(err).Should(Equal(errors.ErrNotFound))
}
})
It("Should only find exact key with Get", func() {
if db, ok := p.(Get); ok {
ShuffledIndex(nil, kv.Len(), 1, func(i int) {
key_, key, value := kv.IndexInexact(i)
// Using exact key.
rvalue, err := db.TestGet(key)
Expect(err).ShouldNot(HaveOccurred(), "Error for key %q", key)
Expect(rvalue).Should(Equal(value), "Value for key %q", key)
// Using inexact key.
if len(key_) > 0 {
_, err = db.TestGet(key_)
Expect(err).Should(HaveOccurred(), "Error for key %q", key_)
Expect(err).Should(Equal(errors.ErrNotFound))
}
})
}
})
It("Should only find present key with Has", func() {
if db, ok := p.(Has); ok {
ShuffledIndex(nil, kv.Len(), 1, func(i int) {
key_, key, _ := kv.IndexInexact(i)
// Using exact key.
ret, err := db.TestHas(key)
Expect(err).ShouldNot(HaveOccurred(), "Error for key %q", key)
Expect(ret).Should(BeTrue(), "False for key %q", key)
// Using inexact key.
if len(key_) > 0 {
ret, err = db.TestHas(key_)
Expect(err).ShouldNot(HaveOccurred(), "Error for key %q", key_)
Expect(ret).ShouldNot(BeTrue(), "True for key %q", key)
}
})
}
})
TestIter := func(r *util.Range, _kv KeyValue) {
if db, ok := p.(NewIterator); ok {
iter := db.TestNewIterator(r)
Expect(iter.Error()).ShouldNot(HaveOccurred())
t := IteratorTesting{
KeyValue: _kv,
Iter: iter,
}
DoIteratorTesting(&t)
iter.Release()
}
}
It("Should iterates and seeks correctly", func(done Done) {
TestIter(nil, kv.Clone())
done <- true
}, 3.0)
RandomIndex(rnd, kv.Len(), Min(kv.Len(), 50), func(i int) {
type slice struct {
r *util.Range
start, limit int
}
key_, _, _ := kv.IndexInexact(i)
for _, x := range []slice{
{&util.Range{Start: key_, Limit: nil}, i, kv.Len()},
{&util.Range{Start: nil, Limit: key_}, 0, i},
} {
It(fmt.Sprintf("Should iterates and seeks correctly of a slice %d .. %d", x.start, x.limit), func(done Done) {
TestIter(x.r, kv.Slice(x.start, x.limit))
done <- true
}, 3.0)
}
})
RandomRange(rnd, kv.Len(), Min(kv.Len(), 50), func(start, limit int) {
It(fmt.Sprintf("Should iterates and seeks correctly of a slice %d .. %d", start, limit), func(done Done) {
r := kv.Range(start, limit)
TestIter(&r, kv.Slice(start, limit))
done <- true
}, 3.0)
})
}
func AllKeyValueTesting(rnd *rand.Rand, body, setup func(KeyValue) DB, teardown func(DB)) {
Test := func(kv *KeyValue) func() {
return func() {
var p DB
if setup != nil {
Defer("setup", func() {
p = setup(*kv)
})
}
if teardown != nil {
Defer("teardown", func() {
teardown(p)
})
}
if body != nil {
p = body(*kv)
}
KeyValueTesting(rnd, *kv, p, func(KeyValue) DB {
return p
}, nil)
}
}
Describe("with no key/value (empty)", Test(&KeyValue{}))
Describe("with empty key", Test(KeyValue_EmptyKey()))
Describe("with empty value", Test(KeyValue_EmptyValue()))
Describe("with one key/value", Test(KeyValue_OneKeyValue()))
Describe("with big value", Test(KeyValue_BigValue()))
Describe("with special key", Test(KeyValue_SpecialKey()))
Describe("with multiple key/value", Test(KeyValue_MultipleKeyValue()))
Describe("with generated key/value", Test(KeyValue_Generate(nil, 120, 1, 50, 10, 120)))
}

View File

@@ -1,586 +0,0 @@
// Copyright (c) 2014, Suryandaru Triandana <syndtr@gmail.com>
// All rights reserved.
//
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
package testutil
import (
"bytes"
"fmt"
"io"
"os"
"path/filepath"
"runtime"
"strings"
"sync"
. "github.com/onsi/gomega"
"github.com/syndtr/goleveldb/leveldb/storage"
"github.com/syndtr/goleveldb/leveldb/util"
)
var (
storageMu sync.Mutex
storageUseFS bool = true
storageKeepFS bool = false
storageNum int
)
type StorageMode int
const (
ModeOpen StorageMode = 1 << iota
ModeCreate
ModeRemove
ModeRead
ModeWrite
ModeSync
ModeClose
)
const (
modeOpen = iota
modeCreate
modeRemove
modeRead
modeWrite
modeSync
modeClose
modeCount
)
const (
typeManifest = iota
typeJournal
typeTable
typeTemp
typeCount
)
const flattenCount = modeCount * typeCount
func flattenType(m StorageMode, t storage.FileType) int {
var x int
switch m {
case ModeOpen:
x = modeOpen
case ModeCreate:
x = modeCreate
case ModeRemove:
x = modeRemove
case ModeRead:
x = modeRead
case ModeWrite:
x = modeWrite
case ModeSync:
x = modeSync
case ModeClose:
x = modeClose
default:
panic("invalid storage mode")
}
x *= typeCount
switch t {
case storage.TypeManifest:
return x + typeManifest
case storage.TypeJournal:
return x + typeJournal
case storage.TypeTable:
return x + typeTable
case storage.TypeTemp:
return x + typeTemp
default:
panic("invalid file type")
}
}
func listFlattenType(m StorageMode, t storage.FileType) []int {
ret := make([]int, 0, flattenCount)
add := func(x int) {
x *= typeCount
switch {
case t&storage.TypeManifest != 0:
ret = append(ret, x+typeManifest)
case t&storage.TypeJournal != 0:
ret = append(ret, x+typeJournal)
case t&storage.TypeTable != 0:
ret = append(ret, x+typeTable)
case t&storage.TypeTemp != 0:
ret = append(ret, x+typeTemp)
}
}
switch {
case m&ModeOpen != 0:
add(modeOpen)
case m&ModeCreate != 0:
add(modeCreate)
case m&ModeRemove != 0:
add(modeRemove)
case m&ModeRead != 0:
add(modeRead)
case m&ModeWrite != 0:
add(modeWrite)
case m&ModeSync != 0:
add(modeSync)
case m&ModeClose != 0:
add(modeClose)
}
return ret
}
func packFile(num uint64, t storage.FileType) uint64 {
if num>>(64-typeCount) != 0 {
panic("overflow")
}
return num<<typeCount | uint64(t)
}
func unpackFile(x uint64) (uint64, storage.FileType) {
return x >> typeCount, storage.FileType(x) & storage.TypeAll
}
type emulatedError struct {
err error
}
func (err emulatedError) Error() string {
return fmt.Sprintf("emulated storage error: %v", err.err)
}
type storageLock struct {
s *Storage
r util.Releaser
}
func (l storageLock) Release() {
l.r.Release()
l.s.logI("storage lock released")
}
type reader struct {
f *file
storage.Reader
}
func (r *reader) Read(p []byte) (n int, err error) {
err = r.f.s.emulateError(ModeRead, r.f.Type())
if err == nil {
r.f.s.stall(ModeRead, r.f.Type())
n, err = r.Reader.Read(p)
}
r.f.s.count(ModeRead, r.f.Type(), n)
if err != nil && err != io.EOF {
r.f.s.logI("read error, num=%d type=%v n=%d err=%v", r.f.Num(), r.f.Type(), n, err)
}
return
}
func (r *reader) ReadAt(p []byte, off int64) (n int, err error) {
err = r.f.s.emulateError(ModeRead, r.f.Type())
if err == nil {
r.f.s.stall(ModeRead, r.f.Type())
n, err = r.Reader.ReadAt(p, off)
}
r.f.s.count(ModeRead, r.f.Type(), n)
if err != nil && err != io.EOF {
r.f.s.logI("readAt error, num=%d type=%v offset=%d n=%d err=%v", r.f.Num(), r.f.Type(), off, n, err)
}
return
}
func (r *reader) Close() (err error) {
return r.f.doClose(r.Reader)
}
type writer struct {
f *file
storage.Writer
}
func (w *writer) Write(p []byte) (n int, err error) {
err = w.f.s.emulateError(ModeWrite, w.f.Type())
if err == nil {
w.f.s.stall(ModeWrite, w.f.Type())
n, err = w.Writer.Write(p)
}
w.f.s.count(ModeWrite, w.f.Type(), n)
if err != nil && err != io.EOF {
w.f.s.logI("write error, num=%d type=%v n=%d err=%v", w.f.Num(), w.f.Type(), n, err)
}
return
}
func (w *writer) Sync() (err error) {
err = w.f.s.emulateError(ModeSync, w.f.Type())
if err == nil {
w.f.s.stall(ModeSync, w.f.Type())
err = w.Writer.Sync()
}
w.f.s.count(ModeSync, w.f.Type(), 0)
if err != nil {
w.f.s.logI("sync error, num=%d type=%v err=%v", w.f.Num(), w.f.Type(), err)
}
return
}
func (w *writer) Close() (err error) {
return w.f.doClose(w.Writer)
}
type file struct {
s *Storage
storage.File
}
func (f *file) pack() uint64 {
return packFile(f.Num(), f.Type())
}
func (f *file) assertOpen() {
ExpectWithOffset(2, f.s.opens).NotTo(HaveKey(f.pack()), "File open, num=%d type=%v writer=%v", f.Num(), f.Type(), f.s.opens[f.pack()])
}
func (f *file) doClose(closer io.Closer) (err error) {
err = f.s.emulateError(ModeClose, f.Type())
if err == nil {
f.s.stall(ModeClose, f.Type())
}
f.s.mu.Lock()
defer f.s.mu.Unlock()
if err == nil {
ExpectWithOffset(2, f.s.opens).To(HaveKey(f.pack()), "File closed, num=%d type=%v", f.Num(), f.Type())
err = closer.Close()
}
f.s.countNB(ModeClose, f.Type(), 0)
writer := f.s.opens[f.pack()]
if err != nil {
f.s.logISkip(1, "file close failed, num=%d type=%v writer=%v err=%v", f.Num(), f.Type(), writer, err)
} else {
f.s.logISkip(1, "file closed, num=%d type=%v writer=%v", f.Num(), f.Type(), writer)
delete(f.s.opens, f.pack())
}
return
}
func (f *file) Open() (r storage.Reader, err error) {
err = f.s.emulateError(ModeOpen, f.Type())
if err == nil {
f.s.stall(ModeOpen, f.Type())
}
f.s.mu.Lock()
defer f.s.mu.Unlock()
if err == nil {
f.assertOpen()
f.s.countNB(ModeOpen, f.Type(), 0)
r, err = f.File.Open()
}
if err != nil {
f.s.logI("file open failed, num=%d type=%v err=%v", f.Num(), f.Type(), err)
} else {
f.s.logI("file opened, num=%d type=%v", f.Num(), f.Type())
f.s.opens[f.pack()] = false
r = &reader{f, r}
}
return
}
func (f *file) Create() (w storage.Writer, err error) {
err = f.s.emulateError(ModeCreate, f.Type())
if err == nil {
f.s.stall(ModeCreate, f.Type())
}
f.s.mu.Lock()
defer f.s.mu.Unlock()
if err == nil {
f.assertOpen()
f.s.countNB(ModeCreate, f.Type(), 0)
w, err = f.File.Create()
}
if err != nil {
f.s.logI("file create failed, num=%d type=%v err=%v", f.Num(), f.Type(), err)
} else {
f.s.logI("file created, num=%d type=%v", f.Num(), f.Type())
f.s.opens[f.pack()] = true
w = &writer{f, w}
}
return
}
func (f *file) Remove() (err error) {
err = f.s.emulateError(ModeRemove, f.Type())
if err == nil {
f.s.stall(ModeRemove, f.Type())
}
f.s.mu.Lock()
defer f.s.mu.Unlock()
if err == nil {
f.assertOpen()
f.s.countNB(ModeRemove, f.Type(), 0)
err = f.File.Remove()
}
if err != nil {
f.s.logI("file remove failed, num=%d type=%v err=%v", f.Num(), f.Type(), err)
} else {
f.s.logI("file removed, num=%d type=%v", f.Num(), f.Type())
}
return
}
type Storage struct {
storage.Storage
closeFn func() error
lmu sync.Mutex
lb bytes.Buffer
mu sync.Mutex
// Open files, true=writer, false=reader
opens map[uint64]bool
counters [flattenCount]int
bytesCounter [flattenCount]int64
emulatedError [flattenCount]error
stallCond sync.Cond
stalled [flattenCount]bool
}
func (s *Storage) log(skip int, str string) {
s.lmu.Lock()
defer s.lmu.Unlock()
_, file, line, ok := runtime.Caller(skip + 2)
if ok {
// Truncate file name at last file name separator.
if index := strings.LastIndex(file, "/"); index >= 0 {
file = file[index+1:]
} else if index = strings.LastIndex(file, "\\"); index >= 0 {
file = file[index+1:]
}
} else {
file = "???"
line = 1
}
fmt.Fprintf(&s.lb, "%s:%d: ", file, line)
lines := strings.Split(str, "\n")
if l := len(lines); l > 1 && lines[l-1] == "" {
lines = lines[:l-1]
}
for i, line := range lines {
if i > 0 {
s.lb.WriteString("\n\t")
}
s.lb.WriteString(line)
}
s.lb.WriteByte('\n')
}
func (s *Storage) logISkip(skip int, format string, args ...interface{}) {
pc, _, _, ok := runtime.Caller(skip + 1)
if ok {
if f := runtime.FuncForPC(pc); f != nil {
fname := f.Name()
if index := strings.LastIndex(fname, "."); index >= 0 {
fname = fname[index+1:]
}
format = fname + ": " + format
}
}
s.log(skip+1, fmt.Sprintf(format, args...))
}
func (s *Storage) logI(format string, args ...interface{}) {
s.logISkip(1, format, args...)
}
func (s *Storage) Log(str string) {
s.log(1, "Log: "+str)
s.Storage.Log(str)
}
func (s *Storage) Lock() (r util.Releaser, err error) {
r, err = s.Storage.Lock()
if err != nil {
s.logI("storage locking failed, err=%v", err)
} else {
s.logI("storage locked")
r = storageLock{s, r}
}
return
}
func (s *Storage) GetFile(num uint64, t storage.FileType) storage.File {
return &file{s, s.Storage.GetFile(num, t)}
}
func (s *Storage) GetFiles(t storage.FileType) (files []storage.File, err error) {
rfiles, err := s.Storage.GetFiles(t)
if err != nil {
s.logI("get files failed, err=%v", err)
return
}
files = make([]storage.File, len(rfiles))
for i, f := range rfiles {
files[i] = &file{s, f}
}
s.logI("get files, type=0x%x count=%d", int(t), len(files))
return
}
func (s *Storage) GetManifest() (f storage.File, err error) {
manifest, err := s.Storage.GetManifest()
if err != nil {
if !os.IsNotExist(err) {
s.logI("get manifest failed, err=%v", err)
}
return
}
s.logI("get manifest, num=%d", manifest.Num())
return &file{s, manifest}, nil
}
func (s *Storage) SetManifest(f storage.File) error {
f_, ok := f.(*file)
ExpectWithOffset(1, ok).To(BeTrue())
ExpectWithOffset(1, f_.Type()).To(Equal(storage.TypeManifest))
err := s.Storage.SetManifest(f_.File)
if err != nil {
s.logI("set manifest failed, err=%v", err)
} else {
s.logI("set manifest, num=%d", f_.Num())
}
return err
}
func (s *Storage) openFiles() string {
out := "Open files:"
for x, writer := range s.opens {
num, t := unpackFile(x)
out += fmt.Sprintf("\n · num=%d type=%v writer=%v", num, t, writer)
}
return out
}
func (s *Storage) Close() error {
s.mu.Lock()
defer s.mu.Unlock()
ExpectWithOffset(1, s.opens).To(BeEmpty(), s.openFiles())
err := s.Storage.Close()
if err != nil {
s.logI("storage closing failed, err=%v", err)
} else {
s.logI("storage closed")
}
if s.closeFn != nil {
if err1 := s.closeFn(); err1 != nil {
s.logI("close func error, err=%v", err1)
}
}
return err
}
func (s *Storage) countNB(m StorageMode, t storage.FileType, n int) {
s.counters[flattenType(m, t)]++
s.bytesCounter[flattenType(m, t)] += int64(n)
}
func (s *Storage) count(m StorageMode, t storage.FileType, n int) {
s.mu.Lock()
defer s.mu.Unlock()
s.countNB(m, t, n)
}
func (s *Storage) ResetCounter(m StorageMode, t storage.FileType) {
for _, x := range listFlattenType(m, t) {
s.counters[x] = 0
s.bytesCounter[x] = 0
}
}
func (s *Storage) Counter(m StorageMode, t storage.FileType) (count int, bytes int64) {
for _, x := range listFlattenType(m, t) {
count += s.counters[x]
bytes += s.bytesCounter[x]
}
return
}
func (s *Storage) emulateError(m StorageMode, t storage.FileType) error {
s.mu.Lock()
defer s.mu.Unlock()
err := s.emulatedError[flattenType(m, t)]
if err != nil {
return emulatedError{err}
}
return nil
}
func (s *Storage) EmulateError(m StorageMode, t storage.FileType, err error) {
s.mu.Lock()
defer s.mu.Unlock()
for _, x := range listFlattenType(m, t) {
s.emulatedError[x] = err
}
}
func (s *Storage) stall(m StorageMode, t storage.FileType) {
x := flattenType(m, t)
s.mu.Lock()
defer s.mu.Unlock()
for s.stalled[x] {
s.stallCond.Wait()
}
}
func (s *Storage) Stall(m StorageMode, t storage.FileType) {
s.mu.Lock()
defer s.mu.Unlock()
for _, x := range listFlattenType(m, t) {
s.stalled[x] = true
}
}
func (s *Storage) Release(m StorageMode, t storage.FileType) {
s.mu.Lock()
defer s.mu.Unlock()
for _, x := range listFlattenType(m, t) {
s.stalled[x] = false
}
s.stallCond.Broadcast()
}
func NewStorage() *Storage {
var stor storage.Storage
var closeFn func() error
if storageUseFS {
for {
storageMu.Lock()
num := storageNum
storageNum++
storageMu.Unlock()
path := filepath.Join(os.TempDir(), fmt.Sprintf("goleveldb-test%d0%d0%d", os.Getuid(), os.Getpid(), num))
if _, err := os.Stat(path); os.IsNotExist(err) {
stor, err = storage.OpenFile(path)
ExpectWithOffset(1, err).NotTo(HaveOccurred(), "creating storage at %s", path)
closeFn = func() error {
if storageKeepFS {
return nil
}
return os.RemoveAll(path)
}
break
}
}
} else {
stor = storage.NewMemStorage()
}
s := &Storage{
Storage: stor,
closeFn: closeFn,
opens: make(map[uint64]bool),
}
s.stallCond.L = &s.mu
return s
}

View File

@@ -1,21 +0,0 @@
// Copyright (c) 2014, Suryandaru Triandana <syndtr@gmail.com>
// All rights reserved.
//
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
// +build go1.3
package util
import (
"sync"
)
type Pool struct {
sync.Pool
}
func NewPool(cap int) *Pool {
return &Pool{}
}

View File

@@ -1,33 +0,0 @@
// Copyright (c) 2014, Suryandaru Triandana <syndtr@gmail.com>
// All rights reserved.
//
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
// +build !go1.3
package util
type Pool struct {
pool chan interface{}
}
func (p *Pool) Get() interface{} {
select {
case x := <-p.pool:
return x
default:
return nil
}
}
func (p *Pool) Put(x interface{}) {
select {
case p.pool <- x:
default:
}
}
func NewPool(cap int) *Pool {
return &Pool{pool: make(chan interface{}, cap)}
}

View File

@@ -1,7 +0,0 @@
language: go
go:
- 1.1
- 1.2
- 1.3
- 1.4
- tip

View File

@@ -1,50 +0,0 @@
Suture
======
[![Build Status](https://travis-ci.org/thejerf/suture.png?branch=master)](https://travis-ci.org/thejerf/suture)
Suture provides Erlang-ish supervisor trees for Go. "Supervisor trees" ->
"sutree" -> "suture" -> holds your code together when it's trying to die.
This library has hit maturity, and isn't expected to be changed
radically. This can also be imported via gopkg.in/thejerf/suture.v1 .
It is intended to deal gracefully with the real failure cases that can
occur with supervision trees (such as burning all your CPU time endlessly
restarting dead services), while also making no unnecessary demands on the
"service" code, and providing hooks to perform adequate logging with in a
production environment.
[A blog post describing the design decisions](http://www.jerf.org/iri/post/2930)
is available.
This module is fully covered with [godoc](http://godoc.org/github.com/thejerf/suture),
including an example, usage, and everything else you might expect from a
README.md on GitHub. (DRY.)
Code Signing
------------
Starting with the commit after ac7cf8591b, I will be signing this repository
with the ["jerf" keybase account](https://keybase.io/jerf).
Aspiration
----------
One of the big wins the Erlang community has with their pervasive OTP
support is that it makes it easy for them to distribute libraries that
easily fit into the OTP paradigm. It ought to someday be considered a good
idea to distribute libraries that provide some sort of supervisor tree
functionality out of the box. It is possible to provide this functionality
without explicitly depending on the Suture library.
Changelog
---------
suture uses semantic versioning.
1. 1.0.0
* Initial release.
2. 1.0.1
* Fixed data race on the .state variable.

View File

@@ -1,12 +0,0 @@
#!/bin/bash
GOLINTOUT=$(golint *go)
if [ ! -z "$GOLINTOUT" -o "$?" != 0 ]; then
echo golint failed:
echo $GOLINTOUT
exit 1
fi
go test

View File

@@ -1,19 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build nacl plan9 solaris
package ipv6
func (c *genericOpt) sysfd() (int, error) {
return 0, errOpNoSupport
}
func (c *dgramOpt) sysfd() (int, error) {
return 0, errOpNoSupport
}
func (c *payloadHandler) sysfd() (int, error) {
return 0, errOpNoSupport
}

View File

@@ -1,46 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build darwin dragonfly freebsd linux netbsd openbsd
package ipv6
import (
"net"
"reflect"
)
func (c *genericOpt) sysfd() (int, error) {
switch p := c.Conn.(type) {
case *net.TCPConn, *net.UDPConn, *net.IPConn:
return sysfd(p)
}
return 0, errInvalidConnType
}
func (c *dgramOpt) sysfd() (int, error) {
switch p := c.PacketConn.(type) {
case *net.UDPConn, *net.IPConn:
return sysfd(p.(net.Conn))
}
return 0, errInvalidConnType
}
func (c *payloadHandler) sysfd() (int, error) {
return sysfd(c.PacketConn.(net.Conn))
}
func sysfd(c net.Conn) (int, error) {
cv := reflect.ValueOf(c)
switch ce := cv.Elem(); ce.Kind() {
case reflect.Struct:
nfd := ce.FieldByName("conn").FieldByName("fd")
switch fe := nfd.Elem(); fe.Kind() {
case reflect.Struct:
fd := fe.FieldByName("sysfd")
return int(fd.Int()), nil
}
}
return 0, errInvalidConnType
}

View File

@@ -1,45 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
import (
"net"
"reflect"
"syscall"
)
func (c *genericOpt) sysfd() (syscall.Handle, error) {
switch p := c.Conn.(type) {
case *net.TCPConn, *net.UDPConn, *net.IPConn:
return sysfd(p)
}
return syscall.InvalidHandle, errInvalidConnType
}
func (c *dgramOpt) sysfd() (syscall.Handle, error) {
switch p := c.PacketConn.(type) {
case *net.UDPConn, *net.IPConn:
return sysfd(p.(net.Conn))
}
return syscall.InvalidHandle, errInvalidConnType
}
func (c *payloadHandler) sysfd() (syscall.Handle, error) {
return sysfd(c.PacketConn.(net.Conn))
}
func sysfd(c net.Conn) (syscall.Handle, error) {
cv := reflect.ValueOf(c)
switch ce := cv.Elem(); ce.Kind() {
case reflect.Struct:
netfd := ce.FieldByName("conn").FieldByName("fd")
switch fe := netfd.Elem(); fe.Kind() {
case reflect.Struct:
fd := fe.FieldByName("sysfd")
return syscall.Handle(fd.Uint()), nil
}
}
return syscall.InvalidHandle, errInvalidConnType
}

View File

@@ -1,21 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
import (
"net"
"os"
"syscall"
"unsafe"
)
func setsockoptIPMreq(fd syscall.Handle, opt *sockOpt, ifi *net.Interface, grp net.IP) error {
var mreq sysIPv6Mreq
copy(mreq.Multiaddr[:], grp)
if ifi != nil {
mreq.setIfindex(ifi.Index)
}
return os.NewSyscallError("setsockopt", syscall.Setsockopt(fd, int32(opt.level), int32(opt.name), (*byte)(unsafe.Pointer(&mreq)), sysSizeofIPv6Mreq))
}

View File

@@ -1,86 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
import (
"net"
"os"
"syscall"
"unsafe"
)
func getInt(fd syscall.Handle, opt *sockOpt) (int, error) {
if opt.name < 1 || opt.typ != ssoTypeInt {
return 0, errOpNoSupport
}
var i int32
l := int32(4)
if err := syscall.Getsockopt(fd, int32(opt.level), int32(opt.name), (*byte)(unsafe.Pointer(&i)), &l); err != nil {
return 0, os.NewSyscallError("getsockopt", err)
}
return int(i), nil
}
func setInt(fd syscall.Handle, opt *sockOpt, v int) error {
if opt.name < 1 || opt.typ != ssoTypeInt {
return errOpNoSupport
}
i := int32(v)
return os.NewSyscallError("setsockopt", syscall.Setsockopt(fd, int32(opt.level), int32(opt.name), (*byte)(unsafe.Pointer(&i)), 4))
}
func getInterface(fd syscall.Handle, opt *sockOpt) (*net.Interface, error) {
if opt.name < 1 || opt.typ != ssoTypeInterface {
return nil, errOpNoSupport
}
var i int32
l := int32(4)
if err := syscall.Getsockopt(fd, int32(opt.level), int32(opt.name), (*byte)(unsafe.Pointer(&i)), &l); err != nil {
return nil, os.NewSyscallError("getsockopt", err)
}
if i == 0 {
return nil, nil
}
ifi, err := net.InterfaceByIndex(int(i))
if err != nil {
return nil, err
}
return ifi, nil
}
func setInterface(fd syscall.Handle, opt *sockOpt, ifi *net.Interface) error {
if opt.name < 1 || opt.typ != ssoTypeInterface {
return errOpNoSupport
}
var i int32
if ifi != nil {
i = int32(ifi.Index)
}
return os.NewSyscallError("setsockopt", syscall.Setsockopt(fd, int32(opt.level), int32(opt.name), (*byte)(unsafe.Pointer(&i)), 4))
}
func getICMPFilter(fd syscall.Handle, opt *sockOpt) (*ICMPFilter, error) {
return nil, errOpNoSupport
}
func setICMPFilter(fd syscall.Handle, opt *sockOpt, f *ICMPFilter) error {
return errOpNoSupport
}
func getMTUInfo(fd syscall.Handle, opt *sockOpt) (*net.Interface, int, error) {
return nil, 0, errOpNoSupport
}
func setGroup(fd syscall.Handle, opt *sockOpt, ifi *net.Interface, grp net.IP) error {
if opt.name < 1 || opt.typ != ssoTypeIPMreq {
return errOpNoSupport
}
return setsockoptIPMreq(fd, opt, ifi, grp)
}
func setSourceGroup(fd syscall.Handle, opt *sockOpt, ifi *net.Interface, grp, src net.IP) error {
// TODO(mikio): implement this
return errOpNoSupport
}

View File

@@ -1,31 +0,0 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
import (
"syscall"
"unsafe"
)
const (
sysGETSOCKOPT = 0xf
sysSETSOCKOPT = 0xe
)
func socketcall(call int, a0, a1, a2, a3, a4, a5 uintptr) (int, syscall.Errno)
func getsockopt(fd, level, name int, v unsafe.Pointer, l *sysSockoptLen) error {
if _, errno := socketcall(sysGETSOCKOPT, uintptr(fd), uintptr(level), uintptr(name), uintptr(v), uintptr(unsafe.Pointer(l)), 0); errno != 0 {
return error(errno)
}
return nil
}
func setsockopt(fd, level, name int, v unsafe.Pointer, l sysSockoptLen) error {
if _, errno := socketcall(sysSETSOCKOPT, uintptr(fd), uintptr(level), uintptr(name), uintptr(v), uintptr(l), 0); errno != 0 {
return error(errno)
}
return nil
}

View File

@@ -1,26 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build darwin dragonfly freebsd linux,!386 netbsd openbsd
package ipv6
import (
"syscall"
"unsafe"
)
func getsockopt(fd, level, name int, v unsafe.Pointer, l *sysSockoptLen) error {
if _, _, errno := syscall.Syscall6(syscall.SYS_GETSOCKOPT, uintptr(fd), uintptr(level), uintptr(name), uintptr(v), uintptr(unsafe.Pointer(l)), 0); errno != 0 {
return error(errno)
}
return nil
}
func setsockopt(fd, level, name int, v unsafe.Pointer, l sysSockoptLen) error {
if _, _, errno := syscall.Syscall6(syscall.SYS_SETSOCKOPT, uintptr(fd), uintptr(level), uintptr(name), uintptr(v), uintptr(l), 0); errno != 0 {
return error(errno)
}
return nil
}

28
ISSUE_TEMPLATE.md Normal file
View File

@@ -0,0 +1,28 @@
Do not report security issues in this bug tracker. Instead, contact
security@syncthing.net directly - see https://syncthing.net/security.html
for more information.
If your issue is a support request ("How do I get my devices to connect?"
or similar), please use the support forum at https://forum.syncthing.net/
where a large number of helpful people hang out. This issue tracker is for
reporting bugs or feature requests directly to the developers.
If your issue is a bug report, replace this boilerplate with a description
of the problem, being sure to include at least:
- what happened,
- what you expected to happen instead, and
- any steps to reproduce the problem.
Also fill out the version information below and add log output or
screenshots as appropriate.
If your issue is a feature request, simply replace this template text in
its entirety.
### Version Information
Syncthing Version: v0.x.y
OS Version: Windows 7 / Ubuntu 14.04 / ...
Browser Version: (if applicable, for GUI issues)

198
NICKS
View File

@@ -1,72 +1,132 @@
# This file maps email addresses used in commits to nicks used the changelog.
# It is auto generated from the AUTHORS file by script/authors.go.
AudriusButkevicius <audrius.butkevicius@gmail.com>
Cathryne <cathryne.linenweaver@gmail.com> <Cathryne@users.noreply.github.com>
KayoticSully <kayoticsully@gmail.com>
LordLandon <lordlandon@gmail.com>
Moter8 <moter8@gmail.com>
Nutomic <me@nutomic.com>
Rewt0r <rewt0r@gmx.com> <Rewt0r@users.noreply.github.com>
Stefan-Code <stefan.github@gmail.com> <Stefan.github@gmail.com>
Vilbrekin <vilbrekin@gmail.com>
Zillode <zillode@zillode.be>
acogdev <jake@acogdev.com>
alex2108 <register-github@alex-graf.de>
andrew-d <andrew@du.nham.ca>
asdil12 <dominik@heidler.eu>
bencurthoys <ben@bencurthoys.com>
bigbear2nd <bigbear2nd@gmail.com>
brbecker <brbecker@gmail.com>
brendanlong <self@brendanlong.com>
brgmnn <dan.arne.bergmann@gmail.com> <brgmnn@users.noreply.github.com>
bsidhom <bsidhom@gmail.com>
burkemw3 <mburke@amplify.com> <burkemw3@gmail.com>
calmh <jakob@nym.se>
canton7 <antony.male@gmail.com>
cdata <chris@scriptolo.gy>
cdhowie <me@chrishowie.com>
ceh <emil@hessman.se>
cqcallaw <enlightened.despot@gmail.com>
dva <denisva@gmail.com>
dzarda <dzardacz@gmail.com>
eipiminus1 <eipiminusone+github@gmail.com> <eipiminus1@users.noreply.github.com>
facastagnini <federico.castagnini@gmail.com>
filoozoom <philippe@schommers.be>
frioux <frew@afoolishmanifesto.com> <frioux@gmail.com>
fti7 <frank@isemann.name>
gillisig <gilli@vx.is>
hadogenes <szafar@linux.pl>
jarlebring <jarlebring@gmail.com>
jedie <github.com@jensdiemer.de> <git@jensdiemer.de>
jpjp <jamespatterson@operamail.com> <jpjp@users.noreply.github.com>
kamadak <kamada@nanohz.org>
kilburn <kilburn@la3.org>
kozec <kozec@kozec.com>
krozycki <rozycki.karol@gmail.com>
marcindziadus <dziadus.marcin@gmail.com>
0x010C <antoine.lamielle@0x010c.fr>
0x010C <gh@0x010c.fr>
acogdev <jake@acogdev.com>
adelq <aqalieh95@gmail.com>
adelq <adelq@users.noreply.github.com>
alessandro.g89 <alessandro.g89@gmail.com>
alex2108 <register-github@alex-graf.de>
andersonvom <andersonvom@gmail.com>
andrew-d <andrew@du.nham.ca>
asdil12 <dominik@heidler.eu>
AudriusButkevicius <audrius.butkevicius@gmail.com>
aviau <alexandre@alexandreviau.net>
aviau <aviau@debian.org>
bencurthoys <ben@bencurthoys.com>
bigbear2nd <bigbear2nd@gmail.com>
brbecker <brbecker@gmail.com>
brendanlong <self@brendanlong.com>
brgmnn <dan.arne.bergmann@gmail.com>
brgmnn <brgmnn@users.noreply.github.com>
bsidhom <bsidhom@gmail.com>
buinsky <vix_booja@tut.by>
burkemw3 <mburke@amplify.com>
burkemw3 <burkemw3@gmail.com>
calmh <jakob@nym.se>
calmh <jakob@kastelo.net>
canton7 <antony.male@gmail.com>
Cathryne <cathryne.linenweaver@gmail.com>
Cathryne <Cathryne@users.noreply.github.com>
cdata <chris@scriptolo.gy>
cdhowie <me@chrishowie.com>
ceh <emil@hessman.se>
cqcallaw <enlightened.despot@gmail.com>
damajor <damajor@gmail.com>
dinosore <dinosore@dbrsoftware.co.uk>
dva <denisva@gmail.com>
dzarda <dzardacz@gmail.com>
eipiminus1 <eipiminusone+github@gmail.com>
eipiminus1 <eipiminus1@users.noreply.github.com>
elopio <yo@elopio.net>
facastagnini <federico.castagnini@gmail.com>
filoozoom <philippe@schommers.be>
frioux <frew@afoolishmanifesto.com>
frioux <frioux@gmail.com>
fti7 <frank@isemann.name>
gillisig <gilli@vx.is>
hadogenes <szafar@linux.pl>
imsodin <freisim93@gmail.com>
ironmig <kma1660@gmail.com>
jarlebring <jarlebring@gmail.com>
jedie <github.com@jensdiemer.de>
jedie <git@jensdiemer.de>
jgke <jgke@jgke.fi>
jpjp <jamespatterson@operamail.com>
jpjp <jpjp@users.noreply.github.com>
kamadak <kamada@nanohz.org>
KayoticSully <kayoticsully@gmail.com>
kc1212 <kc04bc@gmx.com>
kc1212 <kc1212@users.noreply.github.com>
kilburn <kilburn@la3.org>
kluppy <kluppy@going2blue.com>
kozec <kozec@kozec.com>
kralo <max.schulze@online.de>
kralo <kralo@users.noreply.github.com>
krozycki <rozycki.karol@gmail.com>
Kudalufi <kurt@va1der.ca>
kwhite17 <kevinwhite1710@gmail.com>
letiemble <laurent.etiemble@gmail.com>
letiemble <laurent.etiemble@monobjc.net>
lkwg82 <lkwg82@gmx.de>
LordLandon <lordlandon@gmail.com>
majedev <majed.alhajry@gmail.com>
marcindziadus <dziadus.marcin@gmail.com>
marclaporte <marc@marclaporte.com>
mateon1 <matin1111@wp.pl>
mogwa1 <devriesb@gmail.com>
moshen <moshen.colin@gmail.com>
mvdan <mvdan@mvdan.cc>
pascalj <github@pascalj.com> <mail@pascal-jungblut.com>
peterhoeg <peter@speartail.com>
philips <brandon@ifup.org>
piobpl <piotrb10@gmail.com>
pluby <phill.luby@newredo.com>
pyfisch <pyfisch@gmail.com>
qbit <qbit@deftly.net>
ralder <ralder@yandex.ru>
rumpelsepp <stefan@sevenbyte.org> <rumpelsepp@sevenbyte.org>
sciurius <jvromans@squirrel.nl>
seehuhn <voss@seehuhn.de>
snnd <dw@risu.io>
simplypeachy <aD@simplypeachy.co.uk> <simplypeachy@users.noreply.github.com>
timabell <tim@timwise.co.uk>
tnn2 <tnn@nygren.pp.se>
tojrobinson <tully@tojr.org>
tylerbrazier <tyler@tylerbrazier.com>
uok <ueomkail@gmail.com> <uok@users.noreply.github.com>
veeti <veeti.paananen@rojekti.fi>
wsgcsysadmin <e.meitner@willystreet.coo>
zukoo <fxgsell@gmail.com>
marclaporte <marc@laporte.name>
mateon1 <matin1111@wp.pl>
mogwa1 <devriesb@gmail.com>
moshen <moshen.colin@gmail.com>
Moter8 <moter8@gmail.com>
mpx <mark@kyne.com.au>
mvdan <mvdan@mvdan.cc>
norgeous <daniel@harte.me>
norgeous <daniel@danielharte.co.uk>
norgeous <norgeous@users.noreply.github.com>
nrm21 <natemorrison@gmail.com>
Nutomic <me@nutomic.com>
pascalj <github@pascalj.com>
pascalj <mail@pascal-jungblut.com>
peterhoeg <peter@speartail.com>
philips <brandon@ifup.org>
piobpl <piotrb10@gmail.com>
plouj <ploujj@gmail.com>
pluby <phill.luby@newredo.com>
ProactiveServices <aD@simplypeachy.co.uk>
ProactiveServices <simplypeachy@users.noreply.github.com>
ProactiveServices <ProactiveServices@users.noreply.github.com>
pyfisch <pyfisch@gmail.com>
qbit <qbit@deftly.net>
ralder <ralder@yandex.ru>
Rewt0r <rewt0r@gmx.com>
Rewt0r <Rewt0r@users.noreply.github.com>
rumpelsepp <stefan@sevenbyte.org>
rumpelsepp <rumpelsepp@sevenbyte.org>
scienmind <scintertech@cryptolab.net>
sciurius <jvromans@squirrel.nl>
seehuhn <voss@seehuhn.de>
Smiley73 <heiko@zuerker.org>
snnd <dw@risu.io>
Stefan-Code <stefan.github@gmail.com>
Stefan-Code <Stefan.github@gmail.com>
timabell <tim@timwise.co.uk>
timhowes <timhowes@berkeley.edu>
tnn2 <tnn@nygren.pp.se>
tojrobinson <tully@tojr.org>
tpng <benny.tpng@gmail.com>
tylerbrazier <tyler@tylerbrazier.com>
Unrud <unrud@openaliasbox.org>
Unrud <Unrud@users.noreply.github.com>
uok <ueomkail@gmail.com>
uok <uok@users.noreply.github.com>
veeti <veeti.paananen@rojekti.fi>
Vilbrekin <vilbrekin@gmail.com>
wkennington <william@wkennington.com>
WSGCSysadmin <e.meitner@willystreet.coop>
wweich <wweich@users.noreply.github.com>
wweich <wweich@gmx.de>
xduugu <cedric@gmx.ca>
zaynetro <romanznet@gmail.com>
Zillode <zillode@zillode.be>
zukoo <fxgsell@gmail.com>

32
PULL_REQUEST_TEMPLATE.md Normal file
View File

@@ -0,0 +1,32 @@
### Purpose
Describe the purpose of this change. If there is an existing issue that is
resolved by this pull request, ensure that the commit subject is on the form
`Some short description (fixes #1234)` where 1234 is the issue number.
### Testing
Describe what testing has been done, and how the reviewer can test the change
if new tests are not included.
### Screenshots
If this is a GUI change, include screenshots of the change. If not, please
feel free to just delete this section.
### Documentation
If this is a user visible change (including API and protocol changes), add a link here
to the corresponding pull request on https://github.com/syncthing/docs or describe
the documentation changes necessary.
### Authorship
Every author of a code contribution (Go, Javascript, HTML, CSS etc, with the
possible exception of minor typo corrections and similar) is recorded in the
AUTHORS and NICKS files and the in-GUI credits. If this is your first
contribution, a maintainer will add you properly before accepting the
contribution. You need not do so yourself or worry about the fact that the
"authors" automated test fails. However, if your name (such as you want it
presented in the credits) is not visible on your Github profile or in your
commit messages, please assist by providing it here.

View File

@@ -1,46 +1,89 @@
# Syncthing
[![Latest Build (Official)](https://img.shields.io/jenkins/s/http/build.syncthing.net/syncthing.svg?style=flat-square&label=unix%20build)](http://build.syncthing.net/job/syncthing/lastBuild/)
[![AppVeyor Build](https://img.shields.io/appveyor/ci/calmh/syncthing/master.svg?style=flat-square&label=windows%20build)](https://ci.appveyor.com/project/calmh/syncthing)
[![API Documentation](https://img.shields.io/badge/api-Godoc-blue.svg?style=flat-square)](http://godoc.org/github.com/syncthing/syncthing)
[![MPLv2 License](https://img.shields.io/badge/license-MPLv2-blue.svg?style=flat-square)](https://www.mozilla.org/MPL/2.0/)
[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/88/badge)](https://bestpractices.coreinfrastructure.org/projects/88)
This is the Syncthing project which pursues the following goals:
## Goals
1. Define a protocol for synchronization of a folder between a number of
collaborating devices. This protocol should be well defined, unambiguous,
easily understood, free to use, efficient, secure and language neutral.
This is called the [Block Exchange Protocol][1].
Syncthing is a **continous file synchronization program**. It synchronizes
files between two or more computers. We strive to fulfill the goals below.
The goals are listed in order of importance, the most important one being
the first. This is the summary version of the goal list - for more
commentary, see the full [Goals document][13].
2. Provide the reference implementation to demonstrate the usability of
said protocol. This is the `syncthing` utility. We hope that
alternative, compatible implementations of the protocol will arise.
Syncthing should be:
The two are evolving together; the protocol is not to be considered
stable until Syncthing 1.0 is released, at which point it is locked down
for incompatible changes.
1. Safe From Data Loss
Protecting the user's data is paramount. We take every reasonable
precaution to avoid corrupting the user's files.
2. Secure Against Attackers
Again, protecting the user's data is paramount. Regardless of our other
goals we must never allow the user's data to be susceptible to
eavesdropping or modification by unauthorized parties.
3. Easy to Use
Syncthing should be approachable, understandable and inclusive.
4. Automatic
User interaction should be required only when absolutely necessary.
5. Universally Available
Syncthing should run on every common computer. We are mindful that the
latest technology is not always available to any given individual.
6. For Individuals
Syncthing is primarily about empowering the individual user with safe,
secure and easy to use file synchronization.
7. Everything Else
There are many things we care about that don't make it on to the list. It
is fine to optimize for these values, as long as they are not in conflict
with the stated goals above.
## Getting Started
Take a look at the [getting started guide][2].
There are a few examples for keeping Syncthing running in the background
on your system in [the etc directory][3].
on your system in [the etc directory][3]. There are also several [GUI
implementations][11] for Windows, Mac and Linux.
There is an IRC channel, `#syncthing` on [Freenode][4], for talking directly
to developers and users.
## Vote on features/bugs
We'd like to encourage you to [vote][12] on issues that matter to you.
This helps the team understand what are the biggest pain points for our users, and could potentially influence what is being worked on next.
## Getting in Touch
The first and best point of contact is the [Forum][8]. There is also an IRC
channel, `#syncthing` on [freenode][4] (with a [web client][9]), for talking
directly to developers and users. If you've found something that is clearly a
bug, feel free to report it in the [GitHub issue tracker][10].
## Building
Building Syncthing from source is easy, and there's a [guide][5].
Building Syncthing from source is easy, and there's a [guide][5]
that describes it for both Unix and Windows systems.
## Signed Releases
As of v0.10.15 and onwards, git tags and release binaries are GPG signed
with the key D26E6ED000654A3E (see https://syncthing.net/security.html).
For release binaries, MD5 and SHA1 checksums are calculated and signed,
available in the md5sum.txt.asc and sha1sum.txt.asc files.
As of v0.10.15 and onwards release binaries are GPG signed with the key
D26E6ED000654A3E, available from https://syncthing.net/security.html and
most key servers.
There is also a built in automatic upgrade mechanism (disabled in some
distribution channels) which uses a compiled in ECDSA signature. Mac OS
X binaries are also properly code signed.
## Documentation
@@ -48,10 +91,16 @@ Please see the [Syncthing documentation site][6].
All code is licensed under the [MPLv2 License][7].
[1]: http://docs.syncthing.net/specs/bep-v1.html
[2]: http://docs.syncthing.net/intro/getting-started.html
[1]: https://docs.syncthing.net/specs/bep-v1.html
[2]: https://docs.syncthing.net/intro/getting-started.html
[3]: https://github.com/syncthing/syncthing/blob/master/etc
[4]: https://webchat.freenode.net/
[5]: http://docs.syncthing.net/dev/building.html
[6]: http://docs.syncthing.net/
[4]: http://www.freenode.net/irc_servers.shtml
[5]: https://docs.syncthing.net/dev/building.html
[6]: https://docs.syncthing.net/
[7]: https://github.com/syncthing/syncthing/blob/master/LICENSE
[8]: https://forum.syncthing.net/
[9]: https://kiwiirc.com/client/irc.freenode.net/#syncthing
[10]: https://github.com/syncthing/syncthing/issues
[11]: https://docs.syncthing.net/users/contrib.html#gui-wrappers
[12]: https://www.bountysource.com/teams/syncthing/issues
[13]: https://github.com/syncthing/syncthing/blob/master/GOALS.md

View File

@@ -1,12 +0,0 @@
version: '{branch}-{build}'
clone_folder: C:\src\github.com\syncthing\syncthing
init:
- go version
environment:
GOPATH: C:\
build_script:
- go run build.go zip
test_script:
- go run build.go test
artifacts:
- path: '*.zip'

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 9.8 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 23 KiB

After

Width:  |  Height:  |  Size: 20 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.4 KiB

After

Width:  |  Height:  |  Size: 2.2 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 48 KiB

After

Width:  |  Height:  |  Size: 40 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.4 KiB

After

Width:  |  Height:  |  Size: 4.9 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 24 KiB

After

Width:  |  Height:  |  Size: 19 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 47 KiB

After

Width:  |  Height:  |  Size: 38 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 9.8 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 8.2 KiB

View File

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 KiB

View File

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.9 KiB

View File

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.9 KiB

BIN
assets/statusicons/sync.svg Normal file
View File

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.1 KiB

975
build.go
View File

File diff suppressed because it is too large Load Diff

101
build.sh
View File

@@ -27,10 +27,6 @@ case "${1:-default}" in
build "$@"
;;
deps)
build "$@"
;;
assets)
build "$@"
;;
@@ -63,8 +59,9 @@ case "${1:-default}" in
;;
prerelease)
go run script/authors.go
build transifex
git add -A gui/assets/ lib/auto/
git add -A gui/default/assets/ lib/auto/
pushd man ; ./refresh.sh ; popd
git add -A man
;;
@@ -74,27 +71,26 @@ case "${1:-default}" in
;;
all)
build -goos darwin -goarch amd64 tar
platforms=(
darwin-amd64 dragonfly-amd64 freebsd-amd64 linux-amd64 netbsd-amd64 openbsd-amd64 solaris-amd64 windows-amd64
freebsd-386 linux-386 netbsd-386 openbsd-386 windows-386
linux-arm linux-arm64 linux-ppc64 linux-ppc64le
)
build -goos dragonfly -goarch amd64 tar
for plat in "${platforms[@]}"; do
echo Building "$plat"
build -goos freebsd -goarch 386 tar
build -goos freebsd -goarch amd64 tar
goos="${plat%-*}"
goarch="${plat#*-}"
dist="tar"
build -goos linux -goarch 386 tar
build -goos linux -goarch amd64 tar
build -goos linux -goarch arm tar
if [[ $goos == "windows" ]]; then
dist="zip"
fi
build -goos netbsd -goarch 386 tar
build -goos netbsd -goarch amd64 tar
build -goos openbsd -goarch 386 tar
build -goos openbsd -goarch amd64 tar
build -goos solaris -goarch amd64 tar
build -goos windows -goarch 386 zip
build -goos windows -goarch amd64 zip
build -goos "$goos" -goarch "$goarch" "$dist"
echo
done
;;
test-cov)
@@ -106,9 +102,9 @@ case "${1:-default}" in
fail=0
# For every package in the repo
for dir in $(go list ./...) ; do
for dir in $(go list ./lib/... ./cmd/...) ; do
# run the tests
GOPATH="$(pwd)/Godeps/_workspace:$GOPATH" go test -race -coverprofile=profile.out $dir
GOPATH="$(pwd)/Godeps/_workspace:$GOPATH" go test -coverprofile=profile.out $dir
if [ -f profile.out ] ; then
# and if there was test output, append it to coverage.out
grep -v "mode: " profile.out >> coverage.out
@@ -116,6 +112,11 @@ case "${1:-default}" in
fi
done
notCovered=$(egrep -c '\s0$' coverage.out)
total=$(wc -l coverage.out | awk '{print $1}')
coverPct=$(awk "BEGIN{print (1 - $notCovered / $total) * 100}")
echo "Total coverage is $coverPct%"
gocov convert coverage.out | gocov-xml > coverage.xml
# This is usually run from within Jenkins. If it is, we need to
@@ -126,53 +127,13 @@ case "${1:-default}" in
fi
;;
docker-all)
img=${DOCKERIMG:-syncthing/build:latest}
docker run --rm -h syncthing-builder -u $(id -u) -t \
-v $(pwd):/go/src/github.com/syncthing/syncthing \
-w /go/src/github.com/syncthing/syncthing \
-e "STTRACE=$STTRACE" \
"$img" \
sh -c './build.sh clean \
&& ./build.sh test-cov \
&& ./build.sh bench \
&& ./build.sh all'
;;
test-xunit)
ulimit -t 600 &>/dev/null || true
ulimit -d 512000 &>/dev/null || true
ulimit -m 512000 &>/dev/null || true
docker-test)
img=${DOCKERIMG:-syncthing/build:latest}
docker run --rm -h syncthing-builder -u $(id -u) -t \
-v $(pwd):/go/src/github.com/syncthing/syncthing \
-w /go/src/github.com/syncthing/syncthing \
-e "STTRACE=$STTRACE" \
"$img" \
sh -euxc './build.sh clean \
&& go run build.go -race \
&& export GOPATH=$(pwd)/Godeps/_workspace:$GOPATH \
&& cd test \
&& go test -tags integration -v -timeout 90m -short \
&& git clean -fxd .'
;;
docker-lint)
img=${DOCKERIMG:-syncthing/build:latest}
docker run --rm -h syncthing-builder -u $(id -u) -t \
-v $(pwd):/go/src/github.com/syncthing/syncthing \
-w /go/src/github.com/syncthing/syncthing \
-e "STTRACE=$STTRACE" \
"$img" \
sh -euxc 'go run build.go lint'
;;
docker-vet)
img=${DOCKERIMG:-syncthing/build:latest}
docker run --rm -h syncthing-builder -u $(id -u) -t \
-v $(pwd):/go/src/github.com/syncthing/syncthing \
-w /go/src/github.com/syncthing/syncthing \
-e "STTRACE=$STTRACE" \
"$img" \
sh -euxc 'go run build.go vet'
(GOPATH="$(pwd)/Godeps/_workspace:$GOPATH" go test -v -race ./lib/... ./cmd/... || true) > tests.out
go2xunit -output tests.xml -fail < tests.out
;;
*)

143
cmd/stbench/main.go Normal file
View File

@@ -0,0 +1,143 @@
// Copyright (C) 2016 The Syncthing Authors.
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at http://mozilla.org/MPL/2.0/.
// This doesn't build on Windows due to the Rusage stuff.
// +build !windows
package main
import (
"flag"
"fmt"
"log"
"runtime"
"syscall"
"time"
"github.com/syncthing/syncthing/lib/rc"
)
var homeDir = "h1"
var syncthingBin = "./bin/syncthing"
var test = "scan"
func main() {
flag.StringVar(&homeDir, "home", homeDir, "Home directory location")
flag.StringVar(&syncthingBin, "bin", syncthingBin, "Binary location")
flag.StringVar(&test, "test", test, "Test to run")
flag.Parse()
switch test {
case "scan":
// scan measures the resource usage required to perform the initial
// scan, without cleaning away the database first.
testScan()
}
}
// testScan starts a process and reports on the resource usage required to
// perform the initial scan.
func testScan() {
log.Println("Starting...")
p := rc.NewProcess("127.0.0.1:8081")
if err := p.Start(syncthingBin, "-home", homeDir, "-no-browser"); err != nil {
log.Println(err)
return
}
defer p.Stop()
wallTime := awaitScanComplete(p)
report(p, wallTime)
}
// awaitScanComplete waits for a folder to transition idle->scanning and
// then scanning->idle and returns the time taken for the scan.
func awaitScanComplete(p *rc.Process) time.Duration {
log.Println("Awaiting scan completion...")
var t0, t1 time.Time
lastEvent := 0
loop:
for {
evs, err := p.Events(lastEvent)
if err != nil {
continue
}
for _, ev := range evs {
if ev.Type == "StateChanged" {
data := ev.Data.(map[string]interface{})
log.Println(ev)
if data["to"].(string) == "scanning" {
t0 = ev.Time
continue
}
if !t0.IsZero() && data["to"].(string) == "idle" {
t1 = ev.Time
break loop
}
}
lastEvent = ev.ID
}
time.Sleep(250 * time.Millisecond)
}
return t1.Sub(t0)
}
// report stops the given process and reports on it's resource usage in two
// ways: human readable to stderr, and CSV to stdout.
func report(p *rc.Process, wallTime time.Duration) {
sv, err := p.SystemVersion()
if err != nil {
log.Println(err)
return
}
ss, err := p.SystemStatus()
if err != nil {
log.Println(err)
return
}
proc, err := p.Stop()
if err != nil {
return
}
rusage, ok := proc.SysUsage().(*syscall.Rusage)
if !ok {
return
}
log.Println("Version:", sv.Version)
log.Println("Alloc:", ss.Alloc/1024, "KiB")
log.Println("Sys:", ss.Sys/1024, "KiB")
log.Println("Goroutines:", ss.Goroutines)
log.Println("Wall time:", wallTime)
log.Println("Utime:", time.Duration(rusage.Utime.Nano()))
log.Println("Stime:", time.Duration(rusage.Stime.Nano()))
if runtime.GOOS == "darwin" {
// Darwin reports in bytes, Linux seems to report in KiB even
// though the manpage says otherwise.
rusage.Maxrss /= 1024
}
log.Println("MaxRSS:", rusage.Maxrss, "KiB")
fmt.Printf("%s,%d,%d,%d,%.02f,%.02f,%.02f,%d\n",
sv.Version,
ss.Alloc/1024,
ss.Sys/1024,
ss.Goroutines,
wallTime.Seconds(),
time.Duration(rusage.Utime.Nano()).Seconds(),
time.Duration(rusage.Stime.Nano()).Seconds(),
rusage.Maxrss)
}

19
cmd/stcli/LICENSE Normal file
View File

@@ -0,0 +1,19 @@
Copyright (C) 2014 Audrius Butkevičius
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:
- The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

10
cmd/stcli/README.md Normal file
View File

@@ -0,0 +1,10 @@
syncthing-cli
=============
[![Latest Build](http://img.shields.io/jenkins/s/http/build.syncthing.net/syncthing-cli.svg?style=flat-square)](http://build.syncthing.net/job/syncthing-cli/lastBuild/)
A CLI that talks to the Syncthing REST interface.
`go get github.com/syncthing/syncthing-cli`
Or download the [latest build](http://build.syncthing.net/job/syncthing-cli/lastSuccessfulBuild/artifact/).

115
cmd/stcli/client.go Normal file
View File

@@ -0,0 +1,115 @@
// Copyright (C) 2014 Audrius Butkevičius
package main
import (
"bytes"
"crypto/tls"
"net/http"
"strings"
"github.com/AudriusButkevicius/cli"
)
type APIClient struct {
httpClient http.Client
endpoint string
apikey string
username string
password string
id string
csrf string
}
var instance *APIClient
func getClient(c *cli.Context) *APIClient {
if instance != nil {
return instance
}
endpoint := c.GlobalString("endpoint")
if !strings.HasPrefix(endpoint, "http") {
endpoint = "http://" + endpoint
}
httpClient := http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
InsecureSkipVerify: c.GlobalBool("insecure"),
},
},
}
client := APIClient{
httpClient: httpClient,
endpoint: endpoint,
apikey: c.GlobalString("apikey"),
username: c.GlobalString("username"),
password: c.GlobalString("password"),
}
if client.apikey == "" {
request, err := http.NewRequest("GET", client.endpoint, nil)
die(err)
response := client.handleRequest(request)
client.id = response.Header.Get("X-Syncthing-ID")
if client.id == "" {
die("Failed to get device ID")
}
for _, item := range response.Cookies() {
if item.Name == "CSRF-Token-"+client.id[:5] {
client.csrf = item.Value
goto csrffound
}
}
die("Failed to get CSRF token")
csrffound:
}
instance = &client
return &client
}
func (client *APIClient) handleRequest(request *http.Request) *http.Response {
if client.apikey != "" {
request.Header.Set("X-API-Key", client.apikey)
}
if client.username != "" || client.password != "" {
request.SetBasicAuth(client.username, client.password)
}
if client.csrf != "" {
request.Header.Set("X-CSRF-Token-"+client.id[:5], client.csrf)
}
response, err := client.httpClient.Do(request)
die(err)
if response.StatusCode == 404 {
die("Invalid endpoint or API call")
} else if response.StatusCode == 401 {
die("Invalid username or password")
} else if response.StatusCode == 403 {
if client.apikey == "" {
die("Invalid CSRF token")
}
die("Invalid API key")
} else if response.StatusCode != 200 {
body := strings.TrimSpace(string(responseToBArray(response)))
if body != "" {
die(body)
}
die("Unknown HTTP status returned: " + response.Status)
}
return response
}
func httpGet(c *cli.Context, url string) *http.Response {
client := getClient(c)
request, err := http.NewRequest("GET", client.endpoint+"/rest/"+url, nil)
die(err)
return client.handleRequest(request)
}
func httpPost(c *cli.Context, url string, body string) *http.Response {
client := getClient(c)
request, err := http.NewRequest("POST", client.endpoint+"/rest/"+url, bytes.NewBufferString(body))
die(err)
return client.handleRequest(request)
}

188
cmd/stcli/cmd_devices.go Normal file
View File

@@ -0,0 +1,188 @@
// Copyright (C) 2014 Audrius Butkevičius
package main
import (
"fmt"
"strings"
"github.com/AudriusButkevicius/cli"
"github.com/syncthing/syncthing/lib/config"
)
func init() {
cliCommands = append(cliCommands, cli.Command{
Name: "devices",
HideHelp: true,
Usage: "Device command group",
Subcommands: []cli.Command{
{
Name: "list",
Usage: "List registered devices",
Requires: &cli.Requires{},
Action: devicesList,
},
{
Name: "add",
Usage: "Add a new device",
Requires: &cli.Requires{"device id", "device name?"},
Action: devicesAdd,
},
{
Name: "remove",
Usage: "Remove an existing device",
Requires: &cli.Requires{"device id"},
Action: devicesRemove,
},
{
Name: "get",
Usage: "Get a property of a device",
Requires: &cli.Requires{"device id", "property"},
Action: devicesGet,
},
{
Name: "set",
Usage: "Set a property of a device",
Requires: &cli.Requires{"device id", "property", "value..."},
Action: devicesSet,
},
},
})
}
func devicesList(c *cli.Context) {
cfg := getConfig(c)
first := true
writer := newTableWriter()
for _, device := range cfg.Devices {
if !first {
fmt.Fprintln(writer)
}
fmt.Fprintln(writer, "ID:\t", device.DeviceID, "\t")
fmt.Fprintln(writer, "Name:\t", device.Name, "\t(name)")
fmt.Fprintln(writer, "Address:\t", strings.Join(device.Addresses, " "), "\t(address)")
fmt.Fprintln(writer, "Compression:\t", device.Compression, "\t(compression)")
fmt.Fprintln(writer, "Certificate name:\t", device.CertName, "\t(certname)")
fmt.Fprintln(writer, "Introducer:\t", device.Introducer, "\t(introducer)")
first = false
}
writer.Flush()
}
func devicesAdd(c *cli.Context) {
nid := c.Args()[0]
id := parseDeviceID(nid)
newDevice := config.DeviceConfiguration{
DeviceID: id,
Name: nid,
Addresses: []string{"dynamic"},
}
if len(c.Args()) > 1 {
newDevice.Name = c.Args()[1]
}
if len(c.Args()) > 2 {
addresses := c.Args()[2:]
for _, item := range addresses {
if item == "dynamic" {
continue
}
validAddress(item)
}
newDevice.Addresses = addresses
}
cfg := getConfig(c)
for _, device := range cfg.Devices {
if device.DeviceID == id {
die("Device " + nid + " already exists")
}
}
cfg.Devices = append(cfg.Devices, newDevice)
setConfig(c, cfg)
}
func devicesRemove(c *cli.Context) {
nid := c.Args()[0]
id := parseDeviceID(nid)
if nid == getMyID(c) {
die("Cannot remove yourself")
}
cfg := getConfig(c)
for i, device := range cfg.Devices {
if device.DeviceID == id {
last := len(cfg.Devices) - 1
cfg.Devices[i] = cfg.Devices[last]
cfg.Devices = cfg.Devices[:last]
setConfig(c, cfg)
return
}
}
die("Device " + nid + " not found")
}
func devicesGet(c *cli.Context) {
nid := c.Args()[0]
id := parseDeviceID(nid)
arg := c.Args()[1]
cfg := getConfig(c)
for _, device := range cfg.Devices {
if device.DeviceID != id {
continue
}
switch strings.ToLower(arg) {
case "name":
fmt.Println(device.Name)
case "address":
fmt.Println(strings.Join(device.Addresses, "\n"))
case "compression":
fmt.Println(device.Compression.String())
case "certname":
fmt.Println(device.CertName)
case "introducer":
fmt.Println(device.Introducer)
default:
die("Invalid property: " + arg + "\nAvailable properties: name, address, compression, certname, introducer")
}
return
}
die("Device " + nid + " not found")
}
func devicesSet(c *cli.Context) {
nid := c.Args()[0]
id := parseDeviceID(nid)
arg := c.Args()[1]
config := getConfig(c)
for i, device := range config.Devices {
if device.DeviceID != id {
continue
}
switch strings.ToLower(arg) {
case "name":
config.Devices[i].Name = strings.Join(c.Args()[2:], " ")
case "address":
for _, item := range c.Args()[2:] {
if item == "dynamic" {
continue
}
validAddress(item)
}
config.Devices[i].Addresses = c.Args()[2:]
case "compression":
err := config.Devices[i].Compression.UnmarshalText([]byte(c.Args()[2]))
die(err)
case "certname":
config.Devices[i].CertName = strings.Join(c.Args()[2:], " ")
case "introducer":
config.Devices[i].Introducer = parseBool(c.Args()[2])
default:
die("Invalid property: " + arg + "\nAvailable properties: name, address, compression, certname, introducer")
}
setConfig(c, config)
return
}
die("Device " + nid + " not found")
}

67
cmd/stcli/cmd_errors.go Normal file
View File

@@ -0,0 +1,67 @@
// Copyright (C) 2014 Audrius Butkevičius
package main
import (
"encoding/json"
"fmt"
"strings"
"github.com/AudriusButkevicius/cli"
)
func init() {
cliCommands = append(cliCommands, cli.Command{
Name: "errors",
HideHelp: true,
Usage: "Error command group",
Subcommands: []cli.Command{
{
Name: "show",
Usage: "Show pending errors",
Requires: &cli.Requires{},
Action: errorsShow,
},
{
Name: "push",
Usage: "Push an error to active clients",
Requires: &cli.Requires{"error message..."},
Action: errorsPush,
},
{
Name: "clear",
Usage: "Clear pending errors",
Requires: &cli.Requires{},
Action: wrappedHTTPPost("system/error/clear"),
},
},
})
}
func errorsShow(c *cli.Context) {
response := httpGet(c, "system/error")
var data map[string][]map[string]interface{}
json.Unmarshal(responseToBArray(response), &data)
writer := newTableWriter()
for _, item := range data["errors"] {
time := item["time"].(string)[:19]
time = strings.Replace(time, "T", " ", 1)
err := item["error"].(string)
err = strings.TrimSpace(err)
fmt.Fprintln(writer, time+":\t"+err)
}
writer.Flush()
}
func errorsPush(c *cli.Context) {
err := strings.Join(c.Args(), " ")
response := httpPost(c, "system/error", strings.TrimSpace(err))
if response.StatusCode != 200 {
err = fmt.Sprint("Failed to push error\nStatus code: ", response.StatusCode)
body := string(responseToBArray(response))
if body != "" {
err += "\nBody: " + body
}
die(err)
}
}

351
cmd/stcli/cmd_folders.go Normal file
View File

@@ -0,0 +1,351 @@
// Copyright (C) 2014 Audrius Butkevičius
package main
import (
"fmt"
"path/filepath"
"strings"
"github.com/AudriusButkevicius/cli"
"github.com/syncthing/syncthing/lib/config"
)
func init() {
cliCommands = append(cliCommands, cli.Command{
Name: "folders",
HideHelp: true,
Usage: "Folder command group",
Subcommands: []cli.Command{
{
Name: "list",
Usage: "List available folders",
Requires: &cli.Requires{},
Action: foldersList,
},
{
Name: "add",
Usage: "Add a new folder",
Requires: &cli.Requires{"folder id", "directory"},
Action: foldersAdd,
},
{
Name: "remove",
Usage: "Remove an existing folder",
Requires: &cli.Requires{"folder id"},
Action: foldersRemove,
},
{
Name: "override",
Usage: "Override changes from other nodes for a master folder",
Requires: &cli.Requires{"folder id"},
Action: foldersOverride,
},
{
Name: "get",
Usage: "Get a property of a folder",
Requires: &cli.Requires{"folder id", "property"},
Action: foldersGet,
},
{
Name: "set",
Usage: "Set a property of a folder",
Requires: &cli.Requires{"folder id", "property", "value..."},
Action: foldersSet,
},
{
Name: "unset",
Usage: "Unset a property of a folder",
Requires: &cli.Requires{"folder id", "property"},
Action: foldersUnset,
},
{
Name: "devices",
Usage: "Folder devices command group",
HideHelp: true,
Subcommands: []cli.Command{
{
Name: "list",
Usage: "List of devices which the folder is shared with",
Requires: &cli.Requires{"folder id"},
Action: foldersDevicesList,
},
{
Name: "add",
Usage: "Share a folder with a device",
Requires: &cli.Requires{"folder id", "device id"},
Action: foldersDevicesAdd,
},
{
Name: "remove",
Usage: "Unshare a folder with a device",
Requires: &cli.Requires{"folder id", "device id"},
Action: foldersDevicesRemove,
},
{
Name: "clear",
Usage: "Unshare a folder with all devices",
Requires: &cli.Requires{"folder id"},
Action: foldersDevicesClear,
},
},
},
},
})
}
func foldersList(c *cli.Context) {
cfg := getConfig(c)
first := true
writer := newTableWriter()
for _, folder := range cfg.Folders {
if !first {
fmt.Fprintln(writer)
}
fmt.Fprintln(writer, "ID:\t", folder.ID, "\t")
fmt.Fprintln(writer, "Path:\t", folder.RawPath, "\t(directory)")
fmt.Fprintln(writer, "Folder type:\t", folder.Type, "\t(type)")
fmt.Fprintln(writer, "Ignore permissions:\t", folder.IgnorePerms, "\t(permissions)")
fmt.Fprintln(writer, "Rescan interval in seconds:\t", folder.RescanIntervalS, "\t(rescan)")
if folder.Versioning.Type != "" {
fmt.Fprintln(writer, "Versioning:\t", folder.Versioning.Type, "\t(versioning)")
for key, value := range folder.Versioning.Params {
fmt.Fprintf(writer, "Versioning %s:\t %s \t(versioning-%s)\n", key, value, key)
}
}
first = false
}
writer.Flush()
}
func foldersAdd(c *cli.Context) {
cfg := getConfig(c)
abs, err := filepath.Abs(c.Args()[1])
die(err)
folder := config.FolderConfiguration{
ID: c.Args()[0],
RawPath: filepath.Clean(abs),
}
cfg.Folders = append(cfg.Folders, folder)
setConfig(c, cfg)
}
func foldersRemove(c *cli.Context) {
cfg := getConfig(c)
rid := c.Args()[0]
for i, folder := range cfg.Folders {
if folder.ID == rid {
last := len(cfg.Folders) - 1
cfg.Folders[i] = cfg.Folders[last]
cfg.Folders = cfg.Folders[:last]
setConfig(c, cfg)
return
}
}
die("Folder " + rid + " not found")
}
func foldersOverride(c *cli.Context) {
cfg := getConfig(c)
rid := c.Args()[0]
for _, folder := range cfg.Folders {
if folder.ID == rid && folder.Type == config.FolderTypeSendOnly {
response := httpPost(c, "db/override", "")
if response.StatusCode != 200 {
err := fmt.Sprint("Failed to override changes\nStatus code: ", response.StatusCode)
body := string(responseToBArray(response))
if body != "" {
err += "\nBody: " + body
}
die(err)
}
return
}
}
die("Folder " + rid + " not found or folder not master")
}
func foldersGet(c *cli.Context) {
cfg := getConfig(c)
rid := c.Args()[0]
arg := strings.ToLower(c.Args()[1])
for _, folder := range cfg.Folders {
if folder.ID != rid {
continue
}
if strings.HasPrefix(arg, "versioning-") {
arg = arg[11:]
value, ok := folder.Versioning.Params[arg]
if ok {
fmt.Println(value)
return
}
die("Versioning property " + c.Args()[1][11:] + " not found")
}
switch arg {
case "directory":
fmt.Println(folder.RawPath)
case "type":
fmt.Println(folder.Type)
case "permissions":
fmt.Println(folder.IgnorePerms)
case "rescan":
fmt.Println(folder.RescanIntervalS)
case "versioning":
if folder.Versioning.Type != "" {
fmt.Println(folder.Versioning.Type)
}
default:
die("Invalid property: " + c.Args()[1] + "\nAvailable properties: directory, type, permissions, versioning, versioning-<key>")
}
return
}
die("Folder " + rid + " not found")
}
func foldersSet(c *cli.Context) {
rid := c.Args()[0]
arg := strings.ToLower(c.Args()[1])
val := strings.Join(c.Args()[2:], " ")
cfg := getConfig(c)
for i, folder := range cfg.Folders {
if folder.ID != rid {
continue
}
if strings.HasPrefix(arg, "versioning-") {
cfg.Folders[i].Versioning.Params[arg[11:]] = val
setConfig(c, cfg)
return
}
switch arg {
case "directory":
cfg.Folders[i].RawPath = val
case "type":
var t config.FolderType
if err := t.UnmarshalText([]byte(val)); err != nil {
die("Invalid folder type: " + err.Error())
}
cfg.Folders[i].Type = t
case "permissions":
cfg.Folders[i].IgnorePerms = parseBool(val)
case "rescan":
cfg.Folders[i].RescanIntervalS = parseInt(val)
case "versioning":
cfg.Folders[i].Versioning.Type = val
default:
die("Invalid property: " + c.Args()[1] + "\nAvailable properties: directory, master, permissions, versioning, versioning-<key>")
}
setConfig(c, cfg)
return
}
die("Folder " + rid + " not found")
}
func foldersUnset(c *cli.Context) {
rid := c.Args()[0]
arg := strings.ToLower(c.Args()[1])
cfg := getConfig(c)
for i, folder := range cfg.Folders {
if folder.ID != rid {
continue
}
if strings.HasPrefix(arg, "versioning-") {
arg = arg[11:]
if _, ok := folder.Versioning.Params[arg]; ok {
delete(cfg.Folders[i].Versioning.Params, arg)
setConfig(c, cfg)
return
}
die("Versioning property " + c.Args()[1][11:] + " not found")
}
switch arg {
case "versioning":
cfg.Folders[i].Versioning.Type = ""
cfg.Folders[i].Versioning.Params = make(map[string]string)
default:
die("Invalid property: " + c.Args()[1] + "\nAvailable properties: versioning, versioning-<key>")
}
setConfig(c, cfg)
return
}
die("Folder " + rid + " not found")
}
func foldersDevicesList(c *cli.Context) {
rid := c.Args()[0]
cfg := getConfig(c)
for _, folder := range cfg.Folders {
if folder.ID != rid {
continue
}
for _, device := range folder.Devices {
fmt.Println(device.DeviceID)
}
return
}
die("Folder " + rid + " not found")
}
func foldersDevicesAdd(c *cli.Context) {
rid := c.Args()[0]
nid := parseDeviceID(c.Args()[1])
cfg := getConfig(c)
for i, folder := range cfg.Folders {
if folder.ID != rid {
continue
}
for _, device := range folder.Devices {
if device.DeviceID == nid {
die("Device " + c.Args()[1] + " is already part of this folder")
}
}
for _, device := range cfg.Devices {
if device.DeviceID == nid {
cfg.Folders[i].Devices = append(folder.Devices, config.FolderDeviceConfiguration{
DeviceID: device.DeviceID,
})
setConfig(c, cfg)
return
}
}
die("Device " + c.Args()[1] + " not found in device list")
}
die("Folder " + rid + " not found")
}
func foldersDevicesRemove(c *cli.Context) {
rid := c.Args()[0]
nid := parseDeviceID(c.Args()[1])
cfg := getConfig(c)
for ri, folder := range cfg.Folders {
if folder.ID != rid {
continue
}
for ni, device := range folder.Devices {
if device.DeviceID == nid {
last := len(folder.Devices) - 1
cfg.Folders[ri].Devices[ni] = folder.Devices[last]
cfg.Folders[ri].Devices = cfg.Folders[ri].Devices[:last]
setConfig(c, cfg)
return
}
}
die("Device " + c.Args()[1] + " not found")
}
die("Folder " + rid + " not found")
}
func foldersDevicesClear(c *cli.Context) {
rid := c.Args()[0]
cfg := getConfig(c)
for i, folder := range cfg.Folders {
if folder.ID != rid {
continue
}
cfg.Folders[i].Devices = []config.FolderDeviceConfiguration{}
setConfig(c, cfg)
return
}
die("Folder " + rid + " not found")
}

78
cmd/stcli/cmd_general.go Normal file
View File

@@ -0,0 +1,78 @@
// Copyright (C) 2014 Audrius Butkevičius
package main
import (
"encoding/json"
"fmt"
"github.com/AudriusButkevicius/cli"
)
func init() {
cliCommands = append(cliCommands, []cli.Command{
{
Name: "id",
Usage: "Get ID of the Syncthing client",
Requires: &cli.Requires{},
Action: generalID,
},
{
Name: "status",
Usage: "Configuration status, whether or not a restart is required for changes to take effect",
Requires: &cli.Requires{},
Action: generalStatus,
},
{
Name: "restart",
Usage: "Restart syncthing",
Requires: &cli.Requires{},
Action: wrappedHTTPPost("system/restart"),
},
{
Name: "shutdown",
Usage: "Shutdown syncthing",
Requires: &cli.Requires{},
Action: wrappedHTTPPost("system/shutdown"),
},
{
Name: "reset",
Usage: "Reset syncthing deleting all folders and devices",
Requires: &cli.Requires{},
Action: wrappedHTTPPost("system/reset"),
},
{
Name: "upgrade",
Usage: "Upgrade syncthing (if a newer version is available)",
Requires: &cli.Requires{},
Action: wrappedHTTPPost("system/upgrade"),
},
{
Name: "version",
Usage: "Syncthing client version",
Requires: &cli.Requires{},
Action: generalVersion,
},
}...)
}
func generalID(c *cli.Context) {
fmt.Println(getMyID(c))
}
func generalStatus(c *cli.Context) {
response := httpGet(c, "system/config/insync")
var status struct{ ConfigInSync bool }
json.Unmarshal(responseToBArray(response), &status)
if !status.ConfigInSync {
die("Config out of sync")
}
fmt.Println("Config in sync")
}
func generalVersion(c *cli.Context) {
response := httpGet(c, "system/version")
version := make(map[string]interface{})
json.Unmarshal(responseToBArray(response), &version)
prettyPrintJSON(version)
}

127
cmd/stcli/cmd_gui.go Normal file
View File

@@ -0,0 +1,127 @@
// Copyright (C) 2014 Audrius Butkevičius
package main
import (
"fmt"
"strings"
"github.com/AudriusButkevicius/cli"
)
func init() {
cliCommands = append(cliCommands, cli.Command{
Name: "gui",
HideHelp: true,
Usage: "GUI command group",
Subcommands: []cli.Command{
{
Name: "dump",
Usage: "Show all GUI configuration settings",
Requires: &cli.Requires{},
Action: guiDump,
},
{
Name: "get",
Usage: "Get a GUI configuration setting",
Requires: &cli.Requires{"setting"},
Action: guiGet,
},
{
Name: "set",
Usage: "Set a GUI configuration setting",
Requires: &cli.Requires{"setting", "value"},
Action: guiSet,
},
{
Name: "unset",
Usage: "Unset a GUI configuration setting",
Requires: &cli.Requires{"setting"},
Action: guiUnset,
},
},
})
}
func guiDump(c *cli.Context) {
cfg := getConfig(c).GUI
writer := newTableWriter()
fmt.Fprintln(writer, "Enabled:\t", cfg.Enabled, "\t(enabled)")
fmt.Fprintln(writer, "Use HTTPS:\t", cfg.UseTLS(), "\t(tls)")
fmt.Fprintln(writer, "Listen Addresses:\t", cfg.Address(), "\t(address)")
if cfg.User != "" {
fmt.Fprintln(writer, "Authentication User:\t", cfg.User, "\t(username)")
fmt.Fprintln(writer, "Authentication Password:\t", cfg.Password, "\t(password)")
}
if cfg.APIKey != "" {
fmt.Fprintln(writer, "API Key:\t", cfg.APIKey, "\t(apikey)")
}
writer.Flush()
}
func guiGet(c *cli.Context) {
cfg := getConfig(c).GUI
arg := c.Args()[0]
switch strings.ToLower(arg) {
case "enabled":
fmt.Println(cfg.Enabled)
case "tls":
fmt.Println(cfg.UseTLS())
case "address":
fmt.Println(cfg.Address())
case "user":
if cfg.User != "" {
fmt.Println(cfg.User)
}
case "password":
if cfg.User != "" {
fmt.Println(cfg.Password)
}
case "apikey":
if cfg.APIKey != "" {
fmt.Println(cfg.APIKey)
}
default:
die("Invalid setting: " + arg + "\nAvailable settings: enabled, tls, address, user, password, apikey")
}
}
func guiSet(c *cli.Context) {
cfg := getConfig(c)
arg := c.Args()[0]
val := c.Args()[1]
switch strings.ToLower(arg) {
case "enabled":
cfg.GUI.Enabled = parseBool(val)
case "tls":
cfg.GUI.RawUseTLS = parseBool(val)
case "address":
validAddress(val)
cfg.GUI.RawAddress = val
case "user":
cfg.GUI.User = val
case "password":
cfg.GUI.Password = val
case "apikey":
cfg.GUI.APIKey = val
default:
die("Invalid setting: " + arg + "\nAvailable settings: enabled, tls, address, user, password, apikey")
}
setConfig(c, cfg)
}
func guiUnset(c *cli.Context) {
cfg := getConfig(c)
arg := c.Args()[0]
switch strings.ToLower(arg) {
case "user":
cfg.GUI.User = ""
case "password":
cfg.GUI.Password = ""
case "apikey":
cfg.GUI.APIKey = ""
default:
die("Invalid setting: " + arg + "\nAvailable settings: user, password, apikey")
}
setConfig(c, cfg)
}

173
cmd/stcli/cmd_options.go Normal file
View File

@@ -0,0 +1,173 @@
// Copyright (C) 2014 Audrius Butkevičius
package main
import (
"fmt"
"strings"
"github.com/AudriusButkevicius/cli"
)
func init() {
cliCommands = append(cliCommands, cli.Command{
Name: "options",
HideHelp: true,
Usage: "Options command group",
Subcommands: []cli.Command{
{
Name: "dump",
Usage: "Show all Syncthing option settings",
Requires: &cli.Requires{},
Action: optionsDump,
},
{
Name: "get",
Usage: "Get a Syncthing option setting",
Requires: &cli.Requires{"setting"},
Action: optionsGet,
},
{
Name: "set",
Usage: "Set a Syncthing option setting",
Requires: &cli.Requires{"setting", "value..."},
Action: optionsSet,
},
},
})
}
func optionsDump(c *cli.Context) {
cfg := getConfig(c).Options
writer := newTableWriter()
fmt.Fprintln(writer, "Sync protocol listen addresses:\t", strings.Join(cfg.ListenAddresses, " "), "\t(addresses)")
fmt.Fprintln(writer, "Global discovery enabled:\t", cfg.GlobalAnnEnabled, "\t(globalannenabled)")
fmt.Fprintln(writer, "Global discovery servers:\t", strings.Join(cfg.GlobalAnnServers, " "), "\t(globalannserver)")
fmt.Fprintln(writer, "Local discovery enabled:\t", cfg.LocalAnnEnabled, "\t(localannenabled)")
fmt.Fprintln(writer, "Local discovery port:\t", cfg.LocalAnnPort, "\t(localannport)")
fmt.Fprintln(writer, "Outgoing rate limit in KiB/s:\t", cfg.MaxSendKbps, "\t(maxsend)")
fmt.Fprintln(writer, "Incoming rate limit in KiB/s:\t", cfg.MaxRecvKbps, "\t(maxrecv)")
fmt.Fprintln(writer, "Reconnect interval in seconds:\t", cfg.ReconnectIntervalS, "\t(reconnect)")
fmt.Fprintln(writer, "Start browser:\t", cfg.StartBrowser, "\t(browser)")
fmt.Fprintln(writer, "Enable UPnP:\t", cfg.NATEnabled, "\t(nat)")
fmt.Fprintln(writer, "UPnP Lease in minutes:\t", cfg.NATLeaseM, "\t(natlease)")
fmt.Fprintln(writer, "UPnP Renewal period in minutes:\t", cfg.NATRenewalM, "\t(natrenew)")
fmt.Fprintln(writer, "Restart on Wake Up:\t", cfg.RestartOnWakeup, "\t(wake)")
reporting := "unrecognized value"
switch cfg.URAccepted {
case -1:
reporting = "false"
case 0:
reporting = "undecided/false"
case 1:
reporting = "true"
}
fmt.Fprintln(writer, "Anonymous usage reporting:\t", reporting, "\t(reporting)")
writer.Flush()
}
func optionsGet(c *cli.Context) {
cfg := getConfig(c).Options
arg := c.Args()[0]
switch strings.ToLower(arg) {
case "address":
fmt.Println(strings.Join(cfg.ListenAddresses, "\n"))
case "globalannenabled":
fmt.Println(cfg.GlobalAnnEnabled)
case "globalannservers":
fmt.Println(strings.Join(cfg.GlobalAnnServers, "\n"))
case "localannenabled":
fmt.Println(cfg.LocalAnnEnabled)
case "localannport":
fmt.Println(cfg.LocalAnnPort)
case "maxsend":
fmt.Println(cfg.MaxSendKbps)
case "maxrecv":
fmt.Println(cfg.MaxRecvKbps)
case "reconnect":
fmt.Println(cfg.ReconnectIntervalS)
case "browser":
fmt.Println(cfg.StartBrowser)
case "nat":
fmt.Println(cfg.NATEnabled)
case "natlease":
fmt.Println(cfg.NATLeaseM)
case "natrenew":
fmt.Println(cfg.NATRenewalM)
case "reporting":
switch cfg.URAccepted {
case -1:
fmt.Println("false")
case 0:
fmt.Println("undecided/false")
case 1:
fmt.Println("true")
default:
fmt.Println("unknown")
}
case "wake":
fmt.Println(cfg.RestartOnWakeup)
default:
die("Invalid setting: " + arg + "\nAvailable settings: address, globalannenabled, globalannserver, localannenabled, localannport, maxsend, maxrecv, reconnect, browser, upnp, upnplease, upnprenew, reporting, wake")
}
}
func optionsSet(c *cli.Context) {
config := getConfig(c)
arg := c.Args()[0]
val := c.Args()[1]
switch strings.ToLower(arg) {
case "address":
for _, item := range c.Args().Tail() {
validAddress(item)
}
config.Options.ListenAddresses = c.Args().Tail()
case "globalannenabled":
config.Options.GlobalAnnEnabled = parseBool(val)
case "globalannserver":
for _, item := range c.Args().Tail() {
validAddress(item)
}
config.Options.GlobalAnnServers = c.Args().Tail()
case "localannenabled":
config.Options.LocalAnnEnabled = parseBool(val)
case "localannport":
config.Options.LocalAnnPort = parsePort(val)
case "maxsend":
config.Options.MaxSendKbps = parseUint(val)
case "maxrecv":
config.Options.MaxRecvKbps = parseUint(val)
case "reconnect":
config.Options.ReconnectIntervalS = parseUint(val)
case "browser":
config.Options.StartBrowser = parseBool(val)
case "nat":
config.Options.NATEnabled = parseBool(val)
case "natlease":
config.Options.NATLeaseM = parseUint(val)
case "natrenew":
config.Options.NATRenewalM = parseUint(val)
case "reporting":
switch strings.ToLower(val) {
case "u", "undecided", "unset":
config.Options.URAccepted = 0
default:
boolvalue := parseBool(val)
if boolvalue {
config.Options.URAccepted = 1
} else {
config.Options.URAccepted = -1
}
}
case "wake":
config.Options.RestartOnWakeup = parseBool(val)
default:
die("Invalid setting: " + arg + "\nAvailable settings: address, globalannenabled, globalannserver, localannenabled, localannport, maxsend, maxrecv, reconnect, browser, upnp, upnplease, upnprenew, reporting, wake")
}
setConfig(c, config)
}

72
cmd/stcli/cmd_report.go Normal file
View File

@@ -0,0 +1,72 @@
// Copyright (C) 2014 Audrius Butkevičius
package main
import (
"encoding/json"
"fmt"
"github.com/AudriusButkevicius/cli"
)
func init() {
cliCommands = append(cliCommands, cli.Command{
Name: "report",
HideHelp: true,
Usage: "Reporting command group",
Subcommands: []cli.Command{
{
Name: "system",
Usage: "Report system state",
Requires: &cli.Requires{},
Action: reportSystem,
},
{
Name: "connections",
Usage: "Report about connections to other devices",
Requires: &cli.Requires{},
Action: reportConnections,
},
{
Name: "usage",
Usage: "Usage report",
Requires: &cli.Requires{},
Action: reportUsage,
},
},
})
}
func reportSystem(c *cli.Context) {
response := httpGet(c, "system/status")
data := make(map[string]interface{})
json.Unmarshal(responseToBArray(response), &data)
prettyPrintJSON(data)
}
func reportConnections(c *cli.Context) {
response := httpGet(c, "system/connections")
data := make(map[string]map[string]interface{})
json.Unmarshal(responseToBArray(response), &data)
var overall map[string]interface{}
for key, value := range data {
if key == "total" {
overall = value
continue
}
value["Device ID"] = key
prettyPrintJSON(value)
fmt.Println()
}
if overall != nil {
fmt.Println("=== Overall statistics ===")
prettyPrintJSON(overall)
}
}
func reportUsage(c *cli.Context) {
response := httpGet(c, "svc/report")
report := make(map[string]interface{})
json.Unmarshal(responseToBArray(response), &report)
prettyPrintJSON(report)
}

31
cmd/stcli/labels.go Normal file
View File

@@ -0,0 +1,31 @@
// Copyright (C) 2014 Audrius Butkevičius
package main
var jsonAttributeLabels = map[string]string{
"folderMaxMiB": "Largest folder size in MiB",
"folderMaxFiles": "Largest folder file count",
"longVersion": "Long version",
"totMiB": "Total size in MiB",
"totFiles": "Total files",
"uniqueID": "Unique ID",
"numFolders": "Folder count",
"numDevices": "Device count",
"memoryUsageMiB": "Memory usage in MiB",
"memorySize": "Total memory in MiB",
"sha256Perf": "SHA256 Benchmark",
"At": "Last contacted",
"Completion": "Percent complete",
"InBytesTotal": "Total bytes received",
"OutBytesTotal": "Total bytes sent",
"ClientVersion": "Client version",
"alloc": "Memory allocated in bytes",
"sys": "Memory using in bytes",
"cpuPercent": "CPU load in percent",
"extAnnounceOK": "External announcments working",
"goroutines": "Number of Go routines",
"myID": "Client ID",
"tilde": "Tilde expands to",
"arch": "Architecture",
"os": "OS",
}

63
cmd/stcli/main.go Normal file
View File

@@ -0,0 +1,63 @@
// Copyright (C) 2014 Audrius Butkevičius
package main
import (
"sort"
"github.com/AudriusButkevicius/cli"
)
type ByAlphabet []cli.Command
func (a ByAlphabet) Len() int { return len(a) }
func (a ByAlphabet) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
func (a ByAlphabet) Less(i, j int) bool { return a[i].Name < a[j].Name }
var cliCommands []cli.Command
func main() {
app := cli.NewApp()
app.Name = "syncthing-cli"
app.Author = "Audrius Butkevičius"
app.Email = "audrius.butkevicius@gmail.com"
app.Usage = "Syncthing command line interface"
app.Version = "0.1"
app.HideHelp = true
app.Flags = []cli.Flag{
cli.StringFlag{
Name: "endpoint, e",
Value: "http://127.0.0.1:8384",
Usage: "End point to connect to",
EnvVar: "STENDPOINT",
},
cli.StringFlag{
Name: "apikey, k",
Value: "",
Usage: "API Key",
EnvVar: "STAPIKEY",
},
cli.StringFlag{
Name: "username, u",
Value: "",
Usage: "Username",
EnvVar: "STUSERNAME",
},
cli.StringFlag{
Name: "password, p",
Value: "",
Usage: "Password",
EnvVar: "STPASSWORD",
},
cli.BoolFlag{
Name: "insecure, i",
Usage: "Do not verify SSL certificate",
EnvVar: "STINSECURE",
},
}
sort.Sort(ByAlphabet(cliCommands))
app.Commands = cliCommands
app.RunAndExitOnError()
}

165
cmd/stcli/utils.go Normal file
View File

@@ -0,0 +1,165 @@
// Copyright (C) 2014 Audrius Butkevičius
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"os"
"regexp"
"sort"
"strconv"
"strings"
"text/tabwriter"
"unicode"
"github.com/AudriusButkevicius/cli"
"github.com/syncthing/syncthing/lib/config"
"github.com/syncthing/syncthing/lib/protocol"
)
func responseToBArray(response *http.Response) []byte {
defer response.Body.Close()
bytes, err := ioutil.ReadAll(response.Body)
if err != nil {
die(err)
}
return bytes
}
func die(vals ...interface{}) {
if len(vals) > 1 || vals[0] != nil {
os.Stderr.WriteString(fmt.Sprintln(vals...))
os.Exit(1)
}
}
func wrappedHTTPPost(url string) func(c *cli.Context) {
return func(c *cli.Context) {
httpPost(c, url, "")
}
}
func prettyPrintJSON(json map[string]interface{}) {
writer := newTableWriter()
remap := make(map[string]interface{})
for k, v := range json {
key, ok := jsonAttributeLabels[k]
if !ok {
key = firstUpper(k)
}
remap[key] = v
}
jsonKeys := make([]string, 0, len(remap))
for key := range remap {
jsonKeys = append(jsonKeys, key)
}
sort.Strings(jsonKeys)
for _, k := range jsonKeys {
var value string
rvalue := remap[k]
switch rvalue.(type) {
case int, int16, int32, int64, uint, uint16, uint32, uint64, float32, float64:
value = fmt.Sprintf("%.0f", rvalue)
default:
value = fmt.Sprint(rvalue)
}
if value == "" {
continue
}
fmt.Fprintln(writer, k+":\t"+value)
}
writer.Flush()
}
func firstUpper(str string) string {
for i, v := range str {
return string(unicode.ToUpper(v)) + str[i+1:]
}
return ""
}
func newTableWriter() *tabwriter.Writer {
writer := new(tabwriter.Writer)
writer.Init(os.Stdout, 0, 8, 0, '\t', 0)
return writer
}
func getMyID(c *cli.Context) string {
response := httpGet(c, "system/status")
data := make(map[string]interface{})
json.Unmarshal(responseToBArray(response), &data)
return data["myID"].(string)
}
func getConfig(c *cli.Context) config.Configuration {
response := httpGet(c, "system/config")
config := config.Configuration{}
json.Unmarshal(responseToBArray(response), &config)
return config
}
func setConfig(c *cli.Context, cfg config.Configuration) {
body, err := json.Marshal(cfg)
die(err)
response := httpPost(c, "system/config", string(body))
if response.StatusCode != 200 {
die("Unexpected status code", response.StatusCode)
}
}
func parseBool(input string) bool {
val, err := strconv.ParseBool(input)
if err != nil {
die(input + " is not a valid value for a boolean")
}
return val
}
func parseInt(input string) int {
val, err := strconv.ParseInt(input, 0, 64)
if err != nil {
die(input + " is not a valid value for an integer")
}
return int(val)
}
func parseUint(input string) int {
val, err := strconv.ParseUint(input, 0, 64)
if err != nil {
die(input + " is not a valid value for an unsigned integer")
}
return int(val)
}
func parsePort(input string) int {
port := parseUint(input)
if port < 1 || port > 65535 {
die(input + " is not a valid port\nExpected value between 1 and 65535")
}
return port
}
func validAddress(input string) {
tokens := strings.Split(input, ":")
if len(tokens) != 2 {
die(input + " is not a valid value for an address\nExpected format <ip or hostname>:<port>")
}
matched, err := regexp.MatchString("^[a-zA-Z0-9]+([-a-zA-Z0-9.]+[-a-zA-Z0-9]+)?$", tokens[0])
die(err)
if !matched {
die(input + " is not a valid value for an address\nExpected format <ip or hostname>:<port>")
}
parsePort(tokens[1])
}
func parseDeviceID(input string) protocol.DeviceID {
device, err := protocol.DeviceIDFromString(input)
if err != nil {
die(input + " is not a valid device id")
}
return device
}

113
cmd/stdisco/main.go Normal file
View File

@@ -0,0 +1,113 @@
// Copyright (C) 2016 The Syncthing Authors.
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at http://mozilla.org/MPL/2.0/.
package main
import (
"crypto/rand"
"encoding/binary"
"flag"
"log"
"strings"
"time"
"github.com/syncthing/syncthing/lib/beacon"
"github.com/syncthing/syncthing/lib/discover"
"github.com/syncthing/syncthing/lib/protocol"
)
var (
all = false // print all packets, not just first from each device/source
fake = false // send fake packets to lure out other devices faster
mc = "[ff12::8384]:21027"
bc = 21027
)
var (
// Static prefix that we use when generating fake device IDs, so that we
// can recognize them ourselves. Also makes the device ID start with
// "STPROBE-" which is humanly recognizable.
randomPrefix = []byte{148, 223, 23, 4, 148}
// Our random, fake, device ID that we use when sending announcements.
myID = randomDeviceID()
)
func main() {
flag.BoolVar(&all, "all", all, "Print all received announcements (not only first)")
flag.BoolVar(&fake, "fake", fake, "Send fake announcements")
flag.StringVar(&mc, "mc", mc, "IPv6 multicast address")
flag.IntVar(&bc, "bc", bc, "IPv4 broadcast port number")
flag.Parse()
if fake {
log.Println("My ID:", myID)
}
runbeacon(beacon.NewMulticast(mc), fake)
runbeacon(beacon.NewBroadcast(bc), fake)
select {}
}
func runbeacon(bc beacon.Interface, fake bool) {
go bc.Serve()
go recv(bc)
if fake {
go send(bc)
}
}
// receives and prints discovery announcements
func recv(bc beacon.Interface) {
seen := make(map[string]bool)
for {
data, src := bc.Recv()
if m := binary.BigEndian.Uint32(data); m != discover.Magic {
log.Printf("Incorrect magic %x in announcement from %v", m, src)
continue
}
var ann discover.Announce
ann.Unmarshal(data[4:])
if ann.ID == myID {
// This is one of our own fake packets, don't print it.
continue
}
// Print announcement details for the first packet from a given
// device ID and source address, or if -all was given.
key := ann.ID.String() + src.String()
if all || !seen[key] {
log.Printf("Announcement from %v\n", src)
log.Printf(" %v at %s\n", ann.ID, strings.Join(ann.Addresses, ", "))
seen[key] = true
}
}
}
// sends fake discovery announcements once every second
func send(bc beacon.Interface) {
ann := discover.Announce{
ID: myID,
Addresses: []string{"tcp://fake.example.com:12345"},
}
bs, _ := ann.Marshal()
for {
bc.Send(bs)
time.Sleep(time.Second)
}
}
// returns a random but recognizable device ID
func randomDeviceID() protocol.DeviceID {
var id protocol.DeviceID
copy(id[:], randomPrefix)
rand.Read(id[len(randomPrefix):])
return id
}

19
cmd/stdiscosrv/LICENSE Normal file
View File

@@ -0,0 +1,19 @@
Copyright (C) 2014-2015 The Discosrv Authors
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:
- The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

44
cmd/stdiscosrv/README.md Normal file
View File

@@ -0,0 +1,44 @@
stdiscosrv
==========
[![Latest Build](http://img.shields.io/jenkins/s/http/build.syncthing.net/stdiscosrv.svg?style=flat-square)](http://build.syncthing.net/job/stdiscosrv/lastBuild/)
This is the global discovery server for the `syncthing` project.
To get it, run `go get github.com/syncthing/stdiscosrv` or download the
[latest build](http://build.syncthing.net/job/stdiscosrv/lastSuccessfulBuild/artifact/)
from the build server.
Usage
-----
The discovery server supports `ql` and `postgres` backends.
Specify the backend via `-db-backend` and the database DSN via `-db-dsn`.
By default it will use in-memory `ql` backend. If you wish to persist the
information on disk between restarts in `ql`, specify a file DSN:
```bash
$ stdiscosrv -db-dsn="file:///var/run/stdiscosrv.db"
```
For `postgres`, you will need to create a database and a user with permissions
to create tables in it, then start the stdiscosrv as follows:
```bash
$ export STDISCOSRV_DB_DSN="postgres://user:password@localhost/databasename"
$ stdiscosrv -db-backend="postgres"
```
You can pass the DSN as command line option, but the value what you pass in will
be visible in most process managers, potentially exposing the database password
to other users.
In all cases, the appropriate tables and indexes will be created at first
startup. If it doesn't exit with an error, you're fine.
See `stdiscosrv -help` for other options.
##### Third-party attribution
[cznic/lldb](https://github.com/cznic/lldb), Copyright (C) 2014 The lldb Authors.

75
cmd/stdiscosrv/clean.go Normal file
View File

@@ -0,0 +1,75 @@
// Copyright (C) 2014-2015 Jakob Borg and Contributors (see the CONTRIBUTORS file).
package main
import (
"database/sql"
"log"
"time"
)
type cleansrv struct {
intv time.Duration
db *sql.DB
prep map[string]*sql.Stmt
}
func (s *cleansrv) Serve() {
for {
time.Sleep(next(s.intv))
err := s.cleanOldEntries()
if err != nil {
log.Println("Clean:", err)
}
}
}
func (s *cleansrv) Stop() {
panic("stop unimplemented")
}
func (s *cleansrv) cleanOldEntries() (err error) {
var tx *sql.Tx
tx, err = s.db.Begin()
if err != nil {
return err
}
defer func() {
if err == nil {
err = tx.Commit()
} else {
tx.Rollback()
}
}()
res, err := tx.Stmt(s.prep["cleanAddress"]).Exec()
if err != nil {
return err
}
if rows, _ := res.RowsAffected(); rows > 0 {
log.Printf("Clean: %d old addresses", rows)
}
res, err = tx.Stmt(s.prep["cleanDevice"]).Exec()
if err != nil {
return err
}
if rows, _ := res.RowsAffected(); rows > 0 {
log.Printf("Clean: %d old devices", rows)
}
var devs, addrs int
row := tx.Stmt(s.prep["countDevice"]).QueryRow()
if err = row.Scan(&devs); err != nil {
return err
}
row = tx.Stmt(s.prep["countAddress"]).QueryRow()
if err = row.Scan(&addrs); err != nil {
return err
}
log.Printf("Database: %d devices, %d addresses", devs, addrs)
return nil
}

32
cmd/stdiscosrv/db.go Normal file
View File

@@ -0,0 +1,32 @@
// Copyright (C) 2014-2015 Jakob Borg and Contributors (see the CONTRIBUTORS file).
package main
import (
"database/sql"
"fmt"
)
type setupFunc func(db *sql.DB) error
type compileFunc func(db *sql.DB) (map[string]*sql.Stmt, error)
var (
setupFuncs = make(map[string]setupFunc)
compileFuncs = make(map[string]compileFunc)
)
func register(name string, setup setupFunc, compile compileFunc) {
setupFuncs[name] = setup
compileFuncs[name] = compile
}
func setup(backend string, db *sql.DB) (map[string]*sql.Stmt, error) {
setup, ok := setupFuncs[backend]
if !ok {
return nil, fmt.Errorf("Unsupported backend")
}
if err := setup(db); err != nil {
return nil, err
}
return compileFuncs[backend](db)
}

146
cmd/stdiscosrv/main.go Normal file
View File

@@ -0,0 +1,146 @@
// Copyright (C) 2014-2015 Jakob Borg and Contributors (see the CONTRIBUTORS file).
package main
import (
"crypto/tls"
"database/sql"
"flag"
"fmt"
"log"
"os"
"runtime"
"strconv"
"time"
"github.com/syncthing/syncthing/lib/protocol"
"github.com/syncthing/syncthing/lib/tlsutil"
"github.com/thejerf/suture"
)
const (
minNegCache = 60 // seconds
maxNegCache = 3600 // seconds
maxDeviceAge = 7 * 86400 // one week, in seconds
)
var (
Version string
BuildStamp string
BuildUser string
BuildHost string
BuildDate time.Time
LongVersion string
)
func init() {
stamp, _ := strconv.Atoi(BuildStamp)
BuildDate = time.Unix(int64(stamp), 0)
date := BuildDate.UTC().Format("2006-01-02 15:04:05 MST")
LongVersion = fmt.Sprintf(`stdiscosrv %s (%s %s-%s) %s@%s %s`, Version, runtime.Version(), runtime.GOOS, runtime.GOARCH, BuildUser, BuildHost, date)
}
var (
lruSize = 10240
limitAvg = 5
limitBurst = 20
globalStats stats
statsFile string
backend = "ql"
dsn = getEnvDefault("STDISCOSRV_DB_DSN", "memory://stdiscosrv")
certFile = "cert.pem"
keyFile = "key.pem"
debug = false
useHTTP = false
)
func main() {
const (
cleanIntv = 1 * time.Hour
statsIntv = 5 * time.Minute
)
var listen string
log.SetOutput(os.Stdout)
log.SetFlags(0)
flag.StringVar(&listen, "listen", ":8443", "Listen address")
flag.IntVar(&lruSize, "limit-cache", lruSize, "Limiter cache entries")
flag.IntVar(&limitAvg, "limit-avg", limitAvg, "Allowed average package rate, per 10 s")
flag.IntVar(&limitBurst, "limit-burst", limitBurst, "Allowed burst size, packets")
flag.StringVar(&statsFile, "stats-file", statsFile, "File to write periodic operation stats to")
flag.StringVar(&backend, "db-backend", backend, "Database backend to use")
flag.StringVar(&dsn, "db-dsn", dsn, "Database DSN")
flag.StringVar(&certFile, "cert", certFile, "Certificate file")
flag.StringVar(&keyFile, "key", keyFile, "Key file")
flag.BoolVar(&debug, "debug", debug, "Debug")
flag.BoolVar(&useHTTP, "http", useHTTP, "Listen on HTTP (behind an HTTPS proxy)")
flag.Parse()
log.Println(LongVersion)
var cert tls.Certificate
var err error
if !useHTTP {
cert, err = tls.LoadX509KeyPair(certFile, keyFile)
if err != nil {
log.Println("Failed to load keypair. Generating one, this might take a while...")
cert, err = tlsutil.NewCertificate(certFile, keyFile, "stdiscosrv", 3072)
if err != nil {
log.Fatalln("Failed to generate X509 key pair:", err)
}
}
devID := protocol.NewDeviceID(cert.Certificate[0])
log.Println("Server device ID is", devID)
}
db, err := sql.Open(backend, dsn)
if err != nil {
log.Fatalln("sql.Open:", err)
}
prep, err := setup(backend, db)
if err != nil {
log.Fatalln("Setup:", err)
}
main := suture.NewSimple("main")
main.Add(&querysrv{
addr: listen,
cert: cert,
db: db,
prep: prep,
})
main.Add(&cleansrv{
intv: cleanIntv,
db: db,
prep: prep,
})
main.Add(&statssrv{
intv: statsIntv,
file: statsFile,
db: db,
})
globalStats.Reset()
main.Serve()
}
func getEnvDefault(key, def string) string {
if val := os.Getenv(key); val != "" {
return val
}
return def
}
func next(intv time.Duration) time.Duration {
t0 := time.Now()
t1 := t0.Add(intv).Truncate(intv)
return t1.Sub(t0)
}

98
cmd/stdiscosrv/psql.go Normal file
View File

@@ -0,0 +1,98 @@
// Copyright (C) 2014-2015 Jakob Borg and Contributors (see the CONTRIBUTORS file).
package main
import (
"database/sql"
"fmt"
_ "github.com/lib/pq"
)
func init() {
register("postgres", postgresSetup, postgresCompile)
}
func postgresSetup(db *sql.DB) error {
var err error
db.SetMaxIdleConns(4)
db.SetMaxOpenConns(8)
_, err = db.Exec(`CREATE TABLE IF NOT EXISTS Devices (
DeviceID CHAR(63) NOT NULL PRIMARY KEY,
Seen TIMESTAMP NOT NULL
)`)
if err != nil {
return err
}
var tmp string
row := db.QueryRow(`SELECT 'DevicesDeviceIDIndex'::regclass`)
if err = row.Scan(&tmp); err != nil {
_, err = db.Exec(`CREATE INDEX DevicesDeviceIDIndex ON Devices (DeviceID)`)
}
if err != nil {
return err
}
row = db.QueryRow(`SELECT 'DevicesSeenIndex'::regclass`)
if err = row.Scan(&tmp); err != nil {
_, err = db.Exec(`CREATE INDEX DevicesSeenIndex ON Devices (Seen)`)
}
if err != nil {
return err
}
_, err = db.Exec(`CREATE TABLE IF NOT EXISTS Addresses (
DeviceID CHAR(63) NOT NULL,
Seen TIMESTAMP NOT NULL,
Address VARCHAR(2048) NOT NULL
)`)
if err != nil {
return err
}
row = db.QueryRow(`SELECT 'AddressesDeviceIDSeenIndex'::regclass`)
if err = row.Scan(&tmp); err != nil {
_, err = db.Exec(`CREATE INDEX AddressesDeviceIDSeenIndex ON Addresses (DeviceID, Seen)`)
}
if err != nil {
return err
}
row = db.QueryRow(`SELECT 'AddressesDeviceIDAddressIndex'::regclass`)
if err = row.Scan(&tmp); err != nil {
_, err = db.Exec(`CREATE INDEX AddressesDeviceIDAddressIndex ON Addresses (DeviceID, Address)`)
}
if err != nil {
return err
}
return nil
}
func postgresCompile(db *sql.DB) (map[string]*sql.Stmt, error) {
stmts := map[string]string{
"cleanAddress": "DELETE FROM Addresses WHERE Seen < now() - '2 hour'::INTERVAL",
"cleanDevice": fmt.Sprintf("DELETE FROM Devices WHERE Seen < now() - '%d hour'::INTERVAL", maxDeviceAge/3600),
"countAddress": "SELECT count(*) FROM Addresses",
"countDevice": "SELECT count(*) FROM Devices",
"insertAddress": "INSERT INTO Addresses (DeviceID, Seen, Address) VALUES ($1, now(), $2)",
"insertDevice": "INSERT INTO Devices (DeviceID, Seen) VALUES ($1, now())",
"selectAddress": "SELECT Address FROM Addresses WHERE DeviceID=$1 AND Seen > now() - '1 hour'::INTERVAL ORDER BY random() LIMIT 16",
"selectDevice": "SELECT Seen FROM Devices WHERE DeviceID=$1",
"updateAddress": "UPDATE Addresses SET Seen=now() WHERE DeviceID=$1 AND Address=$2",
"updateDevice": "UPDATE Devices SET Seen=now() WHERE DeviceID=$1",
}
res := make(map[string]*sql.Stmt, len(stmts))
for key, stmt := range stmts {
prep, err := db.Prepare(stmt)
if err != nil {
return nil, err
}
res[key] = prep
}
return res, nil
}

81
cmd/stdiscosrv/ql.go Normal file
View File

@@ -0,0 +1,81 @@
// Copyright (C) 2015 Audrius Butkevicius and Contributors (see the CONTRIBUTORS file).
package main
import (
"database/sql"
"fmt"
"log"
"github.com/cznic/ql"
)
func init() {
ql.RegisterDriver()
register("ql", qlSetup, qlCompile)
}
func qlSetup(db *sql.DB) (err error) {
tx, err := db.Begin()
if err != nil {
return
}
defer func() {
if err == nil {
err = tx.Commit()
} else {
tx.Rollback()
}
}()
_, err = tx.Exec(`CREATE TABLE IF NOT EXISTS Devices (
DeviceID STRING NOT NULL,
Seen TIME NOT NULL
)`)
if err != nil {
return
}
if _, err = tx.Exec(`CREATE INDEX IF NOT EXISTS DevicesDeviceIDIndex ON Devices (DeviceID)`); err != nil {
return
}
_, err = tx.Exec(`CREATE TABLE IF NOT EXISTS Addresses (
DeviceID STRING NOT NULL,
Seen TIME NOT NULL,
Address STRING NOT NULL,
)`)
if err != nil {
return
}
_, err = tx.Exec(`CREATE INDEX IF NOT EXISTS AddressesDeviceIDAddressIndex ON Addresses (DeviceID, Address)`)
return
}
func qlCompile(db *sql.DB) (map[string]*sql.Stmt, error) {
stmts := map[string]string{
"cleanAddress": `DELETE FROM Addresses WHERE Seen < now() - duration("2h")`,
"cleanDevice": fmt.Sprintf(`DELETE FROM Devices WHERE Seen < now() - duration("%dh")`, maxDeviceAge/3600),
"countAddress": "SELECT count(*) FROM Addresses",
"countDevice": "SELECT count(*) FROM Devices",
"insertAddress": "INSERT INTO Addresses (DeviceID, Seen, Address) VALUES ($1, now(), $2)",
"insertDevice": "INSERT INTO Devices (DeviceID, Seen) VALUES ($1, now())",
"selectAddress": `SELECT Address from Addresses WHERE DeviceID==$1 AND Seen > now() - duration("1h") LIMIT 16`,
"selectDevice": "SELECT Seen FROM Devices WHERE DeviceID==$1",
"updateAddress": "UPDATE Addresses Seen=now() WHERE DeviceID==$1 AND Address==$2",
"updateDevice": "UPDATE Devices Seen=now() WHERE DeviceID==$1",
}
res := make(map[string]*sql.Stmt, len(stmts))
for key, stmt := range stmts {
prep, err := db.Prepare(stmt)
if err != nil {
log.Println("Failed to compile", stmt)
return nil, err
}
res[key] = prep
}
return res, nil
}

492
cmd/stdiscosrv/querysrv.go Normal file
View File

@@ -0,0 +1,492 @@
// Copyright (C) 2014-2015 Jakob Borg and Contributors (see the CONTRIBUTORS file).
package main
import (
"bytes"
"crypto/tls"
"database/sql"
"encoding/json"
"encoding/pem"
"fmt"
"log"
"math/rand"
"net"
"net/http"
"net/url"
"strconv"
"sync"
"time"
"github.com/golang/groupcache/lru"
"github.com/syncthing/syncthing/lib/protocol"
"golang.org/x/net/context"
"golang.org/x/time/rate"
)
type querysrv struct {
addr string
db *sql.DB
prep map[string]*sql.Stmt
limiter *safeCache
cert tls.Certificate
listener net.Listener
}
type announcement struct {
Seen time.Time `json:"seen"`
Addresses []string `json:"addresses"`
}
type safeCache struct {
*lru.Cache
mut sync.Mutex
}
func (s *safeCache) Get(key string) (val interface{}, ok bool) {
s.mut.Lock()
val, ok = s.Cache.Get(key)
s.mut.Unlock()
return
}
func (s *safeCache) Add(key string, val interface{}) {
s.mut.Lock()
s.Cache.Add(key, val)
s.mut.Unlock()
}
type requestID int64
func (i requestID) String() string {
return fmt.Sprintf("%016x", int64(i))
}
type contextKey int
const idKey contextKey = iota
func negCacheFor(lastSeen time.Time) int {
since := time.Since(lastSeen).Seconds()
if since >= maxDeviceAge {
return maxNegCache
}
if since < 0 {
// That's weird
return minNegCache
}
// Return a value linearly scaled from minNegCache (at zero seconds ago)
// to maxNegCache (at maxDeviceAge seconds ago).
r := since / maxDeviceAge
return int(minNegCache + r*(maxNegCache-minNegCache))
}
func (s *querysrv) Serve() {
s.limiter = &safeCache{
Cache: lru.New(lruSize),
}
if useHTTP {
listener, err := net.Listen("tcp", s.addr)
if err != nil {
log.Println("Listen:", err)
return
}
s.listener = listener
} else {
tlsCfg := &tls.Config{
Certificates: []tls.Certificate{s.cert},
ClientAuth: tls.RequestClientCert,
SessionTicketsDisabled: true,
MinVersion: tls.VersionTLS12,
CipherSuites: []uint16{
tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
},
}
tlsListener, err := tls.Listen("tcp", s.addr, tlsCfg)
if err != nil {
log.Println("Listen:", err)
return
}
s.listener = tlsListener
}
http.HandleFunc("/v2/", s.handler)
http.HandleFunc("/ping", handlePing)
srv := &http.Server{
ReadTimeout: 5 * time.Second,
WriteTimeout: 5 * time.Second,
MaxHeaderBytes: 1 << 10,
}
if err := srv.Serve(s.listener); err != nil {
log.Println("Serve:", err)
}
}
var topCtx = context.Background()
func (s *querysrv) handler(w http.ResponseWriter, req *http.Request) {
reqID := requestID(rand.Int63())
ctx := context.WithValue(topCtx, idKey, reqID)
if debug {
log.Println(reqID, req.Method, req.URL)
}
t0 := time.Now()
defer func() {
diff := time.Since(t0)
var comment string
if diff > time.Second {
comment = "(very slow request)"
} else if diff > 100*time.Millisecond {
comment = "(slow request)"
}
if comment != "" || debug {
log.Println(reqID, req.Method, req.URL, "completed in", diff, comment)
}
}()
var remoteIP net.IP
if useHTTP {
remoteIP = net.ParseIP(req.Header.Get("X-Forwarded-For"))
} else {
addr, err := net.ResolveTCPAddr("tcp", req.RemoteAddr)
if err != nil {
log.Println("remoteAddr:", err)
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
return
}
remoteIP = addr.IP
}
if s.limit(remoteIP) {
if debug {
log.Println(remoteIP, "is limited")
}
w.Header().Set("Retry-After", "60")
http.Error(w, "Too Many Requests", 429)
return
}
switch req.Method {
case "GET":
s.handleGET(ctx, w, req)
case "POST":
s.handlePOST(ctx, remoteIP, w, req)
default:
globalStats.Error()
http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
}
}
func (s *querysrv) handleGET(ctx context.Context, w http.ResponseWriter, req *http.Request) {
reqID := ctx.Value(idKey).(requestID)
deviceID, err := protocol.DeviceIDFromString(req.URL.Query().Get("device"))
if err != nil {
if debug {
log.Println(reqID, "bad device param")
}
globalStats.Error()
http.Error(w, "Bad Request", http.StatusBadRequest)
return
}
var ann announcement
ann.Seen, err = s.getDeviceSeen(deviceID)
negCache := strconv.Itoa(negCacheFor(ann.Seen))
w.Header().Set("Retry-After", negCache)
w.Header().Set("Cache-Control", "public, max-age="+negCache)
if err != nil {
// The device is not in the database.
globalStats.Query()
http.Error(w, "Not Found", http.StatusNotFound)
return
}
t0 := time.Now()
ann.Addresses, err = s.getAddresses(ctx, deviceID)
if err != nil {
log.Println(reqID, "getAddresses:", err)
globalStats.Error()
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
return
}
if debug {
log.Println(reqID, "getAddresses in", time.Since(t0))
}
globalStats.Query()
if len(ann.Addresses) == 0 {
http.Error(w, "Not Found", http.StatusNotFound)
return
}
globalStats.Answer()
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(ann)
}
func (s *querysrv) handlePOST(ctx context.Context, remoteIP net.IP, w http.ResponseWriter, req *http.Request) {
reqID := ctx.Value(idKey).(requestID)
rawCert := certificateBytes(req)
if rawCert == nil {
if debug {
log.Println(reqID, "no certificates")
}
globalStats.Error()
http.Error(w, "Forbidden", http.StatusForbidden)
return
}
var ann announcement
if err := json.NewDecoder(req.Body).Decode(&ann); err != nil {
if debug {
log.Println(reqID, "decode:", err)
}
globalStats.Error()
http.Error(w, "Bad Request", http.StatusBadRequest)
return
}
deviceID := protocol.NewDeviceID(rawCert)
// handleAnnounce returns *two* errors. The first indicates a problem with
// something the client posted to us. We should return a 400 Bad Request
// and not worry about it. The second indicates that the request was fine,
// but something internal messed up. We should log it and respond with a
// more apologetic 500 Internal Server Error.
userErr, internalErr := s.handleAnnounce(ctx, remoteIP, deviceID, ann.Addresses)
if userErr != nil {
if debug {
log.Println(reqID, "handleAnnounce:", userErr)
}
globalStats.Error()
http.Error(w, "Bad Request", http.StatusBadRequest)
return
}
if internalErr != nil {
log.Println(reqID, "handleAnnounce:", internalErr)
globalStats.Error()
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
return
}
globalStats.Announce()
// TODO: Slowly increase this for stable clients
w.Header().Set("Reannounce-After", "1800")
// We could return the lookup result here, but it's kind of unnecessarily
// expensive to go query the database again so we let the client decide to
// do a lookup if they really care.
w.WriteHeader(http.StatusNoContent)
}
func (s *querysrv) Stop() {
s.listener.Close()
}
func (s *querysrv) handleAnnounce(ctx context.Context, remote net.IP, deviceID protocol.DeviceID, addresses []string) (userErr, internalErr error) {
reqID := ctx.Value(idKey).(requestID)
tx, err := s.db.Begin()
if err != nil {
internalErr = err
return
}
defer func() {
// Since we return from a bunch of different places, we handle
// rollback in the defer.
if internalErr != nil || userErr != nil {
tx.Rollback()
}
}()
for _, annAddr := range addresses {
uri, err := url.Parse(annAddr)
if err != nil {
userErr = err
return
}
host, port, err := net.SplitHostPort(uri.Host)
if err != nil {
userErr = err
return
}
ip := net.ParseIP(host)
if host == "" || ip.IsUnspecified() {
// Do not use IPv6 remote address if requested scheme is tcp4
if uri.Scheme == "tcp4" && remote.To4() == nil {
continue
}
// Do not use IPv4 remote address if requested scheme is tcp6
if uri.Scheme == "tcp6" && remote.To4() != nil {
continue
}
host = remote.String()
}
uri.Host = net.JoinHostPort(host, port)
if err := s.updateAddress(ctx, tx, deviceID, uri.String()); err != nil {
internalErr = err
return
}
}
if err := s.updateDevice(ctx, tx, deviceID); err != nil {
internalErr = err
return
}
t0 := time.Now()
internalErr = tx.Commit()
if debug {
log.Println(reqID, "commit in", time.Since(t0))
}
return
}
func (s *querysrv) limit(remote net.IP) bool {
key := remote.String()
bkt, ok := s.limiter.Get(key)
if ok {
bkt := bkt.(*rate.Limiter)
if !bkt.Allow() {
// Rate limit exceeded; ignore packet
return true
}
} else {
// limitAvg is in packets per ten seconds.
s.limiter.Add(key, rate.NewLimiter(rate.Limit(limitAvg)/10, limitBurst))
}
return false
}
func (s *querysrv) updateDevice(ctx context.Context, tx *sql.Tx, device protocol.DeviceID) error {
reqID := ctx.Value(idKey).(requestID)
t0 := time.Now()
res, err := tx.Stmt(s.prep["updateDevice"]).Exec(device.String())
if err != nil {
return err
}
if debug {
log.Println(reqID, "updateDevice in", time.Since(t0))
}
if rows, _ := res.RowsAffected(); rows == 0 {
t0 = time.Now()
_, err := tx.Stmt(s.prep["insertDevice"]).Exec(device.String())
if err != nil {
return err
}
if debug {
log.Println(reqID, "insertDevice in", time.Since(t0))
}
}
return nil
}
func (s *querysrv) updateAddress(ctx context.Context, tx *sql.Tx, device protocol.DeviceID, uri string) error {
res, err := tx.Stmt(s.prep["updateAddress"]).Exec(device.String(), uri)
if err != nil {
return err
}
if rows, _ := res.RowsAffected(); rows == 0 {
_, err := tx.Stmt(s.prep["insertAddress"]).Exec(device.String(), uri)
if err != nil {
return err
}
}
return nil
}
func (s *querysrv) getAddresses(ctx context.Context, device protocol.DeviceID) ([]string, error) {
rows, err := s.prep["selectAddress"].Query(device.String())
if err != nil {
return nil, err
}
defer rows.Close()
var res []string
for rows.Next() {
var addr string
err := rows.Scan(&addr)
if err != nil {
log.Println("Scan:", err)
continue
}
res = append(res, addr)
}
return res, nil
}
func (s *querysrv) getDeviceSeen(device protocol.DeviceID) (time.Time, error) {
row := s.prep["selectDevice"].QueryRow(device.String())
var seen time.Time
if err := row.Scan(&seen); err != nil {
return time.Time{}, err
}
return seen.In(time.UTC), nil
}
func handlePing(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(204)
}
func certificateBytes(req *http.Request) []byte {
if req.TLS != nil && len(req.TLS.PeerCertificates) > 0 {
return req.TLS.PeerCertificates[0].Raw
}
if hdr := req.Header.Get("X-SSL-Cert"); hdr != "" {
bs := []byte(hdr)
// The certificate is in PEM format but with spaces for newlines. We
// need to reinstate the newlines for the PEM decoder. But we need to
// leave the spaces in the BEGIN and END lines - the first and last
// space - alone.
firstSpace := bytes.Index(bs, []byte(" "))
lastSpace := bytes.LastIndex(bs, []byte(" "))
for i := firstSpace + 1; i < lastSpace; i++ {
if bs[i] == ' ' {
bs[i] = '\n'
}
}
block, _ := pem.Decode(bs)
if block == nil {
// Decoding failed
return nil
}
return block.Bytes
}
return nil
}

141
cmd/stdiscosrv/stats.go Normal file
View File

@@ -0,0 +1,141 @@
// Copyright (C) 2014-2015 Jakob Borg and Contributors (see the CONTRIBUTORS file).
package main
import (
"bytes"
"database/sql"
"fmt"
"io/ioutil"
"log"
"os"
"sync/atomic"
"time"
)
type stats struct {
// Incremented atomically
announces int64
queries int64
answers int64
errors int64
}
func (s *stats) Announce() {
atomic.AddInt64(&s.announces, 1)
}
func (s *stats) Query() {
atomic.AddInt64(&s.queries, 1)
}
func (s *stats) Answer() {
atomic.AddInt64(&s.answers, 1)
}
func (s *stats) Error() {
atomic.AddInt64(&s.errors, 1)
}
// Reset returns a copy of the current stats and resets the counters to
// zero.
func (s *stats) Reset() stats {
// Create a copy of the stats using atomic reads
copy := stats{
announces: atomic.LoadInt64(&s.announces),
queries: atomic.LoadInt64(&s.queries),
answers: atomic.LoadInt64(&s.answers),
errors: atomic.LoadInt64(&s.errors),
}
// Reset the stats by subtracting the values that we copied
atomic.AddInt64(&s.announces, -copy.announces)
atomic.AddInt64(&s.queries, -copy.queries)
atomic.AddInt64(&s.answers, -copy.answers)
atomic.AddInt64(&s.errors, -copy.errors)
return copy
}
type statssrv struct {
intv time.Duration
file string
db *sql.DB
}
func (s *statssrv) Serve() {
lastReset := time.Now()
for {
time.Sleep(next(s.intv))
stats := globalStats.Reset()
d := time.Since(lastReset).Seconds()
lastReset = time.Now()
log.Printf("Stats: %.02f announces/s, %.02f queries/s, %.02f answers/s, %.02f errors/s",
float64(stats.announces)/d, float64(stats.queries)/d, float64(stats.answers)/d, float64(stats.errors)/d)
if s.file != "" {
s.writeToFile(stats, d)
}
}
}
func (s *statssrv) Stop() {
panic("stop unimplemented")
}
func (s *statssrv) writeToFile(stats stats, secs float64) {
newLine := []byte("\n")
var addrs int
row := s.db.QueryRow("SELECT COUNT(*) FROM Addresses")
if err := row.Scan(&addrs); err != nil {
log.Println("stats query:", err)
return
}
fd, err := os.OpenFile(s.file, os.O_RDWR|os.O_CREATE, 0666)
if err != nil {
log.Println("stats file:", err)
return
}
defer func() {
err = fd.Close()
if err != nil {
log.Println("stats file:", err)
}
}()
bs, err := ioutil.ReadAll(fd)
if err != nil {
log.Println("stats file:", err)
return
}
lines := bytes.Split(bytes.TrimSpace(bs), newLine)
if len(lines) > 12 {
lines = lines[len(lines)-12:]
}
latest := fmt.Sprintf("%v: %6d addresses, %8.02f announces/s, %8.02f queries/s, %8.02f answers/s, %8.02f errors/s\n",
time.Now().UTC().Format(time.RFC3339), addrs,
float64(stats.announces)/secs, float64(stats.queries)/secs, float64(stats.answers)/secs, float64(stats.errors)/secs)
lines = append(lines, []byte(latest))
_, err = fd.Seek(0, 0)
if err != nil {
log.Println("stats file:", err)
return
}
err = fd.Truncate(0)
if err != nil {
log.Println("stats file:", err)
return
}
_, err = fd.Write(bytes.Join(lines, newLine))
if err != nil {
log.Println("stats file:", err)
return
}
}

View File

@@ -40,7 +40,8 @@ func main() {
log.Println("Lstat:")
log.Printf(" Size: %d bytes", fi.Size())
log.Printf(" Mode: 0%o", fi.Mode())
log.Printf(" Time: %v (%d)", fi.ModTime(), fi.ModTime().Unix())
log.Printf(" Time: %v", fi.ModTime())
log.Printf(" %d.%09d", fi.ModTime().Unix(), fi.ModTime().Nanosecond())
log.Println()
if !fi.Mode().IsDir() && !fi.Mode().IsRegular() {
@@ -52,7 +53,8 @@ func main() {
log.Println("Stat:")
log.Printf(" Size: %d bytes", fi.Size())
log.Printf(" Mode: 0%o", fi.Mode())
log.Printf(" Time: %v (%d)", fi.ModTime(), fi.ModTime().Unix())
log.Printf(" Time: %v", fi.ModTime())
log.Printf(" %d.%09d", fi.ModTime().Unix(), fi.ModTime().Nanosecond())
log.Println()
}
@@ -68,7 +70,7 @@ func main() {
if *standardBlocks || blockSize < protocol.BlockSize {
blockSize = protocol.BlockSize
}
bs, err := scanner.Blocks(fd, blockSize, fi.Size(), nil)
bs, err := scanner.Blocks(fd, blockSize, fi.Size(), nil, true)
if err != nil {
log.Fatal(err)
}

View File

@@ -49,9 +49,8 @@ func main() {
}
type checkResult struct {
server string
direct []string
relays []discover.Relay
server string
addresses []string
error
}
@@ -67,7 +66,7 @@ func checkServers(deviceID protocol.DeviceID, servers ...string) {
}()
}
for _ = range servers {
for range servers {
res := <-resc
u, _ := url.Parse(res.server)
@@ -76,17 +75,14 @@ func checkServers(deviceID protocol.DeviceID, servers ...string) {
if res.error != nil {
fmt.Println(" " + res.error.Error())
}
for _, addr := range res.direct {
for _, addr := range res.addresses {
fmt.Println(" address:", addr)
}
for _, rel := range res.relays {
fmt.Printf(" relay: %s (%d ms)\n", rel.URL, rel.Latency)
}
}
}
func checkServer(deviceID protocol.DeviceID, server string) checkResult {
disco, err := discover.NewGlobal(server, tls.Certificate{}, nil, nil)
disco, err := discover.NewGlobal(server, tls.Certificate{}, nil)
if err != nil {
return checkResult{error: err}
}
@@ -98,8 +94,8 @@ func checkServer(deviceID protocol.DeviceID, server string) checkResult {
})
go func() {
direct, relays, err := disco.Lookup(deviceID)
res <- checkResult{direct: direct, relays: relays, error: err}
addresses, err := disco.Lookup(deviceID)
res <- checkResult{addresses: addresses, error: err}
}()
return <-res

119
cmd/stgenfiles/main.go Normal file
View File

@@ -0,0 +1,119 @@
// Copyright (C) 2016 The Syncthing Authors.
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at http://mozilla.org/MPL/2.0/.
package main
import (
"flag"
"fmt"
"io"
"log"
"math/rand"
"os"
"path/filepath"
"time"
)
func main() {
dir := flag.String("dir", "~/files", "Directory to generate into")
files := flag.Int("files", 1000, "Number of files to create")
maxExp := flag.Int("maxexp", 20, "Max size exponent")
src := flag.String("src", "/dev/urandom", "Source of file data")
flag.Parse()
if err := generateFiles(*dir, *files, *maxExp, *src); err != nil {
log.Println(err)
}
}
func generateFiles(dir string, files, maxexp int, srcname string) error {
fd, err := os.Open(srcname)
if err != nil {
return err
}
for i := 0; i < files; i++ {
n := randomName()
if rand.Float64() < 0.05 {
// Some files and directories are dotfiles
n = "." + n
}
p0 := filepath.Join(dir, string(n[0]), n[0:2])
err = os.MkdirAll(p0, 0755)
if err != nil {
log.Fatal(err)
}
p1 := filepath.Join(p0, n)
s := int64(1 << uint(rand.Intn(maxexp)))
a := int64(128 * 1024)
if a > s {
a = s
}
s += rand.Int63n(a)
if err := generateOneFile(fd, p1, s); err != nil {
return err
}
}
return nil
}
func generateOneFile(fd io.ReadSeeker, p1 string, s int64) error {
src := io.LimitReader(&inifiteReader{fd}, s)
dst, err := os.Create(p1)
if err != nil {
return err
}
_, err = io.Copy(dst, src)
if err != nil {
return err
}
err = dst.Close()
if err != nil {
return err
}
_ = os.Chmod(p1, os.FileMode(rand.Intn(0777)|0400))
t := time.Now().Add(-time.Duration(rand.Intn(30*86400)) * time.Second)
return os.Chtimes(p1, t, t)
}
func randomName() string {
var b [16]byte
readRand(b[:])
return fmt.Sprintf("%x", b[:])
}
func readRand(bs []byte) (int, error) {
var r uint32
for i := range bs {
if i%4 == 0 {
r = uint32(rand.Int63())
}
bs[i] = byte(r >> uint((i%4)*8))
}
return len(bs), nil
}
type inifiteReader struct {
rd io.ReadSeeker
}
func (i *inifiteReader) Read(bs []byte) (int, error) {
n, err := i.rd.Read(bs)
if err == io.EOF {
err = nil
i.rd.Seek(0, 0)
}
return n, err
}

View File

@@ -10,51 +10,71 @@ import (
"encoding/binary"
"fmt"
"log"
"time"
"github.com/syncthing/syncthing/lib/db"
"github.com/syncthing/syncthing/lib/protocol"
"github.com/syndtr/goleveldb/leveldb"
)
func dump(ldb *leveldb.DB) {
func dump(ldb *db.Instance) {
it := ldb.NewIterator(nil, nil)
var dev protocol.DeviceID
for it.Next() {
key := it.Key()
switch key[0] {
case db.KeyTypeDevice:
folder := nulString(key[1 : 1+64])
devBytes := key[1+64 : 1+64+32]
name := nulString(key[1+64+32:])
copy(dev[:], devBytes)
fmt.Printf("[device] F:%q N:%q D:%v\n", folder, name, dev)
folder := binary.BigEndian.Uint32(key[1:])
device := binary.BigEndian.Uint32(key[1+4:])
name := nulString(key[1+4+4:])
fmt.Printf("[device] F:%d D:%d N:%q", folder, device, name)
var f protocol.FileInfo
err := f.UnmarshalXDR(it.Value())
err := f.Unmarshal(it.Value())
if err != nil {
log.Fatal(err)
}
fmt.Printf(" N:%q\n F:%#o\n M:%d\n V:%v\n S:%d\n B:%d\n", f.Name, f.Flags, f.Modified, f.Version, f.Size(), len(f.Blocks))
fmt.Printf(" V:%v\n", f)
case db.KeyTypeGlobal:
folder := nulString(key[1 : 1+64])
name := nulString(key[1+64:])
fmt.Printf("[global] F:%q N:%q V:%x\n", folder, name, it.Value())
folder := binary.BigEndian.Uint32(key[1:])
name := nulString(key[1+4:])
var flv db.VersionList
flv.Unmarshal(it.Value())
fmt.Printf("[global] F:%d N:%q V:%s\n", folder, name, flv)
case db.KeyTypeBlock:
folder := nulString(key[1 : 1+64])
hash := key[1+64 : 1+64+32]
name := nulString(key[1+64+32:])
fmt.Printf("[block] F:%q H:%x N:%q I:%d\n", folder, hash, name, binary.BigEndian.Uint32(it.Value()))
folder := binary.BigEndian.Uint32(key[1:])
hash := key[1+4 : 1+4+32]
name := nulString(key[1+4+32:])
fmt.Printf("[block] F:%d H:%x N:%q I:%d\n", folder, hash, name, binary.BigEndian.Uint32(it.Value()))
case db.KeyTypeDeviceStatistic:
fmt.Printf("[dstat]\n %x\n %x\n", it.Key(), it.Value())
fmt.Printf("[dstat] K:%x V:%x\n", it.Key(), it.Value())
case db.KeyTypeFolderStatistic:
fmt.Printf("[fstat]\n %x\n %x\n", it.Key(), it.Value())
fmt.Printf("[fstat] K:%x V:%x\n", it.Key(), it.Value())
case db.KeyTypeVirtualMtime:
fmt.Printf("[mtime]\n %x\n %x\n", it.Key(), it.Value())
folder := binary.BigEndian.Uint32(key[1:])
name := nulString(key[1+4:])
val := it.Value()
var real, virt time.Time
real.UnmarshalBinary(val[:len(val)/2])
virt.UnmarshalBinary(val[len(val)/2:])
fmt.Printf("[mtime] F:%d N:%q R:%v V:%v\n", folder, name, real, virt)
case db.KeyTypeFolderIdx:
key := binary.BigEndian.Uint32(it.Key()[1:])
fmt.Printf("[folderidx] K:%d V:%q\n", key, it.Value())
case db.KeyTypeDeviceIdx:
key := binary.BigEndian.Uint32(it.Key()[1:])
val := it.Value()
if len(val) == 0 {
fmt.Printf("[deviceidx] K:%d V:<nil>\n", key)
} else {
dev := protocol.DeviceIDFromBytes(val)
fmt.Printf("[deviceidx] K:%d V:%s\n", key, dev)
}
default:
fmt.Printf("[???]\n %x\n %x\n", it.Key(), it.Value())

View File

@@ -8,14 +8,12 @@ package main
import (
"container/heap"
"encoding/binary"
"fmt"
"github.com/syncthing/syncthing/lib/db"
"github.com/syncthing/syncthing/lib/protocol"
"github.com/syndtr/goleveldb/leveldb"
)
// An IntHeap is a min-heap of ints.
type SizedElement struct {
key string
size int
@@ -39,33 +37,31 @@ func (h *ElementHeap) Pop() interface{} {
return x
}
func dumpsize(ldb *leveldb.DB) {
func dumpsize(ldb *db.Instance) {
h := &ElementHeap{}
heap.Init(h)
it := ldb.NewIterator(nil, nil)
var dev protocol.DeviceID
var ele SizedElement
for it.Next() {
key := it.Key()
switch key[0] {
case db.KeyTypeDevice:
folder := nulString(key[1 : 1+64])
devBytes := key[1+64 : 1+64+32]
name := nulString(key[1+64+32:])
copy(dev[:], devBytes)
ele.key = fmt.Sprintf("DEVICE:%s:%s:%s", dev, folder, name)
folder := binary.BigEndian.Uint32(key[1:])
device := binary.BigEndian.Uint32(key[1+4:])
name := nulString(key[1+4+4:])
ele.key = fmt.Sprintf("DEVICE:%d:%d:%s", folder, device, name)
case db.KeyTypeGlobal:
folder := nulString(key[1 : 1+64])
name := nulString(key[1+64:])
ele.key = fmt.Sprintf("GLOBAL:%s:%s", folder, name)
folder := binary.BigEndian.Uint32(key[1:])
name := nulString(key[1+4:])
ele.key = fmt.Sprintf("GLOBAL:%d:%s", folder, name)
case db.KeyTypeBlock:
folder := nulString(key[1 : 1+64])
hash := key[1+64 : 1+64+32]
name := nulString(key[1+64+32:])
ele.key = fmt.Sprintf("BLOCK:%s:%x:%s", folder, hash, name)
folder := binary.BigEndian.Uint32(key[1:])
hash := key[1+4 : 1+4+32]
name := nulString(key[1+4+32:])
ele.key = fmt.Sprintf("BLOCK:%d:%x:%s", folder, hash, name)
case db.KeyTypeDeviceStatistic:
ele.key = fmt.Sprintf("DEVICESTATS:%s", key[1:])
@@ -76,6 +72,14 @@ func dumpsize(ldb *leveldb.DB) {
case db.KeyTypeVirtualMtime:
ele.key = fmt.Sprintf("MTIME:%s", key[1:])
case db.KeyTypeFolderIdx:
id := binary.BigEndian.Uint32(key[1:])
ele.key = fmt.Sprintf("FOLDERIDX:%d", id)
case db.KeyTypeDeviceIdx:
id := binary.BigEndian.Uint32(key[1:])
ele.key = fmt.Sprintf("DEVICEIDX:%d", id)
default:
ele.key = fmt.Sprintf("UNKNOWN:%x", key)
}

View File

@@ -13,8 +13,7 @@ import (
"os"
"path/filepath"
"github.com/syndtr/goleveldb/leveldb"
"github.com/syndtr/goleveldb/leveldb/opt"
"github.com/syncthing/syncthing/lib/db"
)
func main() {
@@ -28,16 +27,12 @@ func main() {
path := flag.Arg(0)
if path == "" {
path = filepath.Join(defaultConfigDir(), "index-v0.11.0.db")
path = filepath.Join(defaultConfigDir(), "index-v0.14.0.db")
}
fmt.Println("Path:", path)
ldb, err := leveldb.OpenFile(path, &opt.Options{
ErrorIfMissing: true,
Strict: opt.StrictAll,
OpenFilesCacheCapacity: 100,
})
ldb, err := db.Open(path)
if err != nil {
log.Fatal(err)
}

View File

@@ -0,0 +1,22 @@
The MIT License (MIT)
Copyright (c) 2015 The Syncthing Project
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -0,0 +1,21 @@
# relaypoolsrv
[![Latest Build](http://img.shields.io/jenkins/s/http/build.syncthing.net/relaypoolsrv.svg?style=flat-square)](http://build.syncthing.net/job/relaypoolsrv/lastBuild/)
This is the relay pool server for the `syncthing` project, which allows community hosted [relaysrv](https://github.com/syncthing/relaysrv)'s to join the public pool.
Servers that join the pool are then advertised to users of `syncthing` as potential connection points for those who are unable to connect directly due to NAT or firewall issues.
There is very little reason why you'd want to run this yourself, as `relaypoolsrv` is just used for announcement and lookup of public relay servers. If you are looking to setup a private or a public relay, please check the documentation for [relaysrv](https://github.com/syncthing/relaysrv), which also explains how to join the default public pool.
If you still want to run it, you can run `go get github.com/syncthing/relaypoolsrv` download it or download the
[latest build](http://build.syncthing.net/job/relaypoolsrv/lastSuccessfulBuild/artifact/)
from the build server.
See `relaypoolsrv -help` for configuration options.
##### Third-party attributions
[oschwald/geoip2-golang](https://github.com/oschwald/geoip2-golang), [oschwald/maxminddb-golang](https://github.com/oschwald/maxminddb-golang), Copyright (C) 2015 [Gregory J. Oschwald](mailto:oschwald@gmail.com).
[lib/pq](https://github.com/lib/pq)</a>, Copyright (C) 2011-2013 'pq' Contributors Portions Copyright (C) 2011 Blake Mizerany.

1
cmd/strelaypoolsrv/auto/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
gui.go

Some files were not shown because too many files have changed in this diff Show More