217 Commits

Author SHA1 Message Date
greatroar
bd0acd04b1 lib/protocol: Use one mutex for rawConnect.awaiting and nextID (#8198)
Having a separate mutex for the three or four instructions needed to
fetch and increment nextID means the overhead exceeds the cost of this
operation.  nextID is now handled inside the critical section for
awaiting instead, while the more expensive channel creation has been
moved outside it.

This is mostly a simplification, though it may have minor performance
benefits in some situations. The single-threaded sender benchmark shows
no significant difference:

name               old speed      new speed      delta
RequestsRawTCP-8   55.3MB/s ± 7%  56.6MB/s ± 6%   ~     (p=0.190 n=10+10)
RequestsTLSoTCP-8  20.5MB/s ±20%  20.8MB/s ± 8%   ~     (p=0.604 n=10+9)
2022-02-28 09:13:30 +01:00
greatroar
a0fd619df3 lib/protocol: Require at least 3.125% savings from compression (#8133)
* lib/protocol: Require at least 3.125% savings from compression

The new lz4 library doesn't need its output buffer to be the maximum
size, unlike the old one (which would allocate if it weren't). It can
take a buffer that is of a smaller size and will report if compressed
data can fit inside the buffer (with a small chance of reporting a false
negative). Use that property to our advantage by requiring compressed
data to be at most n-n/32 = .96875*n bytes long for n input bytes.

* lib/protocol: Remove unused receivers

To make DeepSource happy.

* lib/protocol: Micro-optimize lz4Compress

Only write the length if compression was successful. This is a memory
write, so the compiler can't reorder it.

Only check the return value of lz4.CompressBlock.  Length-zero inputs
are always expanded by LZ4 compression (the library documents this),
so the check on len(src) isn't needed.
2022-01-24 19:36:58 +01:00
Jakob Borg
b6d1e16b4e lib/protocol: Switch to a newer lz4 package (#8122) 2022-01-17 18:52:43 +01:00
André Colomb
0cba3154f0 lib/model: Remove bogus fields from connections API endpoint (fixes #8103) (#8104)
* lib/model: Remove bogus fields from connections API endpoint.

Switch the returned data type for the /rest/system/connections element
"total" to use only the Statistics struct.  The other fields of the
ConnectionInfo struct are not populated and misleading.

* Lowercase JSON field names.

* lib/model: Get rid of ConnectionInfo.MarshalJSON().

It was missing the StartedAt field from the embedded Statistics
struct.  Just lowercasing the JSON attribute names can be done just as
easily with annotations.

* lib/model: Remove bogus startedAt field from totals.

Instead of using the Statistics type with one field empty, just switch
to a free-form map with the three needed fields.
2022-01-10 10:26:45 +01:00
greatroar
ae70046b49 lib/protocol: Remove unused sorting boilerplate 2021-11-27 15:35:07 +01:00
Jakob Borg
4b750b6dc3 all: Remove usage of deprecated io/ioutil (#7971)
As of Go 1.16 io/ioutil is deprecated. This replaces usage with the
corresponding functions in package os and package io.
2021-11-22 08:59:47 +01:00
greatroar
8f8e8a9285 lib/protocol: Simplify codeToError, errorToCode
Also be explicit about the fact that ErrNoError is nil. That name isn't
used anywhere outside this file.
2021-10-22 18:40:40 +02:00
greatroar
46082f194c lib/protocol: Eliminate nativeModel on Unix 2021-10-22 18:40:40 +02:00
Simon Frei
c94b797f00 lib/protocol: Preserve sequence decrypting fileinfos (fixes #7994) (#7995) 2021-10-06 10:26:54 +02:00
Jakob Borg
92eaf52c21 lib/protocol: Test to lock down encryption key derivation
So that we don't inadvertently change how these things work.
2021-10-04 14:36:28 +02:00
greatroar
eafb40460d lib/discover, lib/protocol: Buffer allocation 2021-09-06 15:30:56 +02:00
Jakob Borg
d47745a86b all: Update build constraints to Go 1.17 style (#7894) 2021-08-17 10:10:41 +02:00
Simon Frei
50aacdf1f0 lib/protocol: Ensure correct blocksize on enc. fileinfo (ref #7861) (#7870) 2021-08-04 23:12:01 +02:00
greatroar
bd363fe0b7 lib/protocol: Write uncompressible messages uncompressed (#7790) 2021-06-27 17:59:30 +02:00
Simon Frei
857caf3637 lib/connections: Trigger dialer when connection gets closed (#7753) 2021-06-17 13:57:44 +02:00
Simon Frei
5bdb6798a9 all: Regenerate proto (#7696) 2021-05-19 13:30:20 +02:00
greatroar
3746c899b7 build: List go:generate tools in tools.go (#7599) 2021-05-08 12:52:06 +02:00
Simon Frei
1a00ea7c6e lib: Prevent using protocol method with native path (fixes #7557) (#7563) 2021-04-11 15:29:43 +02:00
Simon Frei
0dcd9794d4 lib/protocol: Deterministic encrypted version (fixes #7533) (#7538) 2021-03-31 08:59:15 +02:00
Simon Frei
924b96856f lib: Handle adding enc folders on an existing conn (fixes #7509) (#7510) 2021-03-22 21:50:19 +01:00
Jakob Borg
8ef504f745 all: Simplify some method calls (#7499)
strings.Replace(a, b, c, -1) -> strings.ReplaceAll(a, b, c)

(Go 1.12) and who knows what was up with that dialQueue.Sort() thing.
2021-03-17 23:12:26 +01:00
Simon Frei
d2d4fcc1df lib/protocol: Improve messages when an error occurs receiving (ref #7466) (#7470) 2021-03-15 19:14:09 +01:00
Simon Frei
f80ee472c2 lib/protocol: Set invalid flag on encrypted file infos (fixes #7466) (#7467) 2021-03-13 16:57:36 +01:00
Jakob Borg
4d979a1ce9 all: Truncate some timestamps (fixes #7457) (#7459)
This truncates times meant for API consumption to second precision,
where fractions won't typically matter or add any value. Exception to
this is timestamps on logs and events, and of course I'm not touching
things like file metadata.

I'm not 100% certain this is an exhaustive change, but it's the things I
found by grepping and following the breadcrumbs from lib/api...

I also considered general-but-ugly solutions, like having the API
serializer itself do reflection magic or even regexps on returned
objects, but decided against it because aurgh...
2021-03-12 10:35:10 +01:00
greatroar
c00520281b lib/protocol: Optimize FileKey (#7440) 2021-03-07 18:44:21 +01:00
Simon Frei
3d91f7c975 lib: Use counterfeiter to mock interfaces in tests (#7375) 2021-03-03 08:53:50 +01:00
greatroar
56b5352f64 all: Use crypt/rand through its buffered version, but not in benchmarks (#7420) 2021-03-02 19:17:20 +01:00
greatroar
ffcaffa32f lib/protocol: Optimize encrypted filename handling + make it more strict (#7408) 2021-02-27 08:57:12 +01:00
Simon Frei
0ffd80f380 lib/protocol: Alwasy return buffers to the pool (#7409) 2021-02-27 08:55:51 +01:00
Simon Frei
c0f353c0e8 lib: Do not set ModifiedBy on meta only changes (#7345) 2021-02-08 15:30:39 +01:00
Jakob Borg
3b7a57d108 lib/protocol: Hide repeated data blocks in a given file (#7319) 2021-02-02 20:15:14 +01:00
Audrius Butkevicius
a7d9268e4d lib/model: Make /browse endpoint return sane objects (#7306) 2021-02-01 09:27:34 +01:00
Jakob Borg
b13b15758d lib/connections, lib/model: Track last connection duration (ref #7223) (#7242)
This adds a statistic to track the last connection duration per device.
It isn't used for much in this PR, but it's available for #7223 to use
in deciding how to order device connection attempts (deprioritizing
devices that just dropped our connection the last time).
2021-01-05 17:45:07 +01:00
Simon Frei
c845e245a1 lib: Close underlying conn in protocol (fixes #7165) (#7212) 2020-12-21 11:40:51 +01:00
Simon Frei
8fd6b1d428 lib/protocol: Handle slashified paths in IsEncryptedParent (fixes #7184) (#7187) 2020-12-09 18:16:14 +01:00
Simon Frei
bbb22c8c80 lib/protocol: Send Close message on read error (#7141) 2020-11-27 11:31:20 +01:00
Simon Frei
5d2c83a7e9 lib/protocol: Fix OOR panic on recv-only folders (#7143) 2020-11-23 18:37:27 +01:00
Simon Frei
31559e908b all: Add untrusted folders behind feature flag (ref #62) (#7055) 2020-11-09 15:33:32 +01:00
Simon Frei
f0f60ba2e7 lib/api: Add /rest/config endpoint (fixes #6540) (#7001) 2020-10-22 19:54:35 +02:00
Simon Frei
a20c6ca868 lib/model, lib/protocol: Send ClusterConfig on config change (fixes #7020) (#7018) 2020-10-02 11:49:51 +02:00
Audrius Butkevicius
e027175446 all: Move remaining protos to use the vanity plugin (#7009) 2020-10-02 08:07:05 +02:00
Simon Frei
1b1d38183d lib: Remove HelloResult type, same as Hello (#7015) 2020-09-29 12:17:38 +01:00
Jakob Borg
674fca3868 lib/db, lib/protocol: Never need empty-version entries (fixes #6961) (#6962)
Avoid havoc when discovering locally-deleted-upgraded files during repair / need calculation...

Co-authored-by: Simon Frei <freisim93@gmail.com>
2020-09-07 20:18:25 +02:00
Jakob Borg
b7986801cd cmd/stindex: Print more hashes and versions 2020-09-04 09:10:33 +02:00
Audrius Butkevicius
d507d932b8 all: Use protobuf to generate config structs (fixes #6734) (#6900) 2020-08-25 08:11:14 +02:00
Simon Frei
ebbe1abe28 lib/protocol: Include WeakHash in BlocksHash (#6799) 2020-06-29 09:33:07 +02:00
Simon Frei
d7fc7008af lib/protocol: Add BlocksHash to FileInfo string (#6802) 2020-06-27 07:44:33 +02:00
Simon Frei
80ada1bb6c lib/protocol: Don't warn when dropping deleted invalid item (fixes #6795) (#6796) 2020-06-25 20:24:31 +02:00
greatroar
dc145bfad7 lib/protocol: Use sha256.Sum256 in NewDeviceID (#6775)
This is shorter, skips two allocations, makes the function inlineable
and is safer, since the compiler now check whether
DeviceIDLength == sha256.Size.
2020-06-21 20:38:06 +02:00
greatroar
273cc9cef8 lib/rand: Various minor fixes (#6752)
crypto/rand output is cryptographically secure by the Go library
documentation's promise. That, rather than strength (= passes randomness
tests) is the property that Syncthing needs).
2020-06-17 10:43:58 +02:00