1544 Commits

Author SHA1 Message Date
Jakob Borg
21da0d7890 fix(stupgrades): return latest stable & pre for each major 2025-06-05 19:19:05 +02:00
ardevd
ebbe57d0ab fix(syncthing): avoid writing panic log to nil fd (#10154)
### Purpose

This change fixes a logical bug in the panic log writing where we could
end up writing to a uninitialized file descriptor.

On the very first iteration, `panicFd` is nil. We enter the if `panicFd
== nil { … }` block, check for “panic:” or “fatal error:”, and if
neither matches, we skip instantiating `panicFd` altogether. However,
immediately after, still within `if panicFd == nil { … }`, we call
`panicFd.WriteString("Panic at ...")`. But `panicFd` would in this case
be `nil`, which will cause a run‐time panic.

It's not clear to me why panicFd is only initialized if the lines start
with "panic:" or "fatal error:" so I've left that logic untouched. With
this change we at least avoid the risk of writing to a nil
filedescriptor.
## Authorship

Your name and email will be added automatically to the AUTHORS file
based on the commit metadata.

---------

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2025-06-03 07:39:21 +02:00
Jakob Borg
1fdf07933c feat(config): expose folder and device info as metrics (fixes #9519) (#10148)
Tihs makes it easier to use metrics based on device and folder labels,
names, and other attributes. Other metrics which are based on folder or
device ID can be joined with these info metrics to enrich their label
sets.

```
# HELP syncthing_config_device_info Provides additional information labels on devices
# TYPE syncthing_config_device_info gauge
syncthing_config_device_info{device="I6KAH76-66SLLLB-5PFXSOA-UFJCDZC-YAOMLEK-CP2GB32-BV5RQST-3PSROAU",introducer="false",name="s1",paused="false",untrusted="false"} 1

# HELP syncthing_config_folder_info Provides additional information labels on folders
# TYPE syncthing_config_folder_info gauge
syncthing_config_folder_info{folder="default",label="The default folder",path="s2",paused="false",type="sendreceive"} 1
```

With this you can e.g. query for

```
syncthing_connections_active * on(device) group_left syncthing_config_device_info
```

Fixes #9519 
Closes #10074 
Closes #10147
2025-05-31 17:09:23 +02:00
Marcel Meyer
598915193a refactor: use slices package for sorting (#10136)
Few more complicated usages of the sort packages are left.

### Purpose

Make progress towards replacing the sort package with slices package.
2025-05-26 20:37:49 +02:00
Marcel Meyer
48b757cac1 refactor: use slices package for sort (#10132)
The sort package is still used in places that were not trivial to
change. Since Go 1.21 slices package can be uswed for sort. See
https://go.dev/doc/go1.21#slices

### Purpose

Make some progress with the migration to a more up-to-date syntax.
2025-05-26 13:37:26 +02:00
Jakob Borg
64b5a1b738 fix(syncthing): ensure both config and data dirs exist at startup (fixes #10126) (#10127)
Previously we'd only ensure the config dir, which is often but not
always the same as the data dir.

Fixes #10126
2025-05-25 08:10:17 +02:00
domain
0bf21d9db2 fix(strelaysrv): make the session limiter session-dependent (fixes #10072) (#10073)
### Purpose

Make the session limiter only apply to current session.

### Testing

Relay 2 or more sessions and check if the sum of the connection speed
can exceed the specified per-session rate.

2 sessions (-global-rate=50000000 and -per-session-rate=6250000):


![图片](https://github.com/user-attachments/assets/133e531a-ed49-4890-aef7-821c628bcfc8)

1 session (-global-rate=50000000 and -per-session-rate=6250000):


![图片](https://github.com/user-attachments/assets/ac89ea53-2d8e-4347-9bbc-4780d85e38d7)
2025-04-30 14:25:01 +00:00
Marcus B Spencer
190dff142c feat(config): add option for audit file (fixes #9481) (#10066) 2025-04-23 22:32:23 +07:00
Jakob Borg
b4776ea4e0 feat(stdiscosrv): configurable desired not-found rate 2025-04-13 09:41:16 +02:00
Jakob Borg
b5ffd0a796 chore(blobs): generalised blob storage 2025-04-13 09:41:16 +02:00
Jakob Borg
c74299b59a chore(stdiscosrv): path style s3 2025-04-13 09:40:14 +02:00
Jakob Borg
8b6d837483 feat(ursv): add os/arch/distribution metric 2025-04-13 09:40:14 +02:00
Jakob Borg
3e74b3dee2 chore(strelaypoolsrv): limit number of returned relays
Avoid unnecessarily enormous responses by returning a random subset of
relays.
2025-04-13 09:40:14 +02:00
Jakob Borg
f6f144bf17 chore(stupgrades): expose latest release as a metric 2025-04-13 09:40:11 +02:00
Jakob Borg
40888c1a66 fix(syncthing): use separate lock file instead of locking the certificate (fixes #10053) (#10054)
Apparently that nukes the cert under some circumstances on some Windows
🤷
2025-04-12 14:46:57 +02:00
Paul Donald
5f5d672a7d fix(strings): differentiate setup(n) and set(v) up (#10024)
Correct GUI strings, translations and comments to use proper grammar.
2025-04-08 12:45:05 +00:00
Jakob Borg
7762e39fb3 chore(syncthing): use file lock on certificate to prevent multiple instances (#10003)
This adds the locking from the SQLite branch, in preparation, so that we
do not inadvertently permit running an instance of each.
2025-03-27 09:26:21 +00:00
Jakob Borg
65923fc255 fix(syncthing): don't auto upgrade to higher major on startup (#9989)
We avoided upgrading to newer major versions during normal auto upgrade
procedures, but currently not in the initial upgrade check on startup.
2025-03-13 07:59:19 +00:00
Jakob Borg
b5a7879eca fix(stdiscosrv): handle announcements properly :p (#9881)
Further protobuf refactor damage, also adding some better debugging
2024-12-19 20:43:46 +00:00
Jakob Borg
77970d5113 refactor: use modern Protobuf encoder (#9817)
At a high level, this is what I've done and why:

- I'm moving the protobuf generation for the `protocol`, `discovery` and
`db` packages to the modern alternatives, and using `buf` to generate
because it's nice and simple.
- After trying various approaches on how to integrate the new types with
the existing code, I opted for splitting off our own data model types
from the on-the-wire generated types. This means we can have a
`FileInfo` type with nicer ergonomics and lots of methods, while the
protobuf generated type stays clean and close to the wire protocol. It
does mean copying between the two when required, which certainly adds a
small amount of inefficiency. If we want to walk this back in the future
and use the raw generated type throughout, that's possible, this however
makes the refactor smaller (!) as it doesn't change everything about the
type for everyone at the same time.
- I have simply removed in cold blood a significant number of old
database migrations. These depended on previous generations of generated
messages of various kinds and were annoying to support in the new
fashion. The oldest supported database version now is the one from
Syncthing 1.9.0 from Sep 7, 2020.
- I changed config structs to be regular manually defined structs.

For the sake of discussion, some things I tried that turned out not to
work...

### Embedding / wrapping

Embedding the protobuf generated structs in our existing types as a data
container and keeping our methods and stuff:

```
package protocol

type FileInfo struct {
  *generated.FileInfo
}
```

This generates a lot of problems because the internal shape of the
generated struct is quite different (different names, different types,
more pointers), because initializing it doesn't work like you'd expect
(i.e., you end up with an embedded nil pointer and a panic), and because
the types of child types don't get wrapped. That is, even if we also
have a similar wrapper around a `Vector`, that's not the type you get
when accessing `someFileInfo.Version`, you get the `*generated.Vector`
that doesn't have methods, etc.

### Aliasing

```
package protocol

type FileInfo = generated.FileInfo
```

Doesn't help because you can't attach methods to it, plus all the above.

### Generating the types into the target package like we do now and
attaching methods

This fails because of the different shape of the generated type (as in
the embedding case above) plus the generated struct already has a bunch
of methods that we can't necessarily override properly (like `String()`
and a bunch of getters).

### Methods to functions

I considered just moving all the methods we attach to functions in a
specific package, so that for example

```
package protocol

func (f FileInfo) Equal(other FileInfo) bool
```

would become

```
package fileinfos

func Equal(a, b *generated.FileInfo) bool
```

and this would mostly work, but becomes quite verbose and cumbersome,
and somewhat limits discoverability (you can't see what methods are
available on the type in auto completions, etc). In the end I did this
in some cases, like in the database layer where a lot of things like
`func (fv *FileVersion) IsEmpty() bool` becomes `func fvIsEmpty(fv
*generated.FileVersion)` because they were anyway just internal methods.

Fixes #8247
2024-12-01 16:50:17 +01:00
Jakob Borg
e82ed6e3d3 style: gofumpt all the things (#9829)
Literally `gofumpt -w .` from the top level dir. Guaranteed to be minor
style changes only and nothing else.

@imsodin per request?
2024-11-19 11:32:56 +01:00
Jakob Borg
47f48faed7 fix(upgrades): avoid clobbering cache when filtering (#9752)
The slice is shared, can't overwrite elements of it. (Upgrade server
only thing.)
2024-10-02 18:56:39 +00:00
Jakob Borg
4d842f7d3b feat(ursrv): new metrics based approach 2024-09-30 14:16:27 -05:00
Jakob Borg
fe01b396ba feat(stupgrades): filter returned releases per compatibility 2024-09-26 10:22:23 +02:00
bt90
9ca101756d chore(ursrv): add Nix detection (#9729)
Classify the builder `nix@nix` as [Nix](https://nixos.org/)

![369684243-172cab09-df6f-449a-a638-1f0a0c080ab3](https://github.com/user-attachments/assets/37a6e0a5-bdcb-4b31-8b36-eaaa42423382)
2024-09-22 14:03:40 +02:00
bt90
a873d12c65 chore(ursrv): extend F-Droid detection (#9728)
Our f-droid apps are currently built using `vagrant@bookworm`:

![grafik](https://github.com/user-attachments/assets/172cab09-df6f-449a-a638-1f0a0c080ab3)
2024-09-22 13:48:38 +02:00
Jakob Borg
d4770ddc77 chore(cmd): clean up commands (#9705)
Move infrastructure related commands to under `cmd/infra` and
development stuff to `cmd/dev`. The default build command builds the
regular user facing binaries: syncthing, stdiscosrv, and strelaysrv.
2024-09-21 09:04:22 +02:00
Jakob Borg
5b9d8a838f chore(stdiscosrv): ensure incoming addresses are sorted and unique 2024-09-15 17:01:16 +02:00
Jakob Borg
8b19cb1e11 chore(stdiscosrv): use zero-allocation merge in the common case 2024-09-15 15:26:40 +02:00
Jakob Borg
ce1e259bb4 chore(stdiscosrv): properly clean out old addresses from memory 2024-09-15 14:20:59 +02:00
Jakob Borg
c8ee2a5cf6 chore(stdiscosrv): calculate IPv6 GUA 2024-09-15 10:48:16 +02:00
Jakob Borg
94d0195b63 chore(stdiscosrv): hide internal/undocumented flags 2024-09-13 08:49:13 +02:00
Jakob Borg
1616edcee3 chore(stdiscosrv): remove legacy replication 2024-09-13 08:49:13 +02:00
Jakob Borg
6505e123bb chore(stdiscosrv): clean up s3 handling 2024-09-13 08:48:04 +02:00
Jakob Borg
63e4659282 chore(stdiscosrv): less garbage in statistics 2024-09-13 08:48:04 +02:00
Jakob Borg
f3f5557c8e chore(stdiscosrv): improve expire, logging 2024-09-13 08:48:04 +02:00
Jakob Borg
b794726e1f chore(stdiscosrv): sched in loop 2024-09-13 08:48:04 +02:00
Jakob Borg
3d59740a0a chore(stdiscosrv): database writing logging 2024-09-13 08:48:04 +02:00
Jakob Borg
66fb65b01f chore(stdiscosrv): use order-preserving expire 2024-09-13 08:48:04 +02:00
Jakob Borg
5c2fcbfd19 chore(stdiscosrv): simplify sorting 2024-09-13 08:48:03 +02:00
Jakob Borg
f9b72330a8 chore(stdiscosrv): reduce allocations in cert handling 2024-09-13 08:48:03 +02:00
Jakob Borg
822b6ac36b chore(stdiscosrv): reduce unnecessary allocations in merge 2024-09-13 08:48:03 +02:00
Jakob Borg
77f7778292 feat(stdiscosrv): enable HTTP profiler 2024-09-13 08:48:03 +02:00
Jakob Borg
aed2c66e52 feat(discosrv): in-memory storage with S3 backing 2024-09-13 08:48:03 +02:00
Jakob Borg
68a1fd010f feat(stdiscosrv): make compression optional (and faster) 2024-09-13 08:33:03 +02:00
Emil Lundberg
7df75e681d gui: Replace global "Panel padding decrease" style with targeted class (#9659)
Transplanted from https://github.com/emlun/syncthing/pull/8 (meta-PR
into https://github.com/syncthing/syncthing/pull/9175) by request of
@acolomb (see:
https://github.com/emlun/syncthing/pull/8#discussion_r1724470574).

This padding decrease currently applies to _all_ collapsible panels, but
this padding decrease may not be appropriate for all collapsible panels.
In particular, it will not be appropriate for the collapsible panels
introduced in https://github.com/emlun/syncthing/pull/8.
2024-08-21 15:02:45 +02:00
Gusted
1243083831 cli: Remove go-shlex dependency (#9644) 2024-08-11 11:37:18 +02:00
Gusted
356c5055ad lib/sha256: Remove it (#9643)
### Purpose

Remove the `lib/sha256` package, because it's no longer necessary. Go's
standard library now has the same performance and is on par with
`sha256-simd` since [Since Go
1.21](1a64574f42).
Therefore using `sha256-simd` has no benefits anymore.

ARM already has optimized sha256 assembly code since
7b8a7f8272,
`sha256-simd` published their results before that optimized assembly was
implemented,
f941fedda8.
The assembly looks very similar and the benchmarks in the Go commit
match that of `sha256-simd`.

This patch removes all of the related code of `lib/sha256` and makes
`crypto/sha256` the 'default'.

Benchmark of `sha256-simd` and `crypto/sha256`:
<details>

```
cpu: AMD Ryzen 5 3600X 6-Core Processor
                │  simd.txt   │               go.txt                │
                │   sec/op    │    sec/op     vs base               │
Hash/8Bytes-12    63.25n ± 1%    73.38n ± 1%  +16.02% (p=0.002 n=6)
Hash/64Bytes-12   98.73n ± 1%   105.30n ± 1%   +6.65% (p=0.002 n=6)
Hash/1K-12        567.2n ± 1%    572.8n ± 1%   +0.99% (p=0.002 n=6)
Hash/8K-12        4.062µ ± 1%    4.062µ ± 1%        ~ (p=0.396 n=6)
Hash/1M-12        512.1µ ± 0%    510.6µ ± 1%        ~ (p=0.485 n=6)
Hash/5M-12        2.556m ± 1%    2.564m ± 0%        ~ (p=0.093 n=6)
Hash/10M-12       5.112m ± 0%    5.127m ± 0%        ~ (p=0.093 n=6)
geomean           13.82µ         14.27µ        +3.28%

                │   simd.txt   │               go.txt                │
                │     B/s      │     B/s       vs base               │
Hash/8Bytes-12    120.6Mi ± 1%   104.0Mi ± 1%  -13.81% (p=0.002 n=6)
Hash/64Bytes-12   618.2Mi ± 1%   579.8Mi ± 1%   -6.22% (p=0.002 n=6)
Hash/1K-12        1.682Gi ± 1%   1.665Gi ± 1%   -0.98% (p=0.002 n=6)
Hash/8K-12        1.878Gi ± 1%   1.878Gi ± 1%        ~ (p=0.310 n=6)
Hash/1M-12        1.907Gi ± 0%   1.913Gi ± 1%        ~ (p=0.485 n=6)
Hash/5M-12        1.911Gi ± 1%   1.904Gi ± 0%        ~ (p=0.093 n=6)
Hash/10M-12       1.910Gi ± 0%   1.905Gi ± 0%        ~ (p=0.093 n=6)
geomean           1.066Gi        1.032Gi        -3.18%
```

</details>


### Testing

Compiled and tested on Linux.

### Documentation

https://github.com/syncthing/docs/pull/874
2024-08-10 12:58:20 +01:00
Ross Smith II
17e60b9e0c Chmod -x non-executable files (fixes #9629) (#9630)
Fixed via
```
git ls-files -s | grep 100755 | grep -E -v '(run|sh)$' | cut -f 2 | xargs git update-index --chmod -x
```
See #9629.
2024-08-05 18:32:23 +02:00
Jakob Borg
2bb5b2244b cmd/stupgrades: Basic process metrics 2024-06-03 19:50:28 +02:00