Commit Graph

2303 Commits

Author SHA1 Message Date
Jakob Borg
10d20c4800 chore(fs): linter complaints 2025-06-06 13:45:44 +02:00
Jakob Borg
700bb75016 chore(model): the easier linter complaints 2025-06-06 13:45:44 +02:00
Jakob Borg
8d37e8f307 Merge branch 'main' into v2
* main:
  feat(config): expose folder and device info as metrics (fixes #9519) (#10148)
  chore: add issue types to GitHub issue templates
  build: remove schedule from PR metadata job
  chore(protocol): only allow enc. password changes on cluster config (#10145)
  chore(protocol): don't start connection routines a second time (#10146)
2025-05-31 17:10:00 +02:00
Jakob Borg
1fdf07933c feat(config): expose folder and device info as metrics (fixes #9519) (#10148)
Tihs makes it easier to use metrics based on device and folder labels,
names, and other attributes. Other metrics which are based on folder or
device ID can be joined with these info metrics to enrich their label
sets.

```
# HELP syncthing_config_device_info Provides additional information labels on devices
# TYPE syncthing_config_device_info gauge
syncthing_config_device_info{device="I6KAH76-66SLLLB-5PFXSOA-UFJCDZC-YAOMLEK-CP2GB32-BV5RQST-3PSROAU",introducer="false",name="s1",paused="false",untrusted="false"} 1

# HELP syncthing_config_folder_info Provides additional information labels on folders
# TYPE syncthing_config_folder_info gauge
syncthing_config_folder_info{folder="default",label="The default folder",path="s2",paused="false",type="sendreceive"} 1
```

With this you can e.g. query for

```
syncthing_connections_active * on(device) group_left syncthing_config_device_info
```

Fixes #9519 
Closes #10074 
Closes #10147
2025-05-31 17:09:23 +02:00
Jakob Borg
8e934a8c69 feat(config): enable multiple connections by default (#10151)
This changes the default number of connections from one to three (one
metadata + two data connections). This should give some advantages of
multiple connections, while also not being an overwhelming change for
larger installations. (Though those may need to tweak their settings
anyway, as always.)
2025-05-30 22:04:55 +02:00
Simon Frei
6765867a2e chore(protocol): only allow enc. password changes on cluster config (#10145)
In practice we already always call SetPassword and ClusterConfig
together. However it's not just "sensible" to do that, it's required: If
the passwords change, the remote device needs to know about that to
check that the enc. setup is valid/consistent (e.g. tokens match,
folder-type is appropriate, ...).
And with the passwords set later, there's no point in adding them as
part of creating a new connection.

This is a "followup" (if one can call it that 4 years later :) ) to
resp. fix for the following commit:
924b96856f

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2025-05-30 09:52:47 +02:00
Simon Frei
4fb8ee6a6f chore(protocol): don't start connection routines a second time (#10146) 2025-05-30 06:28:42 +00:00
Jakob Borg
79bac43800 Merge branch 'main' into v2
* main:
  build: properly propagate build tags to Debian build (#10144)
  fix(protocol): avoid deadlock with concurrent connection start and close (#10140)
  build: add labeler workflow for PRs (#10143)
  build(deps): update our notify package from upstream (#10142)
  build(deps): update dependencies (#10141)
  docs: general notes about v2 coming (#10135)
2025-05-29 17:10:03 +02:00
Jakob Borg
3bd2bff23b fix(protocol): avoid deadlock with concurrent connection start and close (#10140) 2025-05-29 14:56:58 +00:00
Jakob Borg
bb91f53641 Merge branch 'main' into v2
* main:
  refactor: use slices package for sorting (#10136)
  build: handle multiple general release notes
  build: no need to build on the branches that just trigger tags
2025-05-26 21:40:54 +02:00
Marcel Meyer
598915193a refactor: use slices package for sorting (#10136)
Few more complicated usages of the sort packages are left.

### Purpose

Make progress towards replacing the sort package with slices package.
2025-05-26 20:37:49 +02:00
Jakob Borg
54f6b5c2ee Merge branch 'main' into v2
* main:
  build: use specific token for pushing release tags
  fix(gui): update `uncamel()` to handle strings like 'IDs' (fixes #10128) (#10131)
  refactor: use slices package for sort (#10132)
  build: process for automatic release tags (#10133)
  chore(gui, man, authors): update docs, translations, and contributors
2025-05-26 14:22:30 +02:00
Marcel Meyer
48b757cac1 refactor: use slices package for sort (#10132)
The sort package is still used in places that were not trivial to
change. Since Go 1.21 slices package can be uswed for sort. See
https://go.dev/doc/go1.21#slices

### Purpose

Make some progress with the migration to a more up-to-date syntax.
2025-05-26 13:37:26 +02:00
Jakob Borg
99b707c141 fix(versioner): correct fs creation in test 2025-05-25 22:03:19 +02:00
Jakob Borg
39d6692109 Merge branch 'main' into v2
* main:
  fix(syncthing): ensure both config and data dirs exist at startup (fixes #10126) (#10127)
  fix(versioner): fix perms of created folders (fixes #9626) (#10105)
  refactor: use slices.Contains to simplify code (#10121)
2025-05-25 10:00:53 +02:00
Ashish Bhate
1a131a56f2 fix(versioner): fix perms of created folders (fixes #9626) (#10105)
As suggested in the linked issue, I've updated the versioner code to use
the permissions of the corresponding directory in the synced folder,
when creating the folder in the versions directory

### Testing
- Some tests are included with the PR. Happy to add more if you think
there are some edge-cases that we're missing.
- I've tested manually on linux to confirm the permissions of the
created directories.
- I haven't tested on Windows or OSX (I don't have access to these OS)
2025-05-24 07:35:32 +02:00
pullmerge
beda37f28b refactor: use slices.Contains to simplify code (#10121)
There is a [new function](https://pkg.go.dev/slices@go1.21.0#Contains)
added in the go1.21 standard library, which can make the code more
concise and easy to read.
2025-05-23 10:36:06 +00:00
Jakob Borg
78bfe643a8 chore: bump config version, prevent accidental downgrade 2025-05-20 15:37:19 +02:00
Jakob Borg
0a58747eb2 chore: further minor lint fixes 2025-05-20 15:04:33 +02:00
Jakob Borg
96b03fac04 chore: trivial lint fix 2025-05-20 14:34:20 +02:00
Jakob Borg
085455d72e feat: add syncthing debug database-statistics command (#10117)
This adds a command that shows database statistics. Currently it
requires a fork of the sqlite package to add the dbstats virtual table;
the modernc variant already has it.

This also provides the canonical mapping between folder ID and database
file, for tinkerers...

```
% ./bin/syncthing debug database-statistics
DATABASE                 FOLDER ID    TABLE                                  SIZE     FILL
========                 ====== ==    =====                                  ====     ====
main.db                  -            folders                               4 KiB    8.4 %
main.db                  -            folders_database_name                 4 KiB    6.0 %
main.db                  -            kv                                    4 KiB   41.1 %
main.db                  -            schemamigrations                      4 KiB    3.9 %
main.db                  -            sqlite_autoindex_folders_1            4 KiB    3.7 %
...
folder.0007-txpxsvyd.db  w3ejt-fn4dm  indexids                              4 KiB    1.5 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  kv                                    4 KiB    0.8 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  mtimes                              608 KiB   81.5 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  schemamigrations                      4 KiB    3.9 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  sqlite_autoindex_blocklists_1      4108 KiB   89.5 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  sqlite_autoindex_blocks_1        700020 KiB   88.1 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  sqlite_autoindex_devices_1            4 KiB    3.6 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  sqlite_autoindex_kv_1                 4 KiB    0.6 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  sqlite_schema                        12 KiB   45.9 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  sqlite_sequence                       4 KiB    1.0 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  sqlite_stat1                          4 KiB   12.2 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  sqlite_stat4                          4 KiB    0.2 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  (total)                         1906020 KiB   92.8 %
main.db + children       -            (total)                         2205888 KiB   92.0 %
```
2025-05-20 14:27:08 +02:00
Jakob Borg
72849690c9 fix(model): index handler error handling 2025-05-20 14:26:14 +02:00
Jakob Borg
964c8d7d65 fix(model): correct bufferpool handling; simplify (#10113)
The copier routine refactor resulted in bad buffer pool handling,
putting a buffer back into the pool twice. This simplifies and removes
the danger prone Upgrade() method.
2025-05-16 22:50:13 +02:00
Jakob Borg
bacf506e90 Merge branch 'main' into v2
* main:
  chore(gui, man, authors): update docs, translations, and contributors
  fix(config): mark audit log options as needing restart (fixes #10099) (#10100)
  fix(config): deep copy configuration defaults (fixes #9916) (#10101)
2025-05-16 16:17:43 +02:00
Marcus B Spencer
8c3d2f3bc5 fix(config): mark audit log options as needing restart (fixes #10099) (#10100)
### Testing

Change the `auditEnabled` option and you should get a prompt in the Web
GUI.
Restart and change the `auditFile` option, and you should get that same
prompt.

The prompt you should get is shown in the screenshots below.

### Screenshots


![Screenshot_20250507_122546](https://github.com/user-attachments/assets/23ce7c42-5e60-4f88-ac58-f312a9a1f5cc)

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2025-05-09 10:49:11 +00:00
Hazem Krimi
702ed8ecc1 fix(config): deep copy configuration defaults (fixes #9916) (#10101)
### Purpose

Setting default configuration was not working properly since the
defaults struct is not deeply copied.

### Testing

Try running commands to change default configuration and either inspect
`config.xml` or `/rest/config` result to see the applied changed.
Example:
```
./syncthing cli config defaults folder versioning params set keep 5
```
2025-05-09 07:40:32 +02:00
Simon Frei
821d6f43ac chore(model): refactor copier for more flatness (#10094)
Flattened the copier code more. Also removing and moving some
parameters/return values to simplify things. Generally rely less on
return values, e.g. by handling errors right away and using `state` to
do the right thing (e.g. abort on failure).

Supposed to be a refactor without any behaviour changes, except for
fixing a tiny regression on folder order: We used to try copying from
the same folder first, but lost that property at some point (also sent a
PR fixing only that, I'd merge that first making this refactor only).
2025-05-04 09:23:57 +02:00
Simon Frei
fa7b81e1cf fix(model): use same folder first in copier (#10093)
Where `folderFilesystems` and `folders` is built, there's a comment
spelling out the purpose: To have the same folder first, as that's the
most likely to get hits. Plus a copy is possibly more efficient than
from another folder, e.g. if that's on a different filesystem. We lost
that behaviour during some unrelated change.

(Also sneaking in a comment fix on yesterdays change.)
2025-05-02 13:15:26 +02:00
Simon Frei
6b94599467 chore(db, model): simplify per hash DB lookup in copier (#10080)
This is a draft because I haven't adjusted all the tests yet, I'd like
to get feedback on the change overall first, before spending time on
that.

In my opinion the main win of this change is in it's lower complexity
resp. fewer moving parts. It should also be faster as it only does one
query instead of two, but I have no idea if that's practically
relevant.

This also mirrors the v1 DB, where a block map key had the name
appended. Not that this is an argument for the change, it was mostly
reassuring me that I might not be missing something key here
conceptually (I might still be of course, please tell me :) ).

And the change isn't mainly intrinsically motivated, instead it came
up while fixing a bug in the copier. And the nested nature of that code
makes the fix harder, and "un-nesting" it required me to understand
what's happening. This change fell out of that.
2025-05-01 13:44:25 -05:00
xjtdy888
f183d1cbec chore(syncthing): ensure migrated database is closed before exiting (#10076)
After opening the database, we performed some checks, such as whether
the migration had already been successfully completed. If so, the
function returned immediately, and the database was not closed.

---------

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2025-05-01 18:36:35 +00:00
Simon Frei
58bf2b5515 fix(model): close fd immediately in copier (#10079) 2025-05-01 10:15:02 -05:00
Jakob Borg
abe34fc1f6 Merge branch 'main' into v2
* main:
  feat(config): add option for audit file (fixes #9481) (#10066)
  chore(api): log X-Forwarded-For (#10035)
  chore(gui): update dependency copyrights, add script for periodic maintenance (#10067)
  chore(gui, man, authors): update docs, translations, and contributors
  chore(syncthing): remove support for TLS 1.2 sync connections (#10064)
  fix(osutil): give threads same I/O priority on Linux (#10063)
  chore(stun): switch lookup warning to debug level
  chore(gui, man, authors): update docs, translations, and contributors
2025-04-24 08:47:52 +07:00
Simon Frei
be002362b3 fix(model): loop-break regression while block copying in puller (#10069) 2025-04-24 08:29:30 +07:00
Marcus B Spencer
190dff142c feat(config): add option for audit file (fixes #9481) (#10066) 2025-04-23 22:32:23 +07:00
bt90
c667ada63a chore(api): log X-Forwarded-For (#10035)
### Purpose

Fix https://github.com/syncthing/syncthing/issues/9336

The `emitLoginAttempt` function now checks for the presence of an
`X-Forwarded-For` header. The IP from this header is only used if the
connecting host is either on loopback or on the same LAN.

In the case of a host pretending to be a proxy, we'd still have both IPs
in the logs, which should make this much less critical from a security
standpoint.

### Testing

1. directly via localhost
2. via proxy an localhost

#### Logs

```
[3JPXJ] 2025/04/11 15:00:40 INFO: Wrong credentials supplied during API authorization from 127.0.0.1
[3JPXJ] 2025/04/11 15:03:04 INFO: Wrong credentials supplied during API authorization from 192.168.178.5 proxied by 127.0.0.1
```

#### Event API

```
  {
    "id": 23,
    "globalID": 23,
    "time": "2025-04-11T15:00:40.578577402+02:00",
    "type": "LoginAttempt",
    "data": {
      "remoteAddress": "127.0.0.1",
      "success": false,
      "username": "sdfsd"
    }
  },
  {
    "id": 24,
    "globalID": 24,
    "time": "2025-04-11T15:03:04.423403976+02:00",
    "type": "LoginAttempt",
    "data": {
      "proxy": "127.0.0.1",
      "remoteAddress": "192.168.178.5",
      "success": false,
      "username": "sdfsd"
    }
  }
```

### Documentation

https://github.com/syncthing/docs/pull/907

---------

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2025-04-23 06:01:13 +00:00
Jakob Borg
50480b89fc chore(syncthing): remove "default" folder concept (#10068)
This removes the creation of the `default` folder on startup. My feeling
is that the concept is not widely used or appreciated.
2025-04-23 05:59:24 +00:00
Jakob Borg
ff33d976d1 chore(syncthing): remove support for TLS 1.2 sync connections (#10064)
This cleans up the option to allow old TLS 1.2 sync connections. The
flag existed for compatibility with old Syncthing versions that don't
support TLS 1.3, which is approximately Syncthing 1.2.2 (September 2019)
and older. ("Approximately" because it depends on the Go version it's
built with and that's when we switched to building with Go 1.13.)

Ref #10062 because it reminded me this exists.
2025-04-21 10:30:43 +07:00
TheCreeper
69890b4282 fix(osutil): give threads same I/O priority on Linux (#10063) 2025-04-21 02:30:52 +00:00
Jakob Borg
533c9a6ab0 chore(stun): switch lookup warning to debug level 2025-04-17 07:29:10 +07:00
Jakob Borg
ed6575411f Merge branch 'main' into v2
* main:
  feat(stdiscosrv): configurable desired not-found rate
  chore(blobs): generalised blob storage
  chore(stdiscosrv): path style s3
  feat(ursv): add os/arch/distribution metric
  chore(strelaypoolsrv): limit number of returned relays
  build(infra): run in Docker environment for pushes
  chore(stupgrades): expose latest release as a metric
  feat(api, gui): allow authentication bypass for metrics (#10045)
2025-04-13 09:44:09 +02:00
Jakob Borg
b5ffd0a796 chore(blobs): generalised blob storage 2025-04-13 09:41:16 +02:00
Jakob Borg
c74299b59a chore(stdiscosrv): path style s3 2025-04-13 09:40:14 +02:00
Jakob Borg
8b6d837483 feat(ursv): add os/arch/distribution metric 2025-04-13 09:40:14 +02:00
Sébastien WENSKE
ab5c42f4a0 feat(api, gui): allow authentication bypass for metrics (#10045)
### Purpose

Give the ability to skip authentication for prometheus metrics
("/metrics").

### Testing

When authentication is enabled and "Metrics Without Auth" is checked
(not the default), the "/metrics" path remains accessible even when
disconnected.

### Screenshots


![image](https://github.com/user-attachments/assets/144b696b-dd72-46f4-94d5-cd21848e4a4c)

### Documentation

https://github.com/syncthing/docs/pull/906
2025-04-13 07:35:57 +00:00
Jakob Borg
780b8fd3bc Merge branch 'main' into v2
* main:
  build: push artifacts to Azure (#10044)
  fix(syncthing): use separate lock file instead of locking the certificate (fixes #10053) (#10054)
  fix(syncthing): use separate lock file instead of locking the certificate (fixes #10053) (#10054)
2025-04-12 15:16:36 +02:00
Jakob Borg
40888c1a66 fix(syncthing): use separate lock file instead of locking the certificate (fixes #10053) (#10054)
Apparently that nukes the cert under some circumstances on some Windows
🤷
2025-04-12 14:46:57 +02:00
Jakob Borg
ddea2e449c fix(db): version vector serialisation :( (#10050)
ffs
2025-04-09 17:46:49 +02:00
Jakob Borg
95b39a791d Merge branch 'main' into v2
* main:
  fix(gui): fix previous commit
  fix(gui): mark unseen disconnected devices as inactive (#10048)
  fix(strings): differentiate setup(n) and set(v) up (#10024)
  chore(fs): changes to allow Filesystem to be implemented externally (#10040)
  chore(config): resolve primary STUN servers via SRV record (fixes #10029) (#10031)
  build: push artifacts to Azure (#10044)
  chore(gui, man, authors): update docs, translations, and contributors
2025-04-09 15:40:25 +02:00
Paul Donald
5f5d672a7d fix(strings): differentiate setup(n) and set(v) up (#10024)
Correct GUI strings, translations and comments to use proper grammar.
2025-04-08 12:45:05 +00:00
Tommy van der Vorst
d23cd197e1 chore(fs): changes to allow Filesystem to be implemented externally (#10040)
### Purpose

The `fs.Filesystem` interface contains two parts that cannot be
implemented externally because they are private:

* `filesystemWrapperType`: this PR changes `unwrapFilesystem` to
downcast to a specific concrete type
* `underlying`: this PR simply moves it to an unexported interface

### Testing

Regular tests pass.
2025-04-08 12:39:39 +00:00