This reduces the number of file entries we carry in the database,
sometimes significantly. The downside is that if a file is deleted while
a device is offline, and that device comes back more than the cutoff
interval (six months) later, those files will get resurrected at some
point.
The current limit is far too low for our workloads. Perhaps we should
aim even higher than the 64MiB this patch proposes?
Citing the sqlite docs:
> [...] after committing a transaction the rollback journal file may
remain in the file-system. **This increases performance for subsequent
transactions since overwriting an existing file is faster than append to
a file**, but it also consumes file-system space.
tl;dr: if the limit is too low, we're shooting ourselves in the foot in
terms of performance.
(v2 change)
This cleans up the command line parsing a little:
- Remove the hack for supporting legacy single-dash long options (e.g.
`-home`), thus enabling actual short options
- Move legacy imperative flags from under the serve command into
separate commands, e.g. `syncthing serve --paths` to see the paths list
is now `syncthing paths`, `syncthing --upgrade-check` is now `syncthing
upgrade --check`
- Add environment variable support for all remaining flags for the
`serve` command (with one exception, left for the reader to discover),
as these are now all modifiers and not imperative
```
% syncthing --help
Usage: syncthing <command>
Flags:
-h, --help Show context-sensitive help.
Commands:
serve Run Syncthing (default)
cli Command line interface for Syncthing
browser Open GUI in browser, then exit
decrypt Decrypt or verify an encrypted folder
device-id Show device ID, then exit
generate Generate key and config, then exit
paths Show configuration paths, then exit
upgrade Perform or check for upgrade, then exit
version Show current version, then exit
debug Various debugging commands
install-completions Print commands to install shell completions
Run "syncthing <command> --help" for more information on a command.
```
```
% syncthing serve --help
Usage: syncthing serve [flags]
Run Syncthing (default)
Flags:
-h, --help Show context-sensitive help.
-C, --config=PATH Set configuration directory (config and keys) ($STCONFDIR)
-D, --data=PATH Set data directory (database and logs) ($STDATADIR)
-H, --home=PATH Set configuration and data directory ($STHOMEDIR)
--allow-newer-config Allow loading newer than current config version ($STALLOWNEWERCONFIG)
--audit Write events to audit file ($STAUDIT)
--auditfile=PATH Specify audit file (use "-" for stdout, "--" for stderr) ($STAUDITFILE)
--db-maintenance-interval=8h Database maintenance interval ($STDBMAINTINTERVAL)
--gui-address=URL Override GUI address (e.g. "http://192.0.2.42:8443") ($STGUIADDRESS)
--gui-apikey=API-KEY Override GUI API key ($STGUIAPIKEY)
--no-console Hide console window ($STHIDECONSOLE)
--logfile=PATH Log file name (see below) ($STLOGFILE)
--logflags=BITS Select information in log line prefix (see below) ($STLOGFLAGS)
--log-max-old-files=N Number of old files to keep (zero to keep only current) ($STNUMLOGFILES)
--log-max-size=BYTES Maximum size of any file (zero to disable log rotation) ($STLOGMAXSIZE)
--no-browser Do not start browser ($STNOBROWSER)
--no-default-folder Don't create the "default" folder on first startup ($STNODEFAULTFOLDER)
--no-port-probing Don't try to find free ports for GUI and listen addresses on first startup ($STNOPORTPROBING)
--no-restart Do not restart Syncthing when exiting due to API/GUI command, upgrade, or crash ($STNORESTART)
--no-upgrade Disable automatic upgrades ($STNOUPGRADE)
--paused Start with all devices and folders paused ($STPAUSED)
--unpaused Start with all devices and folders unpaused ($STUNPAUSED)
--verbose Print verbose log output ($STVERBOSE)
--debug-gui-assets-dir=PATH Directory to load GUI assets from ($STGUIASSETS)
--debug-perf-stats Write running performance statistics to perf-$pid.csv (Unix only) ($STPERFSTATS)
--debug-profile-block Write block profiles to block-$pid-$timestamp.pprof every 20 seconds ($STBLOCKPROFILE)
--debug-profile-cpu Write a CPU profile to cpu-$pid.pprof on exit ($STCPUPROFILE)
--debug-profile-heap Write heap profiles to heap-$pid-$timestamp.pprof each time heap usage increases ($STHEAPPROFILE)
--debug-profiler-listen=ADDR Network profiler listen address ($STPROFILER)
--debug-reset-delta-idxs Reset delta index IDs, forcing a full index exchange
...
```
Similarly to #10009, we will remove some discontinued STUN servers,
except instead of being the official primary server, it's some
unofficial secondary STUN servers.
### Testing
Use a STUN client (like [`pystun3`](https://pypi.org/project/pystun3))
to probe that the removed STUN servers are inactive.
### Documentation
syncthing/docs#902
The mechanism for primary STUN servers, is still intact, in case this
gets retried with a different domain.
### Purpose
As seen in [stun.syncthing.net doesn’t resolve
anymore](https://forum.syncthing.net/t/stun-syncthing-net-doesnt-resolve-anymore/24075/2?u=marbens)
on the forums, stun.syncthing.net has been shut down, so I think it's
probably a good idea to remove it.
### Testing
1. Have two or more devices
2. Disable Relaying
3. Have no Internet ports open on either end for incoming connections
trigger STUN)
4. Enable the `stun` debugging facility in the Actions -> Logs ->
Debugging Facilities
5. Verify that it doesn't output something like this within a few
seconds:
```
2025-03-30 05:51:32 Enabled debug data for "stun"
2025-03-30 05:51:47 Starting stun for Stun@udp://[::]:22000
2025-03-30 05:51:47 Running stun for Stun@udp://[::]:22000 via stun.syncthing.net:3478
2025-03-30 05:51:47 Stun@udp://[::]:22000 stun addr resolution on stun.syncthing.net:3478: lookup stun.syncthing.net: no such host
```
---------
Co-authored-by: Jakob Borg <jakob@kastelo.net>
### Purpose
In the GUI, the device ID validation was case-sensitive and didn’t
account for dash variations, which allowed users to enter an existing
device ID without receiving proper feedback.
This fix ensures the ID is validated in its canonical form, thus
preventing the user from submitting the request if the device ID already
exists.
### Testing
To test this change, try adding a new device with an ID that matches an
existing device, but with a different case or dashes.
Switch the database from LevelDB to SQLite, for greater stability and
simpler code.
Co-authored-by: Tommy van der Vorst <tommy@pixelspark.nl>
Co-authored-by: bt90 <btom1990@googlemail.com>
We've had weak/rolling hashing in the code for quite a while. It was a
popular request for a while, based on the belief that rsync does this
and we should too. However, the benefit is quite small; we save on
average about 0.8% of transferred blocks over the population as a whole:
<img width="974" alt="Screenshot 2025-03-28 at 17 09 02"
src="https://github.com/user-attachments/assets/bbe10dea-f85e-4043-9823-7cef1220b4a2"
/>
This would be fine if the cost was comparably low, however the downside
of attempting rolling hash matching is that we (by default) do a
complete file read on the destination in order to look for matches
before we starting pulling blocks for the file. For any larger file this
means a sometimes long, I/O-intensive pause before the file starts
syncing, for usually no benefit.
I propose we simply rip off the bandaid and save the effort.
Currently, some options are automatically enabled or disabled depending
on the folder type. However, there is no explanation in the GUI on why
the options are like that. Thus, add short explanatory notes to each
case, where the option is either disabled or enabled according to the
current folder type.
### Purpose
This exposes four methods from `Model` through `Internals`. It allows
apps like Synctrain to obtain information about local/remote need and
sync progress.
### Testing
No testing seems necessary, functions are exported verbatim.
### Screenshots
N/a
### Documentation
Not public API, I am aware this interface may change at any time.
## Authorship
OK.
Co-authored-by: Ross Smith II <ross@smithii.com>
Co-authored-by: Jakob Borg <jakob@kastelo.net>
This allows users to easily disable nightly builds in their forks,
simply by disabling the
build-nightly action.
### Testing
I tested it in my fork, and it works.
### Purpose
Path autocompletion wasn't working when using `~` as a shortcut for the
home directory. The issue occurred because the tilde was expanded to
/home/user, which caused the suggestion to no longer match the input
(thus preventing the autocompletion from appearing in the suggestion
list).
To fix this, I replaced the custom `parentAndBase` function, which
handled path splitting in a more complex way, with `filepath.Split` from
the standard `path/filepath` package. This prevents tilde expansion
while keeping the expected behavior for path splitting.
### Testing
The issue has been tested manually on Linux.
### Screenshots
This is extracted from PR #9175. This deduplicates `SetPassword` calls
and makes `postAdjustGui` a single place where PR #9175 can add
another adjustment step for sanitizing changes to WebAuthn
credentials.
This also adds tests to validate that the refactored logic was not
broken.
These refactorizations were made in [PR #9175][1] to accommodate a few
new variants of authentication method and body content. On request from
reviewers, this PR extracts it as a smaller refactorization to review in
isolation.
### Purpose
This extracts a shared `httpRequest` base function from `httpGet` and
`httpPost`, which will be used in PR #9175 for new helper functions
`httpGetCsrf` (hiding all optional parameters except the CSRF token),
`httpPostCsrf` (same) and `httpPostCsrfAuth` (hiding basic auth
parameters). A `getSessionCookie` function is also extracted from
`hasSessionCookie` and will be used to test that concurrent WebAuthn
authentications result in separate sessions (indicated by different
session cookies).
### Purpose
On iOS, the FSEvents API for watching files (also used on macOS) is not
available, but `kqueue` is. This PR ensures `kqueue` support is built on
iOS instead of the FSEvents based watcher implementation.
Before this PR, you could already use the `kqueue` build option to force
its usage. Unfortunately `gomobile`, the tool that I use to build
Syncthing for iOS and macOS for Synctrain, does not support setting
different build flags for iOS and macOS (unless I build separately for
each, which is a bit of a hassle because XCode nonsense). I am assuming
there are good reasons to support FSEvents even though `kqueue` is also
available on macOS (but I'm not sure why?). I do know FSEvents has been
working fine for me on macOS so it seems best to use FSEvents on macOS
and kqueue on iOS.
Note that this also requires https://github.com/syncthing/notify/pull/4
to be merged in `synchting/notify` (until that is done, this PR will
fail to build on iOS due to `notify` still trying to link to `fsevents`
stuff when the `kqueue` build flag is not set).
### Testing
I compiled both `syncthing/notify` and syncthing with this PR applied,
and used that to successfully build the Synctrain iOS app, which after
this PR works fine and should follow up file changes a bit quicker.
### Screenshots
n/a
### Documentation
n/a
## Authorship
Your name and email will be added automatically to the AUTHORS file
based on the commit metadata.
---------
Co-authored-by: Jakob Borg <jakob@kastelo.net>
### Purpose
This is a [new function](https://pkg.go.dev/slices@go1.21.0#Contains)
added in the go1.21 standard library, which can make the code more
concise and easy to read.
### Testing
Describe what testing has been done, and how the reviewer can test the
change
if new tests are not included.
### Screenshots
If this is a GUI change, include screenshots of the change. If not,
please
feel free to just delete this section.
### Documentation
If this is a user visible change (including API and protocol changes),
add a link here
to the corresponding pull request on https://github.com/syncthing/docs
or describe
the documentation changes necessary.
## Authorship
Your name and email will be added automatically to the AUTHORS file
based on the commit metadata.
Signed-off-by: dashangcun <907225865@qq.com>
Co-authored-by: Jakob Borg <jakob@kastelo.net>
The requirements for Windows code signing changed in 2023, so that newly
generated certificates can only be stored in hardware modules. Luckily,
I managed to snag a three year certificate before that so it hasn't
affected us so much. Now though, it does, because our cert is expiring
in March.
This changes the code signing process for Windows to use a cloud
service, Azure Trusted Signing. This appears to work equally well and
outsources the problem entirely, while also being cheaper than the
actual certificate was to begin with. 🤷
The signing entity will be Kastelo AB and not the Syncthing Foundation,
because the latter is almost impossible to get a certificate for as it's
not a normal corporate entity whose existence can be verified, etc. This
is also how it was prior to the latest certificate; it's not ideal, but
I think it's acceptable under the circumstances.
Currently, this just results in a very ambiguous `setting metadata: lookup
failed` while it could report what it's looking up and why it failed
(not found, etc).
