Avoid:
/_/GOROOT/src/os/user/cgo_lookup_cgo.go:45:(.text+0x54): warning: Using
'getgrgid_r' in statically linked applications requires at runtime the
shared libraries from the glibc version used for linking
and
/tmp/go-build/cgo-gcc-prolog:60:(.text+0x40): warning: Using
'getaddrinfo' in statically linked applications requires at runtime the
shared libraries from the glibc version used for linking
I added these tags as part of the big database PR, but I forget why. I
think it came from an attempt at a static binary using the Go-based
SQLite packages, but that's not the primary build anymore anyway. We can
remove this and go back to the standard resolvers, which gives better
support for primarily Windows and macOS special resolution methods...
chore(gui): remove redundant "authenticated" conditions from Actions
menu (#10235)
Due to previous code changes, the whole Actions menu is only available
when the user is logged in. As such, there is no reason to have the same
ng-if="authenticated" condition repeated in other items belonging to it.
Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
This removes the `debugging` bool under GUI configuration, and two no
longer relevant development endpoints: `httpmetrics` (which I can't
imagine anyone using for anything -- if we need such metrics today, the
right place is the Prometheus exported metrics) and the `peerCompletion`
endpoint (previously used by integration tests).
The debugging bool initially enabled just those two endpoints, which are
not for end users. Then we added profiling and support bundles, which
are very useful indeed for end users to access, and they were hidden
behind the same debug flag. I don't see any reason for keeping that flag
now that these methods are more generally useful.
https://github.com/syncthing/docs/pull/949
This updates our logging framework from legacy freetext strings using
the `log` package to structured log entries using `log/slog`. I have
updated all INFO or higher level entries, but not yet DEBUG (😓)... So,
at a high level:
There is a slight change in log levels, effectively adding a new warning
level:
- DEBUG is still debug (ideally not for users but developers, though
this is something we need to work on)
- INFO is still info, though I've added more data here, effectively
making Syncthing more verbose by default (more on this below)
- WARNING is a new log level that is different from the _old_ WARNING
(more below)
- ERROR is what was WARNING before -- problems that must be dealt with,
and also bubbled as a popup in the GUI.
A new feature is that the logging level can be set per package to
something other than just debug or info, and hence I feel that we can
add a bit more things into INFO while moving some (in fact, most)
current INFO level warnings into WARNING. For example, I think it's
justified to get a log of synced files in INFO and sync failures in
WARNING. These are things that have historically been tricky to debug
properly, and having more information by default will be useful to many,
while still making it possible get close to told level of inscrutability
by setting the log level to WARNING. I'd like to get to a stage where
DEBUG is never necessary to just figure out what's going on, as opposed
to trying to narrow down a likely bug.
Code wise:
- Our logging object, generally known as `l` in each package, is now a
new adapter object that provides the old API on top of the newer one.
(This should go away once all old log entries are migrated.) This is
only for `l.Debugln` and `l.Debugf`.
- There is a new level tracker that keeps the log level for each
package.
- There is a nested setup of handlers, since the structure mandated by
`log/slog` is slightly convoluted (imho). We do this because we need to
do formatting at a "medium" level internally so we can buffer log lines
in text format but with separate timestamp and log level for the API/GUI
to consume.
- The `debug` API call becomes a `loglevels` API call, which can set the
log level to `DEBUG`, `INFO`, `WARNING` or `ERROR` per package. The GUI
is updated to handle this.
- Our custom `sync` package provided some debugging of mutexes quite
strongly integrated into the old logging framework, only turned on when
`STTRACE` was set to certain values at startup, etc. It's been a long
time since this has been useful; I removed it.
- The `STTRACE` env var remains and can be used the same way as before,
while additionally permitting specific log levels to be specified,
`STTRACE=model:WARN,scanner:DEBUG`.
- There is a new command line option `--log-level=INFO` to set the
default log level.
- The command line options `--log-flags` and `--verbose` go away, but
are currently retained as hidden & ignored options since we set them by
default in some of our startup examples and Syncthing would otherwise
fail to start.
Sample format messages:
```
2009-02-13 23:31:30 INF A basic info line (attr1="val with spaces" attr2=2 attr3="val\"quote" a=a log.pkg=slogutil)
2009-02-13 23:31:30 INF An info line with grouped values (attr1=val1 foo.attr2=2 foo.bar.attr3=3 a=a log.pkg=slogutil)
2009-02-13 23:31:30 INF An info line with grouped values via logger (foo.attr1=val1 foo.attr2=2 a=a log.pkg=slogutil)
2009-02-13 23:31:30 INF An info line with nested grouped values via logger (bar.foo.attr1=val1 bar.foo.attr2=2 a=a log.pkg=slogutil)
2009-02-13 23:31:30 WRN A warning entry (a=a log.pkg=slogutil)
2009-02-13 23:31:30 ERR An error (a=a log.pkg=slogutil)
```
---------
Co-authored-by: Ross Smith II <ross@smithii.com>
Based on the discussion in
https://forum.syncthing.net/t/towards-syncthing-2-0/24072/35 This PR
adds the ability for Windows users to use the pipe character (|) to
escape the metacharacters *, ?, [, and { in .stignore files.
Additionally, this PR adds the ability for the user to set the escape
character to backslash, or any character they want, by adding a line in
the form:
#escape=X
(where X is any single rune), to the top of an .stignore file.
This would allow users to use the same .stignore file across platforms,
by simply adding
#escape=\
to the top of the file.
### Testing
All tests pass in CI.
### Documentation
See https://github.com/syncthing/docs/pull/919Fixes#10057: Support escaping in .stignore files on Windows
Fixes#7547: Ignore pattern with \[ and \] does not work
Prior to this fix, the folder would only get marked as "syncing" once we
started downloading data from the network. However in some cases there
will be a lot of data that can be reused locally and we spend
significant time copying blocks before downloading anything; in that
case, the folder would appear as "preparing to sync" while it was in
fact moving lots of data.
This fixes that, making it "syncing" as soon as it begins either copying
or downloading data.
### Purpose
Uses recommended pattern for slice pools to avoid copying the slice
struct, suggested by the linter and actually used in the go stdlib, for
example in `net/http/h2_bundle.go`.
Currently, the Revert Local Changes button for Receive Only folders, and
the Delete Unexpected Items button for Receive Encrypted folders buttons
are shown even when the folder is already performing other operations.
Because of the above, pressing the button seems to have no effect, as
its operation can only proceed after the previous operations have
completed. This confuses the user, who then may keep trying to press the
buttons again and again with no visible result.
Therefore, show the two buttons only when the folders are actually idle,
without performing other operations at the same time. This change makes
them behave similarly to the Override Changes button, which is also only
displayed for Send Only folders when they are idle.
Signed-off-by: Tomasz Wilczyński twilczynski@naver.com
Update the jQuery Fancytree Plugin to the newest version. Apart from
keeping it up-to-date out of principle, this may also help with further
investigation of issue #10155, which is related to the plugin.
Signed-off-by: Tomasz Wilczyński twilczynski@naver.com
chore(gui): Fix "Shut Down" spelling in Actions
Currently, the word is written as "Shutdown". However, similarly to
"Log Out", it is used as a verb here, thus it should be written as two
separate words, i.e. "Shut Down".
Signed-off-by: Tomasz Wilczyński twilczynski@naver.com
Add a wrapper that uses anet on Android, but net on other platforms.
### Purpose
Fixes
https://forum.syncthing.net/t/workaround-for-android-local-discovery/20403/12
### Testing
Run two Syncthing instances with Global Discovery disabled. Pair them
with each other, don't hardcode their addresses, and verify they
connect.
We've always, since the introduction of conflicts, had the policy that
deletes lose against any other change, for safety's sake. This is a
problem, however, because it means the sort order of versions is not a
total order.
That is, given two versions `A` and `B` that are currently in conflict,
we will sort them in a given order (let's say `A, B`, so `A < B` for
ordering purposes: we say "A wins over B" or "A is newer than B") and
consider the first in the list the winner. The loser (who has `B` on
disk) will process the conflict at some point and move the file to a
conflict copy and announce `A'` as the resolved conflict. The winner
(with `A` on disk) doesn't do anything.
However, if `A` is deleted the ordering changes. We still have `A < B`
and, of course, `Adel < A` (this is not even a conflict, just linear
order). In most sane systems this would imply the ordering `Adel < A <
B`, however in our case we in fact have `B < Adel` because any version
wins over a deleted one, so there is no logical ordering at all of the
files at this point. `Adel < A < B < Adel ???` In practice the deleted
version may end up at the head or the tail of the list, depending on the
order we do the compares.
Hence, at this point, "whatever" happens and it's not guaranteed to make
any sense. 😬
I propose that we resolve this my simply letting deletes be versions
like anything else and maintain a total ordering based on just version
vectors with the existing tie breakers like always. That means a delete
can win in a conflict situation, and the result should be that the file
is moved to a conflict copy on the losing device. I think this retains
the data safety to almost the same degree as previously, while removing
probably an entire class of strange out of sync bugs...
---
(A potential wrinkle here is that, ideally, we wouldn't even create the
conflict copy when the delete and the losing version represent the same
data -- same as when we handle normal modification conflicts. However,
the deleted FileInfo doesn't carry any information on what the contents
were, so we can't do that right now. A possible future extension would
be to carry the block list hash of the deleted data in the deleted
FileInfo and use that for this purpose, but I don't want to complicate
this PR with that. The block list hash itself also isn't a
protocol-defined thing at the moment, it's something implementation
dependent that we just use locally.)
### Purpose
As discussed on the forum:
https://forum.syncthing.net/t/reviving-nat-pmp-in-v2-x-on-android-14/24554
TL;DR
Android 14+ only lets java code get the gateway IPv4 address which is
used in SyncthingNative’s NAT-PMP feature.
So I’ve added the java code to the wrapper, got the router IP address
and feeded it to SyncthingNative by setting the env var
“FALLBACK_NET_GATEWAY_IPV4”.
This revives the NAT feature:
> [Z36WU] INFO: Detected 1 NAT service
### Testing
Local build and test via Android emulator (AVD 15).
This makes a couple of backwards compatible changes to the
ClusterConfig:
- Remove the `ignore_permissions` and `ignore_delete` booleans which
we've never read or used for anything
- Remove the `disable_temp_indexes` boolean and option entirely. We did
use this one, and about 1% of users have set the option. The only thing
it does is inhibits sending of periodical DownloadProgress messages
while downloading data, which is a minuscule bandwidth optimisation
given that we're already sending data at the time.
- Change the `read_only` boolean (which indicated send-only folders) to
an enum `FolderType`, where the values zero and one match the existing
usage. Again, we don't actually use this value, but I can see that we
might want to and then it makes more sense for it to be more
comprehensive.
- Change the `paused` boolean to an enum `StopReason`, where zero
indicates not stopped and one indicates paused, exactly the same wire
representation as previously but leaves space for additional stop
reasons (errors etc).
### Purpose
Filesystem watcher errors didnt have any whitespace between the share
name and the error message, making it hard to read. A simple colon and
whitespace solves this issue
I lately wanted some photos on my phone, and watched them sync
excrutiatingly slowly. I am used to android being slow, but not that
slow. This restriction caught my eye and I increased it beyond the
limit (didn't spot it at first), and I did see a clear improvement. Of
course as always with such a one-off test, I might also have
hallucinated it, but it seems plausible with the slow thing in android
being some layer between the actual filesystem and apps.
Also increase the max limit, mostly just because I don't see any reason
to restrict it that low - not that I have a particular reason to want
more.
I also changed the xml default to 0: The `prepare` code will change it
to the actual default - no need to change that anymore if we change the
default in the future.
Before:
- Local discovery on Android 10+ is broken. The phone receives local
discovery packets from other devices running Syncthing on the same
network, e.g. a computer. But it doesn't send its own local discovery
packets.
- Startup of the beacon/broadcast.go and beacon/multicast.go "services"
subsequently fail, see the log entries of "service.go" with "2 of 2
failures, backing off".
Root cause:
- Android 10+ restricts determining the network interfaces for privacy
reasons. The interfaces and IP addresses cannot be determined.
- There's a bug in the go "net" library. I can actually get the
interfaces, but the fix was not implemented by the go team.
Workaround:
- The "community" found a workaround by creating a light wrapper around
"net" called "anet" library.
- "anet" adjusts the behaviour on Android 10+ and gets the interfaces
plus their IP addresses, as required by Syncthing.
After:
- By using the "anet" lib, Syncthing is able to get the interface ip
addresses and put them into the "AllAddresses" string array.
- The "AllAddresses" string array is then announced on the local
discovery multicast and broadcast packets, if enabled in Syncthing's
config.
- By correctly getting the interfaces and IP addresses using "anet" in
"beacon/broadcast.go" and "beacon/multicast.go", the services start up
fine again.
Verification:
- I've built "libSyncthingNative.so" with this PR applied for Android
and put it into Syncthing-Fork v1.29.7.5 for testing. My two phones,
Android 10 and Android 15 (arm64-v8a) immediately discovered each other
using local discovery.
- I can see the "sent XX bytes" and "recv XX bytes" on both phones in
the log filtering for "SyncthingNativeCode" :-).
Personal note:
- Please go light on me, and, if it's not demanded too much of your
time, please help me on this. I am no go programmer. Most things you
think are easy or common sense aren't part of my knowledge set. I'd just
like to help and hope we somehow can drive this home together to fix the
problem.
----
ref: https://github.com/Catfriend1/syncthing-android/pull/1501
ref: https://github.com/Catfriend1/syncthing-android/issues/1500
ref: https://github.com/wlynxg/anet/blob/main/interface.go &
https://github.com/wlynxg/anet/blob/main/interface_android.go
With that fix, I can see the broadcast/multicast lines again and my
phone can be discovered by other phones running the Syncthing app which
wasn't possible before on Android 10+.
```
[ET76H] .346892 broadcast.go:107: DEBUG: sent 185 bytes to 192.168.x.255:21027
[ET76H] .347114 multicast.go:86: DEBUG: sent 185 bytes to [ff12::8384]:21027 on wlan0
```
---------
Co-authored-by: Marcus B Spencer <marcus@marcusspencer.us>
