From 836045ee87c937acf66c01ee849edae465734744 Mon Sep 17 00:00:00 2001 From: Jakob Borg Date: Thu, 7 Aug 2025 11:19:36 +0200 Subject: [PATCH] feat: switch logging framework (#10220) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This updates our logging framework from legacy freetext strings using the `log` package to structured log entries using `log/slog`. I have updated all INFO or higher level entries, but not yet DEBUG (😓)... So, at a high level: There is a slight change in log levels, effectively adding a new warning level: - DEBUG is still debug (ideally not for users but developers, though this is something we need to work on) - INFO is still info, though I've added more data here, effectively making Syncthing more verbose by default (more on this below) - WARNING is a new log level that is different from the _old_ WARNING (more below) - ERROR is what was WARNING before -- problems that must be dealt with, and also bubbled as a popup in the GUI. A new feature is that the logging level can be set per package to something other than just debug or info, and hence I feel that we can add a bit more things into INFO while moving some (in fact, most) current INFO level warnings into WARNING. For example, I think it's justified to get a log of synced files in INFO and sync failures in WARNING. These are things that have historically been tricky to debug properly, and having more information by default will be useful to many, while still making it possible get close to told level of inscrutability by setting the log level to WARNING. I'd like to get to a stage where DEBUG is never necessary to just figure out what's going on, as opposed to trying to narrow down a likely bug. Code wise: - Our logging object, generally known as `l` in each package, is now a new adapter object that provides the old API on top of the newer one. (This should go away once all old log entries are migrated.) This is only for `l.Debugln` and `l.Debugf`. - There is a new level tracker that keeps the log level for each package. - There is a nested setup of handlers, since the structure mandated by `log/slog` is slightly convoluted (imho). We do this because we need to do formatting at a "medium" level internally so we can buffer log lines in text format but with separate timestamp and log level for the API/GUI to consume. - The `debug` API call becomes a `loglevels` API call, which can set the log level to `DEBUG`, `INFO`, `WARNING` or `ERROR` per package. The GUI is updated to handle this. - Our custom `sync` package provided some debugging of mutexes quite strongly integrated into the old logging framework, only turned on when `STTRACE` was set to certain values at startup, etc. It's been a long time since this has been useful; I removed it. - The `STTRACE` env var remains and can be used the same way as before, while additionally permitting specific log levels to be specified, `STTRACE=model:WARN,scanner:DEBUG`. - There is a new command line option `--log-level=INFO` to set the default log level. - The command line options `--log-flags` and `--verbose` go away, but are currently retained as hidden & ignored options since we set them by default in some of our startup examples and Syncthing would otherwise fail to start. Sample format messages: ``` 2009-02-13 23:31:30 INF A basic info line (attr1="val with spaces" attr2=2 attr3="val\"quote" a=a log.pkg=slogutil) 2009-02-13 23:31:30 INF An info line with grouped values (attr1=val1 foo.attr2=2 foo.bar.attr3=3 a=a log.pkg=slogutil) 2009-02-13 23:31:30 INF An info line with grouped values via logger (foo.attr1=val1 foo.attr2=2 a=a log.pkg=slogutil) 2009-02-13 23:31:30 INF An info line with nested grouped values via logger (bar.foo.attr1=val1 bar.foo.attr2=2 a=a log.pkg=slogutil) 2009-02-13 23:31:30 WRN A warning entry (a=a log.pkg=slogutil) 2009-02-13 23:31:30 ERR An error (a=a log.pkg=slogutil) ``` --------- Co-authored-by: Ross Smith II --- .golangci.yml | 10 + cmd/infra/strelaypoolsrv/main.go | 4 +- cmd/infra/strelaypoolsrv/main_test.go | 3 - cmd/infra/strelaypoolsrv/stats.go | 4 +- cmd/infra/stupgrades/main.go | 11 +- cmd/infra/ursrv/serve/serve.go | 25 +- cmd/syncthing/blockprof.go | 7 +- cmd/syncthing/cli/utils.go | 9 - cmd/syncthing/crash_reporting.go | 9 +- cmd/syncthing/debug.go | 6 +- cmd/syncthing/generate/generate.go | 20 +- cmd/syncthing/heapprof.go | 7 +- cmd/syncthing/main.go | 165 +++---- cmd/syncthing/monitor.go | 61 +-- etc/linux-systemd/system/syncthing@.service | 2 +- .../syncthing/core/logViewerModalView.html | 13 +- .../syncthing/core/syncthingController.js | 25 +- internal/db/olddb/smallindex.go | 3 +- internal/db/sqlite/db_folderdb.go | 3 +- internal/db/sqlite/db_open.go | 4 +- internal/db/sqlite/db_service.go | 43 +- internal/db/sqlite/debug.go | 8 +- internal/db/sqlite/folderdb_update.go | 12 +- internal/slogutil/expensive.go | 25 ++ internal/slogutil/formatting.go | 187 ++++++++ internal/slogutil/formatting_test.go | 51 +++ internal/slogutil/leveler.go | 104 +++++ internal/slogutil/line.go | 61 +++ internal/slogutil/recorder.go | 59 +++ internal/slogutil/slogadapter.go | 71 +++ internal/slogutil/sloginit.go | 47 ++ internal/slogutil/slogvalues.go | 40 ++ lib/api/api.go | 89 ++-- lib/api/api_auth.go | 22 +- lib/api/api_statics.go | 3 +- lib/api/api_test.go | 21 +- lib/api/confighandler.go | 8 +- lib/api/debug.go | 6 +- lib/api/tokenmanager.go | 3 +- lib/beacon/beacon.go | 2 +- lib/beacon/broadcast.go | 6 +- lib/beacon/debug.go | 6 +- lib/beacon/multicast.go | 5 +- lib/config/config.go | 11 +- lib/config/debug.go | 6 +- lib/config/deviceconfiguration.go | 5 +- lib/config/folderconfiguration.go | 8 + lib/config/migrations.go | 4 +- lib/config/optionsconfiguration.go | 4 +- lib/config/wrapper.go | 9 +- lib/connections/connections_test.go | 4 +- lib/connections/debug.go | 6 +- lib/connections/limiter.go | 14 +- lib/connections/quic_listen.go | 22 +- lib/connections/registry/registry.go | 3 +- lib/connections/relay_listen.go | 16 +- lib/connections/service.go | 104 ++--- lib/connections/structs.go | 5 + lib/connections/tcp_listen.go | 18 +- lib/dialer/control_unix.go | 5 +- lib/dialer/debug.go | 14 +- lib/dialer/internal.go | 7 +- lib/discover/cache.go | 4 +- lib/discover/debug.go | 6 +- lib/discover/global.go | 24 +- lib/discover/local.go | 14 +- lib/discover/manager.go | 30 +- lib/events/debug.go | 4 +- lib/events/events.go | 9 +- lib/fs/basicfs.go | 3 +- lib/fs/casefs_test.go | 4 - lib/fs/debug.go | 11 +- lib/fs/fakefs.go | 2 + lib/fs/filesystem_copy_range.go | 5 +- lib/ignore/ignore.go | 3 +- lib/logger/LICENSE | 19 - lib/logger/logger.go | 407 ------------------ lib/logger/logger_test.go | 209 --------- lib/logger/mocks/logger.go | 142 ------ lib/model/debug.go | 10 +- lib/model/deviceactivity.go | 4 +- lib/model/devicedownloadstate.go | 4 +- lib/model/folder.go | 43 +- lib/model/folder_recvenc.go | 2 +- lib/model/folder_recvonly.go | 5 +- lib/model/folder_sendonly.go | 4 +- lib/model/folder_sendrecv.go | 94 ++-- lib/model/folder_sendrecv_test.go | 8 +- lib/model/folder_sendrecv_windows.go | 4 +- lib/model/folder_summary.go | 5 +- lib/model/folder_test.go | 2 +- lib/model/folderstate.go | 11 +- lib/model/indexhandler.go | 11 +- lib/model/model.go | 155 ++++--- lib/model/model_test.go | 2 +- lib/model/progressemitter.go | 12 +- lib/model/progressemitter_test.go | 10 - lib/model/queue.go | 7 +- lib/model/requests_test.go | 2 +- lib/model/service_map.go | 2 +- lib/model/sharedpullerstate.go | 30 +- lib/model/sharedpullerstate_test.go | 2 - lib/model/util.go | 8 +- lib/nat/debug.go | 6 +- lib/nat/service.go | 24 +- lib/nat/structs.go | 11 +- lib/osutil/osutil.go | 4 +- lib/pmp/debug.go | 6 +- lib/pmp/pmp.go | 3 +- lib/protocol/bep_clusterconfig.go | 8 + lib/protocol/bep_fileinfo.go | 25 ++ lib/protocol/debug.go | 6 +- lib/protocol/deviceid.go | 9 + lib/protocol/nativemodel_windows.go | 11 +- lib/rc/debug.go | 6 +- lib/rc/rc.go | 8 +- lib/relay/client/debug.go | 6 +- lib/relay/client/static.go | 6 +- lib/scanner/blockqueue.go | 3 +- lib/scanner/debug.go | 6 +- lib/scanner/walk.go | 10 +- lib/stun/debug.go | 6 +- lib/stun/stun.go | 3 +- lib/svcutil/svcutil.go | 40 +- lib/sync/debug.go | 32 -- lib/sync/sync.go | 290 ------------- lib/sync/sync_test.go | 349 --------------- lib/syncthing/debug.go | 8 +- lib/syncthing/superuser_windows.go | 7 +- lib/syncthing/syncthing.go | 64 ++- lib/syncthing/utils.go | 33 +- lib/syncthing/verboseservice.go | 192 --------- lib/syncutil/timeoutcond.go | 79 ++++ lib/syncutil/timeoutcond_test.go | 135 ++++++ lib/upgrade/debug.go | 6 +- lib/upgrade/upgrade_supported.go | 8 +- lib/upnp/debug.go | 6 +- lib/upnp/igd_service.go | 6 +- lib/upnp/upnp.go | 48 +-- lib/ur/debug.go | 6 +- lib/ur/failurereporting.go | 6 +- lib/ur/usage_report.go | 12 +- lib/versioner/debug.go | 6 +- lib/versioner/empty_dir_tracker.go | 4 +- lib/versioner/util.go | 10 +- lib/watchaggregator/debug.go | 6 +- relnotes/v2.0.md | 10 + test/h1/config.xml | 21 +- test/h2/config.xml | 19 +- 149 files changed, 1797 insertions(+), 2641 deletions(-) create mode 100644 internal/slogutil/expensive.go create mode 100644 internal/slogutil/formatting.go create mode 100644 internal/slogutil/formatting_test.go create mode 100644 internal/slogutil/leveler.go create mode 100644 internal/slogutil/line.go create mode 100644 internal/slogutil/recorder.go create mode 100644 internal/slogutil/slogadapter.go create mode 100644 internal/slogutil/sloginit.go create mode 100644 internal/slogutil/slogvalues.go delete mode 100644 lib/logger/LICENSE delete mode 100644 lib/logger/logger.go delete mode 100644 lib/logger/logger_test.go delete mode 100644 lib/logger/mocks/logger.go delete mode 100644 lib/sync/debug.go delete mode 100644 lib/sync/sync.go delete mode 100644 lib/sync/sync_test.go delete mode 100644 lib/syncthing/verboseservice.go create mode 100644 lib/syncutil/timeoutcond.go create mode 100644 lib/syncutil/timeoutcond_test.go diff --git a/.golangci.yml b/.golangci.yml index 470a096f0..9b3a1bf0b 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -60,6 +60,16 @@ linters: - builtin$ - examples$ - _test\.go$ + rules: + # relax the slog rules for debug lines, for now + - linters: [sloglint] + source: Debug + settings: + sloglint: + context: "scope" + static-msg: true + msg-style: capitalized + key-naming-case: camel formatters: enable: - gofumpt diff --git a/cmd/infra/strelaypoolsrv/main.go b/cmd/infra/strelaypoolsrv/main.go index ee982633f..7938105d4 100644 --- a/cmd/infra/strelaypoolsrv/main.go +++ b/cmd/infra/strelaypoolsrv/main.go @@ -17,6 +17,7 @@ import ( "path/filepath" "strconv" "strings" + "sync" "sync/atomic" "time" @@ -31,7 +32,6 @@ import ( "github.com/syncthing/syncthing/lib/protocol" "github.com/syncthing/syncthing/lib/rand" "github.com/syncthing/syncthing/lib/relay/client" - "github.com/syncthing/syncthing/lib/sync" "github.com/syncthing/syncthing/lib/tlsutil" ) @@ -115,7 +115,7 @@ var ( requests chan request - mut = sync.NewRWMutex() + mut sync.RWMutex knownRelays = make([]*relay, 0) permanentRelays = make([]*relay, 0) evictionTimers = make(map[string]*time.Timer) diff --git a/cmd/infra/strelaypoolsrv/main_test.go b/cmd/infra/strelaypoolsrv/main_test.go index 6b1bc9b6e..7c8520f14 100644 --- a/cmd/infra/strelaypoolsrv/main_test.go +++ b/cmd/infra/strelaypoolsrv/main_test.go @@ -13,7 +13,6 @@ import ( "net/http/httptest" "net/url" "strings" - "sync" "testing" ) @@ -28,8 +27,6 @@ func init() { {URL: "known2"}, {URL: "known3"}, } - - mut = new(sync.RWMutex) } // Regression test: handleGetRequest should not modify permanentRelays. diff --git a/cmd/infra/strelaypoolsrv/stats.go b/cmd/infra/strelaypoolsrv/stats.go index 322a92fd7..f964b9b37 100644 --- a/cmd/infra/strelaypoolsrv/stats.go +++ b/cmd/infra/strelaypoolsrv/stats.go @@ -6,10 +6,10 @@ import ( "encoding/json" "net" "net/http" + "sync" "time" "github.com/prometheus/client_golang/prometheus" - "github.com/syncthing/syncthing/lib/sync" ) var ( @@ -104,7 +104,7 @@ func refreshStats() { mut.RUnlock() now := time.Now() - wg := sync.NewWaitGroup() + var wg sync.WaitGroup results := make(chan statsFetchResult, len(relays)) for _, rel := range relays { diff --git a/cmd/infra/stupgrades/main.go b/cmd/infra/stupgrades/main.go index 2a794e370..8c12a3b6c 100644 --- a/cmd/infra/stupgrades/main.go +++ b/cmd/infra/stupgrades/main.go @@ -24,6 +24,7 @@ import ( "github.com/alecthomas/kong" "github.com/prometheus/client_golang/prometheus/promhttp" + "github.com/syncthing/syncthing/internal/slogutil" _ "github.com/syncthing/syncthing/lib/automaxprocs" "github.com/syncthing/syncthing/lib/httpcache" "github.com/syncthing/syncthing/lib/upgrade" @@ -58,10 +59,10 @@ func server(params *cli) error { if err != nil { return fmt.Errorf("metrics: %w", err) } - slog.Info("Metrics listener started", "addr", params.MetricsListen) + slog.Info("Metrics listener started", slogutil.Address(params.MetricsListen)) go func() { if err := http.Serve(metricsListen, mux); err != nil { - slog.Warn("Metrics server returned", "error", err) + slog.Warn("Metrics server returned", slogutil.Error(err)) } }() } @@ -75,9 +76,9 @@ func server(params *cli) error { go func() { for range time.NewTicker(params.CacheTime).C { - slog.Info("Refreshing cached releases", "url", params.URL) + slog.Info("Refreshing cached releases", slogutil.URI(params.URL)) if err := cache.Update(context.Background()); err != nil { - slog.Error("Failed to refresh cached releases", "url", params.URL, "error", err) + slog.Error("Failed to refresh cached releases", slogutil.URI(params.URL), slogutil.Error(err)) } } }() @@ -109,7 +110,7 @@ func server(params *cli) error { if err != nil { return fmt.Errorf("listen: %w", err) } - slog.Info("Main listener started", "addr", params.Listen) + slog.Info("Main listener started", slogutil.Address(params.Listen)) return srv.Serve(srvListener) } diff --git a/cmd/infra/ursrv/serve/serve.go b/cmd/infra/ursrv/serve/serve.go index 49ce2d3cb..3626ef0f2 100644 --- a/cmd/infra/ursrv/serve/serve.go +++ b/cmd/infra/ursrv/serve/serve.go @@ -29,6 +29,7 @@ import ( "github.com/syncthing/syncthing/internal/blob" "github.com/syncthing/syncthing/internal/blob/azureblob" "github.com/syncthing/syncthing/internal/blob/s3" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/build" "github.com/syncthing/syncthing/lib/geoip" "github.com/syncthing/syncthing/lib/ur/contract" @@ -104,23 +105,23 @@ func (cli *CLI) Run() error { urListener, err := net.Listen("tcp", cli.Listen) if err != nil { - slog.Error("Failed to listen (usage reports)", "error", err) + slog.Error("Failed to listen (usage reports)", slogutil.Error(err)) return err } - slog.Info("Listening (usage reports)", "address", urListener.Addr()) + slog.Info("Listening (usage reports)", slogutil.Address(urListener.Addr())) internalListener, err := net.Listen("tcp", cli.ListenInternal) if err != nil { - slog.Error("Failed to listen (internal)", "error", err) + slog.Error("Failed to listen (internal)", slogutil.Error(err)) return err } - slog.Info("Listening (internal)", "address", internalListener.Addr()) + slog.Info("Listening (internal)", slogutil.Address(internalListener.Addr())) var geo *geoip.Provider if cli.GeoIPAccountID != 0 && cli.GeoIPLicenseKey != "" { geo, err = geoip.NewGeoLite2CityProvider(context.Background(), cli.GeoIPAccountID, cli.GeoIPLicenseKey, os.TempDir()) if err != nil { - slog.Error("Failed to load GeoIP", "error", err) + slog.Error("Failed to load GeoIP", slogutil.Error(err)) return err } go geo.Serve(context.TODO()) @@ -132,20 +133,20 @@ func (cli *CLI) Run() error { if cli.S3Endpoint != "" { blobs, err = s3.NewSession(cli.S3Endpoint, cli.S3Region, cli.S3Bucket, cli.S3AccessKeyID, cli.S3SecretKey) if err != nil { - slog.Error("Failed to create S3 session", "error", err) + slog.Error("Failed to create S3 session", slogutil.Error(err)) return err } } else if cli.AzureBlobAccount != "" { blobs, err = azureblob.NewBlobStore(cli.AzureBlobAccount, cli.AzureBlobKey, cli.AzureBlobContainer) if err != nil { - slog.Error("Failed to create Azure blob store", "error", err) + slog.Error("Failed to create Azure blob store", slogutil.Error(err)) return err } } if _, err := os.Stat(cli.DumpFile); err != nil && blobs != nil { if err := cli.downloadDumpFile(blobs); err != nil { - slog.Error("Failed to download dump file", "error", err) + slog.Error("Failed to download dump file", slogutil.Error(err)) } } @@ -167,7 +168,7 @@ func (cli *CLI) Run() error { go func() { for range time.Tick(cli.DumpInterval) { if err := cli.saveDumpFile(srv, blobs); err != nil { - slog.Error("Failed to write dump file", "error", err) + slog.Error("Failed to write dump file", slogutil.Error(err)) } } }() @@ -307,7 +308,7 @@ func (s *server) handleNewData(w http.ResponseWriter, r *http.Request) { lr := &io.LimitedReader{R: r.Body, N: 40 * 1024} bs, _ := io.ReadAll(lr) if err := json.Unmarshal(bs, &rep); err != nil { - log.Error("Failed to decode JSON", "error", err) + log.Error("Failed to decode JSON", slogutil.Error(err)) http.Error(w, "JSON Decode Error", http.StatusInternalServerError) return } @@ -317,7 +318,7 @@ func (s *server) handleNewData(w http.ResponseWriter, r *http.Request) { rep.Address = addr if err := rep.Validate(); err != nil { - log.Error("Failed to validate report", "error", err) + log.Error("Failed to validate report", slogutil.Error(err)) http.Error(w, "Validation Error", http.StatusInternalServerError) return } @@ -394,7 +395,7 @@ func (s *server) load(r io.Reader) { if err := dec.Decode(&rep); errors.Is(err, io.EOF) { break } else if err != nil { - slog.Error("Failed to load record", "error", err) + slog.Error("Failed to load record", slogutil.Error(err)) break } s.addReport(&rep) diff --git a/cmd/syncthing/blockprof.go b/cmd/syncthing/blockprof.go index 0dbee8893..c7ddc4082 100644 --- a/cmd/syncthing/blockprof.go +++ b/cmd/syncthing/blockprof.go @@ -8,11 +8,14 @@ package main import ( "fmt" + "log/slog" "os" "runtime" "runtime/pprof" "syscall" "time" + + "github.com/syncthing/syncthing/internal/slogutil" ) func startBlockProfiler() { @@ -20,10 +23,10 @@ func startBlockProfiler() { if profiler == nil { panic("Couldn't find block profiler") } - l.Debugln("Starting block profiling") + slog.Debug("Starting block profiling") go func() { err := saveBlockingProfiles(profiler) // Only returns on error - l.Warnln("Block profiler failed:", err) + slog.Error("Block profiler failed", slogutil.Error(err)) panic("Block profiler failed") }() } diff --git a/cmd/syncthing/cli/utils.go b/cmd/syncthing/cli/utils.go index fde611575..35f64f970 100644 --- a/cmd/syncthing/cli/utils.go +++ b/cmd/syncthing/cli/utils.go @@ -131,15 +131,6 @@ func prettyPrintResponse(response *http.Response) error { return prettyPrintJSON(data) } -func nulString(bs []byte) string { - for i := range bs { - if bs[i] == 0 { - return string(bs[:i]) - } - } - return string(bs) -} - func normalizePath(path string) string { return filepath.ToSlash(filepath.Clean(path)) } diff --git a/cmd/syncthing/crash_reporting.go b/cmd/syncthing/crash_reporting.go index 279ae11f7..645214235 100644 --- a/cmd/syncthing/crash_reporting.go +++ b/cmd/syncthing/crash_reporting.go @@ -11,12 +11,15 @@ import ( "context" "crypto/sha256" "fmt" + "log/slog" "net/http" "os" "path/filepath" "slices" "strings" "time" + + "github.com/syncthing/syncthing/internal/slogutil" ) const ( @@ -33,7 +36,7 @@ const ( func uploadPanicLogs(ctx context.Context, urlBase, dir string) { files, err := filepath.Glob(filepath.Join(dir, "panic-*.log")) if err != nil { - l.Warnln("Failed to list panic logs:", err) + slog.ErrorContext(ctx, "Failed to list panic logs", slogutil.Error(err)) return } @@ -48,7 +51,7 @@ func uploadPanicLogs(ctx context.Context, urlBase, dir string) { } if err := uploadPanicLog(ctx, urlBase, file); err != nil { - l.Warnln("Reporting crash:", err) + slog.ErrorContext(ctx, "Reporting crash", slogutil.Error(err)) } else { // Rename the log so we don't have to try to report it again. This // succeeds, or it does not. There is no point complaining about it. @@ -71,7 +74,7 @@ func uploadPanicLog(ctx context.Context, urlBase, file string) error { data = filterLogLines(data) hash := fmt.Sprintf("%x", sha256.Sum256(data)) - l.Infof("Reporting crash found in %s (report ID %s) ...\n", filepath.Base(file), hash[:8]) + slog.InfoContext(ctx, "Reporting crash", slogutil.FilePath(filepath.Base(file)), slog.String("id", hash[:8])) url := fmt.Sprintf("%s/%s", urlBase, hash) headReq, err := http.NewRequest(http.MethodHead, url, nil) diff --git a/cmd/syncthing/debug.go b/cmd/syncthing/debug.go index 0ba233416..c8db9e889 100644 --- a/cmd/syncthing/debug.go +++ b/cmd/syncthing/debug.go @@ -6,8 +6,6 @@ package main -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("main", "Main package") +func init() { slogutil.RegisterPackage("Main package") } diff --git a/cmd/syncthing/generate/generate.go b/cmd/syncthing/generate/generate.go index f9d9814f0..ea69ff132 100644 --- a/cmd/syncthing/generate/generate.go +++ b/cmd/syncthing/generate/generate.go @@ -18,9 +18,9 @@ import ( "github.com/syncthing/syncthing/lib/events" "github.com/syncthing/syncthing/lib/fs" "github.com/syncthing/syncthing/lib/locations" - "github.com/syncthing/syncthing/lib/logger" "github.com/syncthing/syncthing/lib/protocol" "github.com/syncthing/syncthing/lib/syncthing" + "golang.org/x/exp/slog" ) type CLI struct { @@ -29,7 +29,7 @@ type CLI struct { NoPortProbing bool `help:"Don't try to find free ports for GUI and listen addresses on first startup" env:"STNOPORTPROBING"` } -func (c *CLI) Run(l logger.Logger) error { +func (c *CLI) Run() error { // Support reading the password from a pipe or similar if c.GUIPassword == "-" { reader := bufio.NewReader(os.Stdin) @@ -40,13 +40,13 @@ func (c *CLI) Run(l logger.Logger) error { c.GUIPassword = string(password) } - if err := Generate(l, locations.GetBaseDir(locations.ConfigBaseDir), c.GUIUser, c.GUIPassword, c.NoPortProbing); err != nil { + if err := Generate(locations.GetBaseDir(locations.ConfigBaseDir), c.GUIUser, c.GUIPassword, c.NoPortProbing); err != nil { return fmt.Errorf("failed to generate config and keys: %w", err) } return nil } -func Generate(l logger.Logger, confDir, guiUser, guiPassword string, skipPortProbing bool) error { +func Generate(confDir, guiUser, guiPassword string, skipPortProbing bool) error { dir, err := fs.ExpandTilde(confDir) if err != nil { return err @@ -61,7 +61,7 @@ func Generate(l logger.Logger, confDir, guiUser, guiPassword string, skipPortPro certFile, keyFile := locations.Get(locations.CertFile), locations.Get(locations.KeyFile) cert, err := tls.LoadX509KeyPair(certFile, keyFile) if err == nil { - l.Warnln("Key exists; will not overwrite.") + slog.Warn("Key exists; will not overwrite.") } else { cert, err = syncthing.GenerateCertificate(certFile, keyFile) if err != nil { @@ -69,7 +69,7 @@ func Generate(l logger.Logger, confDir, guiUser, guiPassword string, skipPortPro } } myID = protocol.NewDeviceID(cert.Certificate[0]) - l.Infoln("Device ID:", myID) + slog.Info("Genereated new keypair", myID.LogAttr()) cfgFile := locations.Get(locations.ConfigFile) cfg, _, err := config.Load(cfgFile, myID, events.NoopLogger) @@ -87,7 +87,7 @@ func Generate(l logger.Logger, confDir, guiUser, guiPassword string, skipPortPro var updateErr error waiter, err := cfg.Modify(func(cfg *config.Configuration) { - updateErr = updateGUIAuthentication(l, &cfg.GUI, guiUser, guiPassword) + updateErr = updateGUIAuthentication(&cfg.GUI, guiUser, guiPassword) }) if err != nil { return fmt.Errorf("modify config: %w", err) @@ -103,17 +103,17 @@ func Generate(l logger.Logger, confDir, guiUser, guiPassword string, skipPortPro return nil } -func updateGUIAuthentication(l logger.Logger, guiCfg *config.GUIConfiguration, guiUser, guiPassword string) error { +func updateGUIAuthentication(guiCfg *config.GUIConfiguration, guiUser, guiPassword string) error { if guiUser != "" && guiCfg.User != guiUser { guiCfg.User = guiUser - l.Infoln("Updated GUI authentication user name:", guiUser) + slog.Info("Updated GUI authentication user", "name", guiUser) } if guiPassword != "" && guiCfg.Password != guiPassword { if err := guiCfg.SetPassword(guiPassword); err != nil { return fmt.Errorf("failed to set GUI authentication password: %w", err) } - l.Infoln("Updated GUI authentication password.") + slog.Info("Updated GUI authentication password") } return nil } diff --git a/cmd/syncthing/heapprof.go b/cmd/syncthing/heapprof.go index ca552d6a4..db23f1ae6 100644 --- a/cmd/syncthing/heapprof.go +++ b/cmd/syncthing/heapprof.go @@ -8,18 +8,21 @@ package main import ( "fmt" + "log/slog" "os" "runtime" "runtime/pprof" "syscall" "time" + + "github.com/syncthing/syncthing/internal/slogutil" ) func startHeapProfiler() { - l.Debugln("Starting heap profiling") + slog.Debug("Starting heap profiling") go func() { err := saveHeapProfiles(1) // Only returns on error - l.Warnln("Heap profiler failed:", err) + slog.Error("Heap profiler failed", slogutil.Error(err)) panic("Heap profiler failed") }() } diff --git a/cmd/syncthing/main.go b/cmd/syncthing/main.go index 455f53088..fa2dd1975 100644 --- a/cmd/syncthing/main.go +++ b/cmd/syncthing/main.go @@ -14,7 +14,8 @@ import ( "errors" "fmt" "io" - "log" + "log/slog" + "maps" "net/http" _ "net/http/pprof" // Need to import this to support STPROFILER. "net/url" @@ -40,6 +41,7 @@ import ( "github.com/syncthing/syncthing/cmd/syncthing/generate" "github.com/syncthing/syncthing/internal/db" "github.com/syncthing/syncthing/internal/db/sqlite" + "github.com/syncthing/syncthing/internal/slogutil" _ "github.com/syncthing/syncthing/lib/automaxprocs" "github.com/syncthing/syncthing/lib/build" "github.com/syncthing/syncthing/lib/config" @@ -47,7 +49,6 @@ import ( "github.com/syncthing/syncthing/lib/events" "github.com/syncthing/syncthing/lib/fs" "github.com/syncthing/syncthing/lib/locations" - "github.com/syncthing/syncthing/lib/logger" "github.com/syncthing/syncthing/lib/osutil" "github.com/syncthing/syncthing/lib/protocol" "github.com/syncthing/syncthing/lib/svcutil" @@ -61,20 +62,8 @@ const ( const ( extraUsage = ` -The --logflags value is a sum of the following: - - 1 Date - 2 Time - 4 Microsecond time - 8 Long filename - 16 Short filename - -I.e. to prefix each log line with time and filename, set --logflags=18 (2 + 16 -from above). The value 0 is used to disable all of the above. The default is -to show date and time (3). - Logging always happens to the command line (stdout) and optionally to the -file at the path specified by --logfile=path. In addition to an path, the special +file at the path specified by --log-file=path. In addition to an path, the special values "default" and "-" may be used. The former logs to DATADIR/syncthing.log (see --data), which is the default on Windows, and the latter only to stdout, no file, which is the default anywhere else. @@ -87,11 +76,10 @@ The following environment variables modify Syncthing's behavior in ways that are mostly useful for developers. Use with care. See also the --debug-* options above. - STTRACE A comma separated string of facilities to trace. The valid - facility strings are listed below. - - STLOCKTHRESHOLD Used for debugging internal deadlocks; sets debug - sensitivity. Use only under direction of a developer. + STTRACE A comma separated string of packages to trace or change log + level for. The valid package strings are listed below. A log + level (DEBUG, INFO, WARN or ERROR) can be added after each + package, separated by a colon. Ex: "model:WARN,nat:DEBUG". STVERSIONEXTRA Add extra information to the version string in logs and the version line in the GUI. Can be set to the name of a wrapper @@ -106,8 +94,8 @@ above. of CPU usage (i.e. performance). -Debugging Facilities --------------------- +Logging Facilities +------------------ The following are valid values for the STTRACE variable: @@ -170,8 +158,9 @@ type serveCmd struct { DBDeleteRetentionInterval time.Duration `help:"Database deleted item retention interval" default:"4320h" env:"STDBDELETERETENTIONINTERVAL"` GUIAddress string `name:"gui-address" help:"Override GUI address (e.g. \"http://192.0.2.42:8443\")" placeholder:"URL" env:"STGUIADDRESS"` GUIAPIKey string `name:"gui-apikey" help:"Override GUI API key" placeholder:"API-KEY" env:"STGUIAPIKEY"` - LogFile string `name:"logfile" help:"Log file name (see below)" default:"${logFile}" placeholder:"PATH" env:"STLOGFILE"` - LogFlags int `name:"logflags" help:"Select information in log line prefix (see below)" default:"${logFlags}" placeholder:"BITS" env:"STLOGFLAGS"` + LogFile string `name:"log-file" aliases:"logfile" help:"Log file name (see below)" default:"${logFile}" placeholder:"PATH" env:"STLOGFILE"` + LogFlags int `name:"logflags" help:"Deprecated option that does nothing, kept for compatibility" hidden:""` + LogLevel slog.Level `help:"Log level for all packages (DEBUG,INFO,WARN,ERROR)" env:"STLOGLEVEL" default:"INFO"` LogMaxFiles int `name:"log-max-old-files" help:"Number of old files to keep (zero to keep only current)" default:"${logMaxFiles}" placeholder:"N" env:"STLOGMAXOLDFILES"` LogMaxSize int `help:"Maximum size of any file (zero to disable log rotation)" default:"${logMaxSize}" placeholder:"BYTES" env:"STLOGMAXSIZE"` NoBrowser bool `help:"Do not start browser" env:"STNOBROWSER"` @@ -180,7 +169,6 @@ type serveCmd struct { NoUpgrade bool `help:"Disable automatic upgrades" env:"STNOUPGRADE"` Paused bool `help:"Start with all devices and folders paused" env:"STPAUSED"` Unpaused bool `help:"Start with all devices and folders unpaused" env:"STUNPAUSED"` - Verbose bool `help:"Print verbose log output" env:"STVERBOSE"` // Debug options below DebugGUIAssetsDir string `help:"Directory to load GUI assets from" placeholder:"PATH" env:"STGUIASSETS"` @@ -199,14 +187,9 @@ type serveCmd struct { func defaultVars() kong.Vars { vars := kong.Vars{} - vars["logFlags"] = strconv.Itoa(logger.DefaultFlags) vars["logMaxSize"] = strconv.Itoa(10 << 20) // 10 MiB vars["logMaxFiles"] = "3" // plus the current one - if os.Getenv("STTRACE") != "" { - vars["logFlags"] = strconv.Itoa(logger.DebugFlags) - } - // On non-Windows, we explicitly default to "-" which means stdout. On // Windows, the "default" options.logFile will later be replaced with the // default path, unless the user has manually specified "-" or @@ -234,13 +217,13 @@ func main() { defaultVars(), ) if err != nil { - log.Fatal(err) + slog.Error("Parsing startup", slogutil.Error(err)) + os.Exit(svcutil.ExitError.AsInt()) } kongplete.Complete(parser) ctx, err := parser.Parse(os.Args[1:]) parser.FatalIfErrorf(err) - ctx.BindTo(l, (*logger.Logger)(nil)) // main logger available to subcommands err = ctx.Run() parser.FatalIfErrorf(err) } @@ -252,15 +235,13 @@ func helpHandler(options kong.HelpOptions, ctx *kong.Context) error { if ctx.Command() == "serve" { // Help was requested for `syncthing serve`, so we add our extra // usage info afte the normal options output. - fmt.Printf(extraUsage, debugFacilities()) + fmt.Printf(extraUsage, logPackages()) } return nil } // serveCmd.Run() is the entrypoint for `syncthing serve` func (c *serveCmd) Run() error { - l.SetFlags(c.LogFlags) - if c.GUIAddress != "" { // The config picks this up from the environment. os.Setenv("STGUIADDRESS", c.GUIAddress) @@ -274,6 +255,9 @@ func (c *serveCmd) Run() error { osutil.HideConsole() } + // The default log level for all packages + slogutil.SetDefaultLevel(c.LogLevel) + // Treat an explicitly empty log file name as no log file if c.LogFile == "" { c.LogFile = "-" @@ -281,7 +265,7 @@ func (c *serveCmd) Run() error { if c.LogFile != "default" { // We must set this *after* expandLocations above. if err := locations.Set(locations.LogFile, c.LogFile); err != nil { - l.Warnln("Setting log file path:", err) + slog.Error("Failed to set log file path", slogutil.Error(err)) os.Exit(svcutil.ExitError.AsInt()) } } @@ -290,7 +274,7 @@ func (c *serveCmd) Run() error { // The asset dir is blank if STGUIASSETS wasn't set, in which case we // should look for extra assets in the default place. if err := locations.Set(locations.GUIAssets, c.DebugGUIAssetsDir); err != nil { - l.Warnln("Setting GUI assets path:", err) + slog.Error("Failed to set GUI assets path", slogutil.Error(err)) os.Exit(svcutil.ExitError.AsInt()) } } @@ -298,7 +282,7 @@ func (c *serveCmd) Run() error { // Ensure that our config and data directories exist. for _, loc := range []locations.BaseDirEnum{locations.ConfigBaseDir, locations.DataBaseDir} { if err := syncthing.EnsureDir(locations.GetBaseDir(loc), 0o700); err != nil { - l.Warnln("Failed to ensure directory exists:", err) + slog.Error("Failed to ensure directory exists", slogutil.Error(err)) os.Exit(svcutil.ExitError.AsInt()) } } @@ -321,29 +305,27 @@ func openGUI() error { return err } } else { - l.Warnln("Browser: GUI is currently disabled") + slog.Error("Browser: GUI is currently disabled") } return nil } -func debugFacilities() string { - facilities := l.Facilities() +func logPackages() string { + packages := slogutil.PackageDescrs() // Get a sorted list of names - var names []string + names := slices.Sorted(maps.Keys(packages)) maxLen := 0 - for name := range facilities { - names = append(names, name) + for _, name := range names { if len(name) > maxLen { maxLen = len(name) } } - slices.Sort(names) // Format the choices b := new(bytes.Buffer) for _, name := range names { - fmt.Fprintf(b, " %-*s - %s\n", maxLen, name, facilities[name]) + fmt.Fprintf(b, " %-*s - %s\n", maxLen, name, packages[name]) } return b.String() } @@ -371,7 +353,7 @@ func checkUpgrade() (upgrade.Release, error) { return upgrade.Release{}, &errNoUpgrade{build.Version, release.Tag} } - l.Infof("Upgrade available (current %q < latest %q)", build.Version, release.Tag) + slog.Info("Upgrade available", "current", build.Version, "latest", release.Tag) return release, nil } @@ -428,13 +410,9 @@ func (c *serveCmd) syncthingMain() { startPerfStats() } - // Set a log prefix similar to the ID we will have later on, or early log - // lines look ugly. - l.SetPrefix("[start] ") - // Print our version information up front, so any crash that happens // early etc. will have it available. - l.Infoln(build.LongVersion) + slog.Info(build.LongVersion) //nolint:sloglint // Ensure that we have a certificate and key. cert, err := syncthing.LoadOrGenerateCertificate( @@ -442,7 +420,7 @@ func (c *serveCmd) syncthingMain() { locations.Get(locations.KeyFile), ) if err != nil { - l.Warnln("Failed to load/generate certificate:", err) + slog.Error("Failed to load/generate certificate", slogutil.Error(err)) os.Exit(1) } @@ -450,10 +428,10 @@ func (c *serveCmd) syncthingMain() { lf := flock.New(locations.Get(locations.LockFile)) locked, err := lf.TryLock() if err != nil { - l.Warnln("Failed to acquire lock:", err) + slog.Error("Failed to acquire lock", slogutil.Error(err)) os.Exit(1) } else if !locked { - l.Warnln("Failed to acquire lock: is another Syncthing instance already running?") + slog.Error("Failed to acquire lock: is another Syncthing instance already running?") os.Exit(1) } @@ -462,7 +440,7 @@ func (c *serveCmd) syncthingMain() { // earlyService is a supervisor that runs the services needed for or // before app startup; the event logger, and the config service. - spec := svcutil.SpecWithDebugLogger(l) + spec := svcutil.SpecWithDebugLogger() earlyService := suture.New("early", spec) earlyService.ServeBackground(ctx) @@ -471,7 +449,7 @@ func (c *serveCmd) syncthingMain() { cfgWrapper, err := syncthing.LoadConfigAtStartup(locations.Get(locations.ConfigFile), cert, evLogger, c.AllowNewerConfig, c.NoPortProbing) if err != nil { - l.Warnln("Failed to initialize config:", err) + slog.Error("Failed to initialize config", slogutil.Error(err)) os.Exit(svcutil.ExitError.AsInt()) } earlyService.Add(cfgWrapper) @@ -483,7 +461,7 @@ func (c *serveCmd) syncthingMain() { if build.IsCandidate && !upgrade.DisabledByCompilation && !c.NoUpgrade { cfgWrapper.Modify(func(cfg *config.Configuration) { - l.Infoln("Automatic upgrade is always enabled for candidate releases.") + slog.Info("Automatic upgrade is always enabled for candidate releases") if cfg.Options.AutoUpgradeIntervalH == 0 || cfg.Options.AutoUpgradeIntervalH > 24 { cfg.Options.AutoUpgradeIntervalH = 12 // Set the option into the config as well, as the auto upgrade @@ -495,13 +473,13 @@ func (c *serveCmd) syncthingMain() { } if err := syncthing.TryMigrateDatabase(c.DBDeleteRetentionInterval); err != nil { - l.Warnln("Failed to migrate old-style database:", err) + slog.Error("Failed to migrate old-style database", slogutil.Error(err)) os.Exit(1) } sdb, err := syncthing.OpenDatabase(locations.Get(locations.Database), c.DBDeleteRetentionInterval) if err != nil { - l.Warnln("Error opening database:", err) + slog.Error("Error opening database", slogutil.Error(err)) os.Exit(1) } @@ -518,12 +496,12 @@ func (c *serveCmd) syncthingMain() { } if err != nil { if _, ok := err.(*errNoUpgrade); ok || err == errTooEarlyUpgradeCheck || err == errTooEarlyUpgrade { - l.Debugln("Initial automatic upgrade:", err) + slog.Debug("Initial automatic upgrade", slogutil.Error(err)) } else { - l.Infoln("Initial automatic upgrade:", err) + slog.Info("Initial automatic upgrade", slogutil.Error(err)) } } else { - l.Infof("Upgraded to %q, should exit now.", release.Tag) + slog.Info("Upgraded, should exit now", "newVersion", release.Tag) os.Exit(svcutil.ExitUpgrade.AsInt()) } } @@ -538,18 +516,17 @@ func (c *serveCmd) syncthingMain() { NoUpgrade: c.NoUpgrade, ProfilerAddr: c.DebugProfilerListen, ResetDeltaIdxs: c.DebugResetDeltaIdxs, - Verbose: c.Verbose, DBMaintenanceInterval: c.DBMaintenanceInterval, } if c.Audit || cfgWrapper.Options().AuditEnabled { - l.Infoln("Auditing is enabled.") + slog.Info("Auditing is enabled") auditFile := cfgWrapper.Options().AuditFile // Ignore config option if command-line option is set if c.AuditFile != "" { - l.Debugln("Using the audit file from the command-line parameter.") + slog.Debug("Using the audit file from the command-line parameter", slogutil.FilePath(c.AuditFile)) auditFile = c.AuditFile } @@ -558,7 +535,7 @@ func (c *serveCmd) syncthingMain() { app, err := syncthing.New(cfgWrapper, sdb, evLogger, cert, appOpts) if err != nil { - l.Warnln("Failed to start Syncthing:", err) + slog.Error("Failed to start Syncthing", slogutil.Error(err)) os.Exit(svcutil.ExitError.AsInt()) } @@ -571,11 +548,11 @@ func (c *serveCmd) syncthingMain() { if c.DebugProfileCPU { f, err := os.Create(fmt.Sprintf("cpu-%d.pprof", os.Getpid())) if err != nil { - l.Warnln("Creating profile:", err) + slog.Error("Failed to create profile", slogutil.Error(err)) os.Exit(svcutil.ExitError.AsInt()) } if err := pprof.StartCPUProfile(f); err != nil { - l.Warnln("Starting profile:", err) + slog.Error("Failed to start profile", slogutil.Error(err)) os.Exit(svcutil.ExitError.AsInt()) } } @@ -595,7 +572,7 @@ func (c *serveCmd) syncthingMain() { status := app.Wait() if status == svcutil.ExitError { - l.Warnln("Syncthing stopped with error:", app.Error()) + slog.Error("Syncthing stopped with error", slogutil.Error(app.Error())) } if c.DebugProfileCPU { @@ -663,13 +640,13 @@ func auditWriter(auditFile string) io.Writer { } fd, err = os.OpenFile(auditFile, auditFlags, 0o600) if err != nil { - l.Warnln("Audit:", err) + slog.Error("Failed to open audit file", slogutil.Error(err)) os.Exit(svcutil.ExitError.AsInt()) } auditDest = auditFile } - l.Infoln("Audit log in", auditDest) + slog.Info("Writing audit log", slogutil.FilePath(auditDest)) return fd } @@ -679,7 +656,7 @@ func (c *serveCmd) autoUpgradePossible() bool { return false } if c.NoUpgrade { - l.Infof("No automatic upgrades; STNOUPGRADE environment variable defined.") + slog.Info("No automatic upgrades; STNOUPGRADE environment variable defined") return false } return true @@ -696,7 +673,7 @@ func autoUpgrade(cfg config.Wrapper, app *syncthing.App, evLogger events.Logger) continue } if cfg.Options().AutoUpgradeEnabled() { - l.Infof("Connected to device %s with a newer version (current %q < remote %q). Checking for upgrades.", data["id"], build.Version, data["clientVersion"]) + slog.Info("Connected to device with a newer version; checking for upgrades", slog.String("device", data["id"]), slog.String("ourVersion", build.Version), slog.String("theirVersion", data["clientVersion"])) } case <-timer.C: } @@ -716,7 +693,7 @@ func autoUpgrade(cfg config.Wrapper, app *syncthing.App, evLogger events.Logger) if err != nil { // Don't complain too loudly here; we might simply not have // internet connectivity, or the upgrade server might be down. - l.Infoln("Automatic upgrade:", err) + slog.Info("Automatic upgrade", slogutil.Error(err)) timer.Reset(checkInterval) continue } @@ -727,15 +704,15 @@ func autoUpgrade(cfg config.Wrapper, app *syncthing.App, evLogger events.Logger) continue } - l.Infof("Automatic upgrade (current %q < latest %q)", build.Version, rel.Tag) + slog.Info("Automatic upgrade", "current", build.Version, "latest", rel.Tag) err = upgrade.To(rel) if err != nil { - l.Warnln("Automatic upgrade:", err) + slog.Error("Automatic upgrade failed", slogutil.Error(err)) timer.Reset(checkInterval) continue } sub.Unsubscribe() - l.Warnf("Automatically upgraded to version %q. Restarting in 1 minute.", rel.Tag) + slog.Error("Automatically upgraded, restarting in 1 minute", slog.String("newVersion", rel.Tag)) time.Sleep(time.Minute) app.Stop(svcutil.ExitUpgrade) return @@ -788,22 +765,22 @@ func cleanConfigDirectory() { fs := fs.NewFilesystem(fs.FilesystemTypeBasic, locations.GetBaseDir(locations.ConfigBaseDir)) files, err := fs.Glob(pat) if err != nil { - l.Infoln("Cleaning:", err) + slog.Warn("Failed to clean config directory", slogutil.Error(err)) continue } for _, file := range files { info, err := fs.Lstat(file) if err != nil { - l.Infoln("Cleaning:", err) + slog.Warn("Failed to clean config directory", slogutil.Error(err)) continue } if time.Since(info.ModTime()) > dur { if err = fs.RemoveAll(file); err != nil { - l.Infoln("Cleaning:", err) + slog.Warn("Failed to clean config directory", slogutil.Error(err)) } else { - l.Infoln("Cleaned away old file", filepath.Base(file)) + slog.Warn("Cleaned away old file", slogutil.FilePath(filepath.Base(file))) } } } @@ -820,7 +797,7 @@ func setPauseState(cfgWrapper config.Wrapper, paused bool) { } }) if err != nil { - l.Warnln("Cannot adjust paused state:", err) + slog.Error("Cannot adjust paused state", slogutil.Error(err)) os.Exit(svcutil.ExitError.AsInt()) } } @@ -847,7 +824,7 @@ func (deviceIDCmd) Run() error { locations.Get(locations.KeyFile), ) if err != nil { - l.Warnln("Error reading device ID:", err) + slog.Error("Failed to read device ID", slogutil.Error(err)) os.Exit(svcutil.ExitError.AsInt()) } @@ -870,7 +847,7 @@ type upgradeCmd struct { func (u upgradeCmd) Run() error { if u.CheckOnly { if _, err := checkUpgrade(); err != nil { - l.Warnln("Checking for upgrade:", err) + slog.Error("Failed to check for upgrade", slogutil.Error(err)) os.Exit(exitCodeForUpgrade(err)) } return nil @@ -879,10 +856,10 @@ func (u upgradeCmd) Run() error { if u.From != "" { err := upgrade.ToURL(u.From) if err != nil { - l.Warnln("Error while Upgrading:", err) + slog.Error("Failed to upgrade", slogutil.Error(err)) os.Exit(svcutil.ExitError.AsInt()) } - l.Infoln("Upgraded from", u.From) + slog.Info("Upgraded", "from", u.From) return nil } @@ -892,7 +869,7 @@ func (u upgradeCmd) Run() error { var locked bool locked, err = lf.TryLock() if err != nil { - l.Warnln("Upgrade:", err) + slog.Error("Failed to lock for upgrade", slogutil.Error(err)) os.Exit(1) } else if locked { err = upgradeViaRest() @@ -901,10 +878,10 @@ func (u upgradeCmd) Run() error { } } if err != nil { - l.Warnln("Upgrade:", err) + slog.Error("Failed to check for upgrade", slogutil.Error(err)) os.Exit(exitCodeForUpgrade(err)) } - l.Infof("Upgraded to %q", release.Tag) + slog.Info("Upgraded", "to", release.Tag) os.Exit(svcutil.ExitUpgrade.AsInt()) return nil } @@ -913,7 +890,7 @@ type browserCmd struct{} func (browserCmd) Run() error { if err := openGUI(); err != nil { - l.Warnln("Failed to open web UI:", err) + slog.Error("Failed to open web UI", slogutil.Error(err)) os.Exit(svcutil.ExitError.AsInt()) } return nil @@ -929,12 +906,12 @@ type debugCmd struct { type resetDatabaseCmd struct{} func (resetDatabaseCmd) Run() error { - l.Infoln("Removing database in", locations.Get(locations.Database)) + slog.Info("Removing database", slogutil.FilePath(locations.Get(locations.Database))) if err := os.RemoveAll(locations.Get(locations.Database)); err != nil { - l.Warnln("Resetting database:", err) + slog.Error("Failed to reset database", slogutil.Error(err)) os.Exit(svcutil.ExitError.AsInt()) } - l.Infoln("Successfully reset database - it will be rebuilt after next start.") + slog.Info("Reset database - it will be rebuilt after next start") return nil } diff --git a/cmd/syncthing/monitor.go b/cmd/syncthing/monitor.go index 4d52dfe8d..5fa22b52d 100644 --- a/cmd/syncthing/monitor.go +++ b/cmd/syncthing/monitor.go @@ -11,26 +11,28 @@ import ( "context" "fmt" "io" + "log/slog" "os" "os/exec" "os/signal" "path/filepath" "strings" + "sync" "syscall" "time" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/build" "github.com/syncthing/syncthing/lib/fs" "github.com/syncthing/syncthing/lib/locations" "github.com/syncthing/syncthing/lib/osutil" "github.com/syncthing/syncthing/lib/svcutil" - "github.com/syncthing/syncthing/lib/sync" ) var ( stdoutFirstLines []string // The first 10 lines of stdout stdoutLastLines []string // The last 50 lines of stdout - stdoutMut = sync.NewMutex() + stdoutMut sync.Mutex ) const ( @@ -44,8 +46,6 @@ const ( ) func (c *serveCmd) monitorMain() { - l.SetPrefix("[monitor] ") - var dst io.Writer = os.Stdout logFile := locations.Get(locations.LogFile) @@ -64,7 +64,7 @@ func (c *serveCmd) monitorMain() { fileDst, err = open(logFile) } if err != nil { - l.Warnln("Failed to set up logging to file, proceeding with logging to stdout only:", err) + slog.Error("Failed to set up logging to file, proceeding with logging to stdout only", slogutil.Error(err)) } else { if build.IsWindows { // Translate line breaks to Windows standard @@ -78,14 +78,14 @@ func (c *serveCmd) monitorMain() { // Log to both stdout and file. dst = io.MultiWriter(dst, fileDst) - l.Infof(`Log output saved to file "%s"`, logFile) + slog.Info("Saved log output", slogutil.FilePath(logFile)) } } args := os.Args binary, err := getBinary(args[0]) if err != nil { - l.Warnln("Error starting the main Syncthing process:", err) + slog.Error("Failed to start the main Syncthing process", slogutil.Error(err)) panic("Error starting the main Syncthing process") } var restarts [restartCounts]time.Time @@ -102,7 +102,7 @@ func (c *serveCmd) monitorMain() { maybeReportPanics() if t := time.Since(restarts[0]); t < restartLoopThreshold { - l.Warnf("%d restarts in %v; not retrying further", restartCounts, t) + slog.Error("Too many restarts; not retrying further", slog.Int("count", restartCounts), slog.Any("interval", t)) os.Exit(svcutil.ExitError.AsInt()) } @@ -122,10 +122,10 @@ func (c *serveCmd) monitorMain() { panic(err) } - l.Debugln("Starting syncthing") + slog.Debug("Starting syncthing") err = cmd.Start() if err != nil { - l.Warnln("Error starting the main Syncthing process:", err) + slog.Error("Failed to start the main Syncthing process", slogutil.Error(err)) panic("Error starting the main Syncthing process") } @@ -134,7 +134,7 @@ func (c *serveCmd) monitorMain() { stdoutLastLines = make([]string, 0, 50) stdoutMut.Unlock() - wg := sync.NewWaitGroup() + var wg sync.WaitGroup wg.Add(1) go func() { @@ -158,13 +158,13 @@ func (c *serveCmd) monitorMain() { stopped := false select { case s := <-stopSign: - l.Infof("Signal %d received; exiting", s) + slog.Info("Received signal; exiting", "signal", s) cmd.Process.Signal(sigTerm) err = <-exit stopped = true case s := <-restartSign: - l.Infof("Signal %d received; restarting", s) + slog.Info("Received signal; restarting", "signal", s) cmd.Process.Signal(sigHup) err = <-exit @@ -184,9 +184,9 @@ func (c *serveCmd) monitorMain() { if exitCode == svcutil.ExitUpgrade.AsInt() { // Restart the monitor process to release the .old // binary as part of the upgrade process. - l.Infoln("Restarting monitor...") + slog.Info("Restarting monitor...") if err = restartMonitor(binary, args); err != nil { - l.Warnln("Restart:", err) + slog.Error("Failed to restart monitor", slogutil.Error(err)) } os.Exit(exitCode) } @@ -196,7 +196,7 @@ func (c *serveCmd) monitorMain() { os.Exit(svcutil.ExitError.AsInt()) } - l.Infoln("Syncthing exited:", err) + slog.Info("Syncthing exited", slogutil.Error(err)) time.Sleep(restartPause) if first { @@ -243,29 +243,13 @@ func copyStderr(stderr io.Reader, dst io.Writer) { if panicFd == nil && (strings.HasPrefix(line, "panic:") || strings.HasPrefix(line, "fatal error:")) { panicFd, err = os.Create(locations.GetTimestamped(locations.PanicLog)) if err != nil { - l.Warnln("Create panic log:", err) + slog.Error("Failed to create panic log", slogutil.Error(err)) continue } - l.Warnf("Panic detected, writing to \"%s\"", panicFd.Name()) - if strings.Contains(line, "leveldb") && strings.Contains(line, "corrupt") { - l.Warnln(` -********************************************************************************* -* Crash due to corrupt database. * -* * -* This crash usually occurs due to one of the following reasons: * -* - Syncthing being stopped abruptly (killed/loss of power) * -* - Bad hardware (memory/disk issues) * -* - Software that affects disk writes (SSD caching software and similar) * -* * -* Please see the following URL for instructions on how to recover: * -* https://docs.syncthing.net/users/faq.html#my-syncthing-database-is-corrupt * -********************************************************************************* -`) - } else { - l.Warnln("Please check for existing issues with similar panic message at https://github.com/syncthing/syncthing/issues/") - l.Warnln("If no issue with similar panic message exists, please create a new issue with the panic log attached") - } + slog.Error("Panic detected, writing to file", slogutil.FilePath(panicFd.Name())) + slog.Info("Please check for existing issues with similar panic message at https://github.com/syncthing/syncthing/issues/") + slog.Info("If no issue with similar panic message exists, please create a new issue with the panic log attached") stdoutMut.Lock() for _, line := range stdoutFirstLines { @@ -446,7 +430,6 @@ func newAutoclosedFile(name string, closeDelay, maxOpenTime time.Duration) (*aut name: name, closeDelay: closeDelay, maxOpenTime: maxOpenTime, - mut: sync.NewMutex(), closed: make(chan struct{}), closeTimer: time.NewTimer(time.Minute), } @@ -554,7 +537,7 @@ func maybeReportPanics() { // Try to get a config to see if/where panics should be reported. cfg, err := loadOrDefaultConfig() if err != nil { - l.Warnln("Couldn't load config; not reporting crash") + slog.Error("Couldn't load config; not reporting crash") return } @@ -574,7 +557,7 @@ func maybeReportPanics() { case <-ctx.Done(): return case <-time.After(panicUploadNoticeWait): - l.Warnln("Uploading crash reports is taking a while, please wait...") + slog.Warn("Uploading crash reports is taking a while, please wait") } }() diff --git a/etc/linux-systemd/system/syncthing@.service b/etc/linux-systemd/system/syncthing@.service index bbdf0d1f5..4e5f42cd8 100644 --- a/etc/linux-systemd/system/syncthing@.service +++ b/etc/linux-systemd/system/syncthing@.service @@ -7,7 +7,7 @@ StartLimitBurst=4 [Service] User=%i -ExecStart=/usr/bin/syncthing serve --no-browser --no-restart --logflags=0 +ExecStart=/usr/bin/syncthing serve --no-browser --no-restart Restart=on-failure RestartSec=1 SuccessExitStatus=3 4 diff --git a/gui/default/syncthing/core/logViewerModalView.html b/gui/default/syncthing/core/logViewerModalView.html index 184a0ecc8..e711910eb 100644 --- a/gui/default/syncthing/core/logViewerModalView.html +++ b/gui/default/syncthing/core/logViewerModalView.html @@ -16,11 +16,16 @@ - - + + -
- {{ name }} +
{{ logging.facilities.packages[key] }} ({{ key }}) + {{ data.description }}
diff --git a/gui/default/syncthing/core/syncthingController.js b/gui/default/syncthing/core/syncthingController.js index 95a1aa190..0ae7102af 100644 --- a/gui/default/syncthing/core/syncthingController.js +++ b/gui/default/syncthing/core/syncthingController.js @@ -1568,16 +1568,8 @@ angular.module('syncthing.core') $scope.logging = { facilities: {}, refreshFacilities: function () { - $http.get(urlbase + '/system/debug').success(function (data) { - var facilities = {}; - data.enabled = data.enabled || []; - $.each(data.facilities, function (key, value) { - facilities[key] = { - description: value, - enabled: data.enabled.indexOf(key) > -1 - } - }) - $scope.logging.facilities = facilities; + $http.get(urlbase + '/system/loglevels').success(function (data) { + $scope.logging.facilities = data; }).error($scope.emitHTTPError); }, show: function () { @@ -1597,13 +1589,10 @@ angular.module('syncthing.core') }); showModal('#logViewer'); }, - onFacilityChange: function (facility) { - var enabled = $scope.logging.facilities[facility].enabled; - // Disable checkboxes while we're in flight. - $.each($scope.logging.facilities, function (key) { - $scope.logging.facilities[key].enabled = null; - }) - $http.post(urlbase + '/system/debug?' + (enabled ? 'enable=' : 'disable=') + facility) + onFacilityChange: function () { + // Disable editing while we're in flight. + $scope.logging.facilities.updating = true; + $http.post(urlbase + '/system/loglevels', $scope.logging.facilities.levels) .success($scope.logging.refreshFacilities) .error($scope.emitHTTPError); }, @@ -1626,7 +1615,7 @@ angular.module('syncthing.core') content: function () { var content = ""; $.each($scope.logging.entries, function (idx, entry) { - content += entry.when.split('.')[0].replace('T', ' ') + ' ' + entry.message + "\n"; + content += entry.when.split('.')[0].replace('T', ' ') + ' ' + entry.level + ' ' + entry.message + "\n"; }); return content; }, diff --git a/internal/db/olddb/smallindex.go b/internal/db/olddb/smallindex.go index 6950f8f2f..c4cdbce9b 100644 --- a/internal/db/olddb/smallindex.go +++ b/internal/db/olddb/smallindex.go @@ -9,9 +9,9 @@ package olddb import ( "encoding/binary" "slices" + "sync" "github.com/syncthing/syncthing/internal/db/olddb/backend" - "github.com/syncthing/syncthing/lib/sync" ) // A smallIndex is an in memory bidirectional []byte to uint32 map. It gives @@ -32,7 +32,6 @@ func newSmallIndex(db backend.Backend, prefix []byte) *smallIndex { prefix: prefix, id2val: make(map[uint32]string), val2id: make(map[string]uint32), - mut: sync.NewMutex(), } idx.load() return idx diff --git a/internal/db/sqlite/db_folderdb.go b/internal/db/sqlite/db_folderdb.go index 5daa67bf0..2431f7576 100644 --- a/internal/db/sqlite/db_folderdb.go +++ b/internal/db/sqlite/db_folderdb.go @@ -12,6 +12,7 @@ import ( "fmt" "io" "iter" + "log/slog" "path/filepath" "strings" "time" @@ -78,7 +79,7 @@ func (s *DB) getFolderDB(folder string, create bool) (*folderDB, error) { } } - l.Debugf("Folder %s in database %s", folder, dbName) + slog.Debug("Folder database opened", "folder", folder, "db", dbName) path := dbName if !filepath.IsAbs(path) { path = filepath.Join(s.pathBase, dbName) diff --git a/internal/db/sqlite/db_open.go b/internal/db/sqlite/db_open.go index 072755f3c..927135dc4 100644 --- a/internal/db/sqlite/db_open.go +++ b/internal/db/sqlite/db_open.go @@ -7,12 +7,14 @@ package sqlite import ( + "log/slog" "os" "path/filepath" "sync" "time" "github.com/syncthing/syncthing/internal/db" + "github.com/syncthing/syncthing/internal/slogutil" ) const maxDBConns = 16 @@ -128,7 +130,7 @@ func OpenTemp() (*DB, error) { return nil, wrap(err) } path := filepath.Join(dir, "db") - l.Debugln("Test DB in", path) + slog.Debug("Test DB", slogutil.FilePath(path)) return Open(path) } diff --git a/internal/db/sqlite/db_service.go b/internal/db/sqlite/db_service.go index 38695785e..de2e68ddf 100644 --- a/internal/db/sqlite/db_service.go +++ b/internal/db/sqlite/db_service.go @@ -9,10 +9,12 @@ package sqlite import ( "context" "fmt" + "log/slog" "time" "github.com/jmoiron/sqlx" "github.com/syncthing/syncthing/internal/db" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/thejerf/suture/v4" ) @@ -56,7 +58,7 @@ func (s *Service) Serve(ctx context.Context) error { if wait < 0 { wait = time.Minute } - l.Debugln("Next periodic run in", wait) + slog.DebugContext(ctx, "Next periodic run due", "after", wait) timer := time.NewTimer(wait) for { @@ -71,17 +73,17 @@ func (s *Service) Serve(ctx context.Context) error { } timer.Reset(s.maintenanceInterval) - l.Debugln("Next periodic run in", s.maintenanceInterval) + slog.DebugContext(ctx, "Next periodic run due", "after", s.maintenanceInterval) _ = s.internalMeta.PutTime(lastMaintKey, time.Now()) } } func (s *Service) periodic(ctx context.Context) error { t0 := time.Now() - l.Debugln("Periodic start") + slog.DebugContext(ctx, "Periodic start") t1 := time.Now() - defer func() { l.Debugln("Periodic done in", time.Since(t1), "+", t1.Sub(t0)) }() + defer func() { slog.DebugContext(ctx, "Periodic done in", "t1", time.Since(t1), "t0t1", t1.Sub(t0)) }() s.sdb.updateLock.Lock() err := tidy(ctx, s.sdb.sql) @@ -94,7 +96,7 @@ func (s *Service) periodic(ctx context.Context) error { fdb.updateLock.Lock() defer fdb.updateLock.Unlock() - if err := garbageCollectOldDeletedLocked(fdb); err != nil { + if err := garbageCollectOldDeletedLocked(ctx, fdb); err != nil { return wrap(err) } if err := garbageCollectBlocklistsAndBlocksLocked(ctx, fdb); err != nil { @@ -118,15 +120,16 @@ func tidy(ctx context.Context, db *sqlx.DB) error { return nil } -func garbageCollectOldDeletedLocked(fdb *folderDB) error { +func garbageCollectOldDeletedLocked(ctx context.Context, fdb *folderDB) error { + l := slog.With("fdb", fdb.baseDB) if fdb.deleteRetention <= 0 { - l.Debugln(fdb.baseName, "delete retention is infinite, skipping cleanup") + slog.DebugContext(ctx, "Delete retention is infinite, skipping cleanup") return nil } // Remove deleted files that are marked as not needed (we have processed // them) and they were deleted more than MaxDeletedFileAge ago. - l.Debugln(fdb.baseName, "forgetting deleted files older than", fdb.deleteRetention) + l.DebugContext(ctx, "Forgetting deleted files", "retention", fdb.deleteRetention) res, err := fdb.stmt(` DELETE FROM files WHERE deleted AND modified < ? AND local_flags & {{.FlagLocalNeeded}} == 0 @@ -135,7 +138,7 @@ func garbageCollectOldDeletedLocked(fdb *folderDB) error { return wrap(err) } if aff, err := res.RowsAffected(); err == nil { - l.Debugln(fdb.baseName, "removed old deleted file records:", aff) + l.DebugContext(ctx, "Removed old deleted file records", "affected", aff) } return nil } @@ -176,9 +179,14 @@ func garbageCollectBlocklistsAndBlocksLocked(ctx context.Context, fdb *folderDB) SELECT 1 FROM files WHERE files.blocklist_hash = blocklists.blocklist_hash )`); err != nil { return wrap(err, "delete blocklists") - } else if shouldDebug() { - rows, err := res.RowsAffected() - l.Debugln(fdb.baseName, "blocklist GC:", rows, err) + } else { + slog.DebugContext(ctx, "Blocklist GC", "fdb", fdb.baseName, "result", slogutil.Expensive(func() any { + rows, err := res.RowsAffected() + if err != nil { + return slogutil.Error(err) + } + return slog.Int64("rows", rows) + })) } if res, err := tx.ExecContext(ctx, ` @@ -187,9 +195,14 @@ func garbageCollectBlocklistsAndBlocksLocked(ctx context.Context, fdb *folderDB) SELECT 1 FROM blocklists WHERE blocklists.blocklist_hash = blocks.blocklist_hash )`); err != nil { return wrap(err, "delete blocks") - } else if shouldDebug() { - rows, err := res.RowsAffected() - l.Debugln(fdb.baseName, "blocks GC:", rows, err) + } else { + slog.DebugContext(ctx, "Blocks GC", "fdb", fdb.baseName, "result", slogutil.Expensive(func() any { + rows, err := res.RowsAffected() + if err != nil { + return slogutil.Error(err) + } + return slog.Int64("rows", rows) + })) } return wrap(tx.Commit()) diff --git a/internal/db/sqlite/debug.go b/internal/db/sqlite/debug.go index acc711a2e..11f3c62bd 100644 --- a/internal/db/sqlite/debug.go +++ b/internal/db/sqlite/debug.go @@ -6,10 +6,6 @@ package sqlite -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("sqlite", "SQLite database") - -func shouldDebug() bool { return l.ShouldDebug("sqlite") } +func init() { slogutil.RegisterPackage("SQLite database") } diff --git a/internal/db/sqlite/folderdb_update.go b/internal/db/sqlite/folderdb_update.go index a02484a1f..76f54d1d3 100644 --- a/internal/db/sqlite/folderdb_update.go +++ b/internal/db/sqlite/folderdb_update.go @@ -10,11 +10,13 @@ import ( "cmp" "context" "fmt" + "log/slog" "slices" "github.com/jmoiron/sqlx" "github.com/syncthing/syncthing/internal/gen/dbproto" "github.com/syncthing/syncthing/internal/itererr" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/osutil" "github.com/syncthing/syncthing/lib/protocol" "github.com/syncthing/syncthing/lib/sliceutil" @@ -486,12 +488,12 @@ func (s *folderDB) periodicCheckpointLocked(fs []protocol.FileInfo) { if s.updatePoints > updatePointsThreshold { conn, err := s.sql.Conn(context.Background()) if err != nil { - l.Debugln(s.baseName, "conn:", err) + slog.Debug("Connection error", slog.String("db", s.baseName), slogutil.Error(err)) return } defer conn.Close() if _, err := conn.ExecContext(context.Background(), `PRAGMA journal_size_limit = 8388608`); err != nil { - l.Debugln(s.baseName, "PRAGMA journal_size_limit:", err) + slog.Debug("PRAGMA journal_size_limit error", slog.String("db", s.baseName), slogutil.Error(err)) } // Every 50th checkpoint becomes a truncate, in an effort to bring @@ -505,11 +507,11 @@ func (s *folderDB) periodicCheckpointLocked(fs []protocol.FileInfo) { var res, modified, moved int if row.Err() != nil { - l.Debugln(s.baseName, cmd+":", err) + slog.Debug("Command error", slog.String("db", s.baseName), slog.String("cmd", cmd), slogutil.Error(err)) } else if err := row.Scan(&res, &modified, &moved); err != nil { - l.Debugln(s.baseName, cmd+" (scan):", err) + slog.Debug("Command scan error", slog.String("db", s.baseName), slog.String("cmd", cmd), slogutil.Error(err)) } else { - l.Debugln(s.baseName, cmd, s.checkpointsCount, "at", s.updatePoints, "returned", res, modified, moved) + slog.Debug("Checkpoint result", "db", s.baseName, "checkpointscount", s.checkpointsCount, "updatepoints", s.updatePoints, "res", res, "modified", modified, "moved", moved) } // Reset the truncate counter when a truncate succeeded. If it diff --git a/internal/slogutil/expensive.go b/internal/slogutil/expensive.go new file mode 100644 index 000000000..881c6383d --- /dev/null +++ b/internal/slogutil/expensive.go @@ -0,0 +1,25 @@ +// Copyright (C) 2025 The Syncthing Authors. +// +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this file, +// You can obtain one at https://mozilla.org/MPL/2.0/. + +package slogutil + +import ( + "log/slog" +) + +// Expensive wraps a log value that is expensive to compute and should only +// be called if the log line is actually emitted. +func Expensive(fn func() any) expensive { + return expensive{fn} +} + +type expensive struct { + fn func() any +} + +func (e expensive) LogValue() slog.Value { + return slog.AnyValue(e.fn()) +} diff --git a/internal/slogutil/formatting.go b/internal/slogutil/formatting.go new file mode 100644 index 000000000..dbf3d90d2 --- /dev/null +++ b/internal/slogutil/formatting.go @@ -0,0 +1,187 @@ +// Copyright (C) 2025 The Syncthing Authors. +// +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this file, +// You can obtain one at https://mozilla.org/MPL/2.0/. + +package slogutil + +import ( + "cmp" + "context" + "io" + "log/slog" + "path" + "runtime" + "strconv" + "strings" + "time" +) + +type formattingHandler struct { + attrs []slog.Attr + groups []string + out io.Writer + recs []*lineRecorder + timeOverride time.Time +} + +var _ slog.Handler = (*formattingHandler)(nil) + +func (h *formattingHandler) Enabled(context.Context, slog.Level) bool { + return true +} + +func (h *formattingHandler) Handle(_ context.Context, rec slog.Record) error { + fr := runtime.CallersFrames([]uintptr{rec.PC}) + var logAttrs []any + if fram, _ := fr.Next(); fram.Function != "" { + pkgName, typeName := funcNameToPkg(fram.Function) + lvl := globalLevels.Get(pkgName) + if lvl > rec.Level { + // Logging not enabled at the record's level + return nil + } + logAttrs = append(logAttrs, slog.String("pkg", pkgName)) + if lvl <= slog.LevelDebug { + // We are debugging, add additional source line data + if typeName != "" { + logAttrs = append(logAttrs, slog.String("type", typeName)) + } + logAttrs = append(logAttrs, slog.Group("src", slog.String("file", path.Base(fram.File)), slog.Int("line", fram.Line))) + } + } + + var prefix string + if len(h.groups) > 0 { + prefix = strings.Join(h.groups, ".") + "." + } + + // Build the message string. + var sb strings.Builder + sb.WriteString(rec.Message) + + // Collect all the attributes, adding the handler prefix. + attrs := make([]slog.Attr, 0, rec.NumAttrs()+len(h.attrs)+1) + rec.Attrs(func(attr slog.Attr) bool { + attr.Key = prefix + attr.Key + attrs = append(attrs, attr) + return true + }) + attrs = append(attrs, h.attrs...) + attrs = append(attrs, slog.Group("log", logAttrs...)) + + // Expand and format attributes + var attrCount int + for _, attr := range attrs { + for _, attr := range expandAttrs("", attr) { + appendAttr(&sb, "", attr, &attrCount) + } + } + if attrCount > 0 { + sb.WriteRune(')') + } + + line := Line{ + When: cmp.Or(h.timeOverride, rec.Time), + Message: sb.String(), + Level: rec.Level, + } + + // If there is a recorder, record the line. + for _, rec := range h.recs { + rec.record(line) + } + + // If there's an output, print the line. + if h.out != nil { + _, _ = line.WriteTo(h.out) + } + return nil +} + +func expandAttrs(prefix string, a slog.Attr) []slog.Attr { + if prefix != "" { + a.Key = prefix + "." + a.Key + } + val := a.Value.Resolve() + if val.Kind() != slog.KindGroup { + return []slog.Attr{a} + } + var attrs []slog.Attr + for _, attr := range val.Group() { + attrs = append(attrs, expandAttrs(a.Key, attr)...) + } + return attrs +} + +func appendAttr(sb *strings.Builder, prefix string, a slog.Attr, attrCount *int) { + if a.Key == "" { + return + } + sb.WriteRune(' ') + if *attrCount == 0 { + sb.WriteRune('(') + } + sb.WriteString(prefix) + sb.WriteString(a.Key) + sb.WriteRune('=') + v := a.Value.Resolve().String() + if strings.ContainsAny(v, ` "`) { + v = strconv.Quote(v) + } + sb.WriteString(v) + *attrCount++ +} + +func (h *formattingHandler) WithAttrs(attrs []slog.Attr) slog.Handler { + if len(h.groups) > 0 { + prefix := strings.Join(h.groups, ".") + "." + for i := range attrs { + attrs[i].Key = prefix + attrs[i].Key + } + } + return &formattingHandler{ + attrs: append(h.attrs, attrs...), + groups: h.groups, + recs: h.recs, + out: h.out, + timeOverride: h.timeOverride, + } +} + +func (h *formattingHandler) WithGroup(name string) slog.Handler { + if name == "" { + return h + } + return &formattingHandler{ + attrs: h.attrs, + groups: append([]string{name}, h.groups...), + recs: h.recs, + out: h.out, + timeOverride: h.timeOverride, + } +} + +func funcNameToPkg(fn string) (string, string) { + fn = strings.ToLower(fn) + fn = strings.TrimPrefix(fn, "github.com/syncthing/syncthing/lib/") + fn = strings.TrimPrefix(fn, "github.com/syncthing/syncthing/internal/") + + pkgTypFn := strings.Split(fn, ".") // [package, type, method] or [package, function] + if len(pkgTypFn) <= 2 { + return pkgTypFn[0], "" + } + + pkg := pkgTypFn[0] + // Remove parenthesis and asterisk from the type name + typ := strings.TrimLeft(strings.TrimRight(pkgTypFn[1], ")"), "(*") + // Skip certain type names that add no value + typ = strings.TrimSuffix(typ, "service") + switch typ { + case pkg, "", "serveparams": + return pkg, "" + default: + return pkg, typ + } +} diff --git a/internal/slogutil/formatting_test.go b/internal/slogutil/formatting_test.go new file mode 100644 index 000000000..a3756304f --- /dev/null +++ b/internal/slogutil/formatting_test.go @@ -0,0 +1,51 @@ +// Copyright (C) 2025 The Syncthing Authors. +// +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this file, +// You can obtain one at https://mozilla.org/MPL/2.0/. + +package slogutil + +import ( + "bytes" + "log/slog" + "strings" + "testing" + "time" +) + +func TestFormattingHandler(t *testing.T) { + buf := new(bytes.Buffer) + h := &formattingHandler{ + out: buf, + timeOverride: time.Unix(1234567890, 0).In(time.UTC), + } + + l := slog.New(h).With("a", "a") + l.Info("A basic info line", "attr1", "val with spaces", "attr2", 2, "attr3", `val"quote`) + l.Info("An info line with grouped values", "attr1", "val1", slog.Group("foo", "attr2", 2, slog.Group("bar", "attr3", "3"))) + + l2 := l.WithGroup("foo") + l2.Info("An info line with grouped values via logger", "attr1", "val1", "attr2", 2) + + l3 := l2.WithGroup("bar") + l3.Info("An info line with nested grouped values via logger", "attr1", "val1", "attr2", 2) + + l3.Debug("A debug entry") + l3.Warn("A warning entry") + l3.Error("An error") + + exp := ` +2009-02-13 23:31:30 INF A basic info line (attr1="val with spaces" attr2=2 attr3="val\"quote" a=a log.pkg=slogutil) +2009-02-13 23:31:30 INF An info line with grouped values (attr1=val1 foo.attr2=2 foo.bar.attr3=3 a=a log.pkg=slogutil) +2009-02-13 23:31:30 INF An info line with grouped values via logger (foo.attr1=val1 foo.attr2=2 a=a log.pkg=slogutil) +2009-02-13 23:31:30 INF An info line with nested grouped values via logger (bar.foo.attr1=val1 bar.foo.attr2=2 a=a log.pkg=slogutil) +2009-02-13 23:31:30 WRN A warning entry (a=a log.pkg=slogutil) +2009-02-13 23:31:30 ERR An error (a=a log.pkg=slogutil)` + + if strings.TrimSpace(buf.String()) != strings.TrimSpace(exp) { + t.Log(buf.String()) + t.Log(exp) + t.Error("mismatch") + } +} diff --git a/internal/slogutil/leveler.go b/internal/slogutil/leveler.go new file mode 100644 index 000000000..5ca1eecd5 --- /dev/null +++ b/internal/slogutil/leveler.go @@ -0,0 +1,104 @@ +// Copyright (C) 2025 The Syncthing Authors. +// +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this file, +// You can obtain one at https://mozilla.org/MPL/2.0/. + +package slogutil + +import ( + "log/slog" + "maps" + "sync" +) + +// A levelTracker keeps track of log level per package. This enables the +// traditional STTRACE variable to set certain packages to debug level, but +// also allows setting packages to other levels such as WARN to silence +// INFO-level messages. +// +// The STTRACE environment variable is one way of controlling this, where +// mentioning a package makes it DEBUG level: +// STTRACE="model,protocol" # model and protocol are at DEBUG level +// however you can also give specific levels after a colon: +// STTRACE="model:WARNING,protocol:DEBUG" + +func PackageDescrs() map[string]string { + return globalLevels.Descrs() +} + +func PackageLevels() map[string]slog.Level { + return globalLevels.Levels() +} + +func SetPackageLevel(pkg string, level slog.Level) { + globalLevels.Set(pkg, level) +} + +func SetDefaultLevel(level slog.Level) { + globalLevels.SetDefault(level) +} + +type levelTracker struct { + mut sync.RWMutex + defLevel slog.Level + descrs map[string]string // package name to description + levels map[string]slog.Level // package name to level +} + +func (t *levelTracker) Get(pkg string) slog.Level { + t.mut.RLock() + defer t.mut.RUnlock() + if level, ok := t.levels[pkg]; ok { + return level + } + return t.defLevel +} + +func (t *levelTracker) Set(pkg string, level slog.Level) { + t.mut.Lock() + changed := t.levels[pkg] != level + t.levels[pkg] = level + t.mut.Unlock() + if changed { + slog.Info("Changed package log level", "package", pkg, "level", level) + } +} + +func (t *levelTracker) SetDefault(level slog.Level) { + t.mut.Lock() + changed := t.defLevel != level + t.defLevel = level + t.mut.Unlock() + if changed { + slog.Info("Changed default log level", "level", level) + } +} + +func (t *levelTracker) SetDescr(pkg, descr string) { + t.mut.Lock() + t.descrs[pkg] = descr + t.mut.Unlock() +} + +func (t *levelTracker) Descrs() map[string]string { + t.mut.RLock() + defer t.mut.RUnlock() + m := make(map[string]string, len(t.descrs)) + maps.Copy(m, t.descrs) + return m +} + +func (t *levelTracker) Levels() map[string]slog.Level { + t.mut.RLock() + defer t.mut.RUnlock() + m := make(map[string]slog.Level, len(t.descrs)) + for pkg := range t.descrs { + if level, ok := t.levels[pkg]; ok { + m[pkg] = level + } else { + m[pkg] = t.defLevel + } + } + return m +} diff --git a/internal/slogutil/line.go b/internal/slogutil/line.go new file mode 100644 index 000000000..2776a5d95 --- /dev/null +++ b/internal/slogutil/line.go @@ -0,0 +1,61 @@ +// Copyright (C) 2025 The Syncthing Authors. +// +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this file, +// You can obtain one at https://mozilla.org/MPL/2.0/. + +package slogutil + +import ( + "encoding/json" + "fmt" + "io" + "log/slog" + "time" +) + +// A Line is our internal representation of a formatted log line. This is +// what we present in the API and what we buffer internally. +type Line struct { + When time.Time `json:"when"` + Message string `json:"message"` + Level slog.Level `json:"level"` +} + +func (l *Line) WriteTo(w io.Writer) (int64, error) { + n, err := fmt.Fprintf(w, "%s %s %s\n", l.timeStr(), l.levelStr(), l.Message) + return int64(n), err +} + +func (l *Line) timeStr() string { + return l.When.Format("2006-01-02 15:04:05") +} + +func (l *Line) levelStr() string { + str := func(base string, val slog.Level) string { + if val == 0 { + return base + } + return fmt.Sprintf("%s%+d", base, val) + } + + switch { + case l.Level < slog.LevelInfo: + return str("DBG", l.Level-slog.LevelDebug) + case l.Level < slog.LevelWarn: + return str("INF", l.Level-slog.LevelInfo) + case l.Level < slog.LevelError: + return str("WRN", l.Level-slog.LevelWarn) + default: + return str("ERR", l.Level-slog.LevelError) + } +} + +func (l *Line) MarshalJSON() ([]byte, error) { + // Custom marshal to get short level strings instead of default JSON serialisation + return json.Marshal(map[string]any{ + "when": l.When, + "message": l.Message, + "level": l.levelStr(), + }) +} diff --git a/internal/slogutil/recorder.go b/internal/slogutil/recorder.go new file mode 100644 index 000000000..68635f4f7 --- /dev/null +++ b/internal/slogutil/recorder.go @@ -0,0 +1,59 @@ +// Copyright (C) 2025 The Syncthing Authors. +// +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this file, +// You can obtain one at https://mozilla.org/MPL/2.0/. + +package slogutil + +import ( + "log/slog" + "sync" + "time" +) + +const maxLogLines = 1000 + +type Recorder interface { + Since(t time.Time) []Line + Clear() +} + +func NewRecorder(level slog.Level) Recorder { + return &lineRecorder{level: level} +} + +type lineRecorder struct { + level slog.Level + mut sync.Mutex + lines []Line +} + +func (r *lineRecorder) record(line Line) { + if line.Level < r.level { + return + } + r.mut.Lock() + r.lines = append(r.lines, line) + if len(r.lines) > maxLogLines { + r.lines = r.lines[len(r.lines)-maxLogLines:] + } + r.mut.Unlock() +} + +func (r *lineRecorder) Clear() { + r.mut.Lock() + r.lines = nil + r.mut.Unlock() +} + +func (r *lineRecorder) Since(t time.Time) []Line { + r.mut.Lock() + defer r.mut.Unlock() + for i := range r.lines { + if r.lines[i].When.After(t) { + return r.lines[i:] + } + } + return nil +} diff --git a/internal/slogutil/slogadapter.go b/internal/slogutil/slogadapter.go new file mode 100644 index 000000000..b30801be9 --- /dev/null +++ b/internal/slogutil/slogadapter.go @@ -0,0 +1,71 @@ +// Copyright (C) 2025 The Syncthing Authors. +// +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this file, +// You can obtain one at https://mozilla.org/MPL/2.0/. + +package slogutil + +import ( + "context" + "fmt" + "log/slog" + "runtime" + "strings" + "time" +) + +// Log levels: +// - DEBUG: programmers only (not user troubleshooting) +// - INFO: most stuff, files syncing properly +// - WARN: errors that can be ignored or will be retried (e.g., sync failures) +// - ERROR: errors that need handling, shown in the GUI + +func RegisterPackage(descr string) { + registerPackage(descr, 2) +} + +func NewAdapter(descr string) *adapter { + registerPackage(descr, 2) + return &adapter{slogDef} +} + +func registerPackage(descr string, frames int) { + var pcs [1]uintptr + runtime.Callers(1+frames, pcs[:]) + pc := pcs[0] + fr := runtime.CallersFrames([]uintptr{pc}) + if fram, _ := fr.Next(); fram.Function != "" { + pkgName, _ := funcNameToPkg(fram.Function) + globalLevels.SetDescr(pkgName, descr) + } +} + +type adapter struct { + l *slog.Logger +} + +func (a adapter) Debugln(vals ...interface{}) { + a.log(strings.TrimSpace(fmt.Sprintln(vals...)), slog.LevelDebug) +} + +func (a adapter) Debugf(format string, vals ...interface{}) { + a.log(fmt.Sprintf(format, vals...), slog.LevelDebug) +} + +func (a adapter) log(msg string, level slog.Level) { + h := a.l.Handler() + if !h.Enabled(context.Background(), level) { + return + } + var pcs [1]uintptr + // skip [runtime.Callers, this function, this function's caller] + runtime.Callers(3, pcs[:]) + pc := pcs[0] + r := slog.NewRecord(time.Now(), level, msg, pc) + _ = h.Handle(context.Background(), r) +} + +func (a adapter) ShouldDebug(facility string) bool { + return globalLevels.Get(facility) <= slog.LevelDebug +} diff --git a/internal/slogutil/sloginit.go b/internal/slogutil/sloginit.go new file mode 100644 index 000000000..b607e75c0 --- /dev/null +++ b/internal/slogutil/sloginit.go @@ -0,0 +1,47 @@ +// Copyright (C) 2025 The Syncthing Authors. +// +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this file, +// You can obtain one at https://mozilla.org/MPL/2.0/. + +package slogutil + +import ( + "log/slog" + "os" + "strings" +) + +var ( + GlobalRecorder = &lineRecorder{level: -1000} + ErrorRecorder = &lineRecorder{level: slog.LevelError} + globalLevels = &levelTracker{ + levels: make(map[string]slog.Level), + descrs: make(map[string]string), + } + slogDef = slog.New(&formattingHandler{ + recs: []*lineRecorder{GlobalRecorder, ErrorRecorder}, + out: os.Stdout, + }) +) + +func init() { + slog.SetDefault(slogDef) + + // Handle legacy STTRACE var + pkgs := strings.Split(os.Getenv("STTRACE"), ",") + for _, pkg := range pkgs { + pkg = strings.TrimSpace(pkg) + if pkg == "" { + continue + } + level := slog.LevelDebug + if cutPkg, levelStr, ok := strings.Cut(pkg, ":"); ok { + pkg = cutPkg + if err := level.UnmarshalText([]byte(levelStr)); err != nil { + slog.Warn("Bad log level requested in STTRACE", slog.String("pkg", pkg), slog.String("level", levelStr), Error(err)) + } + } + globalLevels.Set(pkg, level) + } +} diff --git a/internal/slogutil/slogvalues.go b/internal/slogutil/slogvalues.go new file mode 100644 index 000000000..a9ae86e63 --- /dev/null +++ b/internal/slogutil/slogvalues.go @@ -0,0 +1,40 @@ +// Copyright (C) 2025 The Syncthing Authors. +// +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this file, +// You can obtain one at https://mozilla.org/MPL/2.0/. + +package slogutil + +import ( + "log/slog" + "maps" + "slices" +) + +func Address(v any) slog.Attr { + return slog.Any("address", v) +} + +func Error(err any) slog.Attr { + if err == nil { + return slog.Attr{} + } + return slog.Any("error", err) +} + +func FilePath(path string) slog.Attr { + return slog.String("path", path) +} + +func URI(v any) slog.Attr { + return slog.Any("uri", v) +} + +func Map[T any](m map[string]T) []any { + var attrs []any + for _, key := range slices.Sorted(maps.Keys(m)) { + attrs = append(attrs, slog.Any(key, m[key])) + } + return attrs +} diff --git a/lib/api/api.go b/lib/api/api.go index 4cb7a97ed..a13d9d10d 100644 --- a/lib/api/api.go +++ b/lib/api/api.go @@ -17,6 +17,7 @@ import ( "fmt" "io" "log" + "log/slog" "net" "net/http" "net/url" @@ -28,6 +29,7 @@ import ( "slices" "strconv" "strings" + "sync" "time" "unicode" @@ -42,6 +44,7 @@ import ( "golang.org/x/text/unicode/norm" "github.com/syncthing/syncthing/internal/db" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/build" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/connections" @@ -49,12 +52,10 @@ import ( "github.com/syncthing/syncthing/lib/events" "github.com/syncthing/syncthing/lib/fs" "github.com/syncthing/syncthing/lib/locations" - "github.com/syncthing/syncthing/lib/logger" "github.com/syncthing/syncthing/lib/model" "github.com/syncthing/syncthing/lib/protocol" "github.com/syncthing/syncthing/lib/rand" "github.com/syncthing/syncthing/lib/svcutil" - "github.com/syncthing/syncthing/lib/sync" "github.com/syncthing/syncthing/lib/tlsutil" "github.com/syncthing/syncthing/lib/upgrade" "github.com/syncthing/syncthing/lib/ur" @@ -95,8 +96,8 @@ type service struct { miscDB *db.Typed shutdownTimeout time.Duration - guiErrors logger.Recorder - systemLog logger.Recorder + guiErrors slogutil.Recorder + systemLog slogutil.Recorder } var _ config.Verifier = &service{} @@ -107,7 +108,7 @@ type Service interface { WaitForStart() error } -func New(id protocol.DeviceID, cfg config.Wrapper, assetDir, tlsDefaultCommonName string, m model.Model, defaultSub, diskSub events.BufferedSubscription, evLogger events.Logger, discoverer discover.Manager, connectionsService connections.Service, urService *ur.Service, fss model.FolderSummaryService, errors, systemLog logger.Recorder, noUpgrade bool, miscDB *db.Typed) Service { +func New(id protocol.DeviceID, cfg config.Wrapper, assetDir, tlsDefaultCommonName string, m model.Model, defaultSub, diskSub events.BufferedSubscription, evLogger events.Logger, discoverer discover.Manager, connectionsService connections.Service, urService *ur.Service, fss model.FolderSummaryService, errors, systemLog slogutil.Recorder, noUpgrade bool, miscDB *db.Typed) Service { return &service{ id: id, cfg: cfg, @@ -117,7 +118,6 @@ func New(id protocol.DeviceID, cfg config.Wrapper, assetDir, tlsDefaultCommonNam DefaultEventMask: defaultSub, DiskEventMask: diskSub, }, - eventSubsMut: sync.NewMutex(), evLogger: evLogger, discoverer: discoverer, connectionsService: connectionsService, @@ -151,8 +151,10 @@ func (s *service) getListener(guiCfg config.GUIConfiguration) (net.Listener, err err = shouldRegenerateCertificate(cert) } if err != nil { - l.Infoln("Loading HTTPS certificate:", err) - l.Infoln("Creating new HTTPS certificate") + if !os.IsNotExist(err) { + slog.Warn("Failed to load HTTPS certificate", slogutil.Error(err)) + } + slog.Info("Creating new HTTPS certificate") // When generating the HTTPS certificate, use the system host name per // default. If that isn't available, use the "syncthing" default. @@ -222,7 +224,7 @@ func (s *service) Serve(ctx context.Context) error { case <-s.startedOnce: // We let this be a loud user-visible warning as it may be the only // indication they get that the GUI won't be available. - l.Warnln("Starting API/GUI:", err) + slog.ErrorContext(ctx, "Failed to start API/GUI", slogutil.Error(err)) default: // This is during initialization. A failure here should be fatal @@ -280,7 +282,7 @@ func (s *service) Serve(ctx context.Context) error { restMux.HandlerFunc(http.MethodGet, "/rest/system/status", s.getSystemStatus) // - restMux.HandlerFunc(http.MethodGet, "/rest/system/upgrade", s.getSystemUpgrade) // - restMux.HandlerFunc(http.MethodGet, "/rest/system/version", s.getSystemVersion) // - - restMux.HandlerFunc(http.MethodGet, "/rest/system/debug", s.getSystemDebug) // - + restMux.HandlerFunc(http.MethodGet, "/rest/system/loglevels", s.getSystemDebug) // - restMux.HandlerFunc(http.MethodGet, "/rest/system/log", s.getSystemLog) // [since] restMux.HandlerFunc(http.MethodGet, "/rest/system/log.txt", s.getSystemLogTxt) // [since] @@ -300,7 +302,7 @@ func (s *service) Serve(ctx context.Context) error { restMux.HandlerFunc(http.MethodPost, "/rest/system/upgrade", s.postSystemUpgrade) // - restMux.HandlerFunc(http.MethodPost, "/rest/system/pause", s.makeDevicePauseHandler(true)) // [device] restMux.HandlerFunc(http.MethodPost, "/rest/system/resume", s.makeDevicePauseHandler(false)) // [device] - restMux.HandlerFunc(http.MethodPost, "/rest/system/debug", s.postSystemDebug) // [enable] [disable] + restMux.HandlerFunc(http.MethodPost, "/rest/system/loglevels", s.postSystemDebug) // [enable] [disable] // The DELETE handlers restMux.HandlerFunc(http.MethodDelete, "/rest/cluster/pending/devices", s.deletePendingDevices) // device @@ -409,8 +411,8 @@ func (s *service) Serve(ctx context.Context) error { srv.ErrorLog = log.Default() } - l.Infoln("GUI and API listening on", listener.Addr()) - l.Infoln("Access the GUI via the following URL:", guiCfg.URL()) + slog.InfoContext(ctx, "GUI and API listening", slogutil.Address(listener.Addr())) + slog.InfoContext(ctx, "Access the GUI via the following URL: "+guiCfg.URL()) //nolint:sloglint if s.started != nil { // only set when run by the tests select { @@ -443,14 +445,14 @@ func (s *service) Serve(ctx context.Context) error { select { case <-ctx.Done(): // Shutting down permanently - l.Debugln("shutting down (stop)") + slog.DebugContext(ctx, "Shutting down (stop)") case <-s.configChanged: // Soft restart due to configuration change - l.Debugln("restarting (config changed)") + slog.DebugContext(ctx, "Restarting (config changed)") case err = <-s.exitChan: case err = <-serveError: // Restart due to listen/serve failure - l.Warnln("GUI/API:", err, "(restarting)") + slog.ErrorContext(ctx, "GUI/API error (restarting)", slogutil.Error(err)) } // Give it a moment to shut down gracefully, e.g. if we are restarting // due to a config change through the API, let that finish successfully. @@ -749,31 +751,20 @@ func (*service) getSystemVersion(w http.ResponseWriter, _ *http.Request) { } func (*service) getSystemDebug(w http.ResponseWriter, _ *http.Request) { - names := l.Facilities() - enabled := l.FacilityDebugging() - slices.Sort(enabled) - sendJSON(w, map[string]interface{}{ - "facilities": names, - "enabled": enabled, + sendJSON(w, map[string]any{ + "packages": slogutil.PackageDescrs(), + "levels": slogutil.PackageLevels(), }) } func (*service) postSystemDebug(w http.ResponseWriter, r *http.Request) { - w.Header().Set("Content-Type", "application/json; charset=utf-8") - q := r.URL.Query() - for _, f := range strings.Split(q.Get("enable"), ",") { - if f == "" || l.ShouldDebug(f) { - continue - } - l.SetDebug(f, true) - l.Infof("Enabled debug data for %q", f) + var levelRequest map[string]slog.Level + if err := json.NewDecoder(r.Body).Decode(&levelRequest); err != nil { + http.Error(w, err.Error(), http.StatusBadRequest) + return } - for _, f := range strings.Split(q.Get("disable"), ",") { - if f == "" || !l.ShouldDebug(f) { - continue - } - l.SetDebug(f, false) - l.Infof("Disabled debug data for %q", f) + for pkg, level := range levelRequest { + slogutil.SetPackageLevel(pkg, level) } } @@ -1106,7 +1097,7 @@ func (s *service) getSystemStatus(w http.ResponseWriter, _ *http.Request) { } func (s *service) getSystemError(w http.ResponseWriter, _ *http.Request) { - sendJSON(w, map[string][]logger.Line{ + sendJSON(w, map[string][]slogutil.Line{ "errors": s.guiErrors.Since(time.Time{}), }) } @@ -1114,7 +1105,7 @@ func (s *service) getSystemError(w http.ResponseWriter, _ *http.Request) { func (*service) postSystemError(_ http.ResponseWriter, r *http.Request) { bs, _ := io.ReadAll(r.Body) r.Body.Close() - l.Warnln(string(bs)) + slog.Error("External error report", slogutil.Error(string(bs))) } func (s *service) postSystemErrorClear(_ http.ResponseWriter, _ *http.Request) { @@ -1127,7 +1118,7 @@ func (s *service) getSystemLog(w http.ResponseWriter, r *http.Request) { if err != nil { l.Debugln(err) } - sendJSON(w, map[string][]logger.Line{ + sendJSON(w, map[string][]slogutil.Line{ "messages": s.systemLog.Since(since), }) } @@ -1156,7 +1147,7 @@ func (s *service) getSupportBundle(w http.ResponseWriter, r *http.Request) { // Redacted configuration as a JSON if jsonConfig, err := json.MarshalIndent(getRedactedConfig(s), "", " "); err != nil { - l.Warnln("Support bundle: failed to create config.json:", err) + slog.Warn("Failed to create config.json in support bundle", slogutil.Error(err)) } else { files = append(files, fileEntry{name: "config.json.txt", data: jsonConfig}) } @@ -1171,7 +1162,7 @@ func (s *service) getSupportBundle(w http.ResponseWriter, r *http.Request) { // Errors as a JSON if errs := s.guiErrors.Since(time.Time{}); len(errs) > 0 { if jsonError, err := json.MarshalIndent(errs, "", " "); err != nil { - l.Warnln("Support bundle: failed to create errors.json:", err) + slog.Warn("Failed to create errors.json in support bundle", slogutil.Error(err)) } else { files = append(files, fileEntry{name: "errors.json.txt", data: jsonError}) } @@ -1181,7 +1172,7 @@ func (s *service) getSupportBundle(w http.ResponseWriter, r *http.Request) { if panicFiles, err := filepath.Glob(filepath.Join(locations.GetBaseDir(locations.ConfigBaseDir), "panic*")); err == nil { for _, f := range panicFiles { if panicFile, err := os.ReadFile(f); err != nil { - l.Warnf("Support bundle: failed to load %s: %s", filepath.Base(f), err) + slog.Warn("Failed to load panic file for support bundle", slogutil.FilePath(filepath.Base(f)), slogutil.Error(err)) } else { files = append(files, fileEntry{name: filepath.Base(f), data: panicFile}) } @@ -1204,15 +1195,15 @@ func (s *service) getSupportBundle(w http.ResponseWriter, r *http.Request) { }, "", " "); err == nil { files = append(files, fileEntry{name: "version-platform.json.txt", data: versionPlatform}) } else { - l.Warnln("Failed to create versionPlatform.json: ", err) + slog.Warn("Failed to create versionPlatform.json in support bundle", slogutil.Error(err)) } // Report Data as a JSON if r, err := s.urService.ReportDataPreview(r.Context(), ur.Version); err != nil { - l.Warnln("Support bundle: failed to create usage-reporting.json.txt:", err) + slog.Warn("Failed to create usage-reporting.json.txt in support bundle", slogutil.Error(err)) } else { if usageReportingData, err := json.MarshalIndent(r, "", " "); err != nil { - l.Warnln("Support bundle: failed to serialize usage-reporting.json.txt", err) + slog.Warn("Failed to serialize usage-reporting.json.txt in support bundle", slogutil.Error(err)) } else { files = append(files, fileEntry{name: "usage-reporting.json.txt", data: usageReportingData}) } @@ -1227,7 +1218,7 @@ func (s *service) getSupportBundle(w http.ResponseWriter, r *http.Request) { // Connection data as JSON connStats := s.model.ConnectionStats() if connStatsJSON, err := json.MarshalIndent(connStats, "", " "); err != nil { - l.Warnln("Support bundle: failed to serialize connection-stats.json.txt", err) + slog.Warn("Failed to serialize connection-stats.json.txt in support bundle", slogutil.Error(err)) } else { files = append(files, fileEntry{name: "connection-stats.json.txt", data: connStatsJSON}) } @@ -1273,7 +1264,7 @@ func (s *service) getSupportBundle(w http.ResponseWriter, r *http.Request) { // Add buffer files to buffer zip var zipFilesBuffer bytes.Buffer if err := writeZip(&zipFilesBuffer, files); err != nil { - l.Warnln("Support bundle: failed to create support bundle zip:", err) + slog.Warn("Failed to create support bundle zip (buffer)", slogutil.Error(err)) http.Error(w, err.Error(), http.StatusInternalServerError) return } @@ -1284,7 +1275,7 @@ func (s *service) getSupportBundle(w http.ResponseWriter, r *http.Request) { // Write buffer zip to local zip file (back up) if err := os.WriteFile(zipFilePath, zipFilesBuffer.Bytes(), 0o600); err != nil { - l.Warnln("Support bundle: support bundle zip could not be created:", err) + slog.Warn("Failed to create support bundle zip (file)", slogutil.FilePath(zipFilePath), slogutil.Error(err)) } // Serve the buffer zip to client for download @@ -1534,7 +1525,7 @@ func (s *service) postSystemUpgrade(w http.ResponseWriter, _ *http.Request) { if upgrade.CompareVersions(rel.Tag, build.Version) > upgrade.Equal { err = upgrade.To(rel) if err != nil { - l.Warnln("upgrading:", err) + slog.Error("Failed to upgrade", slogutil.Error(err)) http.Error(w, err.Error(), http.StatusInternalServerError) return } diff --git a/lib/api/api_auth.go b/lib/api/api_auth.go index 8568394a8..0ac71fa8f 100644 --- a/lib/api/api_auth.go +++ b/lib/api/api_auth.go @@ -9,6 +9,7 @@ package api import ( "crypto/tls" "fmt" + "log/slog" "net" "net/http" "slices" @@ -16,6 +17,7 @@ import ( "time" ldap "github.com/go-ldap/ldap/v3" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/events" "github.com/syncthing/syncthing/lib/osutil" @@ -43,11 +45,11 @@ func emitLoginAttempt(success bool, username string, r *http.Request, evLogger e if success { return } + l := slog.Default().With(slogutil.Address(remoteAddress), slog.String("username", username)) if proxy != "" { - l.Infof("Wrong credentials supplied during API authorization from %s proxied by %s", remoteAddress, proxy) - } else { - l.Infof("Wrong credentials supplied during API authorization from %s", remoteAddress) + l = l.With("proxy", proxy) } + l.Warn("Bad credentials supplied during API authorization") } func remoteAddress(r *http.Request) (remoteAddr, proxy string) { @@ -203,7 +205,7 @@ func attemptBasicAuth(r *http.Request, guiCfg config.GUIConfiguration, ldapCfg c return "", false } - l.Debugln("Sessionless HTTP request with authentication; this is expensive.") + slog.Debug("Sessionless HTTP request with authentication; this is expensive.") if auth(username, password, guiCfg, ldapCfg) { return username, true @@ -254,14 +256,14 @@ func authLDAP(username string, password string, cfg config.LDAPConfiguration) bo } if err != nil { - l.Warnln("LDAP Dial:", err) + slog.Error("Failed to dial LDAP server", slogutil.Error(err)) return false } if cfg.Transport == config.LDAPTransportStartTLS { err = connection.StartTLS(&tls.Config{InsecureSkipVerify: cfg.InsecureSkipVerify}) if err != nil { - l.Warnln("LDAP Start TLS:", err) + slog.Error("Failed to handshake start TLS With LDAP server", slogutil.Error(err)) return false } } @@ -271,7 +273,7 @@ func authLDAP(username string, password string, cfg config.LDAPConfiguration) bo bindDN := formatOptionalPercentS(cfg.BindDN, escapeForLDAPDN(username)) err = connection.Bind(bindDN, password) if err != nil { - l.Warnln("LDAP Bind:", err) + slog.Error("Failed to bind with LDAP server", slogutil.Error(err)) return false } @@ -281,7 +283,7 @@ func authLDAP(username string, password string, cfg config.LDAPConfiguration) bo } if cfg.SearchFilter == "" || cfg.SearchBaseDN == "" { - l.Warnln("LDAP configuration: both searchFilter and searchBaseDN must be set, or neither.") + slog.Error("Bad LDAP configuration: both searchFilter and searchBaseDN must be set, or neither") return false } @@ -296,11 +298,11 @@ func authLDAP(username string, password string, cfg config.LDAPConfiguration) bo res, err := connection.Search(searchReq) if err != nil { - l.Warnln("LDAP Search:", err) + slog.Warn("Failed LDAP search", slogutil.Error(err)) return false } if len(res.Entries) != 1 { - l.Infof("Wrong number of LDAP search results, %d != 1", len(res.Entries)) + slog.Warn("Incorrect number of LDAP search results (expected one)", slog.Int("results", len(res.Entries))) return false } diff --git a/lib/api/api_statics.go b/lib/api/api_statics.go index 89b1c4c5a..861565e1d 100644 --- a/lib/api/api_statics.go +++ b/lib/api/api_statics.go @@ -12,12 +12,12 @@ import ( "os" "path/filepath" "strings" + "sync" "time" "github.com/syncthing/syncthing/lib/api/auto" "github.com/syncthing/syncthing/lib/assets" "github.com/syncthing/syncthing/lib/config" - "github.com/syncthing/syncthing/lib/sync" ) const themePrefix = "theme-assets/" @@ -36,7 +36,6 @@ func newStaticsServer(theme, assetDir string) *staticsServer { s := &staticsServer{ assetDir: assetDir, assets: auto.Assets(), - mut: sync.NewRWMutex(), theme: theme, lastThemeChange: time.Now().UTC(), } diff --git a/lib/api/api_test.go b/lib/api/api_test.go index bad4b456c..d6f0b67e4 100644 --- a/lib/api/api_test.go +++ b/lib/api/api_test.go @@ -29,6 +29,7 @@ import ( "github.com/syncthing/syncthing/internal/db" "github.com/syncthing/syncthing/internal/db/sqlite" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/assets" "github.com/syncthing/syncthing/lib/build" "github.com/syncthing/syncthing/lib/config" @@ -38,14 +39,11 @@ import ( eventmocks "github.com/syncthing/syncthing/lib/events/mocks" "github.com/syncthing/syncthing/lib/fs" "github.com/syncthing/syncthing/lib/locations" - "github.com/syncthing/syncthing/lib/logger" - loggermocks "github.com/syncthing/syncthing/lib/logger/mocks" "github.com/syncthing/syncthing/lib/model" modelmocks "github.com/syncthing/syncthing/lib/model/mocks" "github.com/syncthing/syncthing/lib/protocol" "github.com/syncthing/syncthing/lib/rand" "github.com/syncthing/syncthing/lib/svcutil" - "github.com/syncthing/syncthing/lib/sync" "github.com/syncthing/syncthing/lib/tlsutil" "github.com/syncthing/syncthing/lib/ur" ) @@ -96,7 +94,7 @@ func TestStopAfterBrokenConfig(t *testing.T) { srv.started = make(chan string) - sup := suture.New("test", svcutil.SpecWithDebugLogger(l)) + sup := suture.New("test", svcutil.SpecWithDebugLogger()) sup.Add(srv) ctx, cancel := context.WithCancel(context.Background()) sup.ServeBackground(ctx) @@ -150,7 +148,6 @@ func TestAssetsDir(t *testing.T) { e := &staticsServer{ theme: "foo", - mut: sync.NewRWMutex(), assetDir: "testdata", assets: map[string]assets.Asset{ "foo/a": foo, // overridden in foo/a @@ -360,7 +357,7 @@ func TestAPIServiceRequests(t *testing.T) { Prefix: "{", }, { - URL: "/rest/system/debug", + URL: "/rest/system/loglevels", Code: 200, Type: "application/json", Prefix: "{", @@ -1044,16 +1041,8 @@ func startHTTPWithShutdownTimeout(t *testing.T, cfg config.Wrapper, shutdownTime diskEventSub := new(eventmocks.BufferedSubscription) discoverer := new(discovermocks.Manager) connections := new(connmocks.Service) - errorLog := new(loggermocks.Recorder) - systemLog := new(loggermocks.Recorder) - for _, l := range []*loggermocks.Recorder{errorLog, systemLog} { - l.SinceReturns([]logger.Line{ - { - When: time.Now(), - Message: "Test message", - }, - }) - } + errorLog := slogutil.NewRecorder(0) + systemLog := slogutil.NewRecorder(0) addrChan := make(chan string) mockedSummary := &modelmocks.FolderSummaryService{} mockedSummary.SummaryReturns(new(model.FolderSummary), nil) diff --git a/lib/api/confighandler.go b/lib/api/confighandler.go index 9ca6acc32..5bfb32b4c 100644 --- a/lib/api/confighandler.go +++ b/lib/api/confighandler.go @@ -9,10 +9,12 @@ package api import ( "encoding/json" "io" + "log/slog" "net/http" "github.com/julienschmidt/httprouter" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/protocol" "github.com/syncthing/syncthing/lib/structutil" @@ -311,7 +313,7 @@ func (c *configMuxBuilder) adjustConfig(w http.ResponseWriter, r *http.Request) to, err := config.ReadJSON(r.Body, c.id) r.Body.Close() if err != nil { - l.Warnln("Decoding posted config:", err) + slog.Error("Failed to decode posted config", slogutil.Error(err)) http.Error(w, err.Error(), http.StatusBadRequest) return } @@ -415,7 +417,7 @@ func (c *configMuxBuilder) adjustGUI(w http.ResponseWriter, r *http.Request, gui func (c *configMuxBuilder) postAdjustGui(from *config.GUIConfiguration, to *config.GUIConfiguration) error { if to.Password != from.Password { if err := to.SetPassword(to.Password); err != nil { - l.Warnln("hashing password:", err) + slog.Error("Failed to hash password", slogutil.Error(err)) return err } } @@ -456,7 +458,7 @@ func unmarshalToRawMessages(body io.ReadCloser) ([]json.RawMessage, error) { func (c *configMuxBuilder) finish(w http.ResponseWriter, waiter config.Waiter) { waiter.Wait() if err := c.cfg.Save(); err != nil { - l.Warnln("Saving config:", err) + slog.Error("Failed to save config", slogutil.Error(err)) http.Error(w, err.Error(), http.StatusInternalServerError) } } diff --git a/lib/api/debug.go b/lib/api/debug.go index e5325a695..5e1a27355 100644 --- a/lib/api/debug.go +++ b/lib/api/debug.go @@ -6,11 +6,9 @@ package api -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("api", "REST API") +var l = slogutil.NewAdapter("REST API") func shouldDebugHTTP() bool { return l.ShouldDebug("api") diff --git a/lib/api/tokenmanager.go b/lib/api/tokenmanager.go index ddf6b8d28..f0bc7365d 100644 --- a/lib/api/tokenmanager.go +++ b/lib/api/tokenmanager.go @@ -10,6 +10,7 @@ import ( "net/http" "slices" "strings" + "sync" "time" "google.golang.org/protobuf/proto" @@ -19,7 +20,6 @@ import ( "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/events" "github.com/syncthing/syncthing/lib/rand" - "github.com/syncthing/syncthing/lib/sync" ) type tokenManager struct { @@ -49,7 +49,6 @@ func newTokenManager(key string, miscDB *db.Typed, lifetime time.Duration, maxIt lifetime: lifetime, maxItems: maxItems, timeNow: time.Now, - mut: sync.NewMutex(), tokens: &tokens, } } diff --git a/lib/beacon/beacon.go b/lib/beacon/beacon.go index f5182709d..0bc416f04 100644 --- a/lib/beacon/beacon.go +++ b/lib/beacon/beacon.go @@ -45,7 +45,7 @@ type cast struct { // methods to get a functional implementation of Interface. func newCast(name string) *cast { // Only log restarts in debug mode. - spec := svcutil.SpecWithDebugLogger(l) + spec := svcutil.SpecWithDebugLogger() // Don't retry too frenetically: an error to open a socket or // whatever is usually something that is either permanent or takes // a while to get solved... diff --git a/lib/beacon/broadcast.go b/lib/beacon/broadcast.go index 0c96684b8..bba7eef96 100644 --- a/lib/beacon/broadcast.go +++ b/lib/beacon/broadcast.go @@ -8,9 +8,11 @@ package beacon import ( "context" + "log/slog" "net" "time" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/netutil" ) @@ -109,7 +111,7 @@ func writeBroadcasts(ctx context.Context, inbox <-chan []byte, port int) error { } if success == 0 { - l.Debugln("couldn't send any broadcasts") + slog.DebugContext(ctx, "Couldn't send any broadcasts", slogutil.Error(err)) return err } } @@ -146,7 +148,7 @@ func readBroadcasts(ctx context.Context, outbox chan<- recv, port int) error { case <-doneCtx.Done(): return doneCtx.Err() default: - l.Debugln("dropping message") + slog.DebugContext(ctx, "Dropping message") } } } diff --git a/lib/beacon/debug.go b/lib/beacon/debug.go index ab437e643..b1112350d 100644 --- a/lib/beacon/debug.go +++ b/lib/beacon/debug.go @@ -6,8 +6,6 @@ package beacon -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("beacon", "Multicast and broadcast discovery") +var l = slogutil.NewAdapter("Multicast and broadcast discovery") diff --git a/lib/beacon/multicast.go b/lib/beacon/multicast.go index 7e7f3f03f..edaa5df56 100644 --- a/lib/beacon/multicast.go +++ b/lib/beacon/multicast.go @@ -9,6 +9,7 @@ package beacon import ( "context" "errors" + "log/slog" "net" "time" @@ -138,7 +139,7 @@ func readMulticasts(ctx context.Context, outbox chan<- recv, addr string) error } if joined == 0 { - l.Debugln("no multicast interfaces available") + slog.DebugContext(ctx, "No multicast interfaces available") return errors.New("no multicast interfaces available") } @@ -161,7 +162,7 @@ func readMulticasts(ctx context.Context, outbox chan<- recv, addr string) error select { case outbox <- recv{c, addr}: default: - l.Debugln("dropping message") + slog.DebugContext(ctx, "Dropping message") } } } diff --git a/lib/config/config.go b/lib/config/config.go index 29746e4ee..bb3e0390d 100644 --- a/lib/config/config.go +++ b/lib/config/config.go @@ -13,6 +13,7 @@ import ( "errors" "fmt" "io" + "log/slog" "net" "net/url" "os" @@ -21,6 +22,7 @@ import ( "strconv" "strings" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/build" "github.com/syncthing/syncthing/lib/fs" "github.com/syncthing/syncthing/lib/netutil" @@ -121,8 +123,7 @@ func New(myID protocol.DeviceID) Configuration { // Can't happen. if err := cfg.prepare(myID); err != nil { - l.Warnln("bug: error in preparing new folder:", err) - panic("error in preparing new folder") + panic("bug: error in preparing new folder") } return cfg @@ -418,7 +419,7 @@ func (cfg *Configuration) removeDeprecatedProtocols() { func (cfg *Configuration) applyMigrations() { if cfg.Version > 0 && cfg.Version < OldestHandledVersion { - l.Warnf("Configuration version %d is deprecated. Attempting best effort conversion, but please verify manually.", cfg.Version) + slog.Warn("Loaded deprecated configuration version; attempting best effort conversion, but please verify manually", "version", cfg.Version) } // Upgrade configuration versions as appropriate @@ -591,7 +592,7 @@ func ensureNoUntrustedTrustingSharing(f *FolderConfiguration, devices []FolderDe continue } if devCfg := existingDevices[dev.DeviceID]; devCfg.Untrusted { - l.Warnf("Folder %s (%s) is shared in trusted mode with untrusted device %s (%s); unsharing.", f.ID, f.Label, dev.DeviceID.Short(), devCfg.Name) + slog.Error("Folder is shared in trusted mode with untrusted device; unsharing", dev.DeviceID.LogAttr(), f.LogAttr()) devices = sliceutil.RemoveAndZero(devices, i) i-- } @@ -611,7 +612,7 @@ func cleanSymlinks(filesystem fs.Filesystem, dir string) { return err } if info.IsSymlink() { - l.Infoln("Removing incorrectly versioned symlink", path) + slog.Warn("Removing incorrectly versioned symlink", slogutil.FilePath(path)) filesystem.Remove(path) return fs.SkipDir } diff --git a/lib/config/debug.go b/lib/config/debug.go index d85ad98a7..883721aca 100644 --- a/lib/config/debug.go +++ b/lib/config/debug.go @@ -6,8 +6,6 @@ package config -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("config", "Configuration loading and saving") +var l = slogutil.NewAdapter("Configuration loading and saving") diff --git a/lib/config/deviceconfiguration.go b/lib/config/deviceconfiguration.go index 28a51fe9b..18b58e51b 100644 --- a/lib/config/deviceconfiguration.go +++ b/lib/config/deviceconfiguration.go @@ -8,6 +8,7 @@ package config import ( "fmt" + "log/slog" "slices" "github.com/syncthing/syncthing/lib/protocol" @@ -65,11 +66,11 @@ func (cfg *DeviceConfiguration) prepare(sharedFolders []string) { // auto accept folders. if cfg.Untrusted { if cfg.Introducer { - l.Warnf("Device %s (%s) is both untrusted and an introducer, removing introducer flag", cfg.DeviceID.Short(), cfg.Name) + slog.Warn("Device is both untrusted and an introducer, removing introducer flag", cfg.DeviceID.LogAttr()) cfg.Introducer = false } if cfg.AutoAcceptFolders { - l.Warnf("Device %s (%s) is both untrusted and auto-accepting folders, removing auto-accept flag", cfg.DeviceID.Short(), cfg.Name) + slog.Warn("Device is both untrusted and auto-accepting folders, removing auto-accept flag", cfg.DeviceID.LogAttr()) cfg.AutoAcceptFolders = false } } diff --git a/lib/config/folderconfiguration.go b/lib/config/folderconfiguration.go index 698fd6dbc..e318d2ae0 100644 --- a/lib/config/folderconfiguration.go +++ b/lib/config/folderconfiguration.go @@ -13,6 +13,7 @@ import ( "encoding/xml" "errors" "fmt" + "log/slog" "path" "path/filepath" "slices" @@ -268,6 +269,13 @@ func (f FolderConfiguration) Description() string { return fmt.Sprintf("%q (%s)", f.Label, f.ID) } +func (f FolderConfiguration) LogAttr() slog.Attr { + if f.Label == "" || f.Label == f.ID { + return slog.Group("folder", slog.String("id", f.ID), slog.String("type", f.Type.String())) + } + return slog.Group("folder", slog.String("label", f.Label), slog.String("id", f.ID), slog.String("type", f.Type.String())) +} + func (f *FolderConfiguration) DeviceIDs() []protocol.DeviceID { deviceIDs := make([]protocol.DeviceID, len(f.Devices)) for i, n := range f.Devices { diff --git a/lib/config/migrations.go b/lib/config/migrations.go index 99dadb6ed..dcb811b65 100644 --- a/lib/config/migrations.go +++ b/lib/config/migrations.go @@ -8,6 +8,7 @@ package config import ( "cmp" + "log/slog" "net/url" "os" "path" @@ -16,6 +17,7 @@ import ( "strings" "sync" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/build" "github.com/syncthing/syncthing/lib/fs" "github.com/syncthing/syncthing/lib/netutil" @@ -239,7 +241,7 @@ func migrateToConfigV23(cfg *Configuration) { fs.Hide(DefaultMarkerName) // ignore error } if err != nil { - l.Infoln("Failed to upgrade folder marker:", err) + slog.Warn("Failed to upgrade folder marker", slogutil.Error(err)) } } } diff --git a/lib/config/optionsconfiguration.go b/lib/config/optionsconfiguration.go index 173a31e27..ec145e718 100644 --- a/lib/config/optionsconfiguration.go +++ b/lib/config/optionsconfiguration.go @@ -134,11 +134,9 @@ func (opts *OptionsConfiguration) prepare(guiPWIsSet bool) { } if opts.ConnectionPriorityQUICWAN <= opts.ConnectionPriorityQUICLAN { - l.Warnln("Connection priority number for QUIC over WAN must be worse (higher) than QUIC over LAN. Correcting.") opts.ConnectionPriorityQUICWAN = opts.ConnectionPriorityQUICLAN + 1 } if opts.ConnectionPriorityTCPWAN <= opts.ConnectionPriorityTCPLAN { - l.Warnln("Connection priority number for TCP over WAN must be worse (higher) than TCP over LAN. Correcting.") opts.ConnectionPriorityTCPWAN = opts.ConnectionPriorityTCPLAN + 1 } @@ -186,7 +184,7 @@ func (opts OptionsConfiguration) StunServers() []string { case "default": _, records, err := net.LookupSRV("stun", "udp", "syncthing.net") if err != nil { - l.Debugf("Unable to resolve primary STUN servers via DNS:", err) + l.Debugln("Unable to resolve primary STUN servers via DNS:", err) } for _, record := range records { diff --git a/lib/config/wrapper.go b/lib/config/wrapper.go index 645f56d5e..e5120b003 100644 --- a/lib/config/wrapper.go +++ b/lib/config/wrapper.go @@ -12,18 +12,20 @@ package config import ( "context" "errors" + "log/slog" "os" "reflect" + "sync" "sync/atomic" "time" "github.com/thejerf/suture/v4" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/events" "github.com/syncthing/syncthing/lib/osutil" "github.com/syncthing/syncthing/lib/protocol" "github.com/syncthing/syncthing/lib/sliceutil" - "github.com/syncthing/syncthing/lib/sync" ) const ( @@ -151,7 +153,6 @@ func Wrap(path string, cfg Configuration, myID protocol.DeviceID, evLogger event myID: myID, queue: make(chan modifyEntry, maxModifications), waiter: noopWaiter{}, // Noop until first config change - mut: sync.NewMutex(), } return w } @@ -297,7 +298,7 @@ func (w *wrapper) serveSave() { return } if err := w.Save(); err != nil { - l.Warnln("Failed to save config:", err) + slog.Error("Failed to save config", slogutil.Error(err)) } } @@ -328,7 +329,7 @@ func (w *wrapper) replaceLocked(to Configuration) (Waiter, error) { } func (w *wrapper) notifyListeners(from, to Configuration) Waiter { - wg := sync.NewWaitGroup() + wg := new(sync.WaitGroup) wg.Add(len(w.subs)) for _, sub := range w.subs { go func(committer Committer) { diff --git a/lib/connections/connections_test.go b/lib/connections/connections_test.go index 0061d16f9..f04d90a5b 100644 --- a/lib/connections/connections_test.go +++ b/lib/connections/connections_test.go @@ -17,6 +17,7 @@ import ( "net" "net/url" "strings" + "sync" "testing" "time" @@ -27,7 +28,6 @@ import ( "github.com/syncthing/syncthing/lib/events" "github.com/syncthing/syncthing/lib/nat" "github.com/syncthing/syncthing/lib/protocol" - "github.com/syncthing/syncthing/lib/sync" "github.com/syncthing/syncthing/lib/tlsutil" ) @@ -336,7 +336,7 @@ func BenchmarkConnections(b *testing.B) { total := 0 b.ResetTimer() for i := 0; i < b.N; i++ { - wg := sync.NewWaitGroup() + var wg sync.WaitGroup wg.Add(2) errC := make(chan error, 2) go func() { diff --git a/lib/connections/debug.go b/lib/connections/debug.go index f87886309..36d323b5a 100644 --- a/lib/connections/debug.go +++ b/lib/connections/debug.go @@ -6,8 +6,6 @@ package connections -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("connections", "Connection handling") +var l = slogutil.NewAdapter("Connection handling") diff --git a/lib/connections/limiter.go b/lib/connections/limiter.go index 316bf0bc9..6cb61e053 100644 --- a/lib/connections/limiter.go +++ b/lib/connections/limiter.go @@ -10,13 +10,14 @@ import ( "context" "fmt" "io" + "log/slog" + "sync" "sync/atomic" "golang.org/x/time/rate" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/protocol" - "github.com/syncthing/syncthing/lib/sync" ) // limiter manages a read and write rate limit, reacting to config changes @@ -46,7 +47,6 @@ func newLimiter(myId protocol.DeviceID, cfg config.Wrapper) *limiter { myID: myId, write: rate.NewLimiter(rate.Inf, limiterBurstSize), read: rate.NewLimiter(rate.Inf, limiterBurstSize), - mu: sync.NewMutex(), deviceReadLimiters: make(map[protocol.DeviceID]*rate.Limiter), deviceWriteLimiters: make(map[protocol.DeviceID]*rate.Limiter), } @@ -107,15 +107,13 @@ func (lim *limiter) processDevicesConfigurationLocked(from, to config.Configurat writeLimitStr = fmt.Sprintf("limit is %d KiB/s", dev.MaxSendKbps) } - l.Infof("Device %s send rate %s, receive rate %s", dev.DeviceID, writeLimitStr, readLimitStr) + slog.Info("Device is rate limited", dev.DeviceID.LogAttr(), slog.String("send", writeLimitStr), slog.String("recv", readLimitStr)) } } // Delete remote devices which were removed in new configuration for _, dev := range from.Devices { if _, ok := seen[dev.DeviceID]; !ok { - l.Debugf("deviceID: %s should be removed", dev.DeviceID) - delete(lim.deviceWriteLimiters, dev.DeviceID) delete(lim.deviceReadLimiters, dev.DeviceID) } @@ -160,13 +158,13 @@ func (lim *limiter) CommitConfiguration(from, to config.Configuration) bool { lim.limitsLAN.Store(to.Options.LimitBandwidthInLan) - l.Infof("Overall send rate %s, receive rate %s", sendLimitStr, recvLimitStr) + slog.Info("Overall rate limit in use", "send", sendLimitStr, "recv", recvLimitStr) if limited { if to.Options.LimitBandwidthInLan { - l.Infoln("Rate limits apply to LAN connections") + slog.Info("Rate limits apply to LAN connections") } else { - l.Infoln("Rate limits do not apply to LAN connections") + slog.Info("Rate limits do not apply to LAN connections") } } diff --git a/lib/connections/quic_listen.go b/lib/connections/quic_listen.go index 3868fd92c..9a5a8e7d4 100644 --- a/lib/connections/quic_listen.go +++ b/lib/connections/quic_listen.go @@ -13,6 +13,7 @@ import ( "context" "crypto/tls" "errors" + "log/slog" "net" "net/url" "sync" @@ -21,6 +22,7 @@ import ( "github.com/quic-go/quic-go" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/connections/registry" "github.com/syncthing/syncthing/lib/nat" @@ -58,7 +60,7 @@ type quicListener struct { func (t *quicListener) OnNATTypeChanged(natType stun.NATType) { if natType != stun.NATUnknown { - l.Infof("%s detected NAT type: %s", t.uri, natType) + slog.Info("Detected NAT type", slogutil.URI(t.uri), slog.Any("type", natType)) } t.nat.Store(uint64(natType)) } @@ -77,7 +79,7 @@ func (t *quicListener) OnExternalAddressChanged(address *stun.Host, via string) t.mut.Unlock() if uri != nil && (existingAddress == nil || existingAddress.String() != uri.String()) { - l.Infof("%s resolved external address %s (via %s)", t.uri, uri.String(), via) + slog.Info("Resolved external address", slogutil.URI(t.uri), slogutil.Address(uri.String()), slog.String("via", via)) t.notifyAddressesChanged(t) } else if uri == nil && existingAddress != nil { t.notifyAddressesChanged(t) @@ -89,13 +91,13 @@ func (t *quicListener) serve(ctx context.Context) error { udpAddr, err := net.ResolveUDPAddr(network, t.uri.Host) if err != nil { - l.Infoln("Listen (BEP/quic):", err) + slog.WarnContext(ctx, "Failed to listen (QUIC)", slogutil.Error(err)) return err } udpConn, err := net.ListenUDP(network, udpAddr) if err != nil { - l.Infoln("Listen (BEP/quic):", err) + slog.WarnContext(ctx, "Failed to listen (QUIC)", slogutil.Error(err)) return err } defer udpConn.Close() @@ -117,7 +119,7 @@ func (t *quicListener) serve(ctx context.Context) error { listener, err := quicTransport.Listen(t.tlsCfg, quicConfig) if err != nil { - l.Infoln("Listen (BEP/quic):", err) + slog.WarnContext(ctx, "Failed to listen (QUIC)", slogutil.Error(err)) return err } defer listener.Close() @@ -125,8 +127,8 @@ func (t *quicListener) serve(ctx context.Context) error { t.notifyAddressesChanged(t) defer t.clearAddresses(t) - l.Infof("QUIC listener (%v) starting", udpConn.LocalAddr()) - defer l.Infof("QUIC listener (%v) shutting down", udpConn.LocalAddr()) + slog.InfoContext(ctx, "QUIC listener starting", slogutil.Address(udpConn.LocalAddr())) + defer slog.InfoContext(ctx, "QUIC listener shutting down", slogutil.Address(udpConn.LocalAddr())) var ipVersion nat.IPVersion switch t.uri.Scheme { @@ -168,7 +170,7 @@ func (t *quicListener) serve(ctx context.Context) error { if errors.Is(err, context.Canceled) { return nil } else if err != nil { - l.Infoln("Listen (BEP/quic): Accepting connection:", err) + slog.WarnContext(ctx, "Failed to accept QUIC connection", slogutil.Error(err)) acceptFailures++ if acceptFailures > maxAcceptFailures { @@ -185,13 +187,13 @@ func (t *quicListener) serve(ctx context.Context) error { acceptFailures = 0 - l.Debugln("connect from", session.RemoteAddr()) + slog.DebugContext(ctx, "Incoming connection", "from", session.RemoteAddr()) streamCtx, cancel := context.WithTimeout(ctx, quicOperationTimeout) stream, err := session.AcceptStream(streamCtx) cancel() if err != nil { - l.Debugf("failed to accept stream from %s: %v", session.RemoteAddr(), err) + slog.DebugContext(ctx, "Failed to accept stream", slogutil.Address(session.RemoteAddr()), slogutil.Error(err)) _ = session.CloseWithError(1, err.Error()) continue } diff --git a/lib/connections/registry/registry.go b/lib/connections/registry/registry.go index e86fcb5b7..cf99720a8 100644 --- a/lib/connections/registry/registry.go +++ b/lib/connections/registry/registry.go @@ -11,9 +11,9 @@ package registry import ( "strings" + "sync" "github.com/syncthing/syncthing/lib/sliceutil" - "github.com/syncthing/syncthing/lib/sync" ) type Registry struct { @@ -23,7 +23,6 @@ type Registry struct { func New() *Registry { return &Registry{ - mut: sync.NewMutex(), available: make(map[string][]interface{}), } } diff --git a/lib/connections/relay_listen.go b/lib/connections/relay_listen.go index b6e89c813..6e45b3c2d 100644 --- a/lib/connections/relay_listen.go +++ b/lib/connections/relay_listen.go @@ -10,10 +10,12 @@ import ( "context" "crypto/tls" "errors" + "log/slog" "net/url" "sync" "time" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/connections/registry" "github.com/syncthing/syncthing/lib/dialer" @@ -46,7 +48,7 @@ type relayListener struct { func (t *relayListener) serve(ctx context.Context) error { clnt, err := client.NewClient(t.uri, t.tlsCfg.Certificates, 10*time.Second) if err != nil { - l.Infoln("Listen (BEP/relay):", err) + slog.WarnContext(ctx, "Failed to listen (relay)", slogutil.Error(err)) return err } @@ -54,8 +56,8 @@ func (t *relayListener) serve(ctx context.Context) error { t.client = clnt t.mut.Unlock() - l.Infof("Relay listener (%v) starting", t) - defer l.Infof("Relay listener (%v) shutting down", t) + slog.InfoContext(ctx, "Relay listener starting", "id", t.String()) + defer slog.InfoContext(ctx, "Relay listener shutting down", "id", t.String()) defer t.clearAddresses(t) invitationCtx, cancel := context.WithCancel(ctx) @@ -77,19 +79,19 @@ func (t *relayListener) handleInvitations(ctx context.Context, clnt client.Relay conn, err := client.JoinSession(ctx, inv) if err != nil { if !errors.Is(err, context.Canceled) { - l.Infoln("Listen (BEP/relay): joining session:", err) + slog.InfoContext(ctx, "Failed to join session", slogutil.Error(err)) } continue } err = dialer.SetTCPOptions(conn) if err != nil { - l.Debugln("Listen (BEP/relay): setting tcp options:", err) + slog.DebugContext(ctx, "Failed to set TCP options", slogutil.Error(err)) } err = dialer.SetTrafficClass(conn, t.cfg.Options().TrafficClass) if err != nil { - l.Debugln("Listen (BEP/relay): setting traffic class:", err) + slog.DebugContext(ctx, "Failed to set traffic class", slogutil.Error(err)) } var tc *tls.Conn @@ -102,7 +104,7 @@ func (t *relayListener) handleInvitations(ctx context.Context, clnt client.Relay err = tlsTimedHandshake(tc) if err != nil { tc.Close() - l.Infoln("Listen (BEP/relay): TLS handshake:", err) + slog.WarnContext(ctx, "Failed TLS handshake", slogutil.Error(err)) continue } diff --git a/lib/connections/service.go b/lib/connections/service.go index 68a612e8f..bd16be990 100644 --- a/lib/connections/service.go +++ b/lib/connections/service.go @@ -19,17 +19,18 @@ import ( "errors" "fmt" "io" + "log/slog" "math" "net" "net/url" "slices" "strings" - stdsync "sync" + "sync" "time" "github.com/thejerf/suture/v4" - "golang.org/x/time/rate" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/build" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/connections/registry" @@ -42,7 +43,6 @@ import ( "github.com/syncthing/syncthing/lib/sliceutil" "github.com/syncthing/syncthing/lib/stringutil" "github.com/syncthing/syncthing/lib/svcutil" - "github.com/syncthing/syncthing/lib/sync" // Registers NAT service providers _ "github.com/syncthing/syncthing/lib/pmp" @@ -185,7 +185,7 @@ type service struct { } func NewService(cfg config.Wrapper, myID protocol.DeviceID, mdl Model, tlsCfg *tls.Config, discoverer discover.Finder, bepProtocolName string, tlsDefaultCommonName string, evLogger events.Logger, registry *registry.Registry, keyGen *protocol.KeyGenerator) Service { - spec := svcutil.SpecWithInfoLogger(l) + spec := svcutil.SpecWithInfoLogger() service := &service{ Supervisor: suture.New("connections.Service", spec), connectionStatusHandler: newConnectionStatusHandler(), @@ -206,11 +206,9 @@ func NewService(cfg config.Wrapper, myID protocol.DeviceID, mdl Model, tlsCfg *t keyGen: keyGen, lanChecker: &lanChecker{cfg}, - dialNowDevicesMut: sync.NewMutex(), - dialNow: make(chan struct{}, 1), - dialNowDevices: make(map[protocol.DeviceID]struct{}), + dialNow: make(chan struct{}, 1), + dialNowDevices: make(map[protocol.DeviceID]struct{}), - listenersMut: sync.NewRWMutex(), listeners: make(map[string]genericListener), listenerTokens: make(map[string]suture.ServiceToken), } @@ -257,7 +255,7 @@ func (s *service) handleConns(ctx context.Context) error { // because there are implementations out there that don't support // protocol negotiation (iOS for one...). if cs.NegotiatedProtocol != s.bepProtocolName { - l.Infof("Peer at %s did not negotiate bep/1.0", c) + slog.WarnContext(ctx, "Peer at did not negotiate bep/1.0", slogutil.Address(c.RemoteAddr())) } // We should have received exactly one certificate from the other @@ -265,7 +263,7 @@ func (s *service) handleConns(ctx context.Context) error { // connection. certs := cs.PeerCertificates if cl := len(certs); cl != 1 { - l.Infof("Got peer certificate list of length %d != 1 from peer at %s; protocol error", cl, c) + slog.WarnContext(ctx, "Got peer certificate list of incorrect length", slog.Int("length", cl), slogutil.Address(c.RemoteAddr())) c.Close() continue } @@ -276,17 +274,13 @@ func (s *service) handleConns(ctx context.Context) error { // though, especially in the presence of NAT hairpinning, multiple // clients between the same NAT gateway, and global discovery. if remoteID == s.myID { - l.Debugf("Connected to myself (%s) at %s", remoteID, c) + slog.DebugContext(ctx, "Connected to myself", "id", remoteID, "addr", c) c.Close() continue } if err := s.connectionCheckEarly(remoteID, c); err != nil { - if errors.Is(err, errDeviceAlreadyConnected) { - l.Debugf("Connection from %s at %s (%s) rejected: %v", remoteID, c.RemoteAddr(), c.Type(), err) - } else { - l.Infof("Connection from %s at %s (%s) rejected: %v", remoteID, c.RemoteAddr(), c.Type(), err) - } + slog.DebugContext(ctx, "Connection rejected", remoteID.LogAttr(), slogutil.Address(c.RemoteAddr()), slog.String("type", c.Type()), slogutil.Error(err)) c.Close() continue } @@ -382,22 +376,10 @@ func (s *service) handleHellos(ctx context.Context) error { if err != nil { if protocol.IsVersionMismatch(err) { - // The error will be a relatively user friendly description - // of what's wrong with the version compatibility. By - // default identify the other side by device ID and IP. - remote := fmt.Sprintf("%v (%v)", remoteID, c.RemoteAddr()) - if hello.DeviceName != "" { - // If the name was set in the hello return, use that to - // give the user more info about which device is the - // affected one. It probably says more than the remote - // IP. - remote = fmt.Sprintf("%q (%s %s, %v)", hello.DeviceName, hello.ClientName, hello.ClientVersion, remoteID) - } - msg := fmt.Sprintf("Connecting to %s: %s", remote, err) - warningFor(remoteID, msg) + slog.WarnContext(ctx, "Remote device is too old", remoteID.LogAttr(), slogutil.Address(c.RemoteAddr()), slogutil.Error(err)) } else { // It's something else - connection reset or whatever - l.Infof("Failed to exchange Hello messages with %s at %s: %s", remoteID, c, err) + slog.WarnContext(ctx, "Failed to exchange Hello messages", remoteID.LogAttr(), slogutil.Address(c.RemoteAddr()), slogutil.Error(err)) } c.Close() continue @@ -407,14 +389,14 @@ func (s *service) handleHellos(ctx context.Context) error { // The Model will return an error for devices that we don't want to // have a connection with for whatever reason, for example unknown devices. if err := s.model.OnHello(remoteID, c.RemoteAddr(), hello); err != nil { - l.Infof("Connection from %s at %s (%s) rejected: %v", remoteID, c.RemoteAddr(), c.Type(), err) + slog.WarnContext(ctx, "Connection rejected", remoteID.LogAttr(), slogutil.Address(c.RemoteAddr()), slog.Any("type", c.Type()), slogutil.Error(err)) c.Close() continue } deviceCfg, ok := s.cfg.Device(remoteID) if !ok { - l.Infof("Device %s removed from config during connection attempt at %s", remoteID, c) + slog.WarnContext(ctx, "Device removed from config during connection attempt", remoteID.LogAttr(), slogutil.Address(c.RemoteAddr())) c.Close() continue } @@ -434,7 +416,7 @@ func (s *service) handleHellos(ctx context.Context) error { // Incorrect certificate name is something the user most // likely wants to know about, since it's an advanced // config. Warn instead of Info. - l.Warnf("Bad certificate from %s at %s: %v", remoteID, c, err) + slog.ErrorContext(ctx, "Bad certificate from remote", remoteID.LogAttr(), slogutil.Address(c.RemoteAddr()), slogutil.Error(err)) c.Close() continue } @@ -455,7 +437,7 @@ func (s *service) handleHellos(ctx context.Context) error { s.dialNowDevicesMut.Unlock() }() - l.Infof("Established secure connection to %s at %s", remoteID.Short(), c) + slog.InfoContext(ctx, "Established secure connection", remoteID.LogAttr(), slog.Any("connection", c)) s.model.AddConnection(protoConn, hello) continue @@ -477,9 +459,9 @@ func (s *service) connect(ctx context.Context) error { bestDialerPriority := s.bestDialerPriority(cfg) isInitialRampup := initialRampup < stdConnectionLoopSleep - l.Debugln("Connection loop") + slog.DebugContext(ctx, "Connection loop") if isInitialRampup { - l.Debugln("Connection loop in initial rampup") + slog.DebugContext(ctx, "Connection loop in initial rampup") } // Used for consistency throughout this loop run, as time passes @@ -616,9 +598,9 @@ func (s *service) dialDevices(ctx context.Context, now time.Time, cfg config.Con // Perform dials according to the queue, stopping when we've reached the // allowed additional number of connections (if limited). numConns := 0 - var numConnsMut stdsync.Mutex + var numConnsMut sync.Mutex dialSemaphore := semaphore.New(dialMaxParallel) - dialWG := new(stdsync.WaitGroup) + dialWG := new(sync.WaitGroup) dialCtx, dialCancel := context.WithCancel(ctx) defer func() { dialWG.Wait() @@ -675,7 +657,7 @@ func (s *service) resolveDialTargets(ctx context.Context, now time.Time, cfg con uri, err := url.Parse(addr) if err != nil { s.setConnectionStatus(addr, err) - l.Infof("Parsing dialer address %s: %v", addr, err) + slog.WarnContext(ctx, "Failed to parse dialer address", slogutil.Address(addr), slogutil.Error(err)) continue } @@ -695,7 +677,7 @@ func (s *service) resolveDialTargets(ctx context.Context, now time.Time, cfg con l.Debugf("Dialer for %v: %v", uri, err) continue } else if err != nil { - l.Infof("Dialer for %v: %v", uri, err) + slog.WarnContext(ctx, "Failed to get dialer", slogutil.URI(uri), slogutil.Error(err)) continue } @@ -711,7 +693,7 @@ func (s *service) resolveDialTargets(ctx context.Context, now time.Time, cfg con continue } if currentConns >= s.desiredConnectionsToDevice(deviceCfg.DeviceID) && priority == priorityCutoff { - l.Debugf("Not dialing %s at %s using %s as priority is equal and we already have %d/%d connections", deviceID.Short(), addr, dialerFactory, currentConns, deviceCfg.NumConnections) + l.Debugf("Not dialing %s at %s using %s as priority is equal and we already have %d/%d connections", deviceID.Short(), addr, dialerFactory, currentConns, deviceCfg.NumConnections()) continue } @@ -818,7 +800,7 @@ func (s *lanChecker) isLAN(addr net.Addr) bool { func (s *service) createListener(factory listenerFactory, uri *url.URL) bool { // must be called with listenerMut held - l.Debugln("Starting listener", uri) + slog.Debug("Starting listener", "uri", uri) listener := factory.New(uri, s.cfg, s.tlsCfg, s.conns, s.natService, s.registry, s.lanChecker) listener.OnAddressesChanged(s.logListenAddressesChangedEvent) @@ -826,7 +808,7 @@ func (s *service) createListener(factory listenerFactory, uri *url.URL) bool { // Retrying a listener many times in rapid succession is unlikely to help, // thus back off quickly. A listener may soon be functional again, e.g. due // to a network interface coming back online - retry every minute. - spec := svcutil.SpecWithInfoLogger(l) + spec := svcutil.SpecWithInfoLogger() spec.FailureThreshold = 2 spec.FailureBackoff = time.Minute sup := suture.New(fmt.Sprintf("listenerSupervisor@%v", listener), spec) @@ -854,9 +836,6 @@ func (s *service) CommitConfiguration(from, to config.Configuration) bool { for _, dev := range from.Devices { if !newDevices[dev.DeviceID] { - warningLimitersMut.Lock() - delete(warningLimiters, dev.DeviceID) - warningLimitersMut.Unlock() metricDeviceActiveConnections.DeleteLabelValues(dev.DeviceID.String()) } } @@ -875,7 +854,7 @@ func (s *service) CommitConfiguration(from, to config.Configuration) bool { uri, err := url.Parse(addr) if err != nil { - l.Warnf("Skipping malformed listener URL %q: %v", addr, err) + slog.Error("Skipping malformed listener URL", slogutil.URI(addr), slogutil.Error(err)) continue } @@ -886,7 +865,7 @@ func (s *service) CommitConfiguration(from, to config.Configuration) bool { // mean something entirely different to the computer (e.g., // tcp:/127.0.0.1:22000 in fact being equivalent to tcp://:22000). if canonical := uri.String(); canonical != addr { - l.Warnf("Skipping malformed listener URL %q (not canonical)", addr) + slog.Error("Skipping malformed listener URL (not canonical)", slogutil.URI(addr)) continue } @@ -900,7 +879,7 @@ func (s *service) CommitConfiguration(from, to config.Configuration) bool { l.Debugf("Listener for %v: %v", uri, err) continue } else if err != nil { - l.Infof("Listener for %v: %v", uri, err) + slog.Warn("Failed to get listener", slogutil.URI(uri), slogutil.Error(err)) continue } @@ -1007,8 +986,7 @@ type connectionStatusHandler struct { func newConnectionStatusHandler() connectionStatusHandler { return connectionStatusHandler{ - connectionStatusMut: sync.NewRWMutex(), - connectionStatus: make(map[string]ConnectionStatusEntry), + connectionStatus: make(map[string]ConnectionStatusEntry), } } @@ -1082,24 +1060,6 @@ func urlsToStrings(urls []*url.URL) []string { return strings } -var ( - warningLimiters = make(map[protocol.DeviceID]*rate.Limiter) - warningLimitersMut = sync.NewMutex() -) - -func warningFor(dev protocol.DeviceID, msg string) { - warningLimitersMut.Lock() - defer warningLimitersMut.Unlock() - lim, ok := warningLimiters[dev] - if !ok { - lim = rate.NewLimiter(rate.Every(perDeviceWarningIntv), 1) - warningLimiters[dev] = lim - } - if lim.Allow() { - l.Warnln(msg) - } -} - func tlsTimedHandshake(tc *tls.Conn) error { tc.SetDeadline(time.Now().Add(tlsHandshakeTimeout)) defer tc.SetDeadline(time.Time{}) @@ -1156,7 +1116,7 @@ func (s *service) dialParallel(ctx context.Context, deviceID protocol.DeviceID, for _, prio := range priorities { tgts := dialTargetBuckets[prio] res := make(chan internalConn, len(tgts)) - wg := stdsync.WaitGroup{} + wg := sync.WaitGroup{} for _, tgt := range tgts { sema.Take(1) wg.Add(1) @@ -1215,7 +1175,7 @@ func (s *service) validateIdentity(c internalConn, expectedID protocol.DeviceID) // connection. certs := cs.PeerCertificates if cl := len(certs); cl != 1 { - l.Infof("Got peer certificate list of length %d != 1 from peer at %s; protocol error", cl, c) + slog.Warn("Got peer certificate list of incorrect length", slog.Int("length", cl), slogutil.Address(c.RemoteAddr())) c.Close() return fmt.Errorf("expected 1 certificate, got %d", cl) } @@ -1364,7 +1324,7 @@ func (s *service) desiredConnectionsToDevice(deviceID protocol.DeviceID) int { // connected to and how many connections we have to each device. It also // tracks how many connections they are willing to use. type deviceConnectionTracker struct { - connectionsMut stdsync.Mutex + connectionsMut sync.Mutex connections map[protocol.DeviceID][]protocol.Connection // current connections wantConnections map[protocol.DeviceID]int // number of connections they want } diff --git a/lib/connections/structs.go b/lib/connections/structs.go index b9f3dc65c..c40e3b091 100644 --- a/lib/connections/structs.go +++ b/lib/connections/structs.go @@ -11,6 +11,7 @@ import ( "crypto/tls" "fmt" "io" + "log/slog" "net" "net/url" "time" @@ -152,6 +153,10 @@ func (c internalConn) String() string { return fmt.Sprintf("%s-%s/%s/%s/%s-P%d-%s", c.LocalAddr(), c.RemoteAddr(), c.Type(), c.Crypto(), t, c.Priority(), c.connectionID) } +func (c internalConn) LogValue() slog.Value { + return slog.GroupValue(slog.String("local", c.LocalAddr().String()), slog.String("remote", c.RemoteAddr().String()), slog.String("type", c.Type()), slog.Bool("lan", c.isLocal), slog.String("crypto", c.Crypto()), slog.Int("prio", c.priority), slog.String("id", c.ConnectionID())) +} + type dialerFactory interface { New(config.OptionsConfiguration, *tls.Config, *registry.Registry, *lanChecker) genericDialer AlwaysWAN() bool diff --git a/lib/connections/tcp_listen.go b/lib/connections/tcp_listen.go index d487b5f20..2944ca499 100644 --- a/lib/connections/tcp_listen.go +++ b/lib/connections/tcp_listen.go @@ -9,11 +9,14 @@ package connections import ( "context" "crypto/tls" + "errors" + "log/slog" "net" "net/url" "sync" "time" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/connections/registry" "github.com/syncthing/syncthing/lib/dialer" @@ -50,7 +53,7 @@ type tcpListener struct { func (t *tcpListener) serve(ctx context.Context) error { tcaddr, err := net.ResolveTCPAddr(t.uri.Scheme, t.uri.Host) if err != nil { - l.Infoln("Listen (BEP/tcp):", err) + slog.WarnContext(ctx, "Failed to listen (TCP)", slogutil.Error(err)) return err } @@ -60,7 +63,7 @@ func (t *tcpListener) serve(ctx context.Context) error { listener, err := lc.Listen(context.TODO(), t.uri.Scheme, tcaddr.String()) if err != nil { - l.Infoln("Listen (BEP/tcp):", err) + slog.WarnContext(ctx, "Failed to listen (TCP)", slogutil.Error(err)) return err } defer listener.Close() @@ -74,8 +77,8 @@ func (t *tcpListener) serve(ctx context.Context) error { t.registry.Register(t.uri.Scheme, tcaddr) defer t.registry.Unregister(t.uri.Scheme, tcaddr) - l.Infof("TCP listener (%v) starting", tcaddr) - defer l.Infof("TCP listener (%v) shutting down", tcaddr) + slog.InfoContext(ctx, "TCP listener starting", slogutil.Address(tcaddr)) + defer slog.InfoContext(ctx, "TCP listener shutting down", slogutil.Address(tcaddr)) var ipVersion nat.IPVersion if t.uri.Scheme == "tcp4" { @@ -121,8 +124,9 @@ func (t *tcpListener) serve(ctx context.Context) error { default: } if err != nil { - if err, ok := err.(*net.OpError); !ok || !err.Timeout() { - l.Warnln("Listen (BEP/tcp): Accepting connection:", err) + var ne *net.OpError + if ok := errors.As(err, &ne); !ok || !ne.Timeout() { + slog.WarnContext(ctx, "Failed to accept TCP connection", slogutil.Error(err)) acceptFailures++ if acceptFailures > maxAcceptFailures { @@ -152,7 +156,7 @@ func (t *tcpListener) serve(ctx context.Context) error { tc := tls.Server(conn, t.tlsCfg) if err := tlsTimedHandshake(tc); err != nil { - l.Infoln("Listen (BEP/tcp): TLS handshake:", err) + slog.WarnContext(ctx, "Failed TLS handshake", slogutil.Address(tc.RemoteAddr()), slogutil.Error(err)) tc.Close() continue } diff --git a/lib/dialer/control_unix.go b/lib/dialer/control_unix.go index f5060cb09..dc50a52af 100644 --- a/lib/dialer/control_unix.go +++ b/lib/dialer/control_unix.go @@ -10,6 +10,7 @@ package dialer import ( + "log/slog" "syscall" "golang.org/x/sys/unix" @@ -28,11 +29,11 @@ func init() { err = unix.SetsockoptInt(fd, unix.SOL_SOCKET, unix.SO_REUSEPORT, 1) switch { case err == unix.ENOPROTOOPT || err == unix.EINVAL: - l.Debugln("SO_REUSEPORT not supported") + slog.Debug("SO_REUSEPORT not supported") case err != nil: l.Debugln("Unknown error when determining SO_REUSEPORT support", err) default: - l.Debugln("SO_REUSEPORT supported") + slog.Debug("SO_REUSEPORT supported") SupportsReusePort = true } } diff --git a/lib/dialer/debug.go b/lib/dialer/debug.go index 9891be4af..bbac5e32d 100644 --- a/lib/dialer/debug.go +++ b/lib/dialer/debug.go @@ -7,17 +7,7 @@ package dialer import ( - "os" - "strings" - - "github.com/syncthing/syncthing/lib/logger" + "github.com/syncthing/syncthing/internal/slogutil" ) -var ( - l = logger.DefaultLogger.NewFacility("dialer", "Dialing connections") - // To run before init() of other files that log on init. - _ = func() error { - l.SetDebug("dialer", strings.Contains(os.Getenv("STTRACE"), "dialer") || os.Getenv("STTRACE") == "all") - return nil - }() -) +var l = slogutil.NewAdapter("Dialing connections") diff --git a/lib/dialer/internal.go b/lib/dialer/internal.go index c810359ef..b94d91fc7 100644 --- a/lib/dialer/internal.go +++ b/lib/dialer/internal.go @@ -7,6 +7,7 @@ package dialer import ( + "log/slog" "net" "net/http" "net/url" @@ -31,15 +32,15 @@ func init() { // Defer this, so that logging gets set up. go func() { time.Sleep(500 * time.Millisecond) - l.Infoln("Proxy settings detected") + slog.Info("Proxy settings detected") if noFallback { - l.Infoln("Proxy fallback disabled") + slog.Info("Proxy fallback disabled") } }() } else { go func() { time.Sleep(500 * time.Millisecond) - l.Debugln("Dialer logging disabled, as no proxy was detected") + slog.Debug("Dialer logging disabled, as no proxy was detected") }() } } diff --git a/lib/discover/cache.go b/lib/discover/cache.go index 01727382e..5a1a50a15 100644 --- a/lib/discover/cache.go +++ b/lib/discover/cache.go @@ -7,7 +7,7 @@ package discover import ( - stdsync "sync" + "sync" "time" "github.com/syncthing/syncthing/lib/protocol" @@ -34,7 +34,7 @@ type cachedError interface { type cache struct { entries map[protocol.DeviceID]CacheEntry - mut stdsync.Mutex + mut sync.Mutex } func newCache() *cache { diff --git a/lib/discover/debug.go b/lib/discover/debug.go index 45610076c..5f761baff 100644 --- a/lib/discover/debug.go +++ b/lib/discover/debug.go @@ -6,8 +6,6 @@ package discover -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("discover", "Remote device discovery") +var l = slogutil.NewAdapter("Remote device discovery") diff --git a/lib/discover/global.go b/lib/discover/global.go index 94d4994ee..d24b11717 100644 --- a/lib/discover/global.go +++ b/lib/discover/global.go @@ -14,15 +14,17 @@ import ( "errors" "fmt" "io" + "log/slog" "net" "net/http" "net/url" "strconv" - stdsync "sync" + "sync" "time" "golang.org/x/net/http2" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/connections/registry" "github.com/syncthing/syncthing/lib/dialer" "github.com/syncthing/syncthing/lib/events" @@ -181,12 +183,12 @@ func (c *globalClient) Lookup(ctx context.Context, device protocol.DeviceID) (ad resp, err := c.queryClient.Get(ctx, qURL.String()) if err != nil { - l.Debugln("globalClient.Lookup", qURL, err) + slog.DebugContext(ctx, "globalClient.Lookup", "url", qURL, slogutil.Error(err)) return nil, err } if resp.StatusCode != http.StatusOK { resp.Body.Close() - l.Debugln("globalClient.Lookup", qURL, resp.Status) + slog.DebugContext(ctx, "globalClient.Lookup", "url", qURL, "status", resp.Status) err := errors.New(resp.Status) if secs, atoiErr := strconv.Atoi(resp.Header.Get("Retry-After")); atoiErr == nil && secs > 0 { err = &lookupError{ @@ -238,7 +240,7 @@ func (c *globalClient) Serve(ctx context.Context) error { } else if timerResetCount == maxAddressChangesBetweenAnnouncements { // Yet only do it if we haven't had to reset maxAddressChangesBetweenAnnouncements times in a row, // so if something is flip-flopping within 2 seconds, we don't end up in a permanent reset loop. - l.Warnf("Detected a flip-flopping listener") + slog.ErrorContext(ctx, "Detected a flip-flopping listener", slog.String("server", c.server)) c.setError(errors.New("flip flopping listener")) // Incrementing the count above 10 will prevent us from warning or setting the error again // It will also suppress event based resets until we've had a proper round after announceErrorRetryInterval @@ -273,27 +275,27 @@ func (c *globalClient) sendAnnouncement(ctx context.Context, timer *time.Timer) // The marshal doesn't fail, I promise. postData, _ := json.Marshal(ann) - l.Debugf("%s Announcement: %v", c, ann) + slog.DebugContext(ctx, "send announcement", "server", c.server, "announcement", ann) resp, err := c.announceClient.Post(ctx, c.server, "application/json", bytes.NewReader(postData)) if err != nil { - l.Debugln(c, "announce POST:", err) + slog.DebugContext(ctx, "announce POST", "server", c.server, slogutil.Error(err)) c.setError(err) timer.Reset(announceErrorRetryInterval) return } - l.Debugln(c, "announce POST:", resp.Status) + slog.DebugContext(ctx, "announce POST", "server", c.server, "status", resp.Status) resp.Body.Close() if resp.StatusCode < 200 || resp.StatusCode > 299 { - l.Debugln(c, "announce POST:", resp.Status) + slog.DebugContext(ctx, "announce POST", "server", c.server, "status", resp.Status) c.setError(errors.New(resp.Status)) if h := resp.Header.Get("Retry-After"); h != "" { // The server has a recommendation on when we should // retry. Follow it. if secs, err := strconv.Atoi(h); err == nil && secs > 0 { - l.Debugln(c, "announce Retry-After:", secs, err) + slog.DebugContext(ctx, "server sets retry-after", "server", c.server, "seconds", secs) timer.Reset(time.Duration(secs) * time.Second) return } @@ -309,7 +311,7 @@ func (c *globalClient) sendAnnouncement(ctx context.Context, timer *time.Timer) // The server has a recommendation on when we should // reannounce. Follow it. if secs, err := strconv.Atoi(h); err == nil && secs > 0 { - l.Debugln(c, "announce Reannounce-After:", secs, err) + slog.DebugContext(ctx, "announce sets reannounce-after", "server", c.server, "seconds", secs) timer.Reset(time.Duration(secs) * time.Second) return } @@ -424,7 +426,7 @@ func (c *idCheckingHTTPClient) Post(ctx context.Context, url, ctype string, data type errorHolder struct { err error - mut stdsync.Mutex // uses stdlib sync as I want this to be trivially embeddable, and there is no risk of blocking + mut sync.Mutex // uses stdlib sync as I want this to be trivially embeddable, and there is no risk of blocking } func (e *errorHolder) setError(err error) { diff --git a/lib/discover/local.go b/lib/discover/local.go index 483681ebb..eaf0bf5e6 100644 --- a/lib/discover/local.go +++ b/lib/discover/local.go @@ -14,6 +14,7 @@ import ( "errors" "fmt" "io" + "log/slog" "net" "net/url" "strconv" @@ -23,6 +24,7 @@ import ( "google.golang.org/protobuf/proto" "github.com/syncthing/syncthing/internal/gen/discoproto" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/beacon" "github.com/syncthing/syncthing/lib/events" "github.com/syncthing/syncthing/lib/protocol" @@ -54,7 +56,7 @@ const ( func NewLocal(id protocol.DeviceID, addr string, addrList AddressLister, evLogger events.Logger) (FinderService, error) { c := &localClient{ - Supervisor: suture.New("local", svcutil.SpecWithDebugLogger(l)), + Supervisor: suture.New("local", svcutil.SpecWithDebugLogger()), myID: id, addrList: addrList, evLogger: evLogger, @@ -176,7 +178,7 @@ func (c *localClient) recvAnnouncements(ctx context.Context) error { continue } if len(buf) < 4 { - l.Debugf("discover: short packet from %s", addr.String()) + slog.DebugContext(ctx, "received short packet", "address", addr.String()) continue } @@ -188,25 +190,25 @@ func (c *localClient) recvAnnouncements(ctx context.Context) error { case v13Magic: // Old version if !warnedAbout[addr.String()] { - l.Warnf("Incompatible (v0.13) local discovery packet from %v - upgrade that device to connect", addr) + slog.ErrorContext(ctx, "Incompatible (v0.13) local discovery packet - upgrade that device to connect", slogutil.Address(addr)) warnedAbout[addr.String()] = true } continue default: - l.Debugf("discover: Incorrect magic %x from %s", magic, addr) + slog.DebugContext(ctx, "Incorrect magic", "magic", magic, "address", addr) continue } var pkt discoproto.Announce err := proto.Unmarshal(buf[4:], &pkt) if err != nil && !errors.Is(err, io.EOF) { - l.Debugf("discover: Failed to unmarshal local announcement from %s (%s):\n%s", addr, err, hex.Dump(buf[4:])) + slog.DebugContext(ctx, "Failed to unmarshal local announcement", "address", addr, slogutil.Error(err), "packet", hex.Dump(buf[4:])) continue } id, _ := protocol.DeviceIDFromBytes(pkt.Id) - l.Debugf("discover: Received local announcement from %s for %s", addr, id) + slog.DebugContext(ctx, "Received local announcement", "address", addr, "device", id) var newDevice bool if !bytes.Equal(pkt.Id, c.myID[:]) { diff --git a/lib/discover/manager.go b/lib/discover/manager.go index 4c12fbf31..bf0025eb1 100644 --- a/lib/discover/manager.go +++ b/lib/discover/manager.go @@ -13,18 +13,20 @@ import ( "context" "crypto/tls" "fmt" + "log/slog" "slices" + "sync" "time" "github.com/thejerf/suture/v4" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/connections/registry" "github.com/syncthing/syncthing/lib/events" "github.com/syncthing/syncthing/lib/protocol" "github.com/syncthing/syncthing/lib/stringutil" "github.com/syncthing/syncthing/lib/svcutil" - "github.com/syncthing/syncthing/lib/sync" ) // The Manager aggregates results from multiple Finders. Each Finder has @@ -53,7 +55,7 @@ type manager struct { func NewManager(myID protocol.DeviceID, cfg config.Wrapper, cert tls.Certificate, evLogger events.Logger, lister AddressLister, registry *registry.Registry) Manager { m := &manager{ - Supervisor: suture.New("discover.Manager", svcutil.SpecWithDebugLogger(l)), + Supervisor: suture.New("discover.Manager", svcutil.SpecWithDebugLogger()), myID: myID, cfg: cfg, cert: cert, @@ -62,7 +64,6 @@ func NewManager(myID protocol.DeviceID, cfg config.Wrapper, cert tls.Certificate registry: registry, finders: make(map[string]cachedFinder), - mut: sync.NewRWMutex(), } m.Add(svcutil.AsService(m.serve, m.String())) return m @@ -89,7 +90,7 @@ func (m *manager) addLocked(identity string, finder Finder, cacheTime, negCacheT entry.token = &token } m.finders[identity] = entry - l.Infoln("Using discovery mechanism:", identity) + slog.Info("Using discovery mechanism", "identity", identity) } func (m *manager) removeLocked(identity string) { @@ -100,11 +101,11 @@ func (m *manager) removeLocked(identity string) { if entry.token != nil { err := m.Supervisor.Remove(*entry.token) if err != nil { - l.Warnf("removing discovery %s: %s", identity, err) + slog.Warn("Failed to remove discovery mechanism", slog.String("identity", identity), slogutil.Error(err)) } } delete(m.finders, identity) - l.Infoln("Stopped using discovery mechanism: ", identity) + slog.Info("Stopped using discovery mechanism", "identity", identity) } // Lookup attempts to resolve the device ID using any of the added Finders, @@ -117,8 +118,7 @@ func (m *manager) Lookup(ctx context.Context, deviceID protocol.DeviceID) (addre if cacheEntry.found && time.Since(cacheEntry.when) < finder.cacheTime { // It's a positive, valid entry. Use it. - l.Debugln("cached discovery entry for", deviceID, "at", finder) - l.Debugln(" cache:", cacheEntry) + slog.DebugContext(ctx, "Found cached discovery entry", "device", deviceID, "finder", finder, "entry", cacheEntry) addresses = append(addresses, cacheEntry.Addresses...) continue } @@ -127,7 +127,7 @@ func (m *manager) Lookup(ctx context.Context, deviceID protocol.DeviceID) (addre if !cacheEntry.found && valid { // It's a negative, valid entry. We should not make another // attempt right now. - l.Debugln("negative cache entry for", deviceID, "at", finder, "valid until", cacheEntry.when.Add(finder.negCacheTime), "or", cacheEntry.validUntil) + slog.DebugContext(ctx, "Negative cache entry", "device", deviceID, "finder", finder, "until1", cacheEntry.when.Add(finder.negCacheTime), "until2", cacheEntry.validUntil) continue } @@ -136,8 +136,7 @@ func (m *manager) Lookup(ctx context.Context, deviceID protocol.DeviceID) (addre // Perform the actual lookup and cache the result. if addrs, err := finder.Lookup(ctx, deviceID); err == nil { - l.Debugln("lookup for", deviceID, "at", finder) - l.Debugln(" addresses:", addrs) + slog.DebugContext(ctx, "Got finder result", "device", deviceID, "finder", finder, "address", addrs) addresses = append(addresses, addrs...) finder.cache.Set(deviceID, CacheEntry{ Addresses: addrs, @@ -161,8 +160,7 @@ func (m *manager) Lookup(ctx context.Context, deviceID protocol.DeviceID) (addre addresses = stringutil.UniqueTrimmedStrings(addresses) slices.Sort(addresses) - l.Debugln("lookup results for", deviceID) - l.Debugln(" addresses: ", addresses) + slog.DebugContext(ctx, "Final lookup results", "device", deviceID, "addresses", addresses) return addresses, nil } @@ -262,7 +260,7 @@ func (m *manager) CommitConfiguration(_, to config.Configuration) (handled bool) } gd, err := NewGlobal(srv, m.cert, m.addressLister, m.evLogger, m.registry) if err != nil { - l.Warnln("Global discovery:", err) + slog.Warn("Failed to initialize global discovery", slogutil.Error(err)) continue } @@ -279,7 +277,7 @@ func (m *manager) CommitConfiguration(_, to config.Configuration) (handled bool) if _, ok := m.finders[v4Identity]; !ok { bcd, err := NewLocal(m.myID, fmt.Sprintf(":%d", to.Options.LocalAnnPort), m.addressLister, m.evLogger) if err != nil { - l.Warnln("IPv4 local discovery:", err) + slog.Warn("Failed to initialize IPv4 local discovery", slogutil.Error(err)) } else { m.addLocked(v4Identity, bcd, 0, 0) } @@ -290,7 +288,7 @@ func (m *manager) CommitConfiguration(_, to config.Configuration) (handled bool) if _, ok := m.finders[v6Identity]; !ok { mcd, err := NewLocal(m.myID, to.Options.LocalAnnMCAddr, m.addressLister, m.evLogger) if err != nil { - l.Warnln("IPv6 local discovery:", err) + slog.Warn("Failed to initialize IPv6 local discovery", slogutil.Error(err)) } else { m.addLocked(v6Identity, mcd, 0, 0) } diff --git a/lib/events/debug.go b/lib/events/debug.go index 905fcbc0a..92646fed3 100644 --- a/lib/events/debug.go +++ b/lib/events/debug.go @@ -7,7 +7,7 @@ package events import ( - liblogger "github.com/syncthing/syncthing/lib/logger" + "github.com/syncthing/syncthing/internal/slogutil" ) -var dl = liblogger.DefaultLogger.NewFacility("events", "Event generation and logging") +var dl = slogutil.NewAdapter("Event generation and logging") diff --git a/lib/events/events.go b/lib/events/events.go index a3c233761..1650098b1 100644 --- a/lib/events/events.go +++ b/lib/events/events.go @@ -16,11 +16,11 @@ import ( "errors" "fmt" "runtime" + "sync" "time" + "github.com/syncthing/syncthing/lib/syncutil" "github.com/thejerf/suture/v4" - - "github.com/syncthing/syncthing/lib/sync" ) type EventType int64 @@ -474,7 +474,7 @@ type bufferedSubscription struct { next int cur int // Current SubscriptionID mut sync.Mutex - cond *sync.TimeoutCond + cond *syncutil.TimeoutCond } type BufferedSubscription interface { @@ -486,9 +486,8 @@ func NewBufferedSubscription(s Subscription, size int) BufferedSubscription { bs := &bufferedSubscription{ sub: s, buf: make([]Event, size), - mut: sync.NewMutex(), } - bs.cond = sync.NewTimeoutCond(bs.mut) + bs.cond = syncutil.NewTimeoutCond(&bs.mut) go bs.pollingLoop() return bs } diff --git a/lib/fs/basicfs.go b/lib/fs/basicfs.go index 4ed60d10e..bf0ad56f8 100644 --- a/lib/fs/basicfs.go +++ b/lib/fs/basicfs.go @@ -9,6 +9,7 @@ package fs import ( "errors" "fmt" + "log/slog" "os" "os/user" "path/filepath" @@ -32,7 +33,7 @@ type OptionJunctionsAsDirs struct{} func (*OptionJunctionsAsDirs) apply(fs Filesystem) Filesystem { if basic, ok := fs.(*BasicFilesystem); !ok { - l.Warnln("WithJunctionsAsDirs must only be used with FilesystemTypeBasic") + slog.Warn("WithJunctionsAsDirs must only be used with FilesystemTypeBasic") } else { basic.junctionsAsDirs = true } diff --git a/lib/fs/casefs_test.go b/lib/fs/casefs_test.go index 10329b36a..b6c3431ea 100644 --- a/lib/fs/casefs_test.go +++ b/lib/fs/casefs_test.go @@ -300,12 +300,10 @@ func doubleWalkFSWithOtherOps(fsys Filesystem, paths []string, otherOpEvery int, if err := fsys.Walk("/", func(path string, info FileInfo, err error) error { i++ if otherOpEvery != 0 && i%otherOpEvery == 0 { - // l.Infoln("AAA", otherOpPath) if _, err := fsys.Lstat(otherOpPath); err != nil { return err } } - // l.Infoln("CCC", path) return err }); err != nil { return err @@ -316,11 +314,9 @@ func doubleWalkFSWithOtherOps(fsys Filesystem, paths []string, otherOpEvery int, i++ if otherOpEvery != 0 && i%otherOpEvery == 0 { if _, err := fsys.Lstat(otherOpPath); err != nil { - // l.Infoln("AAA", otherOpPath) return err } } - // l.Infoln("CCC", p) if _, err := fsys.Lstat(p); err != nil { return err } diff --git a/lib/fs/debug.go b/lib/fs/debug.go index 0b7970572..90ecb0365 100644 --- a/lib/fs/debug.go +++ b/lib/fs/debug.go @@ -7,14 +7,7 @@ package fs import ( - "github.com/syncthing/syncthing/lib/logger" + "github.com/syncthing/syncthing/internal/slogutil" ) -var l = logger.DefaultLogger.NewFacility("fs", "Filesystem access") - -func init() { - logger.DefaultLogger.NewFacility("walkfs", "Filesystem access while walking") - if logger.DefaultLogger.ShouldDebug("walkfs") { - l.SetDebug("fs", true) - } -} +var l = slogutil.NewAdapter("Filesystem access") diff --git a/lib/fs/fakefs.go b/lib/fs/fakefs.go index 3c056eab6..35d4235c3 100644 --- a/lib/fs/fakefs.go +++ b/lib/fs/fakefs.go @@ -376,6 +376,8 @@ func (fs *fakeFS) Lstat(name string) (FileInfo, error) { } info := &fakeFileInfo{*entry} + info.content = nil + info.children = nil if fs.insens { info.name = filepath.Base(name) } diff --git a/lib/fs/filesystem_copy_range.go b/lib/fs/filesystem_copy_range.go index 8749ad8af..f917729c5 100644 --- a/lib/fs/filesystem_copy_range.go +++ b/lib/fs/filesystem_copy_range.go @@ -7,14 +7,13 @@ package fs import ( + "sync" "syscall" - - "github.com/syncthing/syncthing/lib/sync" ) var ( copyRangeMethods = make(map[CopyRangeMethod]copyRangeImplementation) - mut = sync.NewMutex() + mut sync.Mutex ) type copyRangeImplementation func(src, dst File, srcOffset, dstOffset, size int64) error diff --git a/lib/ignore/ignore.go b/lib/ignore/ignore.go index 765c736e2..062153bfd 100644 --- a/lib/ignore/ignore.go +++ b/lib/ignore/ignore.go @@ -16,6 +16,7 @@ import ( "os" "path/filepath" "strings" + "sync" "time" "unicode/utf8" @@ -26,7 +27,6 @@ import ( "github.com/syncthing/syncthing/lib/fs" "github.com/syncthing/syncthing/lib/ignore/ignoreresult" "github.com/syncthing/syncthing/lib/osutil" - "github.com/syncthing/syncthing/lib/sync" ) const escapePrefix = "#escape" @@ -153,7 +153,6 @@ func New(fs fs.Filesystem, opts ...Option) *Matcher { m := &Matcher{ fs: fs, stop: make(chan struct{}), - mut: sync.NewMutex(), } for _, opt := range opts { opt(m) diff --git a/lib/logger/LICENSE b/lib/logger/LICENSE deleted file mode 100644 index fa5b4e205..000000000 --- a/lib/logger/LICENSE +++ /dev/null @@ -1,19 +0,0 @@ -Copyright (C) 2013 Jakob Borg - -Permission is hereby granted, free of charge, to any person obtaining a copy of -this software and associated documentation files (the "Software"), to deal in -the Software without restriction, including without limitation the rights to -use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies -of the Software, and to permit persons to whom the Software is furnished to do -so, subject to the following conditions: - -- The above copyright notice and this permission notice shall be included in - all copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. diff --git a/lib/logger/logger.go b/lib/logger/logger.go deleted file mode 100644 index 919a5fcf4..000000000 --- a/lib/logger/logger.go +++ /dev/null @@ -1,407 +0,0 @@ -// Copyright (C) 2014 Jakob Borg. All rights reserved. Use of this source code -// is governed by an MIT-style license that can be found in the LICENSE file. - -//go:generate -command counterfeiter go run github.com/maxbrunsfeld/counterfeiter/v6 -//go:generate counterfeiter -o mocks/logger.go --fake-name Recorder . Recorder - -// Package logger implements a standardized logger with callback functionality -package logger - -import ( - "fmt" - "io" - "log" - "os" - "slices" - "strings" - "sync" - "time" -) - -// This package uses stdlib sync as it may be used to debug syncthing/lib/sync -// and that would cause an implosion of the universe. - -type LogLevel int - -const ( - LevelDebug LogLevel = iota - LevelVerbose - LevelInfo - LevelWarn - NumLevels -) - -const ( - DefaultFlags = log.Ltime | log.Ldate - DebugFlags = log.Ltime | log.Ldate | log.Lmicroseconds | log.Lshortfile -) - -// A MessageHandler is called with the log level and message text. -type MessageHandler func(l LogLevel, msg string) - -type Logger interface { - AddHandler(level LogLevel, h MessageHandler) - SetFlags(flag int) - SetPrefix(prefix string) - Debugln(vals ...interface{}) - Debugf(format string, vals ...interface{}) - Verboseln(vals ...interface{}) - Verbosef(format string, vals ...interface{}) - Infoln(vals ...interface{}) - Infof(format string, vals ...interface{}) - Warnln(vals ...interface{}) - Warnf(format string, vals ...interface{}) - ShouldDebug(facility string) bool - SetDebug(facility string, enabled bool) - Facilities() map[string]string - FacilityDebugging() []string - NewFacility(facility, description string) Logger -} - -type logger struct { - logger *log.Logger - handlers [NumLevels][]MessageHandler - facilities map[string]string // facility name => description - debug map[string]struct{} // only facility names with debugging enabled - traces []string - mut sync.Mutex -} - -// DefaultLogger logs to standard output with a time prefix. -var DefaultLogger = New() - -func New() Logger { - if os.Getenv("LOGGER_DISCARD") != "" { - // Hack to completely disable logging, for example when running - // benchmarks. - return newLogger(io.Discard) - } - return newLogger(controlStripper{os.Stdout}) -} - -func newLogger(w io.Writer) Logger { - traces := strings.FieldsFunc(os.Getenv("STTRACE"), func(r rune) bool { - return strings.ContainsRune(",; ", r) - }) - - if len(traces) > 0 { - if slices.Contains(traces, "all") { - traces = []string{"all"} - } else { - slices.Sort(traces) - } - } - - return &logger{ - logger: log.New(w, "", DefaultFlags), - traces: traces, - facilities: make(map[string]string), - debug: make(map[string]struct{}), - } -} - -// AddHandler registers a new MessageHandler to receive messages with the -// specified log level or above. -func (l *logger) AddHandler(level LogLevel, h MessageHandler) { - l.mut.Lock() - defer l.mut.Unlock() - l.handlers[level] = append(l.handlers[level], h) -} - -// See log.SetFlags -func (l *logger) SetFlags(flag int) { - l.logger.SetFlags(flag) -} - -// See log.SetPrefix -func (l *logger) SetPrefix(prefix string) { - l.logger.SetPrefix(prefix) -} - -func (l *logger) callHandlers(level LogLevel, s string) { - for ll := LevelDebug; ll <= level; ll++ { - for _, h := range l.handlers[ll] { - h(level, strings.TrimSpace(s)) - } - } -} - -// Debugln logs a line with a DEBUG prefix. -func (l *logger) Debugln(vals ...interface{}) { - l.debugln(3, vals...) -} - -func (l *logger) debugln(level int, vals ...interface{}) { - s := fmt.Sprintln(vals...) - l.mut.Lock() - defer l.mut.Unlock() - l.logger.Output(level, "DEBUG: "+s) - l.callHandlers(LevelDebug, s) -} - -// Debugf logs a formatted line with a DEBUG prefix. -func (l *logger) Debugf(format string, vals ...interface{}) { - l.debugf(3, format, vals...) -} - -func (l *logger) debugf(level int, format string, vals ...interface{}) { - s := fmt.Sprintf(format, vals...) - l.mut.Lock() - defer l.mut.Unlock() - l.logger.Output(level, "DEBUG: "+s) - l.callHandlers(LevelDebug, s) -} - -// Infoln logs a line with a VERBOSE prefix. -func (l *logger) Verboseln(vals ...interface{}) { - s := fmt.Sprintln(vals...) - l.mut.Lock() - defer l.mut.Unlock() - l.logger.Output(2, "VERBOSE: "+s) - l.callHandlers(LevelVerbose, s) -} - -// Infof logs a formatted line with a VERBOSE prefix. -func (l *logger) Verbosef(format string, vals ...interface{}) { - s := fmt.Sprintf(format, vals...) - l.mut.Lock() - defer l.mut.Unlock() - l.logger.Output(2, "VERBOSE: "+s) - l.callHandlers(LevelVerbose, s) -} - -// Infoln logs a line with an INFO prefix. -func (l *logger) Infoln(vals ...interface{}) { - s := fmt.Sprintln(vals...) - l.mut.Lock() - defer l.mut.Unlock() - l.logger.Output(2, "INFO: "+s) - l.callHandlers(LevelInfo, s) -} - -// Infof logs a formatted line with an INFO prefix. -func (l *logger) Infof(format string, vals ...interface{}) { - s := fmt.Sprintf(format, vals...) - l.mut.Lock() - defer l.mut.Unlock() - l.logger.Output(2, "INFO: "+s) - l.callHandlers(LevelInfo, s) -} - -// Warnln logs a formatted line with a WARNING prefix. -func (l *logger) Warnln(vals ...interface{}) { - s := fmt.Sprintln(vals...) - l.mut.Lock() - defer l.mut.Unlock() - l.logger.Output(2, "WARNING: "+s) - l.callHandlers(LevelWarn, s) -} - -// Warnf logs a formatted line with a WARNING prefix. -func (l *logger) Warnf(format string, vals ...interface{}) { - s := fmt.Sprintf(format, vals...) - l.mut.Lock() - defer l.mut.Unlock() - l.logger.Output(2, "WARNING: "+s) - l.callHandlers(LevelWarn, s) -} - -// ShouldDebug returns true if the given facility has debugging enabled. -func (l *logger) ShouldDebug(facility string) bool { - l.mut.Lock() - _, res := l.debug[facility] - l.mut.Unlock() - return res -} - -// SetDebug enabled or disables debugging for the given facility name. -func (l *logger) SetDebug(facility string, enabled bool) { - l.mut.Lock() - defer l.mut.Unlock() - if _, ok := l.debug[facility]; enabled && !ok { - l.SetFlags(DebugFlags) - l.debug[facility] = struct{}{} - } else if !enabled && ok { - delete(l.debug, facility) - if len(l.debug) == 0 { - l.SetFlags(DefaultFlags) - } - } -} - -// isTraced returns whether the facility name is contained in STTRACE. -func (l *logger) isTraced(facility string) bool { - if len(l.traces) > 0 { - if l.traces[0] == "all" { - return true - } - - _, found := slices.BinarySearch(l.traces, facility) - return found - } - - return false -} - -// FacilityDebugging returns the set of facilities that have debugging -// enabled. -func (l *logger) FacilityDebugging() []string { - enabled := make([]string, 0, len(l.debug)) - l.mut.Lock() - for facility := range l.debug { - enabled = append(enabled, facility) - } - l.mut.Unlock() - return enabled -} - -// Facilities returns the currently known set of facilities and their -// descriptions. -func (l *logger) Facilities() map[string]string { - l.mut.Lock() - res := make(map[string]string, len(l.facilities)) - for facility, descr := range l.facilities { - res[facility] = descr - } - l.mut.Unlock() - return res -} - -// NewFacility returns a new logger bound to the named facility. -func (l *logger) NewFacility(facility, description string) Logger { - l.SetDebug(facility, l.isTraced(facility)) - - l.mut.Lock() - l.facilities[facility] = description - l.mut.Unlock() - - return &facilityLogger{ - logger: l, - facility: facility, - } -} - -// A facilityLogger is a regular logger but bound to a facility name. The -// Debugln and Debugf methods are no-ops unless debugging has been enabled for -// this facility on the parent logger. -type facilityLogger struct { - *logger - facility string -} - -// Debugln logs a line with a DEBUG prefix. -func (l *facilityLogger) Debugln(vals ...interface{}) { - if !l.ShouldDebug(l.facility) { - return - } - l.logger.debugln(3, vals...) -} - -// Debugf logs a formatted line with a DEBUG prefix. -func (l *facilityLogger) Debugf(format string, vals ...interface{}) { - if !l.ShouldDebug(l.facility) { - return - } - l.logger.debugf(3, format, vals...) -} - -// A Recorder keeps a size limited record of log events. -type Recorder interface { - Since(t time.Time) []Line - Clear() -} - -type recorder struct { - lines []Line - initial int - mut sync.Mutex -} - -// A Line represents a single log entry. -type Line struct { - When time.Time `json:"when"` - Message string `json:"message"` - Level LogLevel `json:"level"` -} - -func NewRecorder(l Logger, level LogLevel, size, initial int) Recorder { - r := &recorder{ - lines: make([]Line, 0, size), - initial: initial, - } - l.AddHandler(level, r.append) - return r -} - -func (r *recorder) Since(t time.Time) []Line { - r.mut.Lock() - defer r.mut.Unlock() - - res := r.lines - - for i := 0; i < len(res); i++ { - if res[i].When.After(t) { - // We must copy the result as r.lines can be mutated as soon as the lock - // is released. - res = res[i:] - cp := make([]Line, len(res)) - copy(cp, res) - return cp - } - } - return nil -} - -func (r *recorder) Clear() { - r.mut.Lock() - r.lines = r.lines[:0] - r.mut.Unlock() -} - -func (r *recorder) append(l LogLevel, msg string) { - line := Line{ - When: time.Now(), // intentionally high precision - Message: msg, - Level: l, - } - - r.mut.Lock() - defer r.mut.Unlock() - - if len(r.lines) == cap(r.lines) { - if r.initial > 0 { - // Shift all lines one step to the left, keeping the "initial" first intact. - copy(r.lines[r.initial+1:], r.lines[r.initial+2:]) - } else { - copy(r.lines, r.lines[1:]) - } - // Add the new one at the end - r.lines[len(r.lines)-1] = line - return - } - - r.lines = append(r.lines, line) - if len(r.lines) == r.initial { - r.lines = append(r.lines, Line{time.Now(), "...", l}) - } -} - -// controlStripper is a Writer that replaces control characters -// with spaces. -type controlStripper struct { - io.Writer -} - -func (s controlStripper) Write(data []byte) (int, error) { - for i, b := range data { - if b == '\n' || b == '\r' { - // Newlines are OK - continue - } - if b < 32 { - // Characters below 32 are control characters - data[i] = ' ' - } - } - return s.Writer.Write(data) -} diff --git a/lib/logger/logger_test.go b/lib/logger/logger_test.go deleted file mode 100644 index cc3b72e67..000000000 --- a/lib/logger/logger_test.go +++ /dev/null @@ -1,209 +0,0 @@ -// Copyright (C) 2014 Jakob Borg. All rights reserved. Use of this source code -// is governed by an MIT-style license that can be found in the LICENSE file. - -package logger - -import ( - "bytes" - "fmt" - "io" - "log" - "strings" - "testing" - "time" -) - -func TestAPI(t *testing.T) { - l := New() - l.SetFlags(0) - l.SetPrefix("testing") - - debug := 0 - l.AddHandler(LevelDebug, checkFunc(t, LevelDebug, &debug)) - info := 0 - l.AddHandler(LevelInfo, checkFunc(t, LevelInfo, &info)) - warn := 0 - l.AddHandler(LevelWarn, checkFunc(t, LevelWarn, &warn)) - - l.Debugf("test %d", 0) - l.Debugln("test", 0) - l.Infof("test %d", 1) - l.Infoln("test", 1) - l.Warnf("test %d", 3) - l.Warnln("test", 3) - - if debug != 6 { - t.Errorf("Debug handler called %d != 8 times", debug) - } - if info != 4 { - t.Errorf("Info handler called %d != 6 times", info) - } - if warn != 2 { - t.Errorf("Warn handler called %d != 2 times", warn) - } -} - -func checkFunc(t *testing.T, expectl LogLevel, counter *int) func(LogLevel, string) { - return func(l LogLevel, msg string) { - *counter++ - if l < expectl { - t.Errorf("Incorrect message level %d < %d", l, expectl) - } - } -} - -func TestFacilityDebugging(t *testing.T) { - l := New() - l.SetFlags(0) - - msgs := 0 - l.AddHandler(LevelDebug, func(l LogLevel, msg string) { - msgs++ - if strings.Contains(msg, "f1") { - t.Fatal("Should not get message for facility f1") - } - }) - - f0 := l.NewFacility("f0", "foo#0") - f1 := l.NewFacility("f1", "foo#1") - - l.SetDebug("f0", true) - l.SetDebug("f1", false) - - f0.Debugln("Debug line from f0") - f1.Debugln("Debug line from f1") - - if msgs != 1 { - t.Fatalf("Incorrect number of messages, %d != 1", msgs) - } -} - -func TestRecorder(t *testing.T) { - l := New() - l.SetFlags(0) - - // Keep the last five warnings or higher, no special initial handling. - r0 := NewRecorder(l, LevelWarn, 5, 0) - // Keep the last ten infos or higher, with the first three being permanent. - r1 := NewRecorder(l, LevelInfo, 10, 3) - - // Log a bunch of messages. - for i := 0; i < 15; i++ { - l.Debugf("Debug#%d", i) - l.Infof("Info#%d", i) - l.Warnf("Warn#%d", i) - } - - // r0 should contain the last five warnings - lines := r0.Since(time.Time{}) - if len(lines) != 5 { - t.Fatalf("Incorrect length %d != 5", len(lines)) - } - for i := 0; i < 5; i++ { - expected := fmt.Sprintf("Warn#%d", i+10) - if lines[i].Message != expected { - t.Error("Incorrect warning in r0:", lines[i].Message, "!=", expected) - } - } - - // r0 should contain: - // - The first three messages - // - A "..." marker - // - The last six messages - // (totalling ten) - lines = r1.Since(time.Time{}) - if len(lines) != 10 { - t.Fatalf("Incorrect length %d != 10", len(lines)) - } - expected := []string{ - "Info#0", - "Warn#0", - "Info#1", - "...", - "Info#12", - "Warn#12", - "Info#13", - "Warn#13", - "Info#14", - "Warn#14", - } - for i := 0; i < 10; i++ { - if lines[i].Message != expected[i] { - t.Error("Incorrect warning in r0:", lines[i].Message, "!=", expected[i]) - } - } - - // Check that since works - now := time.Now() - - time.Sleep(time.Millisecond) - - lines = r1.Since(now) - if len(lines) != 0 { - t.Error("unexpected lines") - } - - l.Infoln("hah") - - lines = r1.Since(now) - if len(lines) != 1 { - t.Fatalf("unexpected line count: %d", len(lines)) - } - if lines[0].Message != "hah" { - t.Errorf("incorrect line: %s", lines[0].Message) - } -} - -func TestStackLevel(t *testing.T) { - b := new(bytes.Buffer) - l := newLogger(b) - - l.SetFlags(log.Lshortfile) - l.Infoln("testing") - res := b.String() - - if !strings.Contains(res, "logger_test.go:") { - t.Logf("%q", res) - t.Error("Should identify this file as the source (bad level?)") - } -} - -func TestControlStripper(t *testing.T) { - b := new(bytes.Buffer) - l := newLogger(controlStripper{b}) - - l.Infoln("testing\x07testing\ntesting") - res := b.String() - - if !strings.Contains(res, "testing testing\ntesting") { - t.Logf("%q", res) - t.Error("Control character should become space") - } - if strings.Contains(res, "\x07") { - t.Logf("%q", res) - t.Error("Control character should be removed") - } -} - -func BenchmarkLog(b *testing.B) { - l := newLogger(controlStripper{io.Discard}) - benchmarkLogger(b, l) -} - -func BenchmarkLogNoStripper(b *testing.B) { - l := newLogger(io.Discard) - benchmarkLogger(b, l) -} - -func benchmarkLogger(b *testing.B, l Logger) { - l.SetFlags(log.Lshortfile | log.Lmicroseconds) - l.SetPrefix("ABCDEFG") - - for i := 0; i < b.N; i++ { - l.Infoln("This is a somewhat representative log line") - l.Infof("This is a log line with a couple of formatted things: %d %q", 42, "a file name maybe, who knows?") - } - - b.ReportAllocs() - b.SetBytes(2) // log entries per iteration -} diff --git a/lib/logger/mocks/logger.go b/lib/logger/mocks/logger.go deleted file mode 100644 index b1d5b15d8..000000000 --- a/lib/logger/mocks/logger.go +++ /dev/null @@ -1,142 +0,0 @@ -// Code generated by counterfeiter. DO NOT EDIT. -package mocks - -import ( - "sync" - "time" - - "github.com/syncthing/syncthing/lib/logger" -) - -type Recorder struct { - ClearStub func() - clearMutex sync.RWMutex - clearArgsForCall []struct { - } - SinceStub func(time.Time) []logger.Line - sinceMutex sync.RWMutex - sinceArgsForCall []struct { - arg1 time.Time - } - sinceReturns struct { - result1 []logger.Line - } - sinceReturnsOnCall map[int]struct { - result1 []logger.Line - } - invocations map[string][][]interface{} - invocationsMutex sync.RWMutex -} - -func (fake *Recorder) Clear() { - fake.clearMutex.Lock() - fake.clearArgsForCall = append(fake.clearArgsForCall, struct { - }{}) - stub := fake.ClearStub - fake.recordInvocation("Clear", []interface{}{}) - fake.clearMutex.Unlock() - if stub != nil { - fake.ClearStub() - } -} - -func (fake *Recorder) ClearCallCount() int { - fake.clearMutex.RLock() - defer fake.clearMutex.RUnlock() - return len(fake.clearArgsForCall) -} - -func (fake *Recorder) ClearCalls(stub func()) { - fake.clearMutex.Lock() - defer fake.clearMutex.Unlock() - fake.ClearStub = stub -} - -func (fake *Recorder) Since(arg1 time.Time) []logger.Line { - fake.sinceMutex.Lock() - ret, specificReturn := fake.sinceReturnsOnCall[len(fake.sinceArgsForCall)] - fake.sinceArgsForCall = append(fake.sinceArgsForCall, struct { - arg1 time.Time - }{arg1}) - stub := fake.SinceStub - fakeReturns := fake.sinceReturns - fake.recordInvocation("Since", []interface{}{arg1}) - fake.sinceMutex.Unlock() - if stub != nil { - return stub(arg1) - } - if specificReturn { - return ret.result1 - } - return fakeReturns.result1 -} - -func (fake *Recorder) SinceCallCount() int { - fake.sinceMutex.RLock() - defer fake.sinceMutex.RUnlock() - return len(fake.sinceArgsForCall) -} - -func (fake *Recorder) SinceCalls(stub func(time.Time) []logger.Line) { - fake.sinceMutex.Lock() - defer fake.sinceMutex.Unlock() - fake.SinceStub = stub -} - -func (fake *Recorder) SinceArgsForCall(i int) time.Time { - fake.sinceMutex.RLock() - defer fake.sinceMutex.RUnlock() - argsForCall := fake.sinceArgsForCall[i] - return argsForCall.arg1 -} - -func (fake *Recorder) SinceReturns(result1 []logger.Line) { - fake.sinceMutex.Lock() - defer fake.sinceMutex.Unlock() - fake.SinceStub = nil - fake.sinceReturns = struct { - result1 []logger.Line - }{result1} -} - -func (fake *Recorder) SinceReturnsOnCall(i int, result1 []logger.Line) { - fake.sinceMutex.Lock() - defer fake.sinceMutex.Unlock() - fake.SinceStub = nil - if fake.sinceReturnsOnCall == nil { - fake.sinceReturnsOnCall = make(map[int]struct { - result1 []logger.Line - }) - } - fake.sinceReturnsOnCall[i] = struct { - result1 []logger.Line - }{result1} -} - -func (fake *Recorder) Invocations() map[string][][]interface{} { - fake.invocationsMutex.RLock() - defer fake.invocationsMutex.RUnlock() - fake.clearMutex.RLock() - defer fake.clearMutex.RUnlock() - fake.sinceMutex.RLock() - defer fake.sinceMutex.RUnlock() - copiedInvocations := map[string][][]interface{}{} - for key, value := range fake.invocations { - copiedInvocations[key] = value - } - return copiedInvocations -} - -func (fake *Recorder) recordInvocation(key string, args []interface{}) { - fake.invocationsMutex.Lock() - defer fake.invocationsMutex.Unlock() - if fake.invocations == nil { - fake.invocations = map[string][][]interface{}{} - } - if fake.invocations[key] == nil { - fake.invocations[key] = [][]interface{}{} - } - fake.invocations[key] = append(fake.invocations[key], args) -} - -var _ logger.Recorder = new(Recorder) diff --git a/lib/model/debug.go b/lib/model/debug.go index 9f03e787b..1fc85d28a 100644 --- a/lib/model/debug.go +++ b/lib/model/debug.go @@ -6,12 +6,6 @@ package model -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("model", "The root hub") - -func shouldDebug() bool { - return l.ShouldDebug("model") -} +var l = slogutil.NewAdapter("The root hub") diff --git a/lib/model/deviceactivity.go b/lib/model/deviceactivity.go index 4685f020c..b13733eca 100644 --- a/lib/model/deviceactivity.go +++ b/lib/model/deviceactivity.go @@ -7,8 +7,9 @@ package model import ( + "sync" + "github.com/syncthing/syncthing/lib/protocol" - "github.com/syncthing/syncthing/lib/sync" ) // deviceActivity tracks the number of outstanding requests per device and can @@ -22,7 +23,6 @@ type deviceActivity struct { func newDeviceActivity() *deviceActivity { return &deviceActivity{ act: make(map[protocol.DeviceID]int), - mut: sync.NewMutex(), } } diff --git a/lib/model/devicedownloadstate.go b/lib/model/devicedownloadstate.go index a76fd0c94..d663f7d0c 100644 --- a/lib/model/devicedownloadstate.go +++ b/lib/model/devicedownloadstate.go @@ -8,9 +8,9 @@ package model import ( "slices" + "sync" "github.com/syncthing/syncthing/lib/protocol" - "github.com/syncthing/syncthing/lib/sync" ) // deviceFolderFileDownloadState holds current download state of a file that @@ -122,7 +122,6 @@ func (t *deviceDownloadState) Update(folder string, updates []protocol.FileDownl if !ok { f = &deviceFolderDownloadState{ - mut: sync.NewRWMutex(), files: make(map[string]deviceFolderFileDownloadState), } t.mut.Lock() @@ -186,7 +185,6 @@ func (t *deviceDownloadState) BytesDownloaded(folder string) int64 { func newDeviceDownloadState() *deviceDownloadState { return &deviceDownloadState{ - mut: sync.NewRWMutex(), folders: make(map[string]*deviceFolderDownloadState), } } diff --git a/lib/model/folder.go b/lib/model/folder.go index e33d9596b..9eed29e8e 100644 --- a/lib/model/folder.go +++ b/lib/model/folder.go @@ -10,14 +10,17 @@ import ( "context" "errors" "fmt" + "log/slog" "math/rand" "path/filepath" "slices" "strings" + "sync" "time" "github.com/syncthing/syncthing/internal/db" "github.com/syncthing/syncthing/internal/itererr" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/events" "github.com/syncthing/syncthing/lib/fs" @@ -30,7 +33,6 @@ import ( "github.com/syncthing/syncthing/lib/stats" "github.com/syncthing/syncthing/lib/stringutil" "github.com/syncthing/syncthing/lib/svcutil" - "github.com/syncthing/syncthing/lib/sync" "github.com/syncthing/syncthing/lib/versioner" "github.com/syncthing/syncthing/lib/watchaggregator" ) @@ -54,6 +56,7 @@ type folder struct { modTimeWindow time.Duration ctx context.Context //nolint:containedctx // used internally, only accessible on serve lifetime done chan struct{} // used externally, accessible regardless of serve + sl *slog.Logger scanInterval time.Duration scanTimer *time.Timer @@ -98,7 +101,7 @@ type puller interface { pull() (bool, error) // true when successful and should not be retried } -func newFolder(model *model, ignores *ignore.Matcher, cfg config.FolderConfiguration, evLogger events.Logger, ioLimiter *semaphore.Semaphore, ver versioner.Versioner) folder { +func newFolder(model *model, ignores *ignore.Matcher, cfg config.FolderConfiguration, evLogger events.Logger, ioLimiter *semaphore.Semaphore, ver versioner.Versioner) *folder { f := folder{ stateTracker: newStateTracker(cfg.ID, evLogger), FolderConfiguration: cfg, @@ -112,6 +115,7 @@ func newFolder(model *model, ignores *ignore.Matcher, cfg config.FolderConfigura mtimefs: cfg.Filesystem(fs.NewMtimeOption(model.sdb, cfg.ID)), modTimeWindow: cfg.ModTimeWindow(), done: make(chan struct{}), + sl: slog.Default().With(cfg.LogAttr()), scanInterval: time.Duration(cfg.RescanIntervalS) * time.Second, scanTimer: time.NewTimer(0), // The first scan should be done immediately. @@ -123,17 +127,13 @@ func newFolder(model *model, ignores *ignore.Matcher, cfg config.FolderConfigura pullScheduled: make(chan struct{}, 1), // This needs to be 1-buffered so that we queue a pull if we're busy when it comes. - errorsMut: sync.NewMutex(), - doInSyncChan: make(chan syncRequest), forcedRescanRequested: make(chan struct{}, 1), forcedRescanPaths: make(map[string]struct{}), - forcedRescanPathsMut: sync.NewMutex(), watchCancel: func() {}, restartWatchChan: make(chan struct{}, 1), - watchMut: sync.NewMutex(), versioner: ver, } @@ -143,7 +143,7 @@ func newFolder(model *model, ignores *ignore.Matcher, cfg config.FolderConfigura registerFolderMetrics(f.ID) - return f + return &f } func (f *folder) Serve(ctx context.Context) error { @@ -440,7 +440,7 @@ func (f *folder) pull() (success bool, err error) { // Pulling failed, try again later. delay := f.pullPause + time.Since(startTime) - l.Infof("Folder %v isn't making sync progress - retrying in %v.", f.Description(), stringutil.NiceDurationString(delay)) + f.sl.Info("Folder failed to sync, will be retried", slog.String("wait", stringutil.NiceDurationString(delay))) f.pullFailTimer.Reset(delay) return false, err @@ -948,11 +948,11 @@ func (f *folder) scanTimerFired() error { select { case <-f.initialScanFinished: default: - status := "Completed" if err != nil { - status = "Failed" + f.sl.Error("Failed initial scan", slogutil.Error(err)) + } else { + f.sl.Info("Competed initial scan") } - l.Infoln(status, "initial scan of", f.Type.String(), "folder", f.Description()) close(f.initialScanFinished) } @@ -973,7 +973,7 @@ func (f *folder) versionCleanupTimerFired() { f.setState(FolderCleaning) if err := f.versioner.Clean(f.ctx); err != nil { - l.Infoln("Failed to clean versions in %s: %v", f.Description(), err) + f.sl.Warn("Failed to clean versions", slogutil.Error(err)) } f.versionCleanupTimer.Reset(f.versionCleanupInterval) @@ -1084,7 +1084,7 @@ func (f *folder) monitorWatch(ctx context.Context) { var errOutside *fs.WatchEventOutsideRootError if errors.As(err, &errOutside) { if !warnedOutside { - l.Warnln(err) + slog.WarnContext(ctx, err.Error()) //nolint:sloglint warnedOutside = true } f.evLogger.Log(events.Failure, "watching for changes encountered an event outside of the filesystem root") @@ -1099,7 +1099,7 @@ func (f *folder) monitorWatch(ctx context.Context) { f.warnedKqueue = true summarySub.Unsubscribe() summaryChan = nil - l.Warnf("Filesystem watching (kqueue) is enabled on %v with a lot of files/directories, and that requires a lot of resources and might slow down your system significantly", f.Description()) + slog.WarnContext(ctx, "Filesystem watching (kqueue) is enabled with a lot of files/directories, which requires a lot of resources and might slow down your system significantly", f.LogAttr()) } case <-ctx.Done(): aggrCancel() // for good measure and keeping the linters happy @@ -1130,12 +1130,11 @@ func (f *folder) setWatchError(err error, nextTryIn time.Duration) { if err == nil { return } - msg := fmt.Sprintf("Error while trying to start filesystem watcher for folder %s, trying again in %v: %v", f.Description(), nextTryIn, err) if prevErr != err { //nolint:errorlint - l.Infof(msg) - return + f.sl.Warn("Failed to start filesystem watcher", slog.String("wait", nextTryIn.String()), slogutil.Error(err)) + } else { + f.sl.Debug("Failed to start filesystem watcher", slog.String("wait", nextTryIn.String()), slogutil.Error(err)) } - l.Debugf(msg) } // scanOnWatchErr schedules a full scan immediately if an error occurred while watching. @@ -1162,12 +1161,12 @@ func (f *folder) setError(err error) { if err != nil { if oldErr == nil { - l.Warnf("Error on folder %s: %v", f.Description(), err) + f.sl.Warn("Error on folder", slogutil.Error(err)) } else { - l.Infof("Error on folder %s changed: %q -> %q", f.Description(), oldErr, err) + f.sl.Info("Folder error changed", slogutil.Error(err), slog.Any("previously", oldErr)) } } else { - l.Infoln("Cleared error on folder", f.Description()) + f.sl.Info("Folder error cleared") f.SchedulePull() } @@ -1195,7 +1194,7 @@ func (f *folder) String() string { func (f *folder) newScanError(path string, err error) { f.errorsMut.Lock() - l.Infof("Scanner (folder %s, item %q): %v", f.Description(), path, err) + f.sl.Warn("Failed to scan", slogutil.FilePath(path), slogutil.Error(err)) f.scanErrors = append(f.scanErrors, FileError{ Err: err.Error(), Path: path, diff --git a/lib/model/folder_recvenc.go b/lib/model/folder_recvenc.go index bc5e5b701..6eb0d284c 100644 --- a/lib/model/folder_recvenc.go +++ b/lib/model/folder_recvenc.go @@ -40,7 +40,7 @@ func (f *receiveEncryptedFolder) Revert() { } func (f *receiveEncryptedFolder) revert() error { - l.Infof("Reverting unexpected items in folder %v (receive-encrypted)", f.Description()) + f.sl.Info("Reverting unexpected items") f.setState(FolderScanning) defer f.setState(FolderIdle) diff --git a/lib/model/folder_recvonly.go b/lib/model/folder_recvonly.go index 1ad611b6d..fae413802 100644 --- a/lib/model/folder_recvonly.go +++ b/lib/model/folder_recvonly.go @@ -12,6 +12,7 @@ import ( "time" "github.com/syncthing/syncthing/internal/itererr" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/events" "github.com/syncthing/syncthing/lib/ignore" @@ -69,7 +70,7 @@ func (f *receiveOnlyFolder) Revert() { } func (f *receiveOnlyFolder) revert() error { - l.Infof("Reverting folder %v", f.Description()) + f.sl.Info("Reverting folder") f.setState(FolderScanning) defer f.setState(FolderIdle) @@ -154,7 +155,7 @@ func (f *receiveOnlyFolder) revert() error { // Handle any queued directories deleted, err := delQueue.flush() if err != nil { - l.Infoln("Revert:", err) + f.sl.Warn("Failed to revert directories", slogutil.Error(err)) } now := time.Now() for _, dir := range deleted { diff --git a/lib/model/folder_sendonly.go b/lib/model/folder_sendonly.go index 1273be598..2f04245ae 100644 --- a/lib/model/folder_sendonly.go +++ b/lib/model/folder_sendonly.go @@ -21,7 +21,7 @@ func init() { } type sendOnlyFolder struct { - folder + *folder } func newSendOnlyFolder(model *model, ignores *ignore.Matcher, cfg config.FolderConfiguration, _ versioner.Versioner, evLogger events.Logger, ioLimiter *semaphore.Semaphore) service { @@ -93,7 +93,7 @@ func (f *sendOnlyFolder) Override() { } func (f *sendOnlyFolder) override() error { - l.Infoln("Overriding global state on folder", f.Description()) + f.sl.Info("Overriding global state ") f.setState(FolderScanning) defer f.setState(FolderIdle) diff --git a/lib/model/folder_sendrecv.go b/lib/model/folder_sendrecv.go index a10040baa..c2c6b2972 100644 --- a/lib/model/folder_sendrecv.go +++ b/lib/model/folder_sendrecv.go @@ -13,13 +13,16 @@ import ( "errors" "fmt" "io" + "log/slog" "path/filepath" "slices" "strconv" "strings" + "sync" "time" "github.com/syncthing/syncthing/internal/itererr" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/build" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/events" @@ -29,13 +32,12 @@ import ( "github.com/syncthing/syncthing/lib/protocol" "github.com/syncthing/syncthing/lib/scanner" "github.com/syncthing/syncthing/lib/semaphore" - "github.com/syncthing/syncthing/lib/sync" "github.com/syncthing/syncthing/lib/versioner" ) var ( blockStats = make(map[string]int) - blockStatsMut = sync.NewMutex() + blockStatsMut sync.Mutex ) func init() { @@ -120,7 +122,7 @@ type dbUpdateJob struct { } type sendReceiveFolder struct { - folder + *folder queue *jobQueue blockPullReorderer blockPullReorderer @@ -210,7 +212,7 @@ func (f *sendReceiveFolder) pull() (bool, error) { if pullErrNum > 0 { f.pullErrors = make([]FileError, 0, len(f.tempPullErrors)) for path, err := range f.tempPullErrors { - l.Infof("Puller (folder %s, item %q): %v", f.Description(), path, err) + f.sl.Warn("Failed to sync", slogutil.FilePath(path), slogutil.Error(err)) f.pullErrors = append(f.pullErrors, FileError{ Err: err, Path: path, @@ -221,7 +223,6 @@ func (f *sendReceiveFolder) pull() (bool, error) { f.errorsMut.Unlock() if pullErrNum > 0 { - l.Infof("%v: Failed to sync %v items", f.Description(), pullErrNum) f.evLogger.Log(events.FolderErrors, map[string]interface{}{ "folder": f.folderID, "errors": f.Errors(), @@ -245,10 +246,10 @@ func (f *sendReceiveFolder) pullerIteration(scanChan chan<- string) (int, error) finisherChan := make(chan *sharedPullerState) dbUpdateChan := make(chan dbUpdateJob) - pullWg := sync.NewWaitGroup() - copyWg := sync.NewWaitGroup() - doneWg := sync.NewWaitGroup() - updateWg := sync.NewWaitGroup() + var pullWg sync.WaitGroup + var copyWg sync.WaitGroup + var doneWg sync.WaitGroup + var updateWg sync.WaitGroup l.Debugln(f, "copiers:", f.Copiers, "pullerPendingKiB:", f.PullerMaxPendingKiB) @@ -422,7 +423,6 @@ loop: } default: - l.Warnln(file) panic("unhandleable item type, can't happen") } } @@ -554,6 +554,7 @@ func (f *sendReceiveFolder) handleDir(file protocol.FileInfo, dbUpdateChan chan< }) defer func() { + slog.Info("Created or updated directory", f.LogAttr(), file.LogAttr()) f.evLogger.Log(events.ItemFinished, map[string]interface{}{ "folder": f.folderID, "item": file.Name, @@ -568,11 +569,10 @@ func (f *sendReceiveFolder) handleDir(file protocol.FileInfo, dbUpdateChan chan< mode = 0o777 } - if shouldDebug() { + f.sl.Debug("Need dir", "file", file, "cur", slogutil.Expensive(func() any { curFile, _, _ := f.model.sdb.GetDeviceFile(f.folderID, protocol.LocalDeviceID, file.Name) - l.Debugf("need dir\n\t%v\n\t%v", file, curFile) - } - + return curFile + })) info, err := f.mtimefs.Lstat(file.Name) switch { // There is already something under that name, we need to handle that. @@ -723,6 +723,11 @@ func (f *sendReceiveFolder) handleSymlink(file protocol.FileInfo, dbUpdateChan c }) defer func() { + if err != nil { + slog.Warn("Failed to handle symlink", f.LogAttr(), file.LogAttr(), slogutil.Error(err)) + } else { + slog.Info("Created or updated symlink", f.LogAttr(), file.LogAttr()) + } f.evLogger.Log(events.ItemFinished, map[string]interface{}{ "folder": f.folderID, "item": file.Name, @@ -732,10 +737,10 @@ func (f *sendReceiveFolder) handleSymlink(file protocol.FileInfo, dbUpdateChan c }) }() - if shouldDebug() { - curFile, ok, _ := f.model.sdb.GetDeviceFile(f.folderID, protocol.LocalDeviceID, file.Name) - l.Debugf("need symlink\n\t%v\n\t%v", file, curFile, ok) - } + f.sl.Debug("Need symlink", slogutil.FilePath(file.Name), slog.Any("cur", slogutil.Expensive(func() any { + curFile, _, _ := f.model.sdb.GetDeviceFile(f.folderID, protocol.LocalDeviceID, file.Name) + return curFile + }))) if len(file.SymlinkTarget) == 0 { // Index entry from a Syncthing predating the support for including @@ -814,6 +819,9 @@ func (f *sendReceiveFolder) deleteDir(file protocol.FileInfo, dbUpdateChan chan< defer func() { if err != nil { f.newPullError(file.Name, fmt.Errorf("delete dir: %w", err)) + slog.Info("Failed to delete directory", f.LogAttr(), file.LogAttr(), slogutil.Error(err)) + } else { + slog.Info("Deleted directory", f.LogAttr(), file.LogAttr()) } f.evLogger.Log(events.ItemFinished, map[string]interface{}{ "folder": f.folderID, @@ -859,7 +867,7 @@ func (f *sendReceiveFolder) deleteFileWithCurrent(file, cur protocol.FileInfo, h // care not declare another err. var err error - l.Debugln(f, "Deleting file", file.Name) + l.Debugln(f, "Deleting file or symlink", file.Name) f.evLogger.Log(events.ItemStarted, map[string]string{ "folder": f.folderID, @@ -869,8 +877,15 @@ func (f *sendReceiveFolder) deleteFileWithCurrent(file, cur protocol.FileInfo, h }) defer func() { + kind := "file" + if file.IsSymlink() { + kind = "symlink" + } if err != nil { f.newPullError(file.Name, fmt.Errorf("delete file: %w", err)) + slog.Info("Failed to delete "+kind, f.LogAttr(), file.LogAttr(), slogutil.Error(err)) + } else { + slog.Info("Deleted "+kind, f.LogAttr(), file.LogAttr()) } f.evLogger.Log(events.ItemFinished, map[string]interface{}{ "folder": f.folderID, @@ -927,6 +942,8 @@ func (f *sendReceiveFolder) deleteFileWithCurrent(file, cur protocol.FileInfo, h err = nil dbUpdateChan <- dbUpdateJob{file, dbUpdateDeleteFile} } + + slog.Info("Deleted file", f.LogAttr(), file.LogAttr()) } // renameFile attempts to rename an existing file to a destination @@ -950,6 +967,11 @@ func (f *sendReceiveFolder) renameFile(cur, source, target protocol.FileInfo, db }) defer func() { + if err != nil { + slog.Info("Failed to rename file", f.LogAttr(), target.LogAttr(), slog.String("from", source.Name), slogutil.Error(err)) + } else { + slog.Info("Renamed file", f.LogAttr(), target.LogAttr(), slog.String("from", source.Name)) + } f.evLogger.Log(events.ItemFinished, map[string]interface{}{ "folder": f.folderID, "item": source.Name, @@ -1237,13 +1259,20 @@ func (f *sendReceiveFolder) shortcutFile(file protocol.FileInfo, dbUpdateChan ch }) var err error - defer f.evLogger.Log(events.ItemFinished, map[string]interface{}{ - "folder": f.folderID, - "item": file.Name, - "error": events.Error(err), - "type": "file", - "action": "metadata", - }) + defer func() { + if err != nil { + slog.Info("Failed to update file metadata", f.LogAttr(), file.LogAttr(), slogutil.Error(err)) + } else { + slog.Info("Updated file metadata", f.LogAttr(), file.LogAttr()) + } + f.evLogger.Log(events.ItemFinished, map[string]interface{}{ + "folder": f.folderID, + "item": file.Name, + "error": events.Error(err), + "type": "file", + "action": "metadata", + }) + }() f.queue.Done(file.Name) @@ -1395,7 +1424,7 @@ func (f *sendReceiveFolder) copyBlockFromFolder(folderID string, block protocol. // We just ignore this and continue pulling instead (though // there's a good chance that will fail too, if the DB is // unhealthy). - l.Debugf("Failed to get information from DB about block %v in copier (folderID %v, file %v): %v", block.Hash, f.folderID, state.file.Name) + l.Debugf("Failed to get information from DB about block %v in copier (folderID %v, file %v): %v", block.Hash, f.folderID, state.file.Name, err) return false } @@ -1480,7 +1509,7 @@ func (*sendReceiveFolder) verifyBuffer(buf []byte, block protocol.BlockInfo) err func (f *sendReceiveFolder) pullerRoutine(in <-chan pullBlockState, out chan<- *sharedPullerState) { requestLimiter := semaphore.New(f.PullerMaxPendingKiB * 1024) - wg := sync.NewWaitGroup() + var wg sync.WaitGroup for state := range in { if state.failed() != nil { @@ -1666,6 +1695,8 @@ func (f *sendReceiveFolder) finisherRoutine(in <-chan *sharedPullerState, dbUpda if err != nil { f.newPullError(state.file.Name, fmt.Errorf("finishing: %w", err)) } else { + slog.Info("Synced file", f.LogAttr(), state.file.LogAttr(), slog.Group("blocks", slog.Int("local", state.reused+state.copyTotal), slog.Int("download", state.pullTotal))) + minBlocksPerBlock := state.file.BlockSize() / protocol.MinBlockSize blockStatsMut.Lock() blockStats["total"] += (state.reused + state.copyTotal + state.pullTotal) * minBlocksPerBlock @@ -1673,8 +1704,7 @@ func (f *sendReceiveFolder) finisherRoutine(in <-chan *sharedPullerState, dbUpda blockStats["pulled"] += state.pullTotal * minBlocksPerBlock // copyOriginShifted is counted towards copyOrigin due to progress bar reasons // for reporting reasons we want to separate these. - blockStats["copyOrigin"] += (state.copyOrigin - state.copyOriginShifted) * minBlocksPerBlock - blockStats["copyOriginShifted"] += state.copyOriginShifted * minBlocksPerBlock + blockStats["copyOrigin"] += state.copyOrigin * minBlocksPerBlock blockStats["copyElsewhere"] += (state.copyTotal - state.copyOrigin) * minBlocksPerBlock blockStatsMut.Unlock() } @@ -1777,7 +1807,7 @@ loop: // (resp. whatever caused the error) will cause this file to // change. Log at info level to leave a trace if a user // notices, but no need to warn - l.Infof("Error updating metadata for %v at database commit: %v", job.file.Name, err) + f.sl.Warn("Failed to update metadata at database commit", slogutil.FilePath(job.file.Name), slogutil.Error(err)) } } job.file.Sequence = 0 @@ -1831,7 +1861,7 @@ func (f *sendReceiveFolder) inConflict(current, replacement protocol.Vector) boo func (f *sendReceiveFolder) moveForConflict(name, lastModBy string, scanChan chan<- string) error { if isConflict(name) { - l.Infoln("Conflict for", name, "which is already a conflict copy; not copying again.") + f.sl.Info("Conflict on existing conflict copy; not copying again", slogutil.FilePath(name)) if err := f.mtimefs.Remove(name); err != nil && !fs.IsNotExist(err) { return fmt.Errorf("%s: %w", contextRemovingOldItem, err) } diff --git a/lib/model/folder_sendrecv_test.go b/lib/model/folder_sendrecv_test.go index 5eff7a19c..6b9698419 100644 --- a/lib/model/folder_sendrecv_test.go +++ b/lib/model/folder_sendrecv_test.go @@ -17,6 +17,7 @@ import ( "runtime/pprof" "strconv" "strings" + "sync" "testing" "time" @@ -28,7 +29,6 @@ import ( "github.com/syncthing/syncthing/lib/ignore" "github.com/syncthing/syncthing/lib/protocol" "github.com/syncthing/syncthing/lib/scanner" - "github.com/syncthing/syncthing/lib/sync" ) var blocks = []protocol.BlockInfo{ @@ -471,7 +471,7 @@ func TestDeregisterOnFailInPull(t *testing.T) { dbUpdateChan := make(chan dbUpdateJob, 1) copyChan, copyWg := startCopier(f, pullChan, finisherBufferChan) - pullWg := sync.NewWaitGroup() + var pullWg sync.WaitGroup pullWg.Add(1) go func() { f.pullerRoutine(pullChan, finisherBufferChan) @@ -1268,9 +1268,9 @@ func cleanupSharedPullerState(s *sharedPullerState) { s.writer.mut.Unlock() } -func startCopier(f *sendReceiveFolder, pullChan chan<- pullBlockState, finisherChan chan<- *sharedPullerState) (chan copyBlocksState, sync.WaitGroup) { +func startCopier(f *sendReceiveFolder, pullChan chan<- pullBlockState, finisherChan chan<- *sharedPullerState) (chan copyBlocksState, *sync.WaitGroup) { copyChan := make(chan copyBlocksState) - wg := sync.NewWaitGroup() + wg := new(sync.WaitGroup) wg.Add(1) go func() { f.copierRoutine(copyChan, pullChan, finisherChan) diff --git a/lib/model/folder_sendrecv_windows.go b/lib/model/folder_sendrecv_windows.go index 6028ca522..0e2896d3d 100644 --- a/lib/model/folder_sendrecv_windows.go +++ b/lib/model/folder_sendrecv_windows.go @@ -20,13 +20,13 @@ func (f *sendReceiveFolder) syncOwnership(file *protocol.FileInfo, path string) return nil } - l.Debugln("Owner name for %s is %s (group=%v)", path, file.Platform.Windows.OwnerName, file.Platform.Windows.OwnerIsGroup) + l.Debugf("Owner name for %s is %s (group=%v)", path, file.Platform.Windows.OwnerName, file.Platform.Windows.OwnerIsGroup) usid, gsid, err := lookupUserAndGroup(file.Platform.Windows.OwnerName, file.Platform.Windows.OwnerIsGroup) if err != nil { return err } - l.Debugln("Owner for %s resolved to uid=%q gid=%q", path, usid, gsid) + l.Debugf("Owner for %s resolved to uid=%q gid=%q", path, usid, gsid) return f.mtimefs.Lchown(path, usid, gsid) } diff --git a/lib/model/folder_summary.go b/lib/model/folder_summary.go index c240f0f90..494433231 100644 --- a/lib/model/folder_summary.go +++ b/lib/model/folder_summary.go @@ -14,6 +14,7 @@ import ( "errors" "fmt" "strings" + "sync" "time" "github.com/thejerf/suture/v4" @@ -23,7 +24,6 @@ import ( "github.com/syncthing/syncthing/lib/events" "github.com/syncthing/syncthing/lib/protocol" "github.com/syncthing/syncthing/lib/svcutil" - "github.com/syncthing/syncthing/lib/sync" ) type FolderSummaryService interface { @@ -49,14 +49,13 @@ type folderSummaryService struct { func NewFolderSummaryService(cfg config.Wrapper, m Model, id protocol.DeviceID, evLogger events.Logger) FolderSummaryService { service := &folderSummaryService{ - Supervisor: suture.New("folderSummaryService", svcutil.SpecWithDebugLogger(l)), + Supervisor: suture.New("folderSummaryService", svcutil.SpecWithDebugLogger()), cfg: cfg, model: m, id: id, evLogger: evLogger, immediate: make(chan string), folders: make(map[string]struct{}), - foldersMut: sync.NewMutex(), } service.Add(svcutil.AsService(service.listenForUpdates, fmt.Sprintf("%s/listenForUpdates", service))) diff --git a/lib/model/folder_test.go b/lib/model/folder_test.go index f73c665ff..fda952634 100644 --- a/lib/model/folder_test.go +++ b/lib/model/folder_test.go @@ -171,7 +171,7 @@ func TestSetPlatformData(t *testing.T) { // Minimum required to support setPlatformData sr := &sendReceiveFolder{ - folder: folder{ + folder: &folder{ FolderConfiguration: config.FolderConfiguration{ SyncXattrs: true, }, diff --git a/lib/model/folderstate.go b/lib/model/folderstate.go index 5082a1c51..ac8e201f5 100644 --- a/lib/model/folderstate.go +++ b/lib/model/folderstate.go @@ -7,10 +7,11 @@ package model import ( + "log/slog" + "sync" "time" "github.com/syncthing/syncthing/lib/events" - "github.com/syncthing/syncthing/lib/sync" ) type folderState int @@ -94,7 +95,6 @@ func newStateTracker(id string, evLogger events.Logger) stateTracker { return stateTracker{ folderID: id, evLogger: evLogger, - mut: sync.NewMutex(), } } @@ -115,12 +115,6 @@ func (s *stateTracker) setState(newState folderState) { metricFolderState.WithLabelValues(s.folderID).Set(float64(s.current)) }() - /* This should hold later... - if s.current != FolderIdle && (newState == FolderScanning || newState == FolderSyncing) { - panic("illegal state transition " + s.current.String() + " -> " + newState.String()) - } - */ - eventData := map[string]interface{}{ "folder": s.folderID, "to": newState.String(), @@ -135,6 +129,7 @@ func (s *stateTracker) setState(newState folderState) { s.changed = time.Now().Truncate(time.Second) s.evLogger.Log(events.StateChanged, eventData) + slog.Info("Folder changed state", "folder", s.folderID, "state", newState) } // getState returns the current state, the time when it last changed, and the diff --git a/lib/model/indexhandler.go b/lib/model/indexhandler.go index 097ac09e8..5cf73ab3e 100644 --- a/lib/model/indexhandler.go +++ b/lib/model/indexhandler.go @@ -10,6 +10,7 @@ import ( "context" "errors" "fmt" + "log/slog" "sync" "time" @@ -78,7 +79,7 @@ func newIndexHandler(conn protocol.Connection, downloads *deviceDownloadState, f // the IndexID, or something else weird has // happened. We send a full index to reset the // situation. - l.Infof("Device %v folder %s is delta index compatible, but seems out of sync with reality", conn.DeviceID().Short(), folder.Description()) + slog.Warn("Peer is delta index compatible, but seems out of sync with reality", conn.DeviceID().LogAttr(), folder.LogAttr()) startSequence = 0 } else { l.Debugf("Device %v folder %s is delta index compatible (mlv=%d)", conn.DeviceID().Short(), folder.Description(), startInfo.local.MaxSequence) @@ -93,7 +94,7 @@ func newIndexHandler(conn protocol.Connection, downloads *deviceDownloadState, f // not the right one. Either they are confused or we // must have reset our database since last talking to // them. We'll start with a full index transfer. - l.Infof("Device %v folder %s has mismatching index ID for us (%v != %v)", conn.DeviceID().Short(), folder.Description(), startInfo.local.IndexID, myIndexID) + slog.Warn("Peer has mismatching index ID for us", conn.DeviceID().LogAttr(), folder.LogAttr(), slog.Group("indexid", slog.Any("ours", myIndexID), slog.Any("theirs", startInfo.local.IndexID))) startSequence = 0 } @@ -118,7 +119,7 @@ func newIndexHandler(conn protocol.Connection, downloads *deviceDownloadState, f // will probably send us a full index. We drop any // information we have and remember this new index ID // instead. - l.Infof("Device %v folder %s has a new index ID (%v)", conn.DeviceID().Short(), folder.Description(), startInfo.remote.IndexID) + slog.Info("Peer has a new index ID", conn.DeviceID().LogAttr(), folder.LogAttr(), slog.Any("indexid", startInfo.remote.IndexID)) if err := sdb.DropAllFiles(folder.ID, conn.DeviceID()); err != nil { return nil, err } @@ -361,7 +362,7 @@ func (s *indexHandler) receive(fs []protocol.FileInfo, update bool, op string, p s.cond.L.Unlock() if paused { - l.Infof("%v for paused folder %q", op, s.folder) + slog.Warn("Unexpected operation on paused folder", "op", op, "folder", s.folder) return fmt.Errorf("%v: %w", s.folder, ErrFolderPaused) } @@ -662,7 +663,7 @@ func (r *indexHandlerRegistry) ReceiveIndex(folder string, fs []protocol.FileInf defer r.mut.Unlock() is, isOk := r.indexHandlers.Get(folder) if !isOk { - l.Infof("%v for nonexistent or paused folder %q", op, folder) + slog.Warn("Unexpected operation on nonexistent or paused folder", "op", op, "folder", folder) return fmt.Errorf("%s: %w", folder, ErrFolderMissing) } return is.receive(fs, update, op, prevSequence, lastSequence) diff --git a/lib/model/model.go b/lib/model/model.go index b95a98975..b67e1ff38 100644 --- a/lib/model/model.go +++ b/lib/model/model.go @@ -17,6 +17,7 @@ import ( "fmt" "io" "iter" + "log/slog" "net" "os" "path/filepath" @@ -24,7 +25,7 @@ import ( "runtime" "slices" "strings" - stdsync "sync" + "sync" "sync/atomic" "time" @@ -32,6 +33,7 @@ import ( "github.com/syncthing/syncthing/internal/db" "github.com/syncthing/syncthing/internal/itererr" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/build" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/connections" @@ -45,7 +47,6 @@ import ( "github.com/syncthing/syncthing/lib/semaphore" "github.com/syncthing/syncthing/lib/stats" "github.com/syncthing/syncthing/lib/svcutil" - "github.com/syncthing/syncthing/lib/sync" "github.com/syncthing/syncthing/lib/ur/contract" "github.com/syncthing/syncthing/lib/versioner" ) @@ -213,7 +214,7 @@ var ( // where it sends index information to connected peers and responds to requests // for file data without altering the local folder in any way. func NewModel(cfg config.Wrapper, id protocol.DeviceID, sdb db.DB, protectedFiles []string, evLogger events.Logger, keyGen *protocol.KeyGenerator) Model { - spec := svcutil.SpecWithDebugLogger(l) + spec := svcutil.SpecWithDebugLogger() m := &model{ Supervisor: suture.New("model", spec), @@ -236,7 +237,6 @@ func NewModel(cfg config.Wrapper, id protocol.DeviceID, sdb db.DB, protectedFile observed: db.NewObservedDB(sdb), // fields protected by mut - mut: sync.NewRWMutex(), folderCfgs: make(map[string]config.FolderConfiguration), deviceStatRefs: make(map[protocol.DeviceID]*stats.DeviceStatisticsReference), folderIgnores: make(map[string]*ignore.Matcher), @@ -288,7 +288,7 @@ func (m *model) serve(ctx context.Context) error { l.Debugln(m, "fatal error, stopping", err) return svcutil.AsFatalErr(err, svcutil.ExitError) case <-m.promotionTimer.C: - l.Debugln("promotion timer fired") + slog.Debug("Promotion timer fired") m.promoteConnections() } } @@ -340,7 +340,7 @@ func (m *model) addAndStartFolderLocked(cfg config.FolderConfiguration, cacheIgn ignores := ignore.New(cfg.Filesystem(), ignore.WithCache(cacheIgnoredFiles)) if cfg.Type != config.FolderTypeReceiveEncrypted { if err := ignores.Load(".stignore"); err != nil && !fs.IsNotExist(err) { - l.Warnln("Loading ignores:", err) + slog.Error("Failed to load ignores", slogutil.Error(err)) } } @@ -354,7 +354,7 @@ func (m *model) addAndStartFolderLockedWithIgnores(cfg config.FolderConfiguratio _, ok := m.folderRunners.Get(cfg.ID) if ok { - l.Warnln("Cannot start already running folder", cfg.Description()) + slog.Error("Cannot start already running folder", cfg.LogAttr()) panic("cannot start already running folder") } @@ -388,9 +388,9 @@ func (m *model) addAndStartFolderLockedWithIgnores(cfg config.FolderConfiguratio // it'll show up as errored later. if err := cfg.CreateRoot(); err != nil { - l.Warnln("Failed to create folder root directory:", err) + slog.Error("Failed to create folder root directory", cfg.LogAttr(), slogutil.Error(err)) } else if err = cfg.CreateMarker(); err != nil { - l.Warnln("Failed to create folder marker:", err) + slog.Error("Failed to create folder marker", cfg.LogAttr(), slogutil.Error(err)) } } @@ -398,7 +398,7 @@ func (m *model) addAndStartFolderLockedWithIgnores(cfg config.FolderConfiguratio if encryptionToken, err := readEncryptionToken(cfg); err == nil { m.folderEncryptionPasswordTokens[folder] = encryptionToken } else if !fs.IsNotExist(err) { - l.Warnf("Failed to read encryption token: %v", err) + slog.Error("Failed to read encryption token", cfg.LogAttr(), slogutil.Error(err)) } } @@ -423,7 +423,7 @@ func (m *model) addAndStartFolderLockedWithIgnores(cfg config.FolderConfiguratio p := folderFactory(m, ignores, cfg, ver, m.evLogger, m.folderIOLimiter) m.folderRunners.Add(folder, p) - l.Infof("Ready to synchronize %s (%s)", cfg.Description(), cfg.Type) + slog.Info("Ready to synchronize", cfg.LogAttr()) } func (m *model) warnAboutOverwritingProtectedFiles(cfg config.FolderConfiguration, ignores *ignore.Matcher) { @@ -455,13 +455,13 @@ func (m *model) warnAboutOverwritingProtectedFiles(cfg config.FolderConfiguratio } if len(filesAtRisk) > 0 { - l.Warnln("Some protected files may be overwritten and cause issues. See https://docs.syncthing.net/users/config.html#syncing-configuration-files for more information. The at risk files are:", strings.Join(filesAtRisk, ", ")) + slog.Warn("Some protected files may be overwritten and cause issues; see https://docs.syncthing.net/users/config.html#syncing-configuration-files for more information", slog.Any("filesAtRisk", filesAtRisk)) } } func (m *model) removeFolder(cfg config.FolderConfiguration) { - l.Infoln("Removing folder", cfg.Description()) - defer l.Infoln("Removed folder", cfg.Description()) + slog.Info("Removing folder", cfg.LogAttr()) + defer slog.Info("Removed folder", cfg.LogAttr()) m.mut.RLock() wait := m.folderRunners.StopAndWaitChan(cfg.ID, 0) @@ -515,7 +515,7 @@ func (m *model) restartFolder(from, to config.FolderConfiguration, cacheIgnoredF panic("bug: cannot restart empty folder ID") } if to.ID != from.ID { - l.Warnf("bug: folder restart cannot change ID %q -> %q", from.ID, to.ID) + slog.Error("Bug: folder restart cannot change ID", "from", from.ID, "to", to.ID) panic("bug: folder restart cannot change ID") } folder := to.ID @@ -549,16 +549,14 @@ func (m *model) restartFolder(from, to config.FolderConfiguration, cacheIgnoredF return nil }) - var infoMsg string switch { case to.Paused: - infoMsg = "Paused" + slog.Info("Paused folder", to.LogAttr()) case from.Paused: - infoMsg = "Unpaused" + slog.Info("Unpaused folder", to.LogAttr()) default: - infoMsg = "Restarted" + slog.Info("Restarted folder", to.LogAttr()) } - l.Infof("%v folder %v (%v)", infoMsg, to.Description(), to.Type) return nil } @@ -1172,7 +1170,7 @@ func (m *model) handleIndex(conn protocol.Connection, folder string, fs []protoc l.Debugf("%v (in): %s / %q: %d files", op, deviceID, folder, len(fs)) if cfg, ok := m.cfg.Folder(folder); !ok || !cfg.SharedWith(deviceID) { - l.Warnf("%v for unexpected folder ID %q sent from device %q; ensure that the folder exists and that this device is selected under \"Share With\" in the folder configuration.", op, folder, deviceID) + slog.Warn(`Operation for unexpected folder ID; ensure that the folder exists and that this device is selected under "Share With" in the folder configuration.`, slog.String("operation", op), cfg.LogAttr(), deviceID.LogAttr()) return fmt.Errorf("%s: %w", folder, ErrFolderMissing) } else if cfg.Paused { l.Debugf("%v for paused folder (ID %q) sent from device %q.", op, folder, deviceID) @@ -1243,11 +1241,11 @@ func (m *model) ClusterConfig(conn protocol.Connection, cm *protocol.ClusterConf } } if info.remote.ID == protocol.EmptyDeviceID { - l.Infof("Device %v sent cluster-config without the device info for the remote on folder %v", deviceID.Short(), folder.Description()) + slog.Warn("Device sent cluster-config without the device info for the remote", folder.LogAttr(), deviceID.LogAttr()) return errMissingRemoteInClusterConfig } if info.local.ID == protocol.EmptyDeviceID { - l.Infof("Device %v sent cluster-config without the device info for us locally on folder %v", deviceID.Short(), folder.Description()) + slog.Warn("Device sent cluster-config without the device info for us locally", folder.LogAttr(), deviceID.LogAttr()) return errMissingLocalInClusterConfig } ccDeviceInfos[folder.ID] = info @@ -1255,7 +1253,7 @@ func (m *model) ClusterConfig(conn protocol.Connection, cm *protocol.ClusterConf for _, info := range ccDeviceInfos { if deviceCfg.Introducer && info.local.Introducer { - l.Warnf("Remote %v is an introducer to us, and we are to them - only one should be introducer to the other, see https://docs.syncthing.net/users/introducer.html", deviceCfg.Description()) + slog.Error("Remote is an introducer to us, and we are to them - only one should be introducer to the other, see https://docs.syncthing.net/users/introducer.html", deviceCfg.DeviceID.LogAttr()) } break } @@ -1359,7 +1357,7 @@ func (m *model) ensureIndexHandler(conn protocol.Connection) *indexHandlerRegist // the other side has decided to start using a new primary // connection but we haven't seen it close yet. Ideally it will // close shortly by itself... - l.Infof("Abandoning old index handler for %s (%s) in favour of %s", deviceID.Short(), indexHandlerRegistry.conn.ConnectionID(), connID) + slog.Warn("Abandoning old index handler in favour of new connection", deviceID.LogAttr(), slog.String("old", indexHandlerRegistry.conn.ConnectionID()), slog.String("new", connID)) m.indexHandlers.RemoveAndWait(deviceID, 0) } @@ -1399,7 +1397,7 @@ func (m *model) ccHandleFolders(folders []protocol.Folder, deviceCfg config.Devi deviceID := deviceCfg.DeviceID expiredPending, err := m.observed.PendingFoldersForDevice(deviceID) if err != nil { - l.Infof("Could not get pending folders for cleanup: %v", err) + slog.Warn("Failed to list pending folders for cleanup", slogutil.Error(err)) } of := db.ObservedFolder{Time: time.Now().Truncate(time.Second)} for _, folder := range folders { @@ -1412,7 +1410,7 @@ func (m *model) ccHandleFolders(folders []protocol.Folder, deviceCfg config.Devi if !ok { indexHandlers.Remove(folder.ID) if deviceCfg.IgnoredFolder(folder.ID) { - l.Infof("Ignoring folder %s from device %s since it is in the list of ignored folders", folder.Description(), deviceID) + slog.Info("Ignoring announced folder", folder.LogAttr(), deviceID.LogAttr()) continue } delete(expiredPending, folder.ID) @@ -1420,7 +1418,7 @@ func (m *model) ccHandleFolders(folders []protocol.Folder, deviceCfg config.Devi of.ReceiveEncrypted = len(ccDeviceInfos[folder.ID].local.EncryptionPasswordToken) > 0 of.RemoteEncrypted = len(ccDeviceInfos[folder.ID].remote.EncryptionPasswordToken) > 0 if err := m.observed.AddOrUpdatePendingFolder(folder.ID, of, deviceID); err != nil { - l.Warnf("Failed to persist pending folder entry to database: %v", err) + slog.Warn("Failed to persist pending folder entry to database", slogutil.Error(err)) } if folder.IsRunning() { indexHandlers.AddIndexInfo(folder.ID, ccDeviceInfos[folder.ID]) @@ -1438,7 +1436,7 @@ func (m *model) ccHandleFolders(folders []protocol.Folder, deviceCfg config.Devi "folderLabel": folder.Label, "device": deviceID.String(), }) - l.Infof("Unexpected folder %s sent from device %q; ensure that the folder exists and that this device is selected under \"Share With\" in the folder configuration.", folder.Description(), deviceID) + slog.Warn(`Unexpected folder ID in ClusterConfig; ensure that the folder exists and that this device is selected under "Share With" in the folder configuration.`, folder.LogAttr(), deviceID.LogAttr()) continue } @@ -1463,16 +1461,16 @@ func (m *model) ccHandleFolders(folders []protocol.Folder, deviceCfg config.Devi } m.folderEncryptionFailures[folder.ID][deviceID] = err m.mut.Unlock() - msg := fmt.Sprintf("Failure checking encryption consistency with device %v for folder %v: %v", deviceID, cfg.Description(), err) + const msg = "Failed to verify encryption consistency" if sameError { - l.Debugln(msg) + slog.Debug(msg, cfg.LogAttr(), deviceID.LogAttr(), slogutil.Error(err)) } else { var rerr *redactedError if errors.As(err, &rerr) { err = rerr.redacted } m.evLogger.Log(events.Failure, err.Error()) - l.Warnln(msg) + slog.Error(msg, cfg.LogAttr(), deviceID.LogAttr(), slogutil.Error(err)) } return tempIndexFolders, seenFolders, err } @@ -1507,8 +1505,8 @@ func (m *model) ccHandleFolders(folders []protocol.Folder, deviceCfg config.Devi expiredPendingList := make([]map[string]string, 0, len(expiredPending)) for folder := range expiredPending { if err = m.observed.RemovePendingFolderForDevice(folder, deviceID); err != nil { - msg := "Failed to remove pending folder-device entry" - l.Warnf("%v (%v, %v): %v", msg, folder, deviceID, err) + const msg = "Failed to remove pending folder-device entry" + slog.Warn(msg, slog.String("folder", folder), deviceID.LogAttr(), slogutil.Error(err)) m.evLogger.Log(events.Failure, msg) continue } @@ -1689,13 +1687,13 @@ func (m *model) handleIntroductions(introducerCfg config.DeviceConfiguration, cm } if fcfg.Type != config.FolderTypeReceiveEncrypted && device.EncryptionPasswordToken != nil { - l.Infof("Cannot share folder %s with %v because the introducer %v encrypts data, which requires a password", folder.Description(), device.ID, introducerCfg.DeviceID) + slog.Warn("Cannot share folder in untrusted mode with introduced device because it requires a password", folder.LogAttr(), slog.Any("device", device.ID), slog.Any("introducer", introducerCfg.DeviceID)) continue } // We don't yet share this folder with this device. Add the device // to sharing list of the folder. - l.Infof("Sharing folder %s with %v (vouched for by introducer %v)", folder.Description(), device.ID, introducerCfg.DeviceID) + slog.Info("Sharing folder vouched for by introducer", folder.LogAttr(), slog.Any("device", device.ID), slog.Any("introducer", introducerCfg.DeviceID)) fcfg.Devices = append(fcfg.Devices, config.FolderDeviceConfiguration{ DeviceID: device.ID, IntroducedBy: introducerCfg.DeviceID, @@ -1732,7 +1730,7 @@ func (*model) handleDeintroductions(introducerCfg config.DeviceConfiguration, fo // We could not find that folder shared on the // introducer with the device that was introduced to us. // We should follow and unshare as well. - l.Infof("Unsharing folder %s with %v as introducer %v no longer shares the folder with that device", folderCfg.Description(), folderCfg.Devices[k].DeviceID, folderCfg.Devices[k].IntroducedBy) + slog.Info("Unsharing folder as introducer no longer shares the folder with that device", folderCfg.LogAttr(), slog.Any("device", folderCfg.Devices[k].DeviceID), slog.Any("introducer", folderCfg.Devices[k].IntroducedBy)) folderCfg.Devices = append(folderCfg.Devices[:k], folderCfg.Devices[k+1:]...) folders[folderID] = folderCfg k-- @@ -1750,12 +1748,11 @@ func (*model) handleDeintroductions(introducerCfg config.DeviceConfiguration, fo if _, ok := devicesNotIntroduced[deviceID]; !ok { // The introducer no longer shares any folder with the // device, remove the device. - l.Infof("Removing device %v as introducer %v no longer shares any folders with that device", deviceID, device.IntroducedBy) + slog.Info("Removing device as introducer no longer shares any folders with that device", "device", deviceID, "introducer", device.IntroducedBy) changed = true delete(devices, deviceID) continue } - l.Infof("Would have removed %v as %v no longer shares any folders, yet there are other folders that are shared with this device that haven't been introduced by this introducer.", deviceID, device.IntroducedBy) } } } @@ -1776,7 +1773,7 @@ func (m *model) handleAutoAccepts(deviceID protocol.DeviceID, folder protocol.Fo pathAlternatives = append(pathAlternatives, alt) } if len(pathAlternatives) == 0 { - l.Infof("Failed to auto-accept folder %s from %s due to lack of path alternatives", folder.Description(), deviceID) + slog.Error("Failed to auto-accept folder due to lack of path alternatives", folder.LogAttr(), deviceID.LogAttr()) return config.FolderConfiguration{}, false } for _, path := range pathAlternatives { @@ -1788,7 +1785,7 @@ func (m *model) handleAutoAccepts(deviceID protocol.DeviceID, folder protocol.Fo // Attempt to create it to make sure it does, now. fullPath := filepath.Join(defaultFolderCfg.Path, path) if err := defaultPathFs.MkdirAll(path, 0o700); err != nil { - l.Warnf("Failed to create path for auto-accepted folder %s at path %s: %v", folder.Description(), fullPath, err) + slog.Error("Failed to create path for auto-accepted folder", folder.LogAttr(), slogutil.FilePath(fullPath), slogutil.Error(err)) continue } @@ -1812,14 +1809,14 @@ func (m *model) handleAutoAccepts(deviceID protocol.DeviceID, folder protocol.Fo } else { ignores := m.cfg.DefaultIgnores() if err := m.setIgnores(fcfg, ignores.Lines); err != nil { - l.Warnf("Failed to apply default ignores to auto-accepted folder %s at path %s: %v", folder.Description(), fcfg.Path, err) + slog.Error("Failed to apply default ignores to auto-accepted folder", folder.LogAttr(), slogutil.FilePath(fullPath), slogutil.Error(err)) } } - l.Infof("Auto-accepted %s folder %s at path %s", deviceID, folder.Description(), fcfg.Path) + slog.Info("Auto-accepted folder", fcfg.LogAttr(), slogutil.FilePath(fcfg.Path)) return fcfg, true } - l.Infof("Failed to auto-accept folder %s from %s due to path conflict", folder.Description(), deviceID) + slog.Error("Failed to auto-accept folder due to path conflict", folder.LogAttr(), deviceID.LogAttr()) return config.FolderConfiguration{}, false } else { if slices.Contains(cfg.DeviceIDs(), deviceID) { @@ -1828,19 +1825,19 @@ func (m *model) handleAutoAccepts(deviceID protocol.DeviceID, folder protocol.Fo } if cfg.Type == config.FolderTypeReceiveEncrypted { if len(ccDeviceInfos.remote.EncryptionPasswordToken) == 0 && len(ccDeviceInfos.local.EncryptionPasswordToken) == 0 { - l.Infof("Failed to auto-accept device %s on existing folder %s as the remote wants to send us unencrypted data, but the folder type is receive-encrypted", folder.Description(), deviceID) + slog.Info("Failed to auto-accept device on existing folder as the remote wants to send us unencrypted data, but the folder type is receive-encrypted", folder.LogAttr(), deviceID.LogAttr()) return config.FolderConfiguration{}, false } } else { if len(ccDeviceInfos.remote.EncryptionPasswordToken) > 0 || len(ccDeviceInfos.local.EncryptionPasswordToken) > 0 { - l.Infof("Failed to auto-accept device %s on existing folder %s as the remote wants to send us encrypted data, but the folder type is not receive-encrypted", folder.Description(), deviceID) + slog.Info("Failed to auto-accept device on existing folder as the remote wants to send us encrypted data, but the folder type is not receive-encrypted", folder.LogAttr(), deviceID.LogAttr()) return config.FolderConfiguration{}, false } } cfg.Devices = append(cfg.Devices, config.FolderDeviceConfiguration{ DeviceID: deviceID, }) - l.Infof("Shared %s with %s due to auto-accept", folder.ID, deviceID) + slog.Info("Shared folder due to auto-accept", folder.LogAttr(), deviceID.LogAttr()) return cfg, true } } @@ -1853,7 +1850,7 @@ func (m *model) introduceDevice(device protocol.Device, introducerCfg config.Dev } } - l.Infof("Adding device %v to config (vouched for by introducer %v)", device.ID, introducerCfg.DeviceID) + slog.Info("Adding device to config (vouched for by introducer)", device.ID.LogAttr(), slog.Any("introducer", introducerCfg.DeviceID.Short())) newDeviceCfg := m.cfg.DefaultDevice() newDeviceCfg.DeviceID = device.ID newDeviceCfg.Name = device.Name @@ -1864,7 +1861,7 @@ func (m *model) introduceDevice(device protocol.Device, introducerCfg config.Dev // The introducers' introducers are also our introducers. if device.Introducer { - l.Infof("Device %v is now also an introducer", device.ID) + slog.Info("Device is now also an introducer", device.ID.LogAttr()) newDeviceCfg.Introducer = true newDeviceCfg.SkipIntroductionRemovals = device.SkipIntroductionRemovals } @@ -1921,10 +1918,10 @@ func (m *model) Closed(conn protocol.Connection, err error) { m.mut.RUnlock() k := map[bool]string{false: "secondary", true: "primary"}[removedIsPrimary] - l.Infof("Lost %s connection to %s at %s: %v (%d remain)", k, deviceID.Short(), conn, err, len(remainingConns)) + slog.Info("Lost device connection", slog.String("kind", k), deviceID.LogAttr(), slog.Any("connection", conn), slogutil.Error(err), slog.Int("remaining", len(remainingConns))) if len(remainingConns) == 0 { - l.Infof("Connection to %s at %s closed: %v", deviceID.Short(), conn, err) + slog.Info("Connection closed", deviceID.LogAttr(), slog.Any("connection", conn), slogutil.Error(err)) m.evLogger.Log(events.DeviceDisconnected, map[string]string{ "id": deviceID.String(), "error": err.Error(), @@ -1937,7 +1934,7 @@ func (m *model) Closed(conn protocol.Connection, err error) { type requestResponse struct { data []byte closed chan struct{} - once stdsync.Once + once sync.Once } func newRequestResponse(size int) *requestResponse { @@ -1983,7 +1980,7 @@ func (m *model) Request(conn protocol.Connection, req *protocol.Request) (out pr } if !folderCfg.SharedWith(deviceID) { - l.Warnf("Request from %s for file %s in unshared folder %q", deviceID.Short(), req.Name, req.Folder) + slog.Warn("Request for file in unshared folder", slog.String("folder", req.Folder), deviceID.LogAttr(), slogutil.FilePath(req.Name)) return nil, protocol.ErrGeneric } if folderCfg.Paused { @@ -2248,7 +2245,7 @@ func (m *model) setIgnores(cfg config.FolderConfiguration, content []string) err } if err := ignore.WriteIgnores(cfg.Filesystem(), ".stignore", content); err != nil { - l.Warnln("Saving .stignore:", err) + slog.Error("Failed to save .stignore", slogutil.Error(err)) return err } @@ -2267,7 +2264,7 @@ func (m *model) setIgnores(cfg config.FolderConfiguration, content []string) err func (m *model) OnHello(remoteID protocol.DeviceID, addr net.Addr, hello protocol.Hello) error { if _, ok := m.cfg.Device(remoteID); !ok { if err := m.observed.AddOrUpdatePendingDevice(remoteID, hello.DeviceName, addr.String()); err != nil { - l.Warnf("Failed to persist pending device entry to database: %v", err) + slog.Warn("Failed to persist pending device entry to database", slogutil.Error(err)) } m.evLogger.Log(events.PendingDevicesChanged, map[string][]interface{}{ "added": {map[string]string{ @@ -2294,7 +2291,7 @@ func (m *model) AddConnection(conn protocol.Connection, hello protocol.Hello) { deviceID := conn.DeviceID() deviceCfg, ok := m.cfg.Device(deviceID) if !ok { - l.Infoln("Trying to add connection to unknown device") + slog.Info("Trying to add connection to unknown device") return } @@ -2327,9 +2324,9 @@ func (m *model) AddConnection(conn protocol.Connection, hello protocol.Hello) { m.evLogger.Log(events.DeviceConnected, event) if len(m.deviceConnIDs[deviceID]) == 1 { - l.Infof(`Device %s client is "%s %s" named "%s" at %s`, deviceID.Short(), hello.ClientName, hello.ClientVersion, hello.DeviceName, conn) + slog.Info("New device connection", deviceID.LogAttr(), slogutil.Address(conn.RemoteAddr()), slog.Group("remote", slog.String("name", hello.DeviceName), slog.String("client", hello.ClientName), slog.String("version", hello.ClientVersion))) } else { - l.Infof(`Additional connection (+%d) for device %s at %s`, len(m.deviceConnIDs[deviceID])-1, deviceID.Short(), conn) + slog.Info("Additional device connection", deviceID.LogAttr(), slogutil.Address(conn.RemoteAddr()), slog.Int("count", len(m.deviceConnIDs[deviceID])-1)) } m.mut.Unlock() @@ -2500,9 +2497,9 @@ func (m *model) ScanFolders() map[string]error { m.mut.RUnlock() errors := make(map[string]error, len(m.folderCfgs)) - errorsMut := sync.NewMutex() + var errorsMut sync.Mutex - wg := sync.NewWaitGroup() + var wg sync.WaitGroup wg.Add(len(folders)) for _, folder := range folders { go func() { @@ -2922,7 +2919,7 @@ func (m *model) ResetFolder(folder string) error { if ok { return errors.New("folder must be paused when resetting") } - l.Infof("Cleaning metadata for reset folder %q", folder) + slog.Info("Cleaning metadata for reset folder", "folder", folder) return m.sdb.DropFolder(folder) } @@ -2969,9 +2966,9 @@ func (m *model) CommitConfiguration(from, to config.Configuration) bool { if _, ok := fromFolders[folderID]; !ok { // A folder was added. if cfg.Paused { - l.Infoln("Paused folder", cfg.Description()) + slog.Info("Paused folder", cfg.LogAttr()) } else { - l.Infoln("Adding folder", cfg.Description()) + slog.Info("Adding folder", cfg.LogAttr()) if err := m.newFolder(cfg, to.Options.CacheIgnoredFiles); err != nil { m.fatal(err) return true @@ -3049,7 +3046,7 @@ func (m *model) CommitConfiguration(from, to config.Configuration) bool { } if toCfg.Paused { - l.Infoln("Pausing", deviceID) + slog.Info("Pausing device", deviceID.LogAttr()) closeDevices = append(closeDevices, deviceID) m.evLogger.Log(events.DevicePaused, map[string]string{"device": deviceID.String()}) } else { @@ -3058,7 +3055,7 @@ func (m *model) CommitConfiguration(from, to config.Configuration) bool { closeDevices = append(closeDevices, deviceID) } - l.Infoln("Resuming", deviceID) + slog.Info("Resuming device", deviceID.LogAttr()) m.evLogger.Log(events.DeviceResumed, map[string]string{"device": deviceID.String()}) } @@ -3132,8 +3129,8 @@ func (m *model) cleanPending(existingDevices map[protocol.DeviceID]config.Device var removedPendingFolders []map[string]string pendingFolders, err := m.observed.PendingFolders() if err != nil { - msg := "Could not iterate through pending folder entries for cleanup" - l.Warnf("%v: %v", msg, err) + const msg = "Could not iterate through pending folder entries for cleanup" + slog.Warn(msg, slogutil.Error(err)) m.evLogger.Log(events.Failure, msg) // Continue with pending devices below, loop is skipped. } @@ -3144,8 +3141,8 @@ func (m *model) cleanPending(existingDevices map[protocol.DeviceID]config.Device // at all (but might become pending again). l.Debugf("Discarding pending removed folder %v from all devices", folderID) if err := m.observed.RemovePendingFolder(folderID); err != nil { - msg := "Failed to remove pending folder entry" - l.Warnf("%v (%v): %v", msg, folderID, err) + const msg = "Failed to remove pending folder entry" + slog.Warn(msg, slog.String("folder", folderID), slogutil.Error(err)) m.evLogger.Log(events.Failure, msg) } else { removedPendingFolders = append(removedPendingFolders, map[string]string{ @@ -3171,8 +3168,8 @@ func (m *model) cleanPending(existingDevices map[protocol.DeviceID]config.Device continue removeFolderForDevice: if err := m.observed.RemovePendingFolderForDevice(folderID, deviceID); err != nil { - msg := "Failed to remove pending folder-device entry" - l.Warnf("%v (%v, %v): %v", msg, folderID, deviceID, err) + const msg = "Failed to remove pending folder-device entry" + slog.Warn(msg, slog.String("folder", folderID), deviceID.LogAttr(), slogutil.Error(err)) m.evLogger.Log(events.Failure, msg) continue } @@ -3191,8 +3188,8 @@ func (m *model) cleanPending(existingDevices map[protocol.DeviceID]config.Device var removedPendingDevices []map[string]string pendingDevices, err := m.observed.PendingDevices() if err != nil { - msg := "Could not iterate through pending device entries for cleanup" - l.Warnf("%v: %v", msg, err) + const msg = "Could not iterate through pending device entries for cleanup" + slog.Warn(msg, slogutil.Error(err)) m.evLogger.Log(events.Failure, msg) return } @@ -3208,8 +3205,8 @@ func (m *model) cleanPending(existingDevices map[protocol.DeviceID]config.Device continue removeDevice: if err := m.observed.RemovePendingDevice(deviceID); err != nil { - msg := "Failed to remove pending device entry" - l.Warnf("%v: %v", msg, err) + const msg = "Failed to remove pending device entry" + slog.Warn(msg, slogutil.Error(err)) m.evLogger.Log(events.Failure, msg) continue } @@ -3379,12 +3376,12 @@ func (s folderDeviceSet) hasDevice(dev protocol.DeviceID) bool { // syncMutexMap is a type safe wrapper for a sync.Map that holds mutexes type syncMutexMap struct { - inner stdsync.Map + inner sync.Map } -func (m *syncMutexMap) Get(key string) sync.Mutex { - v, _ := m.inner.LoadOrStore(key, sync.NewMutex()) - return v.(sync.Mutex) +func (m *syncMutexMap) Get(key string) *sync.Mutex { + v, _ := m.inner.LoadOrStore(key, new(sync.Mutex)) + return v.(*sync.Mutex) } type deviceIDSet map[protocol.DeviceID]struct{} diff --git a/lib/model/model_test.go b/lib/model/model_test.go index 98d290514..c4e6df377 100644 --- a/lib/model/model_test.go +++ b/lib/model/model_test.go @@ -3620,7 +3620,7 @@ func TestIssue6961(t *testing.T) { if info, err := tfs.Lstat(name); err != nil { t.Fatal(err) } else { - l.Infoln("intest", info.Mode) + t.Log(info.Mode()) } m.ScanFolders() diff --git a/lib/model/progressemitter.go b/lib/model/progressemitter.go index 6242c33b8..2dd15bf6d 100644 --- a/lib/model/progressemitter.go +++ b/lib/model/progressemitter.go @@ -9,12 +9,13 @@ package model import ( "context" "fmt" + "log/slog" + "sync" "time" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/events" "github.com/syncthing/syncthing/lib/protocol" - "github.com/syncthing/syncthing/lib/sync" ) type ProgressEmitter struct { @@ -53,7 +54,6 @@ func NewProgressEmitter(cfg config.Wrapper, evLogger events.Logger) *ProgressEmi connections: make(map[protocol.DeviceID]protocol.Connection), foldersByConns: make(map[protocol.DeviceID][]string), evLogger: evLogger, - mut: sync.NewMutex(), } t.CommitConfiguration(config.Configuration{}, cfg.RawCopy()) @@ -72,7 +72,7 @@ func (t *ProgressEmitter) Serve(ctx context.Context) error { for { select { case <-ctx.Done(): - l.Debugln("progress emitter: stopping") + slog.Debug("Progress emitter: stopping") return nil case <-t.timer.C: t.mut.Lock() @@ -218,16 +218,16 @@ func (t *ProgressEmitter) CommitConfiguration(_, to config.Configuration) bool { if newInterval > 0 { if t.disabled { t.disabled = false - l.Debugln("progress emitter: enabled") + slog.Debug("Progress emitter: enabled") } if t.interval != newInterval { t.interval = newInterval - l.Debugln("progress emitter: updated interval", t.interval) + l.Debugln("Progress emitter: updated interval", t.interval) } } else if !t.disabled { t.clearLocked() t.disabled = true - l.Debugln("progress emitter: disabled") + slog.Debug("Progress emitter: disabled") } t.minBlocks = to.Options.TempIndexMinBlocks if t.interval < time.Second { diff --git a/lib/model/progressemitter_test.go b/lib/model/progressemitter_test.go index bac8dff7a..e5c10849c 100644 --- a/lib/model/progressemitter_test.go +++ b/lib/model/progressemitter_test.go @@ -18,7 +18,6 @@ import ( "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/events" "github.com/syncthing/syncthing/lib/protocol" - "github.com/syncthing/syncthing/lib/sync" ) var timeout = 100 * time.Millisecond @@ -79,7 +78,6 @@ func TestProgressEmitter(t *testing.T) { s := sharedPullerState{ updated: time.Now(), - mut: sync.NewRWMutex(), } p.Register(&s) @@ -222,7 +220,6 @@ func TestSendDownloadProgressMessages(t *testing.T) { Version: v1, Blocks: blocks, }, - mut: sync.NewRWMutex(), availableUpdated: time.Now(), } p.registry["folder"]["1"] = state1 @@ -305,7 +302,6 @@ func TestSendDownloadProgressMessages(t *testing.T) { Version: v1, Blocks: blocks, }, - mut: sync.NewRWMutex(), available: []int{1, 2, 3}, availableUpdated: time.Now(), } @@ -316,7 +312,6 @@ func TestSendDownloadProgressMessages(t *testing.T) { Version: v1, Blocks: blocks, }, - mut: sync.NewRWMutex(), available: []int{1, 2, 3}, availableUpdated: time.Now(), } @@ -327,7 +322,6 @@ func TestSendDownloadProgressMessages(t *testing.T) { Version: v1, Blocks: blocks, }, - mut: sync.NewRWMutex(), available: []int{1, 2, 3}, availableUpdated: time.Now(), } @@ -375,7 +369,6 @@ func TestSendDownloadProgressMessages(t *testing.T) { Type: protocol.FileInfoTypeDirectory, Blocks: blocks, }, - mut: sync.NewRWMutex(), available: []int{1, 2, 3}, availableUpdated: time.Now(), } @@ -387,7 +380,6 @@ func TestSendDownloadProgressMessages(t *testing.T) { Version: v1, Type: protocol.FileInfoTypeSymlink, }, - mut: sync.NewRWMutex(), available: []int{1, 2, 3}, availableUpdated: time.Now(), } @@ -399,7 +391,6 @@ func TestSendDownloadProgressMessages(t *testing.T) { Version: v1, Blocks: blocks, }, - mut: sync.NewRWMutex(), available: []int{1, 2, 3}, availableUpdated: time.Now(), } @@ -411,7 +402,6 @@ func TestSendDownloadProgressMessages(t *testing.T) { Version: v1, Blocks: blocks[:3], }, - mut: sync.NewRWMutex(), available: []int{1, 2, 3}, availableUpdated: time.Now(), } diff --git a/lib/model/queue.go b/lib/model/queue.go index 7ca129c26..a00c792d4 100644 --- a/lib/model/queue.go +++ b/lib/model/queue.go @@ -7,9 +7,8 @@ package model import ( + "sync" "time" - - "github.com/syncthing/syncthing/lib/sync" ) type jobQueue struct { @@ -25,9 +24,7 @@ type jobQueueEntry struct { } func newJobQueue() *jobQueue { - return &jobQueue{ - mut: sync.NewMutex(), - } + return &jobQueue{} } func (q *jobQueue) Push(file string, size int64, modified time.Time) { diff --git a/lib/model/requests_test.go b/lib/model/requests_test.go index d51982746..eb9472ede 100644 --- a/lib/model/requests_test.go +++ b/lib/model/requests_test.go @@ -1043,7 +1043,7 @@ func TestIgnoreDeleteUnignore(t *testing.T) { if !f.Version.Equal(protocol.Vector{}) && f.Deleted { t.Error("Received deleted index entry with non-empty version") } - l.Infoln(f) + t.Log(f) close(done) return nil }) diff --git a/lib/model/service_map.go b/lib/model/service_map.go index cc681e9d1..59d07d996 100644 --- a/lib/model/service_map.go +++ b/lib/model/service_map.go @@ -37,7 +37,7 @@ func newServiceMap[K comparable, S suture.Service](eventLogger events.Logger) *s tokens: make(map[K]suture.ServiceToken), eventLogger: eventLogger, } - m.supervisor = suture.New(m.String(), svcutil.SpecWithDebugLogger(l)) + m.supervisor = suture.New(m.String(), svcutil.SpecWithDebugLogger()) return m } diff --git a/lib/model/sharedpullerstate.go b/lib/model/sharedpullerstate.go index f0b66f685..d089da09f 100644 --- a/lib/model/sharedpullerstate.go +++ b/lib/model/sharedpullerstate.go @@ -10,6 +10,7 @@ import ( "encoding/binary" "fmt" "io" + "sync" "time" "google.golang.org/protobuf/proto" @@ -18,7 +19,6 @@ import ( "github.com/syncthing/syncthing/lib/fs" "github.com/syncthing/syncthing/lib/osutil" "github.com/syncthing/syncthing/lib/protocol" - "github.com/syncthing/syncthing/lib/sync" ) // A sharedPullerState is kept for each file that is being synced and is kept @@ -39,19 +39,18 @@ type sharedPullerState struct { fsync bool // Mutable, must be locked for access - err error // The first error we hit - writer *lockedWriterAt // Wraps fd to prevent fd closing at the same time as writing - copyTotal int // Total number of copy actions for the whole job - pullTotal int // Total number of pull actions for the whole job - copyOrigin int // Number of blocks copied from the original file - copyOriginShifted int // Number of blocks copied from the original file but shifted - copyNeeded int // Number of copy actions still pending - pullNeeded int // Number of block pulls still pending - updated time.Time // Time when any of the counters above were last updated - closed bool // True if the file has been finalClosed. - available []int // Indexes of the blocks that are available in the temporary file - availableUpdated time.Time // Time when list of available blocks was last updated - mut sync.RWMutex // Protects the above + err error // The first error we hit + writer *lockedWriterAt // Wraps fd to prevent fd closing at the same time as writing + copyTotal int // Total number of copy actions for the whole job + pullTotal int // Total number of pull actions for the whole job + copyOrigin int // Number of blocks copied from the original file + copyNeeded int // Number of copy actions still pending + pullNeeded int // Number of block pulls still pending + updated time.Time // Time when any of the counters above were last updated + closed bool // True if the file has been finalClosed. + available []int // Indexes of the blocks that are available in the temporary file + availableUpdated time.Time // Time when list of available blocks was last updated + mut sync.RWMutex // Protects the above } func newSharedPullerState(file protocol.FileInfo, fs fs.Filesystem, folderID, tempName string, blocks []protocol.BlockInfo, reused []int, ignorePerms, hasCurFile bool, curFile protocol.FileInfo, sparse bool, fsync bool) *sharedPullerState { @@ -70,7 +69,6 @@ func newSharedPullerState(file protocol.FileInfo, fs fs.Filesystem, folderID, te ignorePerms: ignorePerms, hasCurFile: hasCurFile, curFile: curFile, - mut: sync.NewRWMutex(), sparse: sparse, fsync: fsync, created: time.Now(), @@ -225,7 +223,7 @@ func (s *sharedPullerState) tempFileInWritableDir(_ string) error { } // Same fd will be used by all writers - s.writer = &lockedWriterAt{sync.NewRWMutex(), fd} + s.writer = &lockedWriterAt{fd: fd} return nil } diff --git a/lib/model/sharedpullerstate_test.go b/lib/model/sharedpullerstate_test.go index 1a3d8eb5f..7841f95d7 100644 --- a/lib/model/sharedpullerstate_test.go +++ b/lib/model/sharedpullerstate_test.go @@ -11,7 +11,6 @@ import ( "github.com/syncthing/syncthing/lib/fs" "github.com/syncthing/syncthing/lib/rand" - "github.com/syncthing/syncthing/lib/sync" ) // Test creating temporary file inside read-only directory @@ -22,7 +21,6 @@ func TestReadOnlyDir(t *testing.T) { s := sharedPullerState{ fs: ffs, tempName: "testdir/.temp_name", - mut: sync.NewRWMutex(), } fd, err := s.tempFile() diff --git a/lib/model/util.go b/lib/model/util.go index 2fefd3100..9cd60de22 100644 --- a/lib/model/util.go +++ b/lib/model/util.go @@ -10,10 +10,12 @@ import ( "context" "errors" "fmt" + "log/slog" "path/filepath" "time" "github.com/prometheus/client_golang/prometheus" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/fs" ) @@ -46,11 +48,11 @@ func inWritableDir(fn func(string) error, targetFs fs.Filesystem, path string, i // caller is inappropriate.) defer func() { if err := targetFs.Chmod(dir, mode); err != nil && !fs.IsNotExist(err) { - logFn := l.Warnln + logFn := slog.Warn if ignorePerms { - logFn = l.Debugln + logFn = slog.Debug } - logFn("Failed to restore directory permissions after gaining write access:", err) + logFn("Failed to restore directory permissions after gaining write access", slogutil.Error(err)) } }() } diff --git a/lib/nat/debug.go b/lib/nat/debug.go index 127a3c9ee..03aa59eb6 100644 --- a/lib/nat/debug.go +++ b/lib/nat/debug.go @@ -6,8 +6,6 @@ package nat -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("nat", "NAT discovery and port mapping") +var l = slogutil.NewAdapter("NAT discovery and port mapping") diff --git a/lib/nat/service.go b/lib/nat/service.go index 9a196abf0..59646a261 100644 --- a/lib/nat/service.go +++ b/lib/nat/service.go @@ -10,15 +10,16 @@ import ( "context" "fmt" "hash/fnv" + "log/slog" "math/rand" "net" "slices" - stdsync "sync" + "sync" "time" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/protocol" - "github.com/syncthing/syncthing/lib/sync" ) // Service runs a loop for discovery of IGDs (Internet Gateway Devices) and @@ -38,8 +39,6 @@ func NewService(id protocol.DeviceID, cfg config.Wrapper) *Service { id: id, cfg: cfg, processScheduled: make(chan struct{}, 1), - - mut: sync.NewRWMutex(), } cfgCopy := cfg.RawCopy() s.CommitConfiguration(cfgCopy, cfgCopy) @@ -49,11 +48,11 @@ func NewService(id protocol.DeviceID, cfg config.Wrapper) *Service { func (s *Service) CommitConfiguration(_, to config.Configuration) bool { s.mut.Lock() if !s.enabled && to.Options.NATEnabled { - l.Debugln("Starting NAT service") + slog.Debug("Starting NAT service") s.enabled = true s.scheduleProcess() } else if s.enabled && !to.Options.NATEnabled { - l.Debugln("Stopping NAT service") + slog.Debug("Stopping NAT service") s.enabled = false } s.mut.Unlock() @@ -64,7 +63,7 @@ func (s *Service) Serve(ctx context.Context) error { s.cfg.Subscribe(s) defer s.cfg.Unsubscribe(s) - announce := stdsync.Once{} + var announce sync.Once timer := time.NewTimer(0) @@ -97,11 +96,7 @@ func (s *Service) Serve(ctx context.Context) error { timer.Reset(renewIn) if found != -1 { announce.Do(func() { - suffix := "s" - if found == 1 { - suffix = "" - } - l.Infoln("Detected", found, "NAT service"+suffix) + slog.Info("Detected NAT services", "count", found) }) } } @@ -171,7 +166,6 @@ func (s *Service) NewMapping(protocol Protocol, ipVersion IPVersion, ip net.IP, Port: port, }, extAddresses: make(map[string][]Address), - mut: sync.NewRWMutex(), ipVersion: ipVersion, } @@ -259,7 +253,7 @@ func (s *Service) verifyExistingLocked(ctx context.Context, mapping *Mapping, na // entry always has the external port. responseAddrs, err := s.tryNATDevice(ctx, nat, mapping.address, extAddrs[0].Port, mapping.protocol, leaseTime) if err != nil { - l.Infof("Failed to renew %s -> %v open port on %s: %s", mapping, extAddrs, id, err) + slog.WarnContext(ctx, "Failed to renew open port", slog.String("mapping", mapping.String()), slog.Any("addresses", extAddrs), slog.String("id", id), slogutil.Error(err)) mapping.removeAddressLocked(id) change = true continue @@ -311,7 +305,7 @@ func (s *Service) acquireNewLocked(ctx context.Context, mapping *Mapping, nats m addrs, err := s.tryNATDevice(ctx, nat, mapping.address, 0, mapping.protocol, leaseTime) if err != nil { - l.Infof("Failed to acquire %s open port on %s: %s", mapping, id, err) + slog.WarnContext(ctx, "Failed to acquire open port", slog.String("mapping", mapping.String()), slog.String("id", id), slogutil.Error(err)) continue } diff --git a/lib/nat/structs.go b/lib/nat/structs.go index f4f8cbaf4..77facc9b2 100644 --- a/lib/nat/structs.go +++ b/lib/nat/structs.go @@ -8,10 +8,11 @@ package nat import ( "fmt" + "log/slog" "net" + "strconv" + "sync" "time" - - "github.com/syncthing/syncthing/lib/sync" ) type MappingChangeSubscriber func() @@ -28,14 +29,14 @@ type Mapping struct { } func (m *Mapping) setAddressLocked(id string, addresses []Address) { - l.Infof("New external port opened: external %s address(es) %v to local address %s.", m.protocol, addresses, m.address) + slog.Info("New external port opened", "protocol", m.protocol, "external", addresses, "local", m.address, "gateway", id) m.extAddresses[id] = addresses } func (m *Mapping) removeAddressLocked(id string) { addresses, ok := m.extAddresses[id] if ok { - l.Infof("Removing external open port: %s address(es) %v for gateway %s.", m.protocol, addresses, id) + slog.Info("Removing external open port", "protocol", m.protocol, "external", addresses, "gateway", id) delete(m.extAddresses, id) } } @@ -123,7 +124,7 @@ func (a Address) String() string { } else { ipStr = a.IP.String() } - return net.JoinHostPort(ipStr, fmt.Sprintf("%d", a.Port)) + return net.JoinHostPort(ipStr, strconv.Itoa(a.Port)) } func (a Address) GoString() string { diff --git a/lib/osutil/osutil.go b/lib/osutil/osutil.go index 572bfe919..047b5101d 100644 --- a/lib/osutil/osutil.go +++ b/lib/osutil/osutil.go @@ -10,15 +10,15 @@ package osutil import ( "path/filepath" "strings" + "sync" "github.com/syncthing/syncthing/lib/build" "github.com/syncthing/syncthing/lib/fs" - "github.com/syncthing/syncthing/lib/sync" ) // Try to keep this entire operation atomic-like. We shouldn't be doing this // often enough that there is any contention on this lock. -var renameLock = sync.NewMutex() +var renameLock sync.Mutex // RenameOrCopy renames a file, leaving source file intact in case of failure. // Tries hard to succeed on various systems by temporarily tweaking directory diff --git a/lib/pmp/debug.go b/lib/pmp/debug.go index 91e4d5b0a..3a76691c5 100644 --- a/lib/pmp/debug.go +++ b/lib/pmp/debug.go @@ -6,8 +6,6 @@ package pmp -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("pmp", "NAT-PMP discovery and port mapping") +var l = slogutil.NewAdapter("NAT-PMP discovery and port mapping") diff --git a/lib/pmp/pmp.go b/lib/pmp/pmp.go index e5acf1409..feee6972d 100644 --- a/lib/pmp/pmp.go +++ b/lib/pmp/pmp.go @@ -10,6 +10,7 @@ import ( "context" "errors" "fmt" + "log/slog" "net" "strings" "time" @@ -55,7 +56,7 @@ func Discover(ctx context.Context, renewal, timeout time.Duration) []nat.Device return nil } if strings.Contains(err.Error(), "Timed out") { - l.Debugln("Timeout trying to get external address, assume no NAT-PMP available") + slog.Debug("Timeout trying to get external address, assume no NAT-PMP available") return nil } } diff --git a/lib/protocol/bep_clusterconfig.go b/lib/protocol/bep_clusterconfig.go index 3b6dee1c5..dd5f485ef 100644 --- a/lib/protocol/bep_clusterconfig.go +++ b/lib/protocol/bep_clusterconfig.go @@ -8,6 +8,7 @@ package protocol import ( "fmt" + "log/slog" "github.com/syncthing/syncthing/internal/gen/bep" ) @@ -110,6 +111,13 @@ func (f Folder) Description() string { return fmt.Sprintf("%q (%s)", f.Label, f.ID) } +func (f Folder) LogAttr() slog.Attr { + if f.Label == "" || f.Label == f.ID { + return slog.Group("folder", slog.String("id", f.ID)) + } + return slog.Group("folder", slog.String("label", f.Label), slog.String("id", f.ID)) +} + func (f Folder) IsRunning() bool { switch f.StopReason { case FolderStopReasonPaused: diff --git a/lib/protocol/bep_fileinfo.go b/lib/protocol/bep_fileinfo.go index 2fd6dbfa1..775891062 100644 --- a/lib/protocol/bep_fileinfo.go +++ b/lib/protocol/bep_fileinfo.go @@ -11,6 +11,7 @@ import ( "crypto/sha256" "encoding/binary" "fmt" + "log/slog" "slices" "strings" "time" @@ -213,6 +214,30 @@ func (f *FileInfo) WinsConflict(other FileInfo) bool { return f.FileVersion().Compare(other.FileVersion()) == ConcurrentGreater } +func (f *FileInfo) LogAttr() slog.Attr { + attrs := []any{slog.String("name", f.Name)} + var kind string + switch f.Type { + case FileInfoTypeFile: + kind = "file" + if !f.Deleted { + attrs = append(attrs, + slog.Any("modified", f.ModTime()), + slog.String("permissions", fmt.Sprintf("0%03o", f.Permissions)), + slog.Int64("size", f.Size), + slog.Int("blocksize", f.BlockSize()), + ) + } + case FileInfoTypeDirectory: + kind = "dir" + attrs = append(attrs, slog.String("permissions", fmt.Sprintf("0%03o", f.Permissions))) + case FileInfoTypeSymlink: + kind = "symlink" + attrs = append(attrs, slog.String("target", string(f.SymlinkTarget))) + } + return slog.Group(kind, attrs...) +} + func FileInfoFromWire(w *bep.FileInfo) FileInfo { var blocks []BlockInfo if len(w.Blocks) > 0 { diff --git a/lib/protocol/debug.go b/lib/protocol/debug.go index 4acd6c74c..96dc997a5 100644 --- a/lib/protocol/debug.go +++ b/lib/protocol/debug.go @@ -6,8 +6,6 @@ package protocol -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("protocol", "The BEP protocol") +var l = slogutil.NewAdapter("The BEP protocol") diff --git a/lib/protocol/deviceid.go b/lib/protocol/deviceid.go index a4ff04a53..950712d69 100644 --- a/lib/protocol/deviceid.go +++ b/lib/protocol/deviceid.go @@ -13,6 +13,7 @@ import ( "encoding/binary" "errors" "fmt" + "log/slog" "strings" ) @@ -81,6 +82,14 @@ func (n DeviceID) String() string { return id } +func (n DeviceID) LogAttr() slog.Attr { + return slog.Any("device", n.LogValue()) +} + +func (n DeviceID) LogValue() slog.Value { + return slog.StringValue(n.Short().String()) +} + func (n DeviceID) GoString() string { return n.String() } diff --git a/lib/protocol/nativemodel_windows.go b/lib/protocol/nativemodel_windows.go index 88bc2ca6b..96c6dd57b 100644 --- a/lib/protocol/nativemodel_windows.go +++ b/lib/protocol/nativemodel_windows.go @@ -12,9 +12,11 @@ package protocol // Windows uses backslashes as file separator import ( - "fmt" + "log/slog" "path/filepath" "strings" + + "github.com/syncthing/syncthing/internal/slogutil" ) func makeNative(m rawModel) rawModel { return nativeModel{m} } @@ -35,7 +37,7 @@ func (m nativeModel) IndexUpdate(idxUp *IndexUpdate) error { func (m nativeModel) Request(req *Request) (RequestResponse, error) { if strings.Contains(req.Name, `\`) { - l.Warnf("Dropping request for %s, contains invalid path separator", req.Name) + slog.Debug("Dropping request containing invalid path separator", slogutil.FilePath(req.Name)) return nil, ErrNoSuchFile } @@ -47,12 +49,11 @@ func fixupFiles(files []FileInfo) []FileInfo { var out []FileInfo for i := range files { if strings.Contains(files[i].Name, `\`) { - msg := fmt.Sprintf("Dropping index entry for %s, contains invalid path separator", files[i].Name) if files[i].Deleted { // Dropping a deleted item doesn't have any consequences. - l.Debugln(msg) + slog.Debug("Dropping index entry containing invalid path separator", slogutil.FilePath(files[i].Name)) } else { - l.Warnln(msg) + slog.Error("Dropping index entry containing invalid path separator", slogutil.FilePath(files[i].Name)) } if out == nil { // Most incoming updates won't contain anything invalid, so diff --git a/lib/rc/debug.go b/lib/rc/debug.go index 37c53f235..f8b454ce9 100644 --- a/lib/rc/debug.go +++ b/lib/rc/debug.go @@ -6,8 +6,6 @@ package rc -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("rc", "Remote control package") +var l = slogutil.NewAdapter("Remote control package") diff --git a/lib/rc/rc.go b/lib/rc/rc.go index 919631cb3..5b57f4d4b 100644 --- a/lib/rc/rc.go +++ b/lib/rc/rc.go @@ -15,12 +15,14 @@ import ( "fmt" "io" "log" + "log/slog" "net/http" "net/url" "os" "os/exec" "path/filepath" "strconv" + "sync" "time" "github.com/syncthing/syncthing/lib/config" @@ -28,7 +30,6 @@ import ( "github.com/syncthing/syncthing/lib/events" "github.com/syncthing/syncthing/lib/model" "github.com/syncthing/syncthing/lib/protocol" - "github.com/syncthing/syncthing/lib/sync" ) // APIKey is set via the STGUIAPIKEY variable when we launch the binary, to @@ -60,7 +61,6 @@ func NewProcess(addr string) *Process { addr: addr, sequence: make(map[string]map[string]int64), done: make(map[string]bool), - eventMut: sync.NewMutex(), startComplete: make(chan struct{}), stopped: make(chan struct{}), } @@ -481,7 +481,7 @@ func (p *Process) eventLoop() { for _, ev := range evs { if ev.ID != since+1 { - l.Warnln("Event ID jumped", since, "to", ev.ID) + slog.Warn("Event ID jumped", "from", since, "to", ev.ID) } since = ev.ID @@ -573,7 +573,7 @@ func (p *Process) eventLoop() { folder := data["folder"].(string) p.eventMut.Lock() m := p.updateSequenceLocked(folder, device, data["sequence"]) - l.Debugf("FolderCompletion %v\n\t%+v", p.id, folder, m) + l.Debugln("FolderCompletion", p.id, folder, m) p.eventMut.Unlock() } } diff --git a/lib/relay/client/debug.go b/lib/relay/client/debug.go index d6bb82e4f..50f6d1638 100644 --- a/lib/relay/client/debug.go +++ b/lib/relay/client/debug.go @@ -2,8 +2,6 @@ package client -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("relay", "") +var l = slogutil.NewAdapter("Relay client") diff --git a/lib/relay/client/static.go b/lib/relay/client/static.go index 11f50083b..081491f68 100644 --- a/lib/relay/client/static.go +++ b/lib/relay/client/static.go @@ -7,10 +7,12 @@ import ( "crypto/tls" "errors" "fmt" + "log/slog" "net" "net/url" "time" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/dialer" "github.com/syncthing/syncthing/lib/osutil" syncthingprotocol "github.com/syncthing/syncthing/lib/protocol" @@ -65,7 +67,7 @@ func (c *staticClient) serve(ctx context.Context) error { return err } - l.Infof("Joined relay %s://%s", c.uri.Scheme, c.uri.Host) + slog.InfoContext(ctx, "Joined relay", slogutil.URI(fmt.Sprintf("%s://%s", c.uri.Scheme, c.uri.Host))) messages := make(chan interface{}) errorsc := make(chan error, 1) @@ -105,7 +107,7 @@ func (c *staticClient) serve(ctx context.Context) error { return errors.New("relay full") default: - l.Debugln("Relay: protocol error: unexpected message %v", msg) + l.Debugf("Relay: protocol error: unexpected message %v", msg) return fmt.Errorf("protocol error: unexpected message %v", msg) } diff --git a/lib/scanner/blockqueue.go b/lib/scanner/blockqueue.go index 8182ef74d..97e4799e9 100644 --- a/lib/scanner/blockqueue.go +++ b/lib/scanner/blockqueue.go @@ -9,10 +9,10 @@ package scanner import ( "context" "errors" + "sync" "github.com/syncthing/syncthing/lib/fs" "github.com/syncthing/syncthing/lib/protocol" - "github.com/syncthing/syncthing/lib/sync" ) // HashFile hashes the files and returns a list of blocks representing the file. @@ -81,7 +81,6 @@ func newParallelHasher(ctx context.Context, folderID string, fs fs.Filesystem, w inbox: inbox, counter: counter, done: done, - wg: sync.NewWaitGroup(), } ph.wg.Add(workers) diff --git a/lib/scanner/debug.go b/lib/scanner/debug.go index 80f9b617b..c0114ab1d 100644 --- a/lib/scanner/debug.go +++ b/lib/scanner/debug.go @@ -6,8 +6,6 @@ package scanner -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("scanner", "File change detection and hashing") +var l = slogutil.NewAdapter("File change detection and hashing") diff --git a/lib/scanner/walk.go b/lib/scanner/walk.go index ebdf65c9f..26017ea1d 100644 --- a/lib/scanner/walk.go +++ b/lib/scanner/walk.go @@ -10,6 +10,7 @@ import ( "context" "errors" "fmt" + "log/slog" "path/filepath" "strings" "sync/atomic" @@ -19,6 +20,7 @@ import ( metrics "github.com/rcrowley/go-metrics" "golang.org/x/text/unicode/norm" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/build" "github.com/syncthing/syncthing/lib/events" "github.com/syncthing/syncthing/lib/fs" @@ -245,7 +247,7 @@ func (w *walker) scan(ctx context.Context, toHashChan chan<- protocol.FileInfo, if len(w.Subs) == 0 { if err := w.Filesystem.Walk(".", hashFiles); isWarnableError(err) { w.EventLogger.Log(events.Failure, walkFailureEventDesc) - l.Warnf("Aborted scan due to an unexpected error: %v", err) + slog.ErrorContext(ctx, "Aborted scan due to an unexpected error", slogutil.Error(err)) } } else { for _, sub := range w.Subs { @@ -255,7 +257,7 @@ func (w *walker) scan(ctx context.Context, toHashChan chan<- protocol.FileInfo, } if err := w.Filesystem.Walk(sub, hashFiles); isWarnableError(err) { w.EventLogger.Log(events.Failure, walkFailureEventDesc) - l.Warnf("Aborted scan of path '%v' due to an unexpected error: %v", sub, err) + slog.ErrorContext(ctx, "Aborted scan due to an unexpected error", slogutil.FilePath(sub), slogutil.Error(err)) } } } @@ -613,7 +615,7 @@ func (w *walker) applyNormalization(path, normPath string, info fs.FileInfo) (st if err = w.Filesystem.Rename(path, normPath); err != nil { return "", err } - l.Infof(`Normalized UTF8 encoding of file name "%s".`, path) + slog.Info("Normalized UTF8 encoding of file name", slogutil.FilePath(path)) return normPath, nil } if w.Filesystem.SameFile(info, normInfo) { @@ -631,7 +633,7 @@ func (w *walker) applyNormalization(path, normPath string, info fs.FileInfo) (st if err = w.Filesystem.Rename(tempPath, normPath); err != nil { // I don't ever expect this to happen, but if it does, we should probably tell our caller that the normalized // path is the temp path: that way at least the user's data still gets synced. - l.Warnf(`Error renaming "%s" to "%s" while normalizating UTF8 encoding: %v. You will want to rename this file back manually`, tempPath, normPath, err) + slog.Error("Failed to rename while normalizating UTF8 encoding; please rename temp file manually", slog.String("from", tempPath), slog.String("to", normPath), slogutil.Error(err)) return tempPath, nil } return normPath, nil diff --git a/lib/stun/debug.go b/lib/stun/debug.go index fe67d5b0d..5c6400ec9 100644 --- a/lib/stun/debug.go +++ b/lib/stun/debug.go @@ -6,8 +6,6 @@ package stun -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("stun", "STUN functionality") +var l = slogutil.NewAdapter("STUN functionality") diff --git a/lib/stun/stun.go b/lib/stun/stun.go index b15f84b59..a99eb94a7 100644 --- a/lib/stun/stun.go +++ b/lib/stun/stun.go @@ -10,6 +10,7 @@ import ( "context" "errors" "fmt" + "log/slog" "net" "time" @@ -129,7 +130,7 @@ func (s *Service) Serve(ctx context.Context) error { // Are we disabled? if s.cfg.Options().IsStunDisabled() { - l.Infoln("STUN disabled") + slog.InfoContext(ctx, "STUN disabled") s.setNATType(NATUnknown) s.setExternalAddress(nil, "") goto disabled diff --git a/lib/svcutil/svcutil.go b/lib/svcutil/svcutil.go index 1db3d800a..72d1825be 100644 --- a/lib/svcutil/svcutil.go +++ b/lib/svcutil/svcutil.go @@ -10,11 +10,11 @@ import ( "context" "errors" "fmt" + "log/slog" + "sync" "time" - "github.com/syncthing/syncthing/lib/logger" - "github.com/syncthing/syncthing/lib/sync" - + "github.com/syncthing/syncthing/internal/slogutil" "github.com/thejerf/suture/v4" ) @@ -106,7 +106,6 @@ func AsService(fn func(ctx context.Context) error, creator string) ServiceWithEr return &service{ creator: creator, serve: fn, - mut: sync.NewMutex(), } } @@ -159,12 +158,12 @@ func OnSupervisorDone(sup *suture.Supervisor, fn func()) { sup.Add(doneService(fn)) } -func SpecWithDebugLogger(l logger.Logger) suture.Spec { - return spec(func(e suture.Event) { l.Debugln(e) }) +func SpecWithDebugLogger() suture.Spec { + return spec(func(e suture.Event) { slog.Debug(e.String()) }) } -func SpecWithInfoLogger(l logger.Logger) suture.Spec { - return spec(infoEventHook(l)) +func SpecWithInfoLogger() suture.Spec { + return spec(infoEventHook()) } func spec(eventHook suture.EventHook) suture.Spec { @@ -179,31 +178,32 @@ func spec(eventHook suture.EventHook) suture.Spec { // infoEventHook prints service failures and failures to stop services at level // info. All other events and identical, consecutive failures are logged at // debug only. -func infoEventHook(l logger.Logger) suture.EventHook { +func infoEventHook() suture.EventHook { var prevTerminate suture.EventServiceTerminate return func(ei suture.Event) { + m := ei.Map() + l := slog.Default().With("supervisor", m["supervisor_name"], "service", m["service_name"]) switch e := ei.(type) { case suture.EventStopTimeout: - l.Infof("%s: Service %s failed to terminate in a timely manner", e.SupervisorName, e.ServiceName) + l.Warn("Service failed to terminate in a timely manner") case suture.EventServicePanic: - l.Warnln("Caught a service panic, which shouldn't happen") - l.Infoln(e) + l.Error("Caught a service panic, which shouldn't happen") + l.Warn(e.String()) //nolint:sloglint case suture.EventServiceTerminate: - msg := fmt.Sprintf("%s: service %s failed: %s", e.SupervisorName, e.ServiceName, e.Err) if e.ServiceName == prevTerminate.ServiceName && e.Err == prevTerminate.Err { - l.Debugln(msg) + l.Debug("Service failed repeatedly", slogutil.Error(e.Err)) } else { - l.Infoln(msg) + l.Warn("Service failed", slogutil.Error(e.Err)) } prevTerminate = e - l.Debugln(e) // Contains some backoff statistics + l.Debug(e.String()) // Contains some backoff statistics case suture.EventBackoff: - l.Debugf("%s: exiting the backoff state.", e.SupervisorName) + l.Debug("Exiting the backoff state") case suture.EventResume: - l.Debugf("%s: too many service failures - entering the backoff state.", e.SupervisorName) + l.Debug("Too many service failures - entering the backoff state") default: - l.Warnln("Unknown suture supervisor event type", e.Type()) - l.Infoln(e) + l.Warn("Unknown suture supervisor event", slog.Any("type", e.Type())) + l.Warn(e.String()) //nolint:sloglint } } } diff --git a/lib/sync/debug.go b/lib/sync/debug.go deleted file mode 100644 index a1c73a379..000000000 --- a/lib/sync/debug.go +++ /dev/null @@ -1,32 +0,0 @@ -// Copyright (C) 2015 The Syncthing Authors. -// -// This Source Code Form is subject to the terms of the Mozilla Public -// License, v. 2.0. If a copy of the MPL was not distributed with this file, -// You can obtain one at https://mozilla.org/MPL/2.0/. - -package sync - -import ( - "os" - "strconv" - "time" - - "github.com/syncthing/syncthing/lib/logger" -) - -var ( - threshold = 100 * time.Millisecond - l = logger.DefaultLogger.NewFacility("sync", "Mutexes") - - // We make an exception in this package and have an actual "if debug { ... - // }" variable, as it may be rather performance critical and does - // nonstandard things (from a debug logging PoV). - debug = logger.DefaultLogger.ShouldDebug("sync") -) - -func init() { - if n, _ := strconv.Atoi(os.Getenv("STLOCKTHRESHOLD")); n > 0 { - threshold = time.Duration(n) * time.Millisecond - } - l.Debugf("Enabling lock logging at %v threshold", threshold) -} diff --git a/lib/sync/sync.go b/lib/sync/sync.go deleted file mode 100644 index a3ea396de..000000000 --- a/lib/sync/sync.go +++ /dev/null @@ -1,290 +0,0 @@ -// Copyright (C) 2015 The Syncthing Authors. -// -// This Source Code Form is subject to the terms of the Mozilla Public -// License, v. 2.0. If a copy of the MPL was not distributed with this file, -// You can obtain one at https://mozilla.org/MPL/2.0/. - -package sync - -import ( - "fmt" - "path/filepath" - "runtime" - "strconv" - "strings" - "sync" - "sync/atomic" - "time" -) - -var timeNow = time.Now - -type Mutex interface { - Lock() - Unlock() -} - -type RWMutex interface { - Mutex - RLock() - RUnlock() -} - -type WaitGroup interface { - Add(int) - Done() - Wait() -} - -func NewMutex() Mutex { - if debug { - mutex := &loggedMutex{} - mutex.holder.Store(holder{}) - return mutex - } - return &sync.Mutex{} -} - -func NewRWMutex() RWMutex { - if debug { - mutex := &loggedRWMutex{ - readHolders: make(map[int][]holder), - unlockers: make(chan holder, 1024), - } - mutex.holder.Store(holder{}) - return mutex - } - return &sync.RWMutex{} -} - -func NewWaitGroup() WaitGroup { - if debug { - return &loggedWaitGroup{} - } - return &sync.WaitGroup{} -} - -type holder struct { - at string - time time.Time - goid int -} - -func (h holder) String() string { - if h.at == "" { - return "not held" - } - return fmt.Sprintf("at %s goid: %d for %s", h.at, h.goid, timeNow().Sub(h.time)) -} - -type loggedMutex struct { - sync.Mutex - holder atomic.Value -} - -func (m *loggedMutex) Lock() { - m.Mutex.Lock() - m.holder.Store(getHolder()) -} - -func (m *loggedMutex) Unlock() { - currentHolder := m.holder.Load().(holder) - duration := timeNow().Sub(currentHolder.time) - if duration >= threshold { - l.Debugf("Mutex held for %v. Locked at %s unlocked at %s", duration, currentHolder.at, getHolder().at) - } - m.holder.Store(holder{}) - m.Mutex.Unlock() -} - -func (m *loggedMutex) Holders() string { - return m.holder.Load().(holder).String() -} - -type loggedRWMutex struct { - sync.RWMutex - holder atomic.Value - - readHolders map[int][]holder - readHoldersMut sync.Mutex - - logUnlockers atomic.Bool - unlockers chan holder -} - -func (m *loggedRWMutex) Lock() { - start := timeNow() - - m.logUnlockers.Store(true) - m.RWMutex.Lock() - m.logUnlockers.Store(false) - - holder := getHolder() - m.holder.Store(holder) - - duration := holder.time.Sub(start) - - if duration > threshold { - var unlockerStrings []string - loop: - for { - select { - case holder := <-m.unlockers: - unlockerStrings = append(unlockerStrings, holder.String()) - default: - break loop - } - } - l.Debugf("RWMutex took %v to lock. Locked at %s. RUnlockers while locking:\n%s", duration, holder.at, strings.Join(unlockerStrings, "\n")) - } -} - -func (m *loggedRWMutex) Unlock() { - currentHolder := m.holder.Load().(holder) - duration := timeNow().Sub(currentHolder.time) - if duration >= threshold { - l.Debugf("RWMutex held for %v. Locked at %s unlocked at %s", duration, currentHolder.at, getHolder().at) - } - m.holder.Store(holder{}) - m.RWMutex.Unlock() -} - -func (m *loggedRWMutex) RLock() { - m.RWMutex.RLock() - holder := getHolder() - m.readHoldersMut.Lock() - m.readHolders[holder.goid] = append(m.readHolders[holder.goid], holder) - m.readHoldersMut.Unlock() -} - -func (m *loggedRWMutex) RUnlock() { - id := goid() - m.readHoldersMut.Lock() - current := m.readHolders[id] - if len(current) > 0 { - m.readHolders[id] = current[:len(current)-1] - } - m.readHoldersMut.Unlock() - if m.logUnlockers.Load() { - holder := getHolder() - select { - case m.unlockers <- holder: - default: - l.Debugf("Dropped holder %s as channel full", holder) - } - } - m.RWMutex.RUnlock() -} - -func (m *loggedRWMutex) Holders() string { - output := m.holder.Load().(holder).String() + " (writer)" - m.readHoldersMut.Lock() - for _, holders := range m.readHolders { - for _, holder := range holders { - output += "\n" + holder.String() + " (reader)" - } - } - m.readHoldersMut.Unlock() - return output -} - -type loggedWaitGroup struct { - sync.WaitGroup -} - -func (wg *loggedWaitGroup) Wait() { - start := timeNow() - wg.WaitGroup.Wait() - duration := timeNow().Sub(start) - if duration >= threshold { - l.Debugf("WaitGroup took %v at %s", duration, getHolder()) - } -} - -func getHolder() holder { - _, file, line, _ := runtime.Caller(2) - file = filepath.Join(filepath.Base(filepath.Dir(file)), filepath.Base(file)) - return holder{ - at: fmt.Sprintf("%s:%d", file, line), - goid: goid(), - time: timeNow(), - } -} - -func goid() int { - var buf [64]byte - n := runtime.Stack(buf[:], false) - idField := strings.Fields(strings.TrimPrefix(string(buf[:n]), "goroutine "))[0] - id, err := strconv.Atoi(idField) - if err != nil { - return -1 - } - return id -} - -// TimeoutCond is a variant on Cond. It has roughly the same semantics regarding 'L' - it must be held -// both when broadcasting and when calling TimeoutCondWaiter.Wait() -// Call Broadcast() to broadcast to all waiters on the TimeoutCond. Call SetupWait to create a -// TimeoutCondWaiter configured with the given timeout, which can then be used to listen for -// broadcasts. -type TimeoutCond struct { - L sync.Locker - ch chan struct{} -} - -// TimeoutCondWaiter is a type allowing a consumer to wait on a TimeoutCond with a timeout. Wait() may be called multiple times, -// and will return true every time that the TimeoutCond is broadcast to. Once the configured timeout -// expires, Wait() will return false. -// Call Stop() to release resources once this TimeoutCondWaiter is no longer needed. -type TimeoutCondWaiter struct { - c *TimeoutCond - timer *time.Timer -} - -func NewTimeoutCond(l sync.Locker) *TimeoutCond { - return &TimeoutCond{ - L: l, - } -} - -func (c *TimeoutCond) Broadcast() { - // ch.L must be locked when calling this function - - if c.ch != nil { - close(c.ch) - c.ch = nil - } -} - -func (c *TimeoutCond) SetupWait(timeout time.Duration) *TimeoutCondWaiter { - timer := time.NewTimer(timeout) - - return &TimeoutCondWaiter{ - c: c, - timer: timer, - } -} - -func (w *TimeoutCondWaiter) Wait() bool { - // ch.L must be locked when calling this function - - // Ensure that the channel exists, since we're going to be waiting on it - if w.c.ch == nil { - w.c.ch = make(chan struct{}) - } - ch := w.c.ch - - w.c.L.Unlock() - defer w.c.L.Lock() - - select { - case <-w.timer.C: - return false - case <-ch: - return true - } -} - -func (w *TimeoutCondWaiter) Stop() { - w.timer.Stop() -} diff --git a/lib/sync/sync_test.go b/lib/sync/sync_test.go deleted file mode 100644 index 54e11a641..000000000 --- a/lib/sync/sync_test.go +++ /dev/null @@ -1,349 +0,0 @@ -// Copyright (C) 2015 The Syncthing Authors. -// -// This Source Code Form is subject to the terms of the Mozilla Public -// License, v. 2.0. If a copy of the MPL was not distributed with this file, -// You can obtain one at https://mozilla.org/MPL/2.0/. - -package sync - -import ( - "strings" - "sync" - "testing" - "time" - - "github.com/syncthing/syncthing/lib/logger" -) - -const ( - logThreshold = 100 * time.Millisecond - shortWait = 5 * time.Millisecond - longWait = 125 * time.Millisecond -) - -func TestTypes(t *testing.T) { - debug = false - l.SetDebug("sync", false) - - if _, ok := NewMutex().(*sync.Mutex); !ok { - t.Error("Wrong type") - } - - if _, ok := NewRWMutex().(*sync.RWMutex); !ok { - t.Error("Wrong type") - } - - if _, ok := NewWaitGroup().(*sync.WaitGroup); !ok { - t.Error("Wrong type") - } - - debug = true - l.SetDebug("sync", true) - - if _, ok := NewMutex().(*loggedMutex); !ok { - t.Error("Wrong type") - } - - if _, ok := NewRWMutex().(*loggedRWMutex); !ok { - t.Error("Wrong type") - } - - if _, ok := NewWaitGroup().(*loggedWaitGroup); !ok { - t.Error("Wrong type") - } - - debug = false - l.SetDebug("sync", false) -} - -func TestMutex(t *testing.T) { - oldClock := timeNow - clock := newTestClock() - timeNow = clock.Now - defer func() { timeNow = oldClock }() - - debug = true - l.SetDebug("sync", true) - threshold = logThreshold - - msgmut := sync.Mutex{} - var messages []string - - l.AddHandler(logger.LevelDebug, func(_ logger.LogLevel, message string) { - msgmut.Lock() - messages = append(messages, message) - msgmut.Unlock() - }) - - mut := NewMutex() - mut.Lock() - clock.wind(shortWait) - mut.Unlock() - - if len(messages) > 0 { - t.Errorf("Unexpected message count") - } - - mut.Lock() - clock.wind(longWait) - mut.Unlock() - - if len(messages) != 1 { - t.Errorf("Unexpected message count") - } - - debug = false - l.SetDebug("sync", false) -} - -func TestRWMutex(t *testing.T) { - oldClock := timeNow - clock := newTestClock() - timeNow = clock.Now - defer func() { timeNow = oldClock }() - - debug = true - l.SetDebug("sync", true) - threshold = logThreshold - - msgmut := sync.Mutex{} - var messages []string - - l.AddHandler(logger.LevelDebug, func(_ logger.LogLevel, message string) { - msgmut.Lock() - messages = append(messages, message) - msgmut.Unlock() - }) - - mut := NewRWMutex() - mut.Lock() - clock.wind(shortWait) - mut.Unlock() - - if len(messages) > 0 { - t.Errorf("Unexpected message count") - } - - mut.Lock() - clock.wind(longWait) - mut.Unlock() - - if len(messages) != 1 { - t.Errorf("Unexpected message count") - } - - // Testing rlocker logging - wait := make(chan struct{}) - locking := make(chan struct{}) - - mut.RLock() - go func() { - close(locking) - mut.Lock() - close(wait) - }() - - <-locking - clock.wind(longWait) - mut.RUnlock() - <-wait - - mut.Unlock() - - if len(messages) != 2 { - t.Errorf("Unexpected message count") - } else if !strings.Contains(messages[1], "RUnlockers while locking:\nat sync") || !strings.Contains(messages[1], "sync_test.go:") { - t.Error("Unexpected message") - } - - // Testing multiple rlockers - mut.RLock() - mut.RLock() - mut.RLock() - _ = 1 // skip empty critical section check - mut.RUnlock() - mut.RUnlock() - mut.RUnlock() - - debug = false - l.SetDebug("sync", false) -} - -func TestWaitGroup(t *testing.T) { - oldClock := timeNow - clock := newTestClock() - timeNow = clock.Now - defer func() { timeNow = oldClock }() - - debug = true - l.SetDebug("sync", true) - threshold = logThreshold - - msgmut := sync.Mutex{} - var messages []string - - l.AddHandler(logger.LevelDebug, func(_ logger.LogLevel, message string) { - msgmut.Lock() - messages = append(messages, message) - msgmut.Unlock() - }) - - wg := NewWaitGroup() - wg.Add(1) - waiting := make(chan struct{}) - - go func() { - <-waiting - clock.wind(shortWait) - wg.Done() - }() - - close(waiting) - wg.Wait() - - if len(messages) > 0 { - t.Errorf("Unexpected message count") - } - - wg = NewWaitGroup() - waiting = make(chan struct{}) - - wg.Add(1) - go func() { - <-waiting - clock.wind(longWait) - wg.Done() - }() - - close(waiting) - wg.Wait() - - if len(messages) != 1 { - t.Errorf("Unexpected message count") - } - - debug = false - l.SetDebug("sync", false) -} - -func TestTimeoutCond(t *testing.T) { - // WARNING this test relies heavily on threads not being stalled at particular points. - // As such, it's pretty unstable on the build server. It has been left in as it still - // exercises the deadlock detector, and one of the two things it tests is still functional. - // See the comments in runLocks - - const ( - // Low values to avoid being intrusive in continuous testing. Can be - // increased significantly for stress testing. - iterations = 100 - routines = 10 - - timeMult = 2 - ) - - c := NewTimeoutCond(NewMutex()) - - // Start a routine to periodically broadcast on the cond. - - go func() { - d := time.Duration(routines) * timeMult * time.Millisecond / 2 - t.Log("Broadcasting every", d) - for i := 0; i < iterations; i++ { - time.Sleep(d) - - c.L.Lock() - c.Broadcast() - c.L.Unlock() - } - }() - - // Start several routines that wait on it with different timeouts. - - var results [routines][2]int - var wg sync.WaitGroup - for i := 0; i < routines; i++ { - i := i - wg.Add(1) - go func() { - d := time.Duration(i) * timeMult * time.Millisecond - t.Logf("Routine %d waits for %v\n", i, d) - succ, fail := runLocks(t, iterations, c, d) - results[i][0] = succ - results[i][1] = fail - wg.Done() - }() - } - - wg.Wait() - - // Print a table of routine number: successes, failures. - - for i, v := range results { - t.Logf("%4d: %4d %4d\n", i, v[0], v[1]) - } -} - -func runLocks(t *testing.T, iterations int, c *TimeoutCond, d time.Duration) (succ, fail int) { - for i := 0; i < iterations; i++ { - c.L.Lock() - - // The thread may be stalled, so we can't test the 'succeeded late' case reliably. - // Therefore make sure that we start t0 before starting the timeout, and only test - // the 'failed early' case. - - t0 := time.Now() - w := c.SetupWait(d) - - res := w.Wait() - waited := time.Since(t0) - - // Allow 20% slide in either direction, and a five milliseconds of - // scheduling delay... In tweaking these it was clear that things - // worked like the should, so if this becomes a spurious failure - // kind of thing feel free to remove or give significantly more - // slack. - - if !res && waited < d*8/10 { - t.Errorf("Wait failed early, %v < %v", waited, d) - } - if res && waited > d*11/10+5*time.Millisecond { - // Ideally this would be t.Errorf - t.Logf("WARNING: Wait succeeded late, %v > %v. This is probably a thread scheduling issue", waited, d) - } - - w.Stop() - - if res { - succ++ - } else { - fail++ - } - c.L.Unlock() - } - return -} - -type testClock struct { - time time.Time - mut sync.Mutex -} - -func newTestClock() *testClock { - return &testClock{ - time: time.Now(), - } -} - -func (t *testClock) Now() time.Time { - t.mut.Lock() - now := t.time - t.time = t.time.Add(time.Nanosecond) - t.mut.Unlock() - return now -} - -func (t *testClock) wind(d time.Duration) { - t.mut.Lock() - t.time = t.time.Add(d) - t.mut.Unlock() -} diff --git a/lib/syncthing/debug.go b/lib/syncthing/debug.go index e88fce98b..f0015d642 100644 --- a/lib/syncthing/debug.go +++ b/lib/syncthing/debug.go @@ -6,12 +6,10 @@ package syncthing -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("app", "Main run facility") +var l = slogutil.NewAdapter("Main run facility") func shouldDebug() bool { - return l.ShouldDebug("app") + return l.ShouldDebug("syncthing") } diff --git a/lib/syncthing/superuser_windows.go b/lib/syncthing/superuser_windows.go index 96ef3e44d..928479f5f 100644 --- a/lib/syncthing/superuser_windows.go +++ b/lib/syncthing/superuser_windows.go @@ -6,7 +6,10 @@ package syncthing -import "syscall" +import ( + "log/slog" + "syscall" +) // https://docs.microsoft.com/windows/win32/secauthz/well-known-sids const securityLocalSystemRID = "S-1-5-18" @@ -26,7 +29,7 @@ func isSuperUser() bool { } if user.User.Sid == nil { - l.Debugln("sid is nil") + slog.Debug("Sid is nil") return false } diff --git a/lib/syncthing/syncthing.go b/lib/syncthing/syncthing.go index 019aeb47f..99afbaf2e 100644 --- a/lib/syncthing/syncthing.go +++ b/lib/syncthing/syncthing.go @@ -12,6 +12,7 @@ import ( "errors" "fmt" "io" + "log/slog" "net/http" "os" "runtime" @@ -23,6 +24,7 @@ import ( "github.com/thejerf/suture/v4" "github.com/syncthing/syncthing/internal/db" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/api" "github.com/syncthing/syncthing/lib/build" "github.com/syncthing/syncthing/lib/config" @@ -31,7 +33,6 @@ import ( "github.com/syncthing/syncthing/lib/discover" "github.com/syncthing/syncthing/lib/events" "github.com/syncthing/syncthing/lib/locations" - "github.com/syncthing/syncthing/lib/logger" "github.com/syncthing/syncthing/lib/model" "github.com/syncthing/syncthing/lib/osutil" "github.com/syncthing/syncthing/lib/protocol" @@ -55,7 +56,6 @@ type Options struct { NoUpgrade bool ProfilerAddr string ResetDeltaIdxs bool - Verbose bool DBMaintenanceInterval time.Duration } @@ -96,7 +96,7 @@ func New(cfg config.Wrapper, sdb db.DB, evLogger events.Logger, cert tls.Certifi func (a *App) Start() error { // Create a main service manager. We'll add things to this as we go along. // We want any logging it does to go through our log system. - spec := svcutil.SpecWithDebugLogger(l) + spec := svcutil.SpecWithDebugLogger() a.mainService = suture.New("main", spec) // Start the supervisor and wait for it to stop to handle cleanup. @@ -123,13 +123,6 @@ func (a *App) startup() error { a.mainService.Add(newAuditService(a.opts.AuditWriter, a.evLogger)) } - if a.opts.Verbose { - a.mainService.Add(newVerboseService(a.evLogger)) - } - - errors := logger.NewRecorder(l, logger.LevelWarn, maxSystemErrors, 0) - systemLog := logger.NewRecorder(l, logger.LevelDebug, maxSystemLog, initialSystemLog) - // Event subscription for the API; must start early to catch the early // events. The LocalChangeDetected event might overwhelm the event // receiver in some situations so we will not subscribe to it here. @@ -141,10 +134,9 @@ func (a *App) startup() error { // report the error if there is one. osutil.MaximizeOpenFileLimit() - // Figure out our device ID, set it as the log prefix and log it. + // Figure out our device ID and log it. a.myID = protocol.NewDeviceID(a.cert.Certificate[0]) - l.SetPrefix(fmt.Sprintf("[%s] ", a.myID.String()[:5])) - l.Infoln("My ID:", a.myID) + slog.Info("Calculated our device ID", a.myID.LogAttr()) // Emit the Starting event, now that we know who we are. @@ -154,7 +146,7 @@ func (a *App) startup() error { }) if err := checkShortIDs(a.cfg); err != nil { - l.Warnln("Short device IDs are in conflict. Unlucky!\n Regenerate the device ID of one of the following:\n ", err) + slog.Error("Short device IDs are in conflict; regenerate the device ID of one of the conflicting devices", slogutil.Error(err)) return err } @@ -164,19 +156,19 @@ func (a *App) startup() error { runtime.SetBlockProfileRate(1) err := http.ListenAndServe(a.opts.ProfilerAddr, nil) if err != nil { - l.Warnln(err) + slog.Warn("Failed to listen and serve for profiles", slogutil.Error(err)) return } }() } perf := ur.CpuBench(context.Background(), 3, 150*time.Millisecond) - l.Infof("Hashing performance is %.02f MB/s", perf) + slog.Info("Measured hashing performance", "perf", fmt.Sprintf("%.02f MB/s", perf)) if a.opts.ResetDeltaIdxs { - l.Infoln("Reinitializing delta index IDs") + slog.Info("Reinitializing delta index IDs") if err := a.sdb.DropAllIndexIDs(); err != nil { - l.Warnln("Drop index IDs:", err) + slog.Error("Failed to drop index IDs", slogutil.Error(err)) return err } } @@ -192,12 +184,12 @@ func (a *App) startup() error { cfgFolders := a.cfg.Folders() dbFolders, err := a.sdb.ListFolders() if err != nil { - l.Warnln("Listing folders:", err) + slog.Warn("Failed to list folders", slogutil.Error(err)) return err } for _, folder := range dbFolders { if _, ok := cfgFolders[folder]; !ok { - l.Infof("Cleaning metadata for dropped folder %q", folder) + slog.Info("Cleaning metadata for dropped folder", "folder", folder) a.sdb.DropFolder(folder) } } @@ -207,7 +199,7 @@ func (a *App) startup() error { miscDB := db.NewMiscDB(a.sdb) prevVersion, _, err := miscDB.String("prevVersion") if err != nil { - l.Warnln("Database:", err) + slog.Error("Database error when getting previous version", slogutil.Error(err)) return err } @@ -219,14 +211,14 @@ func (a *App) startup() error { curParts := strings.Split(build.Version, "-") if rel := upgrade.CompareVersions(prevParts[0], curParts[0]); rel != upgrade.Equal { if prevVersion != "" { - l.Infoln("Detected upgrade from", prevVersion, "to", build.Version) + slog.Info("Detected upgrade", "from", prevVersion, "to", build.Version) } if a.cfg.Options().SendFullIndexOnUpgrade { // Drop delta indexes in case we've changed random stuff we // shouldn't have. We will resend our index on next connect. if err := a.sdb.DropAllIndexIDs(); err != nil { - l.Warnln("Drop index IDs:", err) + slog.Warn("Failed to drop index IDs", slogutil.Error(err)) return err } } @@ -238,7 +230,7 @@ func (a *App) startup() error { } if err := globalMigration(a.sdb, a.cfg); err != nil { - l.Warnln("Global migration:", err) + slog.Warn("Failed to perform global migration", slogutil.Error(err)) return err } @@ -277,7 +269,7 @@ func (a *App) startup() error { a.cfg.Modify(func(cfg *config.Configuration) { // Candidate builds always run with usage reporting. if build.IsCandidate { - l.Infoln("Anonymous usage reporting is always enabled for candidate releases.") + slog.Info("Anonymous usage reporting is always enabled for candidate releases") if cfg.Options.URAccepted != ur.Version { cfg.Options.URAccepted = ur.Version // Unique ID will be set and config saved below if necessary. @@ -290,21 +282,21 @@ func (a *App) startup() error { // GUI - if err := a.setupGUI(m, defaultSub, diskSub, discoveryManager, connectionsService, usageReportingSvc, errors, systemLog, miscDB); err != nil { - l.Warnln("Failed starting API:", err) + if err := a.setupGUI(m, defaultSub, diskSub, discoveryManager, connectionsService, usageReportingSvc, slogutil.ErrorRecorder, slogutil.GlobalRecorder, miscDB); err != nil { + slog.Error("Failed to start API", slogutil.Error(err)) return err } myDev, _ := a.cfg.Device(a.myID) - l.Infof(`My name is "%v"`, myDev.Name) + slog.Info("Loaded configuration", "name", myDev.Name) for _, device := range a.cfg.Devices() { if device.DeviceID != a.myID { - l.Infof(`Device %s is "%v" at %v`, device.DeviceID, device.Name, device.Addresses) + slog.Info("Loaded peer device configuration", device.DeviceID.LogAttr(), slog.String("name", device.Name), slogutil.Address(device.Addresses)) } } if isSuperUser() { - l.Warnln("Syncthing should not run as a privileged or system user. Please consider using a normal user account.") + slog.Warn("Syncthing should not run as a privileged or system user; please consider using a normal user account") } a.evLogger.Log(events.StartupComplete, map[string]string{ @@ -313,7 +305,7 @@ func (a *App) startup() error { if a.cfg.Options().SetLowPriority { if err := osutil.SetLowPriority(); err != nil { - l.Warnln("Failed to lower process priority:", err) + slog.Warn("Failed to lower process priority", slogutil.Error(err)) } } @@ -332,10 +324,10 @@ func (a *App) wait(errChan <-chan error) { select { case <-done: case <-time.After(10 * time.Second): - l.Warnln("Database failed to stop within 10s") + slog.Warn("Database failed to stop within 10s") } - l.Infoln("Exiting") + slog.Info("Exiting") close(a.stopped) } @@ -383,7 +375,7 @@ func (a *App) stopWithErr(stopReason svcutil.ExitStatus, err error) svcutil.Exit a.exitStatus = stopReason a.err = err if shouldDebug() { - l.Debugln("Services before stop:") + slog.Debug("Services before stop:") printServiceTree(os.Stdout, a.mainService, 0) } a.mainServiceCancel() @@ -392,7 +384,7 @@ func (a *App) stopWithErr(stopReason svcutil.ExitStatus, err error) svcutil.Exit return a.exitStatus } -func (a *App) setupGUI(m model.Model, defaultSub, diskSub events.BufferedSubscription, discoverer discover.Manager, connectionsService connections.Service, urService *ur.Service, errors, systemLog logger.Recorder, miscDB *db.Typed) error { +func (a *App) setupGUI(m model.Model, defaultSub, diskSub events.BufferedSubscription, discoverer discover.Manager, connectionsService connections.Service, urService *ur.Service, errors, systemLog slogutil.Recorder, miscDB *db.Typed) error { guiCfg := a.cfg.GUI() if !guiCfg.Enabled { @@ -400,7 +392,7 @@ func (a *App) setupGUI(m model.Model, defaultSub, diskSub events.BufferedSubscri } if guiCfg.InsecureAdminAccess { - l.Warnln("Insecure admin access is enabled.") + slog.Warn("Insecure admin access is enabled") } summaryService := model.NewFolderSummaryService(a.cfg, m, a.myID, a.evLogger) diff --git a/lib/syncthing/utils.go b/lib/syncthing/utils.go index 79ffd8c46..f4e8469bc 100644 --- a/lib/syncthing/utils.go +++ b/lib/syncthing/utils.go @@ -11,6 +11,7 @@ import ( "errors" "fmt" "io" + "log/slog" "os" "sync" "time" @@ -19,6 +20,7 @@ import ( "github.com/syncthing/syncthing/internal/db/olddb" "github.com/syncthing/syncthing/internal/db/olddb/backend" "github.com/syncthing/syncthing/internal/db/sqlite" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/build" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/events" @@ -44,7 +46,7 @@ func EnsureDir(dir string, mode fs.FileMode) error { err := fs.Chmod(".", mode) // This can fail on crappy filesystems, nothing we can do about it. if err != nil { - l.Warnln(err) + slog.Warn("Failed to correct directory permissions", slogutil.Error(err)) } } } @@ -60,7 +62,7 @@ func LoadOrGenerateCertificate(certFile, keyFile string) (tls.Certificate, error } func GenerateCertificate(certFile, keyFile string) (tls.Certificate, error) { - l.Infof("Generating key and certificate for %s...", tlsDefaultCommonName) + slog.Info("Generating key and certificate", "cn", tlsDefaultCommonName) return tlsutil.NewCertificate(certFile, keyFile, tlsDefaultCommonName, deviceCertLifetimeDays, false) } @@ -68,7 +70,7 @@ func DefaultConfig(path string, myID protocol.DeviceID, evLogger events.Logger, newCfg := config.New(myID) if skipPortProbing { - l.Infoln("Using default network port numbers instead of probing for free ports") + slog.Info("Using default network port numbers instead of probing for free ports") // Record address override initially newCfg.GUI.RawAddress = newCfg.GUI.Address() } else if err := newCfg.ProbeFreePorts(); err != nil { @@ -94,19 +96,16 @@ func LoadConfigAtStartup(path string, cert tls.Certificate, evLogger events.Logg if err != nil { return nil, fmt.Errorf("failed to save default config: %w", err) } - l.Infof("Default config saved. Edit %s to taste (with Syncthing stopped) or use the GUI", cfg.ConfigPath()) - } else if err == io.EOF { + slog.Info("Default config saved; edit to taste (with Syncthing stopped) or use the GUI", slogutil.FilePath(cfg.ConfigPath())) + } else if errors.Is(err, io.EOF) { return nil, errors.New("failed to load config: unexpected end of file. Truncated or empty configuration?") } else if err != nil { return nil, fmt.Errorf("failed to load config: %w", err) } if originalVersion != config.CurrentVersion { - if originalVersion == config.CurrentVersion+1101 { - l.Infof("Now, THAT's what we call a config from the future! Don't worry. As long as you hit that wire with the connecting hook at precisely eighty-eight miles per hour the instant the lightning strikes the tower... everything will be fine.") - } if originalVersion > config.CurrentVersion && !allowNewerConfig { - return nil, fmt.Errorf("config file version (%d) is newer than supported version (%d). If this is expected, use --allow-newer-config to override.", originalVersion, config.CurrentVersion) + return nil, fmt.Errorf("config file version (%d) is newer than supported version (%d); if this is expected, use --allow-newer-config to override", originalVersion, config.CurrentVersion) } err = archiveAndSaveConfig(cfg, originalVersion) if err != nil { @@ -120,7 +119,7 @@ func LoadConfigAtStartup(path string, cert tls.Certificate, evLogger events.Logg func archiveAndSaveConfig(cfg config.Wrapper, originalVersion int) error { // Copy the existing config to an archive copy archivePath := cfg.ConfigPath() + fmt.Sprintf(".v%d", originalVersion) - l.Infoln("Archiving a copy of old config file format at:", archivePath) + slog.Info("Archiving a copy of old config file format", slogutil.FilePath(archivePath)) if err := copyFile(cfg.ConfigPath(), archivePath); err != nil { return err } @@ -179,11 +178,11 @@ func TryMigrateDatabase(deleteRetention time.Duration) error { miscDB := db.NewMiscDB(sdb) if when, ok, err := miscDB.Time("migrated-from-leveldb-at"); err == nil && ok { - l.Warnf("Old-style database present but already migrated at %v; please manually move or remove %s.", when, oldDBDir) + slog.Error("Old-style database present but already migrated; please manually move or remove.", slog.Any("migratedAt", when), slogutil.FilePath(oldDBDir)) return nil } - l.Infoln("Migrating old-style database to SQLite; this may take a while...") + slog.Info("Migrating old-style database to SQLite; this may take a while...") t0 := time.Now() ll, err := olddb.NewLowlevel(be) @@ -217,7 +216,7 @@ func TryMigrateDatabase(deleteRetention time.Duration) error { if time.Since(t1) > 10*time.Second { d := time.Since(t0) + 1 t1 = time.Now() - l.Infof("Migrating folder %s... (%d files and %dk blocks in %v, %.01f files/s)", folder, files, blocks/1000, d.Truncate(time.Second), float64(files)/d.Seconds()) + slog.Info("Still migrating folder", "folder", folder, "files", files, "blocks", blocks, "duration", d.Truncate(time.Second), "filesrate", float64(files)/d.Seconds()) } } } @@ -225,7 +224,7 @@ func TryMigrateDatabase(deleteRetention time.Duration) error { writeErr = sdb.Update(folder, protocol.LocalDeviceID, batch) } d := time.Since(t0) + 1 - l.Infof("Migrated folder %s; %d files and %dk blocks in %v, %.01f files/s", folder, files, blocks/1000, d.Truncate(time.Second), float64(files)/d.Seconds()) + slog.Info("Migrated folder", "folder", folder, "files", files, "blocks", blocks, "duration", d.Truncate(time.Second), "filesrate", float64(files)/d.Seconds()) totFiles += files totBlocks += blocks }() @@ -258,9 +257,9 @@ func TryMigrateDatabase(deleteRetention time.Duration) error { } } - l.Infoln("Migrating virtual mtimes...") + slog.Info("Migrating virtual mtimes...") if err := ll.IterateMtimes(sdb.PutMtime); err != nil { - l.Warnln("Failed to migrate mtimes:", err) + slog.Warn("Failed to migrate mtimes", slogutil.Error(err)) } _ = miscDB.PutTime("migrated-from-leveldb-at", time.Now()) @@ -269,6 +268,6 @@ func TryMigrateDatabase(deleteRetention time.Duration) error { _ = be.Close() _ = os.Rename(oldDBDir, oldDBDir+"-migrated") - l.Infof("Migration complete, %d files and %dk blocks in %s", totFiles, totBlocks/1000, time.Since(t0).Truncate(time.Second)) + slog.Info("Migration complete", "files", totFiles, "blocks", totBlocks/1000, "duration", time.Since(t0).Truncate(time.Second)) return nil } diff --git a/lib/syncthing/verboseservice.go b/lib/syncthing/verboseservice.go deleted file mode 100644 index 2cf905ddd..000000000 --- a/lib/syncthing/verboseservice.go +++ /dev/null @@ -1,192 +0,0 @@ -// Copyright (C) 2015 The Syncthing Authors. -// -// This Source Code Form is subject to the terms of the Mozilla Public -// License, v. 2.0. If a copy of the MPL was not distributed with this file, -// You can obtain one at https://mozilla.org/MPL/2.0/. - -package syncthing - -import ( - "context" - "fmt" - "regexp" - - "github.com/syncthing/syncthing/lib/events" - "github.com/syncthing/syncthing/lib/model" -) - -// The verbose logging service subscribes to events and prints these in -// verbose format to the console using INFO level. -type verboseService struct { - evLogger events.Logger -} - -func newVerboseService(evLogger events.Logger) *verboseService { - return &verboseService{ - evLogger: evLogger, - } -} - -// serve runs the verbose logging service. -func (s *verboseService) Serve(ctx context.Context) error { - sub := s.evLogger.Subscribe(events.AllEvents) - defer sub.Unsubscribe() - for { - select { - case ev, ok := <-sub.C(): - if !ok { - <-ctx.Done() - return ctx.Err() - } - formatted := s.formatEvent(ev) - if formatted != "" { - l.Verboseln(formatted) - } - case <-ctx.Done(): - return ctx.Err() - } - } -} - -var folderSummaryRemoveDeprecatedRe = regexp.MustCompile(`(Invalid|IgnorePatterns|StateChanged):\S+\s?`) - -func (*verboseService) formatEvent(ev events.Event) string { - switch ev.Type { - case events.DownloadProgress: - // Skip - return "" - - case events.Starting: - return fmt.Sprintf("Starting up (%s)", ev.Data.(map[string]string)["home"]) - - case events.StartupComplete: - return "Startup complete" - - case events.DeviceDiscovered: - data := ev.Data.(map[string]interface{}) - return fmt.Sprintf("Discovered device %v at %v", data["device"], data["addrs"]) - - case events.DeviceConnected: - data := ev.Data.(map[string]string) - return fmt.Sprintf("Connected to device %v at %v (type %s)", data["id"], data["addr"], data["type"]) - - case events.DeviceDisconnected: - data := ev.Data.(map[string]string) - return fmt.Sprintf("Disconnected from device %v", data["id"]) - - case events.StateChanged: - data := ev.Data.(map[string]interface{}) - return fmt.Sprintf("Folder %q is now %v", data["folder"], data["to"]) - - case events.LocalChangeDetected: - data := ev.Data.(map[string]string) - return fmt.Sprintf("Local change detected in folder %q: %s %s %s", data["folder"], data["action"], data["type"], data["path"]) - - case events.RemoteChangeDetected: - data := ev.Data.(map[string]string) - return fmt.Sprintf("Remote change detected in folder %q: %s %s %s", data["folder"], data["action"], data["type"], data["path"]) - - case events.LocalIndexUpdated: - data := ev.Data.(map[string]interface{}) - return fmt.Sprintf("Local index update for %q with %d items (seq: %d)", data["folder"], data["items"], data["sequence"]) - - case events.RemoteIndexUpdated: - data := ev.Data.(map[string]interface{}) - return fmt.Sprintf("Device %v sent an index update for %q with %d items (seq: %d)", data["device"], data["folder"], data["items"], data["sequence"]) - - case events.DeviceRejected: - data := ev.Data.(map[string]string) - return fmt.Sprintf("Rejected connection from device %v at %v", data["device"], data["address"]) - - case events.FolderRejected: - data := ev.Data.(map[string]string) - return fmt.Sprintf("Rejected unshared folder %q from device %v", data["folder"], data["device"]) - - case events.ItemStarted: - data := ev.Data.(map[string]string) - return fmt.Sprintf("Started syncing %q / %q (%v %v)", data["folder"], data["item"], data["action"], data["type"]) - - case events.ItemFinished: - data := ev.Data.(map[string]interface{}) - if err, ok := data["error"].(*string); ok && err != nil { - // If the err interface{} is not nil, it is a string pointer. - // Dereference it to get the actual error or Sprintf will print - // the pointer value.... - return fmt.Sprintf("Finished syncing %q / %q (%v %v): %v", data["folder"], data["item"], data["action"], data["type"], *err) - } - return fmt.Sprintf("Finished syncing %q / %q (%v %v): Success", data["folder"], data["item"], data["action"], data["type"]) - - case events.ConfigSaved: - return "Configuration was saved" - - case events.FolderCompletion: - data := ev.Data.(map[string]interface{}) - return fmt.Sprintf("Completion for folder %q on device %v is %v%% (state: %s, seq: %d)", data["folder"], data["device"], data["completion"], data["remoteState"], data["sequence"]) - - case events.FolderSummary: - data := ev.Data.(model.FolderSummaryEventData) - return folderSummaryRemoveDeprecatedRe.ReplaceAllString(fmt.Sprintf("Summary for folder %q is %+v", data.Folder, data.Summary), "") - - case events.FolderScanProgress: - data := ev.Data.(map[string]interface{}) - folder := data["folder"].(string) - current := data["current"].(int64) - total := data["total"].(int64) - rate := data["rate"].(float64) / 1024 / 1024 - var pct int64 - if total > 0 { - pct = 100 * current / total - } - return fmt.Sprintf("Scanning folder %q, %d%% done (%.01f MiB/s)", folder, pct, rate) - - case events.DevicePaused: - data := ev.Data.(map[string]string) - device := data["device"] - return fmt.Sprintf("Device %v was paused", device) - - case events.DeviceResumed: - data := ev.Data.(map[string]string) - device := data["device"] - return fmt.Sprintf("Device %v was resumed", device) - - case events.ClusterConfigReceived: - data := ev.Data.(model.ClusterConfigReceivedEventData) - return fmt.Sprintf("Received ClusterConfig from device %v", data.Device) - - case events.FolderPaused: - data := ev.Data.(map[string]string) - id := data["id"] - label := data["label"] - return fmt.Sprintf("Folder %v (%v) was paused", id, label) - - case events.FolderResumed: - data := ev.Data.(map[string]string) - id := data["id"] - label := data["label"] - return fmt.Sprintf("Folder %v (%v) was resumed", id, label) - - case events.ListenAddressesChanged: - data := ev.Data.(map[string]interface{}) - address := data["address"] - lan := data["lan"] - wan := data["wan"] - return fmt.Sprintf("Listen address %s resolution has changed: lan addresses: %s wan addresses: %s", address, lan, wan) - - case events.LoginAttempt: - data := ev.Data.(map[string]interface{}) - username := data["username"].(string) - var success string - if data["success"].(bool) { - success = "successful" - } else { - success = "failed" - } - return fmt.Sprintf("Login %s for username %s.", success, username) - } - - return fmt.Sprintf("%s %#v", ev.Type, ev) -} - -func (s *verboseService) String() string { - return fmt.Sprintf("verboseService@%p", s) -} diff --git a/lib/syncutil/timeoutcond.go b/lib/syncutil/timeoutcond.go new file mode 100644 index 000000000..a379b1236 --- /dev/null +++ b/lib/syncutil/timeoutcond.go @@ -0,0 +1,79 @@ +// Copyright (C) 2015 The Syncthing Authors. +// +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this file, +// You can obtain one at https://mozilla.org/MPL/2.0/. + +package syncutil + +import ( + "sync" + "time" +) + +// TimeoutCond is a variant on Cond. It has roughly the same semantics regarding 'L' - it must be held +// both when broadcasting and when calling TimeoutCondWaiter.Wait() +// Call Broadcast() to broadcast to all waiters on the TimeoutCond. Call SetupWait to create a +// TimeoutCondWaiter configured with the given timeout, which can then be used to listen for +// broadcasts. +type TimeoutCond struct { + L sync.Locker + ch chan struct{} +} + +// TimeoutCondWaiter is a type allowing a consumer to wait on a TimeoutCond with a timeout. Wait() may be called multiple times, +// and will return true every time that the TimeoutCond is broadcast to. Once the configured timeout +// expires, Wait() will return false. +// Call Stop() to release resources once this TimeoutCondWaiter is no longer needed. +type TimeoutCondWaiter struct { + c *TimeoutCond + timer *time.Timer +} + +func NewTimeoutCond(l sync.Locker) *TimeoutCond { + return &TimeoutCond{ + L: l, + } +} + +func (c *TimeoutCond) Broadcast() { + // ch.L must be locked when calling this function + + if c.ch != nil { + close(c.ch) + c.ch = nil + } +} + +func (c *TimeoutCond) SetupWait(timeout time.Duration) *TimeoutCondWaiter { + timer := time.NewTimer(timeout) + + return &TimeoutCondWaiter{ + c: c, + timer: timer, + } +} + +func (w *TimeoutCondWaiter) Wait() bool { + // ch.L must be locked when calling this function + + // Ensure that the channel exists, since we're going to be waiting on it + if w.c.ch == nil { + w.c.ch = make(chan struct{}) + } + ch := w.c.ch + + w.c.L.Unlock() + defer w.c.L.Lock() + + select { + case <-w.timer.C: + return false + case <-ch: + return true + } +} + +func (w *TimeoutCondWaiter) Stop() { + w.timer.Stop() +} diff --git a/lib/syncutil/timeoutcond_test.go b/lib/syncutil/timeoutcond_test.go new file mode 100644 index 000000000..5872a2716 --- /dev/null +++ b/lib/syncutil/timeoutcond_test.go @@ -0,0 +1,135 @@ +// Copyright (C) 2015 The Syncthing Authors. +// +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this file, +// You can obtain one at https://mozilla.org/MPL/2.0/. + +package syncutil + +import ( + "sync" + "testing" + "time" +) + +func TestTimeoutCond(t *testing.T) { + // WARNING this test relies heavily on threads not being stalled at particular points. + // As such, it's pretty unstable on the build server. It has been left in as it still + // exercises the deadlock detector, and one of the two things it tests is still functional. + // See the comments in runLocks + + const ( + // Low values to avoid being intrusive in continuous testing. Can be + // increased significantly for stress testing. + iterations = 100 + routines = 10 + + timeMult = 2 + ) + + c := NewTimeoutCond(new(sync.Mutex)) + + // Start a routine to periodically broadcast on the cond. + + go func() { + d := time.Duration(routines) * timeMult * time.Millisecond / 2 + t.Log("Broadcasting every", d) + for i := 0; i < iterations; i++ { + time.Sleep(d) + + c.L.Lock() + c.Broadcast() + c.L.Unlock() + } + }() + + // Start several routines that wait on it with different timeouts. + + var results [routines][2]int + var wg sync.WaitGroup + for i := 0; i < routines; i++ { + i := i + wg.Add(1) + go func() { + d := time.Duration(i) * timeMult * time.Millisecond + t.Logf("Routine %d waits for %v\n", i, d) + succ, fail := runLocks(t, iterations, c, d) + results[i][0] = succ + results[i][1] = fail + wg.Done() + }() + } + + wg.Wait() + + // Print a table of routine number: successes, failures. + + for i, v := range results { + t.Logf("%4d: %4d %4d\n", i, v[0], v[1]) + } +} + +func runLocks(t *testing.T, iterations int, c *TimeoutCond, d time.Duration) (succ, fail int) { + for i := 0; i < iterations; i++ { + c.L.Lock() + + // The thread may be stalled, so we can't test the 'succeeded late' case reliably. + // Therefore make sure that we start t0 before starting the timeout, and only test + // the 'failed early' case. + + t0 := time.Now() + w := c.SetupWait(d) + + res := w.Wait() + waited := time.Since(t0) + + // Allow 20% slide in either direction, and a five milliseconds of + // scheduling delay... In tweaking these it was clear that things + // worked like the should, so if this becomes a spurious failure + // kind of thing feel free to remove or give significantly more + // slack. + + if !res && waited < d*8/10 { + t.Errorf("Wait failed early, %v < %v", waited, d) + } + if res && waited > d*11/10+5*time.Millisecond { + // Ideally this would be t.Errorf + t.Logf("WARNING: Wait succeeded late, %v > %v. This is probably a thread scheduling issue", waited, d) + } + + w.Stop() + + if res { + succ++ + } else { + fail++ + } + c.L.Unlock() + } + return +} + +type testClock struct { + time time.Time + mut sync.Mutex +} + +func newTestClock() *testClock { + return &testClock{ + time: time.Now(), + } +} + +func (t *testClock) Now() time.Time { + t.mut.Lock() + now := t.time + t.time = t.time.Add(time.Nanosecond) + t.mut.Unlock() + return now +} + +func (t *testClock) wind(d time.Duration) { + t.mut.Lock() + t.time = t.time.Add(d) + t.mut.Unlock() +} diff --git a/lib/upgrade/debug.go b/lib/upgrade/debug.go index b84e29f06..52fed6373 100644 --- a/lib/upgrade/debug.go +++ b/lib/upgrade/debug.go @@ -6,8 +6,6 @@ package upgrade -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("upgrade", "Binary upgrades") +var l = slogutil.NewAdapter("Binary upgrades") diff --git a/lib/upgrade/upgrade_supported.go b/lib/upgrade/upgrade_supported.go index 6ec4e58aa..bdea9e59d 100644 --- a/lib/upgrade/upgrade_supported.go +++ b/lib/upgrade/upgrade_supported.go @@ -18,6 +18,7 @@ import ( "errors" "fmt" "io" + "log/slog" "net/http" "os" "path" @@ -28,6 +29,7 @@ import ( "time" "github.com/shirou/gopsutil/v4/host" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/dialer" "github.com/syncthing/syncthing/lib/signature" "github.com/syncthing/syncthing/lib/tlsutil" @@ -96,18 +98,18 @@ func upgradeClientGet(url, version string) (*http.Response, error) { func FetchLatestReleases(releasesURL, current string) []Release { resp, err := upgradeClientGet(releasesURL, current) if err != nil { - l.Infoln("Couldn't fetch release information:", err) + slog.Warn("Failed to fetch latest release information", slogutil.Error(err)) return nil } if resp.StatusCode > 299 { - l.Infoln("API call returned HTTP error:", resp.Status) + slog.Warn("Failed to fetch latest release information", slogutil.Error(resp.Status)) return nil } var rels []Release err = json.NewDecoder(io.LimitReader(resp.Body, maxMetadataSize)).Decode(&rels) if err != nil { - l.Infoln("Fetching release information:", err) + slog.Warn("Failed to decode latest release information", slogutil.Error(err)) } resp.Body.Close() diff --git a/lib/upnp/debug.go b/lib/upnp/debug.go index f98ecb542..bf650860d 100644 --- a/lib/upnp/debug.go +++ b/lib/upnp/debug.go @@ -6,8 +6,6 @@ package upnp -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("upnp", "UPnP discovery and port mapping") +var l = slogutil.NewAdapter("UPnP discovery and port mapping") diff --git a/lib/upnp/igd_service.go b/lib/upnp/igd_service.go index a89474fb0..22cbaf777 100644 --- a/lib/upnp/igd_service.go +++ b/lib/upnp/igd_service.go @@ -37,9 +37,11 @@ import ( "encoding/xml" "errors" "fmt" + "log/slog" "net" "time" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/netutil" "github.com/syncthing/syncthing/lib/nat" @@ -105,7 +107,7 @@ func (s *IGDService) AddPinhole(ctx context.Context, protocol nat.Protocol, intA for _, addr := range addrs { ip, _, err := net.ParseCIDR(addr.String()) if err != nil { - l.Infof("Couldn't parse address %s: %s", addr, err) + slog.WarnContext(ctx, "Couldn't parse interface address", slogutil.Address(addr), slogutil.Error(err)) continue } @@ -115,7 +117,7 @@ func (s *IGDService) AddPinhole(ctx context.Context, protocol nat.Protocol, intA } if err := s.tryAddPinholeForIP6(ctx, protocol, intAddr.Port, duration, ip); err != nil { - l.Infof("Couldn't add pinhole for [%s]:%d/%s. %s", ip, intAddr.Port, protocol, err) + slog.WarnContext(ctx, "Couldn't add pinhole", slogutil.Address(ip), slog.Int("port", intAddr.Port), slog.Any("protocol", protocol), slogutil.Error(err)) returnErr = err } else { successfulIPs = append(successfulIPs, ip) diff --git a/lib/upnp/upnp.go b/lib/upnp/upnp.go index cc49c2539..432d410c2 100644 --- a/lib/upnp/upnp.go +++ b/lib/upnp/upnp.go @@ -40,6 +40,7 @@ import ( "errors" "fmt" "io" + "log/slog" "net" "net/http" "net/url" @@ -48,6 +49,7 @@ import ( "sync" "time" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/netutil" "github.com/syncthing/syncthing/lib/build" @@ -108,7 +110,7 @@ func Discover(ctx context.Context, _, timeout time.Duration) []nat.Device { interfaces, err := netutil.Interfaces() if err != nil { - l.Infoln("Listing network interfaces:", err) + slog.WarnContext(ctx, "Failed to list network interfaces", slogutil.Error(err)) return results } @@ -215,7 +217,7 @@ USER-AGENT: syncthing/%s if err != nil { if runtime.GOOS == "windows" && ip6 { // Requires https://github.com/golang/go/issues/63529 to be fixed. - l.Infoln("Support for IPv6 UPnP is currently not available on Windows:", err) + slog.InfoContext(ctx, "Support for IPv6 UPnP is currently not available on Windows", slogutil.Error(err)) } else { l.Debugln("UPnP discovery: listening to udp multicast:", err) } @@ -244,7 +246,7 @@ USER-AGENT: syncthing/%s loop: for { if err := socket.SetDeadline(time.Now().Add(250 * time.Millisecond)); err != nil { - l.Infoln("UPnP socket:", err) + slog.WarnContext(ctx, "Failed to set UPnP socket deadline", slogutil.Error(err)) break } @@ -255,22 +257,21 @@ loop: break loop default: } - if e, ok := err.(net.Error); ok && e.Timeout() { + var ne net.Error + if ok := errors.As(err, &ne); ok && ne.Timeout() { continue // continue reading } - l.Infoln("UPnP read:", err) // legitimate error, not a timeout. + slog.WarnContext(ctx, "Failed to read from UPnP socket", slogutil.Error(err)) // legitimate error, not a timeout. break } igds, err := parseResponse(ctx, deviceType, udpAddr, resp[:n], intf) if err != nil { - switch err.(type) { - case *UnsupportedDeviceTypeError: + var unsupp *UnsupportedDeviceTypeError + if errors.As(err, &unsupp) { l.Debugln(err.Error()) - default: - if !errors.Is(err, context.Canceled) { - l.Infoln("UPnP parse:", err) - } + } else if !errors.Is(err, context.Canceled) { + slog.WarnContext(ctx, "Failed to parse UPnP response", slogutil.Error(err)) } continue } @@ -308,16 +309,11 @@ func parseResponse(ctx context.Context, deviceType string, addr *net.UDPAddr, re deviceDescriptionURL, err := url.Parse(deviceDescriptionLocation) if err != nil { - l.Infoln("Invalid IGD location: " + err.Error()) + slog.WarnContext(ctx, "Got invalid IGD location", slogutil.Error(err)) return nil, err } - if err != nil { - l.Infoln("Invalid source IP for IGD: " + err.Error()) - return nil, err - } - - deviceUSN := response.Header.Get("USN") + deviceUSN := response.Header.Get("Usn") if deviceUSN == "" { return nil, errors.New("invalid IGD response: USN not specified") } @@ -368,7 +364,7 @@ func parseResponse(ctx context.Context, deviceType string, addr *net.UDPAddr, re // we are on an IPv6-only network though, so don't error out in case pinholing is available. localIPv4Address, err = localIPv4Fallback(ctx, deviceDescriptionURL) if err != nil { - l.Infoln("Unable to determine local IPv4 address for IGD: " + err.Error()) + slog.WarnContext(ctx, "Unable to determine local IPv4 address for IGD", slogutil.Error(err)) } } @@ -496,7 +492,7 @@ func getIGDServices(deviceUUID string, localIPAddress net.IP, rootURL string, de devices := getChildDevices(device, wanDeviceURN) if len(devices) < 1 { - l.Infoln(rootURL, "- malformed InternetGatewayDevice description: no WANDevices specified.") + slog.Warn("Got malformed InternetGatewayDevice description: no WANDevices specified") return result } @@ -504,20 +500,20 @@ func getIGDServices(deviceUUID string, localIPAddress net.IP, rootURL string, de connections := getChildDevices(device, wanConnectionURN) if len(connections) < 1 { - l.Infoln(rootURL, "- malformed ", wanDeviceURN, "description: no WANConnectionDevices specified.") + slog.Warn("Got malformed WAN device description: no WANConnectionDevices specified", "urn", wanDeviceURN) } for _, connection := range connections { - for _, URN := range URNs { - services := getChildServices(connection, URN) + for _, urn := range URNs { + services := getChildServices(connection, urn) if len(services) == 0 { - l.Debugln(rootURL, "- no services of type", URN, " found on connection.") + l.Debugln(rootURL, "- no services of type", urn, " found on connection.") } for _, service := range services { if service.ControlURL == "" { - l.Infoln(rootURL+"- malformed", service.Type, "description: no control URL.") + slog.Warn("Gor malformed service description: no control URL", "service", service.Type) } else { u, _ := url.Parse(rootURL) replaceRawPath(u, service.ControlURL) @@ -615,7 +611,7 @@ func soapRequestWithIP(ctx context.Context, url, service, function, message stri resp, err = io.ReadAll(r.Body) if err != nil { - l.Debugf("Error reading SOAP response: %s, partial response (if present):\n\n%s", resp) + l.Debugf("Error reading SOAP response: %v, partial response (if present):\n\n%s", err, resp) return resp, err } diff --git a/lib/ur/debug.go b/lib/ur/debug.go index aed6c3ce6..17ca8d1f1 100644 --- a/lib/ur/debug.go +++ b/lib/ur/debug.go @@ -6,8 +6,6 @@ package ur -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("ur", "Usage reporting") +func init() { slogutil.RegisterPackage("Usage reporting") } diff --git a/lib/ur/failurereporting.go b/lib/ur/failurereporting.go index 92c54173d..f2b92e278 100644 --- a/lib/ur/failurereporting.go +++ b/lib/ur/failurereporting.go @@ -10,11 +10,13 @@ import ( "bytes" "context" "encoding/json" + "log/slog" "net/http" "runtime/pprof" "strings" "time" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/build" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/dialer" @@ -219,14 +221,14 @@ func sendFailureReports(ctx context.Context, reports []FailureReport, url string defer reqCancel() req, err := http.NewRequestWithContext(reqCtx, http.MethodPost, url, &b) if err != nil { - l.Infoln("Failed to send failure report:", err) + slog.WarnContext(ctx, "Failed to send failure report", slogutil.Error(err)) return } req.Header.Set("Content-Type", "application/json") resp, err := client.Do(req) if err != nil { - l.Infoln("Failed to send failure report:", err) + slog.WarnContext(ctx, "Failed to send failure report", slogutil.Error(err)) return } resp.Body.Close() diff --git a/lib/ur/usage_report.go b/lib/ur/usage_report.go index bee2f3a8d..d0ed2657a 100644 --- a/lib/ur/usage_report.go +++ b/lib/ur/usage_report.go @@ -11,6 +11,7 @@ import ( "context" "crypto/tls" "encoding/json" + "log/slog" "math/rand" "net" "net/http" @@ -23,6 +24,7 @@ import ( "github.com/shirou/gopsutil/v4/process" "github.com/syncthing/syncthing/internal/db" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/build" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/connections" @@ -146,8 +148,6 @@ func (s *Service) reportData(ctx context.Context, urVersion int, preview bool) ( report.FolderUses.AutoNormalize++ } switch cfg.Versioning.Type { - case "": - // None case "simple": report.FolderUses.SimpleVersioning++ case "staggered": @@ -156,8 +156,6 @@ func (s *Service) reportData(ctx context.Context, urVersion int, preview bool) ( report.FolderUses.ExternalVersioning++ case "trashcan": report.FolderUses.TrashcanVersioning++ - default: - l.Warnf("Unhandled versioning type for usage reports: %s", cfg.Versioning.Type) } } slices.Sort(report.RescanIntvs) @@ -176,8 +174,6 @@ func (s *Service) reportData(ctx context.Context, urVersion int, preview bool) ( report.DeviceUses.CompressMetadata++ case config.CompressionNever: report.DeviceUses.CompressNever++ - default: - l.Warnf("Unhandled versioning type for usage reports: %s", cfg.Compression) } for _, addr := range cfg.Addresses { @@ -401,9 +397,9 @@ func (s *Service) Serve(ctx context.Context) error { if s.cfg.Options().URAccepted >= 2 { err := s.sendUsageReport(ctx) if err != nil { - l.Infoln("Usage report:", err) + slog.WarnContext(ctx, "Failed to send usage report", slogutil.Error(err)) } else { - l.Infof("Sent usage report (version %d)", s.cfg.Options().URAccepted) + slog.InfoContext(ctx, "Sent usage report", "version", s.cfg.Options().URAccepted) } } t.Reset(24 * time.Hour) // next report tomorrow diff --git a/lib/versioner/debug.go b/lib/versioner/debug.go index ddafb7395..50401b855 100644 --- a/lib/versioner/debug.go +++ b/lib/versioner/debug.go @@ -6,8 +6,6 @@ package versioner -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("versioner", "File versioning") +var l = slogutil.NewAdapter("File versioning") diff --git a/lib/versioner/empty_dir_tracker.go b/lib/versioner/empty_dir_tracker.go index 963e5f0fd..6925fc45c 100644 --- a/lib/versioner/empty_dir_tracker.go +++ b/lib/versioner/empty_dir_tracker.go @@ -7,10 +7,12 @@ package versioner import ( + "log/slog" "path/filepath" "slices" "strings" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/fs" ) @@ -49,7 +51,7 @@ func (t emptyDirTracker) deleteEmptyDirs(fs fs.Filesystem) { l.Debugln("Cleaner: deleting empty directory", path) err := fs.Remove(path) if err != nil { - l.Warnln("Versioner: can't remove directory", path, err) + slog.Warn("Failed to remove versioned directory", slogutil.FilePath(path), slogutil.Error(err)) } } } diff --git a/lib/versioner/util.go b/lib/versioner/util.go index 2286f762c..ebb358f1f 100644 --- a/lib/versioner/util.go +++ b/lib/versioner/util.go @@ -10,6 +10,7 @@ import ( "context" "errors" "fmt" + "log/slog" "os" "path/filepath" "regexp" @@ -17,6 +18,7 @@ import ( "strings" "time" + "github.com/syncthing/syncthing/internal/slogutil" "github.com/syncthing/syncthing/lib/config" "github.com/syncthing/syncthing/lib/fs" "github.com/syncthing/syncthing/lib/osutil" @@ -152,7 +154,7 @@ func archiveFile(method fs.CopyRangeMethod, srcFs, dstFs fs.Filesystem, filePath _, err = dstFs.Stat(".") if err != nil { if fs.IsNotExist(err) { - l.Debugln("creating versions dir") + slog.Debug("Creating versions dir") err := dstFs.MkdirAll(".", 0o755) if err != nil { return err @@ -335,7 +337,7 @@ func findAllVersions(fs fs.Filesystem, filePath string) []string { pattern := filepath.Join(inFolderPath, TagFilename(file, timeGlob)) versions, err := fs.Glob(pattern) if err != nil { - l.Warnln("globbing:", err, "for", pattern) + slog.Warn("Failed to glob for versions", slog.String("pattern", pattern), slogutil.Error(err)) return nil } versions = stringutil.UniqueTrimmedStrings(versions) @@ -385,7 +387,7 @@ func clean(ctx context.Context, versionsFs fs.Filesystem, toRemove func([]string if err := versionsFs.Walk(".", walkFn); err != nil { if !errors.Is(err, context.Canceled) { - l.Warnln("Versioner: scanning versions dir:", err) + slog.WarnContext(ctx, "Failed to scan versions directory", slogutil.Error(err)) } return err } @@ -409,7 +411,7 @@ func cleanVersions(versionsFs fs.Filesystem, versions []string, toRemove func([] l.Debugln("Versioner: Expiring versions", versions) for _, file := range toRemove(versions, time.Now()) { if err := versionsFs.Remove(file); err != nil { - l.Warnf("Versioner: can't remove %q: %v", file, err) + slog.Warn("Failed to remove versioned file during cleanup", slogutil.FilePath(file), slogutil.Error(err)) } } } diff --git a/lib/watchaggregator/debug.go b/lib/watchaggregator/debug.go index 63857c239..182692648 100644 --- a/lib/watchaggregator/debug.go +++ b/lib/watchaggregator/debug.go @@ -6,8 +6,6 @@ package watchaggregator -import ( - "github.com/syncthing/syncthing/lib/logger" -) +import "github.com/syncthing/syncthing/internal/slogutil" -var l = logger.DefaultLogger.NewFacility("watchaggregator", "Filesystem event watcher") +var l = slogutil.NewAdapter("Filesystem event watcher") diff --git a/relnotes/v2.0.md b/relnotes/v2.0.md index c422f60db..24c192b99 100644 --- a/relnotes/v2.0.md +++ b/relnotes/v2.0.md @@ -4,6 +4,16 @@ first launch which can be lengthy for larger setups. The new database is easier to understand and maintain and, hopefully, less buggy. +- The logging format has changed to use structured log entries (a message + plus several key-value pairs). Additionally, we can now control the log + level per package, and a new log level WARNING has been inserted between + INFO and ERROR (which was previously known as WARNING...). The INFO level + has become more verbose, indicating the sync actions taken by Syncthing. A + new command line flag `--log-level` sets the default log level for all + packages, and the `STTRACE` environment variable and GUI has been updated + to set log levels per package. The `--verbose` and `--logflags` command + line options have been removed and will be ignored if given. + - Deleted items are no longer kept forever in the database, instead they are forgotten after six months. If your use case require deletes to take effect after more than a six month delay, set the diff --git a/test/h1/config.xml b/test/h1/config.xml index c7e965a88..c71d4cbd5 100644 --- a/test/h1/config.xml +++ b/test/h1/config.xml @@ -1,5 +1,5 @@ - - + + basic @@ -20,14 +20,14 @@ false 0 0 + 1 -1 false - false false .stfolder false 0 - 2 + 16 false standard standard @@ -65,10 +65,11 @@ 0 3 - +
127.0.0.1:8081
testuser $2a$10$7tKL5uvLDGn5s2VLPM2yWOK/II45az0mTel8hxAUJDRQN1Tk2QYwu + false abc123 default @@ -115,13 +116,13 @@ 180 20 default - auto 0 true false + false + 0 0 - false 10 20 30 @@ -130,7 +131,7 @@ 0 - + basic @@ -148,14 +149,14 @@ false 0 0 + 1 10 false - false false .stfolder false 0 - 2 + 16 false standard standard diff --git a/test/h2/config.xml b/test/h2/config.xml index 81276c1d1..4cfcc661b 100644 --- a/test/h2/config.xml +++ b/test/h2/config.xml @@ -1,5 +1,5 @@ - - + + basic @@ -20,9 +20,9 @@ false 0 0 + 1 -1 false - false false .stfolder false @@ -65,8 +65,9 @@ 0 3 - +
127.0.0.1:8082
+ false abc123 default @@ -113,13 +114,13 @@ 180 20 default - auto 0 true false + false + 0 0 - false 10 20 30 @@ -128,7 +129,7 @@ 0 - + basic @@ -146,14 +147,14 @@ false 0 0 + 1 10 false - false false .stfolder false 0 - 2 + 16 false standard standard