Commit Graph

1590 Commits

Author SHA1 Message Date
Jakob Borg
415adfbae6 lib/db: Don't hang on double iterator release with Badger (#6960)
Since iterators must be released before committing or discarding a
transaction we have the pattern of both deferring a release plus doing
it manually. But we can't release twice because we track this with a
WaitGroup that will panic when there are more Done()s than Add()s. This
just adds a boolean to let an iterator keep track.
2020-09-07 10:31:33 +02:00
Simon Frei
3dd13c3994 test, lib/model: Various integration test updates & improvements (#6956) 2020-09-07 09:35:37 +02:00
Simon Frei
52d1681fe6 lib/db: Copy returned, old FileVersion (#6952) 2020-09-04 14:13:38 +02:00
Simon Frei
7b821d2550 lib/db: Add local need check&repair (#6950) 2020-09-04 14:01:46 +02:00
Simon Frei
0e3e0a7c9e cmd/stindex, lib/db: Use same condition to check for needed files (#6954) 2020-09-04 13:59:04 +02:00
Jakob Borg
71bfad0bc6 cmd/stindex: Add dumping folder meta 2020-09-04 11:39:21 +02:00
Jakob Borg
b7986801cd cmd/stindex: Print more hashes and versions 2020-09-04 09:10:33 +02:00
Simon Frei
6365e6108f lib/model: Don't acquire I/O token for send-only pull (#6951) 2020-09-03 13:59:45 +02:00
Simon Frei
540518a7b7 lib/model: Add missing lock on download-state (fixes #6880) (#6945) 2020-08-30 08:03:10 +02:00
Jakob Borg
3ffbe45a6d lib/db, lib/model: Avoid accounting mishap on folder restart (fixes #6938) (#6939)
We created a new fileset before stopping the folder during restart. When
we create that fileset it loads the current metadata and sequence
numbers from the database. But the folder may have time to update those
before stopping, leaving the new fileset with bad data.

This would cause wrong accounting (forgotten files) and potentially
sequence reuse causing files not sent to other devices.

This change reuses the fileset on restart, skipping the issue entirely.
It also moves the creation of the fileset back under the lock so there
should be no chance of concurrency issues here.
2020-08-27 18:33:27 +02:00
Jakob Borg
3393db1f69 lib/model: Save config after auto accepting folders (fixes #6922) (#6937) 2020-08-27 16:28:19 +02:00
Jakob Borg
145d87ce70 lib/model: Fixup spelling of protocol.FileIntf in test (ref #6463) 2020-08-27 16:21:04 +02:00
Simon Frei
06ac631351 lib/model: Reduce fmut locking in ClusterConfig (#6913)
The FileSet.Drop operation in there needs to potentially update a whole lot of global lists, which can take a while (longer than the deadlock interval apparently)
2020-08-27 16:01:46 +02:00
Simon Frei
1fc2dbdeeb lib/model: Add test for recv-only changes undone by remote (#6463)
Co-authored-by: Audrius Butkevicius <audrius.butkevicius@gmail.com>
2020-08-27 15:54:00 +02:00
Simon Frei
3f0eba388c lib/syncthing: Also cleanup on startup error (#6926) 2020-08-27 15:52:51 +02:00
Audrius Butkevicius
b19b5c95d3 lib/connections: Announce LAN addresses by default (fixes #6928) (#6896) 2020-08-25 11:48:14 +02:00
Audrius Butkevicius
d507d932b8 all: Use protobuf to generate config structs (fixes #6734) (#6900) 2020-08-25 08:11:14 +02:00
Rahmi Pruitt
5b953033c7 Created cleanup functionality for syncthing (#6884)
* Add clean up for Simple File Versioning pt.1

created test

* Add clean up for Simple File Versioning pt.2

Passing the test

* stuck on how javascript communicates with backend

* Add trash clean up for Simple File Versioning

Add trash clean up functionality of to allow the user to delete backups
after specified amount of days.

* Fixed html and js style

* Refactored cleanup test cases

Refactored cleanup test cases to one file and deleted duplicated code.

* Added copyright to test file

* Refactor folder cleanout to utility function

* change utility function to package private

* refactored utility function; fixed build errors

* Updated copyright year.

* refactor test and logging

* refactor html and js

* revert style change in html

* reverted changes in html and some js

* checkout origin head version edit...html

* checkout upstream master and correct file
2020-08-24 12:14:30 +01:00
Simon Frei
a65b46debd lib/model: Don't mark globally deleted items as recv-only (fixes #6910) (#6911) 2020-08-21 12:34:22 +02:00
Simon Frei
b628460930 lib/model: Prevent localflag flipflopping on del. item (ref #6865) (#6919) 2020-08-21 10:09:48 +02:00
Simon Frei
cc1f6e4d4a lib/db, lib/model: Cover exec-paths with debug logging (#6918) 2020-08-20 16:11:20 +02:00
Simon Frei
88599bc154 lib/model: Handle del. dir with locally changed items on pull (fixes #6873) (#6914) 2020-08-20 10:56:29 +02:00
Simon Frei
fc2c46e82f lib/config, lib/fs: Make junction behaviour configurable (ref #6606) (#6907) 2020-08-19 19:58:51 +02:00
Simon Frei
ce4d149bf5 lib/nat: Don't hang on draining timer chan (fixes #6908) (#6912) 2020-08-19 15:58:44 +01:00
Audrius Butkevicius
cbbc262161 lib/db: Dont recurse on flush (fixes #6905) (#6906)
Or else we crash.

Co-authored-by: Jakob Borg <jakob@kastelo.net>
Co-authored-by: Simon Frei <freisim93@gmail.com>
2020-08-19 13:13:44 +02:00
Audrius Butkevicius
758a1a6a37 lib/fs: Disable ioctl on ppc (fixes #6898) (#6901) 2020-08-19 07:56:35 +02:00
Audrius Butkevicius
d0c6c18b4f lib/dialer: Try dialing without reuse in parallel (fixes #6892) (#6893) 2020-08-18 13:11:12 +02:00
Audrius Butkevicius
bf9ff17267 all: Remove need to restart syncthing (#6883) 2020-08-18 09:26:33 +02:00
Simon Frei
8f5215878b lib/db: Don't put truncated files (ref #6855, ref #6501) (#6888) 2020-08-18 09:20:12 +02:00
Simon Frei
9c2117f08e lib/osutil: Check returned error instead of info (ref #6885) (#6887) 2020-08-12 11:33:56 +02:00
Simon Frei
fd8bea6179 lib/osutil: Preserve perms in AtomicWriter (fixes #tbd) (#6885) 2020-08-10 18:48:37 +02:00
Audrius Butkevicius
e9bb17307d lib/fs: Unwrap mtimeFile, get fd the "correct" way (ref #6875) (#6877) 2020-08-07 07:47:48 +02:00
Jakob Borg
5e1f39b6f6 lib/fs: Fix WatchRename test for FreeBSD (fixes #6613) 2020-08-03 23:24:01 +02:00
Audrius Butkevicius
96e197e502 lib/model: Don't close file early (fixes #6875) (#6876) 2020-08-03 21:54:42 +02:00
Simon Frei
bbf25e2d02 lib/db: Log context on panic (#6872) 2020-08-01 17:32:36 +02:00
Simon Frei
a85bc1c3a6 lib/model: Check folder error before sync-waiting (fixes #6793) (#6847) 2020-07-31 19:26:09 +02:00
Simon Frei
850dd4cd25 lib/db: Don't count invalid items to global state (ref #6850) (#6863) 2020-07-30 13:49:14 +02:00
Simon Frei
e46c8ab9ee lib/model: Detect deleted RO items scanning on non-RO (fixes #6864) (#6865) 2020-07-30 13:41:45 +02:00
Jakob Borg
2dc2aa5d21 lib/connections, lib/tlsutil: Handle certName in Go 1.15 (fixes #6867) (#6868)
Our authentication is based on device ID (certificate fingerprint) but
we also check the certificate name for ... historical extra security
reasons. (I don't think this adds anything but it is what it is.) Since
that check breaks in Go 1.15 this change does two things:

- Adds a manual check for the peer certificate CommonName, and if they
  are equal we are happy and don't call the more advanced
  VerifyHostname() function. This allows our old style certificates to
  still pass the check.

- Adds the cert name "syncthing" as a DNS SAN when generating the
  certificate. This is the correct way nowadays and makes VerifyHostname()
  happy in Go 1.15 as well, even without the above patch.
2020-07-30 13:36:11 +02:00
Jakob Borg
d53a2567a4 build: Actually set build tags (#6866)
Apparently our Tags field depended on having specific files react to
tags and add themselves there. This, instead, works for all tags.

Also, pass tags to the test command line.
2020-07-30 10:58:43 +02:00
Jakob Borg
6f4671ed27 lib/connections: Add noquic tag
The QUIC package is notorious for being incompatible with either too
old or too new Go releases. Currently it doesn't build with Go 1.15 RC
and I want to test the rest with Go 1.15. With this I can do `go run
build.go --tags noquic` to do that.
2020-07-30 09:42:02 +02:00
Simon Frei
424d1b1608 lib/db: Commit meta when dropping device (#6862) 2020-07-28 16:46:42 +02:00
Simon Frei
1b9e5c0937 lib/db: Include blocks in db check (ref #6855) (#6861) 2020-07-28 16:25:07 +02:00
Jakob Borg
62cff26edf lib/config: Repair versioning XML serialization (fixes #6859, ref #6693) 2020-07-28 12:08:37 +02:00
Simon Frei
932d8c69de lib/fs: Properly handle case insensitive systems (fixes #1787, fixes #2739, fixes #5708)
With this change we emulate a case sensitive filesystem on top of
insensitive filesystems. This means we correctly pick up case-only renames
and throw a case conflict error when there would be multiple files differing
only in case.

This safety check has a small performance hit (about 20% more filesystem
operations when scanning for changes). The new advanced folder option
`caseSensitiveFS` can be used to disable the safety checks, retaining the
previous behavior on systems known to be fully case sensitive.

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2020-07-28 11:15:11 +02:00
Simon Frei
08e0f938a9 lib/db: Update global meta even if unchanged (fixes #6850) (#6852) 2020-07-24 12:36:16 +02:00
Simon Frei
ebead944b5 lib/fs: Pass infinite recursion error on instead of warning (#6846)
Prompted by https://forum.syncthing.net/t/infinite-filesystem-recursion-detected/15285. In my opinion the filesystem shouldn't throw warnings but pass on errors for the caller to decide what's to be happening with it. Right now in this PR an infinite recursion is a normal scan error, i.e. folder is in failed state and displays failed items, but no warning. I think that's appropriate but if deemed appropriate an additional warning can be thrown in the scanner.
2020-07-22 22:10:24 +02:00
Audrius Butkevicius
55147f5901 lib/db: Rework flush hooks (#6838) 2020-07-19 08:55:27 +02:00
Jakob Borg
aedc2d788f lib/model, lib/versioner: Drive version cleanup from scanner (fixes #6313) (#6693)
This change adds a separate config for the cleanup interval, and runs that cleanup from the main folder loop.
2020-07-14 10:48:50 +02:00
Simon Frei
16f4921c50 lib/nat: Make sure nat keeps being disabled (fixes #6823) (#6824) 2020-07-14 08:16:08 +02:00