mirror of
https://github.com/syncthing/syncthing.git
synced 2025-12-23 22:18:14 -05:00
c918299eab
This just removes an unnecessary foreign key constraint, where we
already do the garbage collection manually in the database service.
However, as part of getting here I tried a couple of other variants
along the way:
- Changing the order of the primary key from `(hash, blocklist_hash,
idx)` to `(blocklist_hash, idx, hash)` so that inserts would be
naturally ordered. However this requires a new index `on blocks (hash)`
so that we can still look up blocks by hash, and turns out to be
strictly worse than what we already have.
- Removing the primary key entirely and the `WITHOUT ROWID` to make it a
rowid table without any required order, and an index as above. This is
faster when the table is small, but becomes slower when it's large (due
to dual indexes I guess).
These are the benchmark results from current `main`, the second
alternative below ("Index(hash)") and this proposal that retains the
combined primary key ("combined"). Overall it ends up being about 65%
faster.
<img width="764" height="452" alt="Screenshot 2025-08-29 at 14 36 28"
src="https://github.com/user-attachments/assets/bff3f9d1-916a-485f-91b7-b54b477f5aac"
/>
Ref #10264
35 lines
1.3 KiB
SQL
35 lines
1.3 KiB
SQL
-- Copyright (C) 2025 The Syncthing Authors.
|
|
--
|
|
-- This Source Code Form is subject to the terms of the Mozilla Public
|
|
-- License, v. 2.0. If a copy of the MPL was not distributed with this file,
|
|
-- You can obtain one at https://mozilla.org/MPL/2.0/.
|
|
|
|
-- Block lists
|
|
--
|
|
-- The block lists are extracted from FileInfos and stored separately. This
|
|
-- reduces the database size by reusing the same block list entry for all
|
|
-- devices announcing the same file. Doing it for all block lists instead of
|
|
-- using a size cutoff simplifies queries. Block lists are garbage collected
|
|
-- "manually", not using a trigger as that was too performance impacting.
|
|
CREATE TABLE IF NOT EXISTS blocklists (
|
|
blocklist_hash BLOB NOT NULL PRIMARY KEY,
|
|
blprotobuf BLOB NOT NULL
|
|
) STRICT, WITHOUT ROWID
|
|
;
|
|
|
|
-- Blocks
|
|
--
|
|
-- For all local files we store the blocks individually for quick lookup. A
|
|
-- given block can exist in multiple blocklists and at multiple offsets in a
|
|
-- blocklist. We eschew most indexes here as inserting millions of blocks is
|
|
-- common and performance is critical.
|
|
CREATE TABLE IF NOT EXISTS blocks (
|
|
hash BLOB NOT NULL,
|
|
blocklist_hash BLOB NOT NULL,
|
|
idx INTEGER NOT NULL,
|
|
offset INTEGER NOT NULL,
|
|
size INTEGER NOT NULL,
|
|
PRIMARY KEY(hash, blocklist_hash, idx)
|
|
) STRICT, WITHOUT ROWID
|
|
;
|