* Resume interrupted replica fetches from a .part file
A replica transfer now lands in <source>.<generation>.<blake3 prefix>.part
and stays there when the stream breaks or the daemon stops. The next
attempt asks the owner for the artifact's tail from the part's length over
the new FetchSnapshotFrom and FetchDatabaseFrom requests, whose header
still names the whole artifact; a header that names different bytes
discards the part and starts over, and the finished file is validated
against the header checksum before it replaces the live replica as before.
Parts are removed only when the owner's listing no longer carries their
generation. A nonzero offset serves the on-disk snapshot without the
pre-fetch save, so a resumed copy finishes against the same bytes.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Cap replica transfer bandwidth with a persisted daemon setting
replication.max_bytes_per_sec in spacedrive.json bounds every replica
artifact served and received through one token bucket (zero is
unlimited); the cap applies to transfers already in flight. It is set
through config.app.update and sd config set replication.max_bytes_per_sec
(bytes, or K/M/G suffixes), read back through sd config get and show.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Pause and resume replica fetching
sd sources replication pause stops this device starting replica
transfers and halts any in flight at its next chunk, keeping the partial
file; resume continues from it and runs a replication pass against every
connected device at once. The switch is replication.paused in the daemon
config, so a paused daemon stays paused across a restart, and a pass that
runs while paused still refreshes listings, owner facts and fleet totals.
A pause does not count as a fetch failure, so no backoff accrues.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Show replica fetch progress in sources list, status and events
Every fetch in flight is tracked with bytes on disk, total and a recent
rate. sources.list carries it on the replica's row (or a replica_fetching
row for a first copy that has no share yet), core.status and the new
mounts.replication_status query summarize it with the pause switch and
cap, and the daemon emits a typed ReplicationProgress event about once a
second per transfer plus a finished one when it stops. sd sources list and
sd status print the figures; ts-client types regenerated.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Document resumable replica fetch, the cap, pause and progress
Source replication gets its own section in networking.mdx with the .part
contract, the wire change, the cap, the pause switch and the progress
fields; file-sync.mdx points there; the replication plan marks its cleanup
item superseded on the replica side and distinct from the restart refetch;
PROJECT_STATUS.md gains the register row.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Keep the part when a resumed request gets no header; cap sidecar pages
A resumed request that fails outright (dropped link, or an owner too old
to know the offset variants) says nothing about the bytes on disk, so the
part now stays and the whole-file retry lands on the same name. Sidecar
pages count against the bandwidth cap on both ends, as the setting's
contract says.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Add the replication field to the test harness config
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Check the pause switch after the cap's wait, not before
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
---------
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Networking: serve streams on connections a protocol dials
The connection cache is shared between the dial side and the accept side,
so once a protocol dials a peer, the peer's own send path finds that
connection and opens its streams on it. Nothing read those streams: only
connections the event loop accepted, or those the reconnect loop handed
over, had a stream handler. The peer's request then sat unanswered until
its 30 s timeout while requests in the other direction answered in
milliseconds. This is the reverse RegisterDeviceRequest timeout SPAC-12
saw after join-remote.
get_or_create_connection now hands every connection it dials to the
event loop as TrackOutboundConnection, which spawns the same stream
handler an inbound connection gets. Messaging, sync, and job activity
pass their event loop handle; pairing keeps its own connection handling.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Sync: the device that joins a library is the one that backfills
Every sync service started Uninitialized and backfilled from the first
connected partner, whatever state it already held. When the Mac Studio
joined the MacBook's library the MacBook was the one with a partner in
its device table, so it went Uninitialized -> Backfilling{Studio} and
applied the empty copy's snapshot over its own rows (LWW on the
deterministic default-space UUID). The Studio never backfilled because
the reverse registration had timed out and its device table held only
itself.
A copy of a library created from another device's library is the only
thing that may start Uninitialized. create_library_with_id now writes an
initial_backfill_pending marker into sync.db; PeerSync reads it on open
and starts Ready otherwise. A Ready device never requests a snapshot
(SharedChangeRequest.include_snapshot = false on catch-up) and repeated
catch-up failures no longer escalate to a full backfill. A copy still
waiting for its own first backfill refuses to serve as a join target.
The join carries the rule explicitly: RegisterDeviceRequest has
needs_initial_state, and RegisterDeviceResponse returns the responder's
own device record so the joiner's table is complete when the command
returns, replacing the spawned reverse request. sync_setup fails
instead of reporting devices_registered: true when that record is
missing.
Catch-up completion is recorded per peer in sync.db even when no rows
arrived; the Ready loop catches up once with a peer it has no record
for and no longer re-polls every peer whose watermark is 60 s old. A
missing watermark means never synced, not ancient, so the forced full
sync warning only fires for a real watermark past retention.
library_join_test pairs two daemons for real, joins Bob to Alice's
library, and asserts Alice's rows are unchanged, Bob holds Alice's
device row and spaces, and Alice runs no further catch-ups.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Sync: test that catch-up with an empty peer runs once
Alice holds a volume, Bob holds nothing of his own, so Alice never
writes a per-resource watermark for Bob. Before the per-peer catch-up
record, that made Bob a new peer on every loop iteration and Alice
caught up with him forever (2 -> 5 sessions in 16 s on 5e285d3). The
test asserts the session count stops moving and the record exists.
Both new suites join the xtask integration list.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Library info: fleet totals count the same devices as device_count
add_device_summaries added every cached peer summary to a library's
statistics, whatever library the peer belonged to, and the summaries
carried no source count. A joined device therefore reported the member's
3 million files next to Sources 0 and Devices 1.
Summaries now carry the device's own source_count, and only summaries
of devices in the library's device table are added, so files, content,
sources, and capacity describe the members that device_count counts.
The CLI labels the block "Statistics (all library members)".
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Docs: describe the pairing and library join flow as it runs
pairing.mdx and devices.mdx described a confirmation prompt that does
not exist and pointed at `sd devices list` for pairing status, which
reads the library's device table and does not show a peer until it
joins the library. The CLI overview lacked the global `--device` flag,
and the sync setup walkthrough never named `library sync-setup
--action join-remote` or `--node-id`. All four now describe the flow
SPAC-12 ran, and library-sync.mdx gains sections on joining, catch-up,
and fleet totals. One PROJECT_STATUS register row.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Sync: catch up once per connection session, never serve a snapshot while pending
Review of #3110 found two gaps. The per-peer catch-up record was never
cleared, so a peer that changed rows while disconnected was never caught
up with again after reconnecting (the live broadcast only lives in the
in-memory retry queue). The record is now cleared when the sync service
starts and when the peer's connection is lost; ConnectionEstablished is
not used because the event loop emits it per accepted stream.
The SharedChangeRequest handler attached a full snapshot without
checking the first-backfill marker, so two copies that both joined from
the same member could seed each other. It now refuses the snapshot while
pending, and the register handler reads the marker from sync.db directly
when the library's sync service is not up yet.
Also: a failed backfill or catch-up left active_backfill_sessions
incremented; the mock transport no longer lists a peer as a partner
before its sync service is registered; the catch-up test now starts
Alice Ready (marker cleared) and Bob pending, the shape of a real join,
and asserts exactly one catch-up and one backfill.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Sync: move past a peer that refuses a snapshot, fail the guard closed, refresh stale catch-ups
A joiner whose chosen peer refused the snapshot kept choosing it: every
partner was scored has_complete_state: true and RTT is only recorded on
success, so the order never changed. A peer whose backfill failed now
scores without complete state until a backfill succeeds, so a connected
member holding the library is tried next.
The snapshot guard read an unreadable marker as "holds state"; it now
refuses, since a wrong refusal only delays a join.
A broadcast whose retries ran out while the QUIC session stayed up had
no recovery under the once-per-session rule. The Ready loop refreshes a
catch-up record older than ten minutes; with per-peer watermarks an
empty refresh is two small requests.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
---------
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
Close the iroh endpoint on networking shutdown, bound every reconnection
dial, give the deferring side of the EndpointId rule a grace period after
which it dials anyway, and re-arm the periodic sweep for any device still
unconnected.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
Spacebot ships separately. This drops its UI, the voice overlay and the
Alt+Space shortcut that opened it, the spacebot app config and window, the
@spacebot/api-client and @spacedrive/ai wiring, and the Spacebot and
Console design docs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The plan's status and what landed, with the SMB defects and the loopback decision: pairing covers the hop between devices, and the share guards against browsers with a Host check rather than a key, since RPC on 6969 is open to local processes anyway. The mounts design's 2026-09-24 status, the macOS volume detection note on the daemon's own mount, and the register row with titan on this build since 2026-09-24 19:52 PDT and its databases backed up first.
Every job that changes the filesystem records effects (created, moved, trashed with where it went, replaced with where the previous bytes went, removed, attributes before and after) in a job_journal table through ctx.record. The trash reports locations: NSFileManager on macOS, the trash crate's listing elsewhere, and a .spacedrive/trash/<job>/ directory on volumes with none. Overwrites in copy, merge and extract stash the previous file there. files.undo previews the reverse as an FsPlan, refuses what changed since, and its job writes its own journal.
Also: files.rename on preflight with per-filesystem name rules and case probing, files.rename_batch with ordered rules and a job that never overwrites; remove_extras on files.merge for a mirror; files.organize and files.flatten as in-place renames; files.archive and files.extract for zip and tar.zst, planned from the archive's directory; files.set_attributes and files.link; files.trash_list and files.trash_empty; jobs.journal and a journal summary on jobs.list. The copy job now honors the skip policy and merges a folder onto an existing folder instead of overwriting inside it, and the copy preview plans a keep-both copy at its numbered name.
30 tests over the fixture. Docs, the plan's status table and PROJECT_STATUS updated.
files.delete takes a Duplicates target: a scope with keep first (walk order) or a list of copies to keep with every other copy of their content going. The job reads each surplus copy and its keeper in full before removing, records the hashes it learns, and checkpoints the compare cursor so it resumes. Preflight validates the scope and keep list and previews keepers as skips and surplus copies as deletes.
Store gains duplicated_contents_beneath and copies_beneath. files.duplicates is registered as a core query and reports each copy's SdPath. CLI: sd file dedupe with --keep, --keep-under, --min-size, --dry-run.
Two opt-in methods per action, validate: and preview:, over the exact
input the action takes. Validation answers findings with stable codes and
the facts of the execution, runs again at dispatch and refuses on an
error. Preview answers an FsPlan read from the index, kept under a handle
so files.directory_listing can serve the directory as it would look after
the operation.
Copy, move, merge and delete answer both. Merge is new: a resumable job
that recurses into matching folders, reads both sides before skipping a
duplicate, applies a per-leaf conflict policy, leaves conflicts alone and
can consume its sources. Delete's validation counts, across every store,
which files are the last copy of their content.
The ValidationResult and ConfirmationRequest hook is gone, and so is the
CLI's local conflict detection. file copy, merge and delete render
preflight before the prompt and take --dry-run.
paths.compare answers what differs between two folders from the source
stores at query time. By location it merges both sides in index order
into only-left, only-right, changed and same. By content it streams one
side and asks the other side's stores which content ids they hold, so a
renamed or moved copy still matches. Pages resume from a cursor and the
first page carries every set's count. Unhashed pairs fall back to size
and mtime and get counted, so a page says how much rests on that.
sd-cli file compare wraps it.
On the store side, files_beneath takes a Start, including one that
resumes a nested store at its first directory, and contents_beneath
answers the content lookup a batch at a time. Working out which stores a
path reaches moved into ops/paths/reach.rs, shared with search.media and
paths.context.
No catalog.db: the P7 row, the filesystem-intelligence plan, the previs
and the design docs now treat cross-source questions as store queries.
The hot cache is raw BGRA and disposable, so nothing durable held a tile:
a wiped cache meant decoding every original again, and a replica could
only show a peer's thumbnails while that peer was awake to send them.
Every tile baked now also goes into its source's sidecars.db as WebP,
5 to 15 KB against around 300 KB in the cache, keyed by record uuid and
tile version. Content hashes would have deduped copies, but a content id
is re-derived when a full hash lands, which would orphan anything keyed
by it. The cache refills from a sidecar at the current version instead
of baking; a forced bake still reads the file. thumbs generate also
encodes a sidecar from the cache for any cached tile without one, so an
existing cache converts without touching the originals.
Each store counts its writes and stamps rows with the count, paired with
an id drawn at creation. Listings advertise both, and after each
replication pass a replica pulls anything newer with FetchSidecars, a
page at a time, into mounts-remote/<device>/<source>.sidecars.db. Remote
tile requests restore from that copy before asking the owner, so they
work with the owner offline. An owner answering FetchTiles sends a
current sidecar's WebP as stored, and tiles fetched on demand land in
the replica's copy too.
Titan converted 97,728 cached tiles into 1.05 GB of sidecars, down from
21 GB of cache. The Mac copied all of them and drew titan tiles it had
never fetched with titan's daemon stopped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A nested source's generation came from the size and mtime of data.db and
its WAL. SQLite deletes the WAL when a pool's last connection closes and
recreates it on the next read, and sqlx closes idle connections after 10
minutes, so the generation moved without any row changing. Titan re-copied
the Mac's unchanged home store in full four times overnight, each round a
736 MB transfer and a 57-minute arena rebuild.
Stores now keep a _revision row, a store id and a count of committed
changes, maintained by triggers on every tracked table. Updates that change
nothing, or only restamp scan_epoch or indexed_at, don't count. Listings and
database headers use the revision instead of file metadata.
The triggers put each single-row store write in a statement journal: the
300k-file scale test builds in 16-18s instead of 12-14s. temp_store=MEMORY
brings it to 9-10s but moves large sorts on the writable pool into memory,
so that's an open decision in the new incremental replication plan.
Verified live: after one switch-over copy, titan made no transfer for twelve
minutes, including across a Mac daemon restart.
Paired devices answer a new DeviceFacts request with their hardware and
mounted volumes, under the ids their sources anchor to. The peer fetches
them on every sync beside the device summary and keeps them in the replica
manifest with its own receipt time, so an offline device still describes
itself as it last did.
volumes.list lists those volumes under their owner, mounted only while it's
connected, and devices.list fills in a paired device's hardware. The
overview folds replicas into their owner's volume bars the way local
sources fold into theirs. Another device's volumes show when they were last
seen and hide the local volume actions on desktop and mobile.
Volume classifications travel as names, so one a peer's build doesn't know
reads as unknown instead of failing the response.
AGENTS.md and CLAUDE.md flip their current direction from tearing down the
entries world to it being gone, and drop the ephemeral note T6.7 retired.
The locations page is deleted; the overview page that explained pins as
location rows becomes Add Sources and Pins, and the data model, library,
architecture, sync, and CLI pages describe pins as Space items. The React
explorer and normalized cache guides use sources and spaces for their
examples, the snapshot format doc points at the moved module, and
CONTRIBUTING tracks a source instead of adding a location.
The final-drop plan records FD5 with what still fails and why, the
convergence plan notes the old world is gone, and the registers move FD5 to
done with FDA ready.
Library statistics counted the legacy tag table, which would have read zero
forever; the instance count now reads definitions across staging and the
stores, and the pre-open static count reads staging with the instance
recount behind it. tagging.mdx describes the built system instead of the
old design sketch, library.mdx stops claiming the library owns tags, and
the design doc carries its three reconciliations. The plans and register
record stretch 1 closed: delivery verified against titan, FD4's
preconditions banked, phase 4 landed.
Locations go in the breaking release without row migration or API
compatibility. Policies accept SdPath and persist a separate
{ source_id, relative_path } target resolved through the source layer;
SdPath gains no source variant; Space items own navigation. The
final-drop boundaries, teardown status, sync research, and core docs
pages now agree. Register brought current through 5a65d0998 and the
search refinement bar.
The source runtime reliability plan graduates from draft to the
executing plan its register row already tracks. Library-sync research
records the split-library state between the Mac and titan and proposes
the small library-state contract; locations research records the
revised direction of configured source subtrees with continuous
processing rules over pins alone, cross-referenced from the
entries-teardown T4.4 row. The working register carries today's landed
work: search phases, fleet statistics, storage replicas, the hashing
loop fix, and the first reliability slices, with the code baseline
brought current.
From a parallel session. thumbs.generate accepts a file or an indexed
folder with missing, stale, and force modes, runs in resumable batches,
and versions video tiles against host FFmpeg availability so a tile
regenerated with better tooling replaces its predecessor in place. File
and folder context menus and the new sd thumbs CLI domain call the same
operation. paths.context now reports whether the exact path has a
committed source record, and the path bar surfaces it with a refresh
affordance. Coverage and resume semantics are documented in
docs/core/thumbnail-generation.mdx.
README and CONTRIBUTING updates, the working register brought current,
locations and data-model pages reconciled, redundancy and overview
polish, media view tweaks, regenerated schemas, and the new Spacedrive
icon asset.
Library loading no longer dispatches a filesystem discovery pass. The
desktop requests one idempotent pass after the connected shell
renders, a bare daemon initiates nothing, and sd index start
--defaults gives CLI and server operators the same explicit path.
Restored volume snapshots prevent repeat full walks.
A machine-scoped registry discovers host FFmpeg and companion tools,
surfaced in Services settings. The thumbnail service uses them for
video posters and on-demand scrub sheets: thumbstrips bake per request
into the volume hot tier and serve over the local HTTP servers with
immutable content-versioned URLs, wired to the timeline scrubber.
The path bar reads a typed path-context query backed by the daemon's
source, volume map, watcher, persistence, and pin state, replacing the
location/index-mode indicator. Containment resolves on the daemon with
path components after volume alias normalization; a pin no longer
implies indexing or watching, and the docs say so.
Replaces the Mintlify setup (mint.json, docs.json, snippets) with a
standalone Fumadocs app in docs/: Next.js app router, meta.json nav per
section, its own bun install. Run with `just dev-docs`.
Content pass alongside the migration: core architecture, data model,
CLI, and networking pages rewritten around the daemon/source-store/
library split, new product-direction.mdx defining the product boundary,
new drive-catalog plan, and the physical-drives plan updated to
reference it. Whitepaper page retired.
A store has two halves. The generation is rebuildable for as long as its
origin still answers, the assertion layer never is, and the origin stops
answering without an event. So the invariant is a rule about what code may
assume rather than a claim about stores, and the registry row tracks what is
actually known about the origin.
Falls out of that: reindex and evict are transactions over one half, and sync
replicates the generation while merging assertions. Which means the assertion
tables need their key and their merge columns settled before P2 fills them,
so P1 gains the re-key, convergent content ids and the uuid narrowing.
New doc is docs/core/design/source-durability.md. Also records two
contradictions found on the way: whether sources may nest (contract 1 and the
previs disagree), and that nothing reconciles record uuids when two devices
index the same origin.
The README still advertised a screening pipeline — Prompt Guard 2, quarantine,
trust metadata on results — that was a stub and is gone. Cut the section
rather than describe an intention as a feature.
The two archive docs went stale from later commits in the same session: they
still put durable assertions in registry.db, still said deleting a source
leaves them behind, and did not mention record_overlay in the store layout.
Both now describe one file per source with no shared durable layer.
Also: the crate layering is stated in both docs, AGENTS.md points at
crates/store for the record table, and the crate docs stop promising
cross-source edges — edge relates records within one source, and anything
spanning sources arrives with the catalog.
docs/core/design/archive.md and docs/archive/README.md described archive as
a second data plane beside the VDFS. The README also documented an
adapter.toml format and a JSONL protocol that never existed in this tree,
and claimed a subprocess sandbox that doesn't. Both rewritten from source.
"spine" is now "the record table" across the remaining docs, and
2026-08-17-record-spine-transplant.md renamed to match.
Adds docs/plans/2026-08-22-source-convergence.md: the register for getting
to one store shape and one source registry before the entries teardown
continues. AGENTS.md and the teardown execution plan point at it.
Photos no longer runs its own demo bake; it follows a Spacedrive explorer
window via a new navigation-focus resource and renders that window's
folder as a media grid, sourced from thumbnails baked by a new daemon-owned
thumbs service. Supporting changes: pvcache moves to a v2 format that
stores each tile's true aspect (envelope + frame) instead of forcing a
square crop, peer mounts now refresh on a generation check instead of only
at reconnect, and the archive crate's `spine` module is renamed to `record`
throughout.
Absolute paths from one developer's machine sat in an example, a shell script,
recorded benchmark metadata, task docs and design docs. The example and the
script were broken for anyone else as a result: the example now resolves against
CARGO_MANIFEST_DIR and the script derives the repo root from git. The duplicate
copy of that script under core/scripts is gone.
Also drops a pointer to a repository that is not public.
The entry schema is the last thing standing from the old model and nothing
needs it. These cover the cutover, the destination written as though built, and
the hazards for work landing next to it.
Image thumbs no longer sit behind ffmpeg gates, sidecar versions bust the cache,
the sidecar layout moves to a shared crate, and metadata backfill is separated
from generation. Video dimensions are read for real; the cas_id tier and dead
settings are removed.
Read-only share on loopback, a byte-range peer protocol for remote sources, and
a block cache in front of both so a second read of the same file does not
re-fetch it.
Sources resolve at startup without onboarding. Adds the Apple Photos native
adapter, collections and the analyzer, the bundle lens, and assertions with
read-time binding.
Each source owns its partition with detached restore, volume fingerprints are
machine-independent so an external drive keeps its identity across hosts, and
directory-size rollups update incrementally. Snapshots gain a restore gate, a
save invariant, unique tmp writes and an fsync before rename.
crates/archive carries record, content and edge tables with a durable overlay
beside them, so the rebuildable generation can be dropped and recreated without
losing anything written through the assertion surface.
* feat: add redundancy awareness & cross-volume file comparison
Surface content redundancy data so users can answer "if this drive dies,
what do I lose?" — builds on existing content identity and volume systems.
Backend:
- New `redundancy.summary` library query with per-volume at-risk vs
redundant byte/file counts and a library-wide replication score
- Extend `SearchFilters` with `at_risk`, `on_volumes`, `not_on_volumes`,
`min_volume_count`, `max_volume_count` filters
- Add composite index migration on entries(content_id, volume_id)
Frontend:
- `/redundancy` dashboard with replication score, volume bars, at-risk callout
- `/redundancy/at-risk` paginated file list sorted by size
- `/redundancy/compare` two-volume comparison (unique/shared toggle)
- Sidebar ShieldCheck button linking to redundancy view
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* redundancy UI improvements + ZFS volume detection fix
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix ZFS pool capacity reporting and stats filtering
- ZFS: override total_capacity for pool-root volumes using zfs list
used+available. df under-reports pool-root Size because it only
counts the root dataset's own used bytes plus avail — on a 60 TB
raidz2 pool this shows as ~15 TB instead of ~62 TB. The pool root's
own used property includes descendants, so used+available is the
real usable capacity.
- Library stats: drop volumes where is_user_visible=false AND
re-apply should_hide_by_mount_path retroactively so stale DB rows
(detected before the Linux visibility filters existed) don't
inflate reported capacity.
- Extract should_hide_by_mount_path into volume/utils as a shared
helper used by both the list query and the stats calculation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* show capacity and visibility in sd volume list
Makes it possible to verify library-level capacity aggregation from
the CLI — previously the list only showed mount, fingerprint, and
tracked/mounted state, which meant debugging the ZFS pool capacity
issue required querying the library DB directly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* document filesystem support matrix and detection
New docs/core/filesystems.mdx covering per-filesystem capabilities
(CoW, pool-awareness, visibility filtering, capacity correction),
platform detection strategies, the FilesystemHandler trait, Linux/
macOS/ZFS visibility rules, the ZFS pool-root capacity problem and
fix, copy strategy selection, and known limitations.
Registered under File Management in both mint.json and docs.json.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* WIP: redundancy filter wiring across search, CLI, and UI
- core/src/ops/search: wire redundancy filters (at_risk, on_volumes,
not_on_volumes, min/max volume_count) through the search query;
fix UUID-to-SQLite BLOB literal so volume UUID comparisons actually
match (volumes.uuid is stored as a 16-byte BLOB, quoted-string
comparison silently returned zero rows).
- apps/cli: new redundancy subcommand + populate the new
SearchFilters fields from search args.
- packages/interface: redundancy at-risk and compare pages reworked
to consume the new filter surface; explorer context/hook updates
to support redundancy-scoped views.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* add web context menu renderer and UI polish
- New WebContextMenuProvider + Radix DropdownMenu-based renderer anchored
at cursor via a 1x1 virtual trigger. Handles separators, submenus,
disabled, and the danger variant via text-status-error.
- useContextMenu now routes web clicks through the provider instead of
parking data in unused local state, and trims leading/trailing/adjacent
separators so condition-filtered menus don't render orphaned lines.
- Drop app-frame corner rounding on the web build.
- Add shrink-0 to the sidebar space switcher so the scrollable sibling
can't compress it vertically.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* sd-server dev workflow: auto web build, shutdown watchdog, stable data dir
- build.rs runs `bun run build` in apps/web so `just dev-server` always
embeds the latest UI. rerun-if-changed covers apps/web/src,
packages/interface/src, and packages/ts-client/src so Rust-only edits
skip the rebuild. Skips gracefully when bun isn't on PATH or
SD_SKIP_WEB_BUILD is set; Dockerfile sets the latter since dist is
pre-built and bun isn't in the Rust stage.
- Graceful shutdown was hanging because the browser holds the /events
SSE stream open forever and axum waits for all connections to drain.
After the first signal, arm a background force-exit on second Ctrl+C
or 5s timeout so the process can't stick.
- Debug builds were starting from a fresh tempfile::tempdir() on every
run (the TempDir handle dropped at end of the closure, deleting the
dir we just took a path to). Default to ~/.spacedrive in debug so data
persists and `just dev-server` shares a data dir with the Tauri app.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* add Sources space item alongside Redundancy in default library layout
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* add TrueNAS native build script
Uses zig cc as C/C++ compiler on TrueNAS Scale where /usr is
read-only and no system gcc exists. Dev tools live at
/mnt/pool/dev-tools/ (zig, cmake, make, extracted deb headers).
Builds sd-server + sd-cli in ~4 min on a 12-core NAS. AI feature
disabled (whisper.cpp C11 atomics incompatible with zig clang-18).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* never block RPC on synchronous statistics calculation
On first load (fresh library, all stats zero), libraries.info used
to calculate statistics synchronously before responding. On large
libraries during active indexing this hangs indefinitely — the
closure-table walk in calculate_file_statistics loads every
descendant ID into a Vec then issues a WHERE IN(...) with millions
of entries, which SQLite can't finish while the indexer is writing.
Now always return cached (possibly zero) stats and let the
background recalculate_statistics task fill them in. The UI
refreshes via the ResourceChanged event when the calculation
completes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* self-heal protocol handler registration on re-init
Core::new() registers default protocol handlers after starting networking,
but swallows any failure (error is only logged). If the initial registration
fails — e.g. on a host where start_networking hasn't fully set up the event
loop command sender by the time register_default_protocol_handlers runs —
the registry is left empty. A subsequent call to Core::init_networking()
would see `services.networking().is_some()` and skip re-registration,
permanently leaving protocols unregistered for the life of the process.
sd-server calls init_networking() right after Core::new(), so it's the
client most exposed to this. Symptom: pairing over the web UI returns
"Pairing protocol not registered" while the same library works fine
from Tauri and mobile.
Fix: init_networking now queries the registry directly for the pairing
handler and re-registers the default set if it's missing, independent of
whether networking is already initialized.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fall back to pkarr+DNS discovery when mDNS port is unavailable
Iroh's endpoint.bind() fails wholesale if any configured discovery service
fails to initialize. MdnsDiscovery requires binding UDP :5353, which on
most Linux systems (including TrueNAS) is already owned by avahi-daemon.
Result: endpoint creation errors out with "Service 'mdns' error", the
event loop never starts, command_sender stays None, and protocol
registration fails — so sd-server has no working networking at all.
Make mDNS best-effort: on any error whose message mentions "mdns",
retry endpoint creation with only pkarr + DNS discovery. Local-network
auto-discovery is lost but remote pairing via node ID (which uses n0's
DNS infrastructure, not mDNS) continues to work normally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* succeed pairing if either mDNS or relay discovery wins
The dual-path discovery in start_pairing_as_joiner_with_code used
tokio::select! to race mDNS and relay. select! resolves on the first
branch to complete — including errors — so a host that can't bind
mDNS (e.g. a Linux box where avahi already owns UDP :5353) would fail
pairing wholesale: mDNS discovery errors out in <1ms with
"Failed to create mDNS discovery: Service 'mdns' error", that Err
wins the race, and relay discovery gets cancelled before it can even
begin.
Switch to futures::select_ok so we only return the error if EVERY
discovery path has failed. mDNS failing immediately now leaves relay
running to completion, which is the common case for remote pairing
into a NAS.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Added a new GitHub Actions workflow (`mobile.yml`) for manual dispatch to build and release the React Native app for iOS and Android.
- Updated the publish artifacts action to change Windows bundle type from MSI to NSIS and adjusted related paths.
- Modified the front-end bundle path to reflect the new structure in the mobile app.
- Enhanced documentation to include details about the new mobile release process and required secrets.
- Updated device versioning to use the `updated_at` timestamp for conflict resolution.
- Refactored sync methods to support shared changes with HLC-based conflict resolution.
- Implemented automatic device discovery via shared sync, allowing devices to propagate changes to all members in the library.
- Added migration to transition existing devices to shared sync.
- Enhanced documentation to reflect changes in device ownership and sync methods.
This commit refines the volume ownership model by ensuring that entries and locations inherit ownership from their respective volumes. It updates the documentation to clarify the sync ownership flow, emphasizing the seamless transfer of ownership when external drives are connected to different devices. Additionally, it improves the overall clarity of the sync state machine and related processes, ensuring that the documentation accurately reflects the system's behavior and enhances understanding for developers and users.
This commit updates the data model to replace the `device_id` field with `volume_id` in the `entries` and `locations` tables. The change allows entries to inherit ownership from their associated volumes, simplifying ownership management during volume transfers. Additionally, migrations have been added to facilitate this transition, ensuring that existing data is correctly updated and indexed. The codebase has been adjusted to reflect these changes across various modules, including sync and query functionalities.
- Added device_id parameter to DatabaseAdapter and DatabaseAdapterForJob for improved context handling.
- Updated ChangeDetector to utilize device_id when creating persistence instances, ensuring accurate indexing behavior.
- Refactored related tests to verify correct parent-child relationships and prevent duplicate entries during folder moves.
