mirror of
https://github.com/spacedriveapp/spacedrive.git
synced 2026-10-30 12:59:38 -04:00
Jamie Pineandcapy-ai-staging[bot]
f9bdf724b6
* Networking: serve streams on connections a protocol dials
The connection cache is shared between the dial side and the accept side,
so once a protocol dials a peer, the peer's own send path finds that
connection and opens its streams on it. Nothing read those streams: only
connections the event loop accepted, or those the reconnect loop handed
over, had a stream handler. The peer's request then sat unanswered until
its 30 s timeout while requests in the other direction answered in
milliseconds. This is the reverse RegisterDeviceRequest timeout SPAC-12
saw after join-remote.
get_or_create_connection now hands every connection it dials to the
event loop as TrackOutboundConnection, which spawns the same stream
handler an inbound connection gets. Messaging, sync, and job activity
pass their event loop handle; pairing keeps its own connection handling.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Sync: the device that joins a library is the one that backfills
Every sync service started Uninitialized and backfilled from the first
connected partner, whatever state it already held. When the Mac Studio
joined the MacBook's library the MacBook was the one with a partner in
its device table, so it went Uninitialized -> Backfilling{Studio} and
applied the empty copy's snapshot over its own rows (LWW on the
deterministic default-space UUID). The Studio never backfilled because
the reverse registration had timed out and its device table held only
itself.
A copy of a library created from another device's library is the only
thing that may start Uninitialized. create_library_with_id now writes an
initial_backfill_pending marker into sync.db; PeerSync reads it on open
and starts Ready otherwise. A Ready device never requests a snapshot
(SharedChangeRequest.include_snapshot = false on catch-up) and repeated
catch-up failures no longer escalate to a full backfill. A copy still
waiting for its own first backfill refuses to serve as a join target.
The join carries the rule explicitly: RegisterDeviceRequest has
needs_initial_state, and RegisterDeviceResponse returns the responder's
own device record so the joiner's table is complete when the command
returns, replacing the spawned reverse request. sync_setup fails
instead of reporting devices_registered: true when that record is
missing.
Catch-up completion is recorded per peer in sync.db even when no rows
arrived; the Ready loop catches up once with a peer it has no record
for and no longer re-polls every peer whose watermark is 60 s old. A
missing watermark means never synced, not ancient, so the forced full
sync warning only fires for a real watermark past retention.
library_join_test pairs two daemons for real, joins Bob to Alice's
library, and asserts Alice's rows are unchanged, Bob holds Alice's
device row and spaces, and Alice runs no further catch-ups.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Sync: test that catch-up with an empty peer runs once
Alice holds a volume, Bob holds nothing of his own, so Alice never
writes a per-resource watermark for Bob. Before the per-peer catch-up
record, that made Bob a new peer on every loop iteration and Alice
caught up with him forever (2 -> 5 sessions in 16 s on 5e285d3). The
test asserts the session count stops moving and the record exists.
Both new suites join the xtask integration list.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Library info: fleet totals count the same devices as device_count
add_device_summaries added every cached peer summary to a library's
statistics, whatever library the peer belonged to, and the summaries
carried no source count. A joined device therefore reported the member's
3 million files next to Sources 0 and Devices 1.
Summaries now carry the device's own source_count, and only summaries
of devices in the library's device table are added, so files, content,
sources, and capacity describe the members that device_count counts.
The CLI labels the block "Statistics (all library members)".
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Docs: describe the pairing and library join flow as it runs
pairing.mdx and devices.mdx described a confirmation prompt that does
not exist and pointed at `sd devices list` for pairing status, which
reads the library's device table and does not show a peer until it
joins the library. The CLI overview lacked the global `--device` flag,
and the sync setup walkthrough never named `library sync-setup
--action join-remote` or `--node-id`. All four now describe the flow
SPAC-12 ran, and library-sync.mdx gains sections on joining, catch-up,
and fleet totals. One PROJECT_STATUS register row.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Sync: catch up once per connection session, never serve a snapshot while pending
Review of #3110 found two gaps. The per-peer catch-up record was never
cleared, so a peer that changed rows while disconnected was never caught
up with again after reconnecting (the live broadcast only lives in the
in-memory retry queue). The record is now cleared when the sync service
starts and when the peer's connection is lost; ConnectionEstablished is
not used because the event loop emits it per accepted stream.
The SharedChangeRequest handler attached a full snapshot without
checking the first-backfill marker, so two copies that both joined from
the same member could seed each other. It now refuses the snapshot while
pending, and the register handler reads the marker from sync.db directly
when the library's sync service is not up yet.
Also: a failed backfill or catch-up left active_backfill_sessions
incremented; the mock transport no longer lists a peer as a partner
before its sync service is registered; the catch-up test now starts
Alice Ready (marker cleared) and Bob pending, the shape of a real join,
and asserts exactly one catch-up and one backfill.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Sync: move past a peer that refuses a snapshot, fail the guard closed, refresh stale catch-ups
A joiner whose chosen peer refused the snapshot kept choosing it: every
partner was scored has_complete_state: true and RTT is only recorded on
success, so the order never changed. A peer whose backfill failed now
scores without complete state until a backfill succeeds, so a connected
member holding the library is tried next.
The snapshot guard read an unreadable marker as "holds state"; it now
refuses, since a wrong refusal only delays a join.
A broadcast whose retries ran out while the QUIC session stayed up had
no recovery under the once-per-session rule. The Ready loop refreshes a
catch-up record older than ten minutes; with per-peer watermarks an
empty refresh is two small requests.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
---------
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
288 lines
8.9 KiB
Plaintext
288 lines
8.9 KiB
Plaintext
---
|
|
title: Spacedrive CLI
|
|
description: Operate the Spacedrive daemon from a terminal or script
|
|
sidebarTitle: Overview
|
|
---
|
|
|
|
The Spacedrive CLI is a typed client of the same daemon used by the desktop and
|
|
web interfaces. It can manage libraries, sources, files, jobs, devices, logs,
|
|
and remote operations without bypassing core permissions or durability rules.
|
|
|
|
## Install from source
|
|
|
|
~~~bash
|
|
cargo build --release --bin sd-cli --bin sd-daemon
|
|
~~~
|
|
|
|
The binaries are written to **target/release**. Keep **sd-cli** and
|
|
**sd-daemon** together if you use daemon auto-start.
|
|
|
|
## Start the daemon
|
|
|
|
~~~bash
|
|
sd-cli start
|
|
sd-cli status
|
|
sd-cli stop
|
|
~~~
|
|
|
|
Run the daemon in the foreground while diagnosing startup:
|
|
|
|
~~~bash
|
|
sd-cli start --foreground
|
|
~~~
|
|
|
|
The desktop app starts automatic discovery once its window shows, which tracks
|
|
the system volume, adds the home folder as a source and maps every attached
|
|
drive. From a terminal, request the same pass:
|
|
|
|
~~~bash
|
|
sd-cli index start --defaults
|
|
~~~
|
|
|
|
For a library that should hold only what you add to it, such as one whose data
|
|
directory lives on an external drive, start the daemon without those defaults.
|
|
The pass still restores, heals and hashes the sources you added:
|
|
|
|
~~~bash
|
|
sd-cli --data-dir "/Volumes/Archive/.spacedrive" start --no-default-sources
|
|
~~~
|
|
|
|
`restart` takes the same flag.
|
|
|
|
## Select a library
|
|
|
|
~~~bash
|
|
sd-cli library create Personal
|
|
sd-cli library list
|
|
sd-cli library switch --library-id <LIBRARY_ID>
|
|
sd-cli library info
|
|
~~~
|
|
|
|
The selected library is remembered for later commands. Use **--library** where
|
|
a command supports an explicit library override.
|
|
|
|
## Track and preserve sources
|
|
|
|
~~~bash
|
|
sd-cli sources track ~/Documents
|
|
sd-cli sources track /Volumes/Archive --unfiltered
|
|
sd-cli sources list
|
|
sd-cli sources update <SOURCE_ID> --name "Archive 2026"
|
|
sd-cli sources verify <SOURCE_ID>
|
|
sd-cli sources freeze <SOURCE_ID>
|
|
~~~
|
|
|
|
A source owns capture and its portable store. Locations are named pins inside a
|
|
source. There are no shallow, content, or deep location index modes.
|
|
|
|
## Inspect durable work
|
|
|
|
~~~bash
|
|
sd-cli job list
|
|
sd-cli job info <JOB_ID>
|
|
sd-cli job monitor --job-id <JOB_ID>
|
|
sd-cli job pause <JOB_ID>
|
|
sd-cli job resume <JOB_ID>
|
|
sd-cli job cancel <JOB_ID>
|
|
~~~
|
|
|
|
Not every job can resume safely. A filesystem walk starts fresh after process
|
|
loss because its sweep evidence is process-local. Transfer and analysis jobs
|
|
resume only when their implementation preserves enough state.
|
|
|
|
## Read logs
|
|
|
|
~~~bash
|
|
sd-cli logs show --timestamps
|
|
sd-cli logs show --job-id <JOB_ID>
|
|
sd-cli logs follow --timestamps --show-job-id
|
|
~~~
|
|
|
|
Logs carry job and library context when the daemon has it. Machine-readable
|
|
operation output is separate from diagnostic logging.
|
|
|
|
## Pair devices
|
|
|
|
~~~bash
|
|
sd-cli network pair generate
|
|
sd-cli network pair join "<PAIRING_CODE>"
|
|
sd-cli network pair status
|
|
sd-cli network devices
|
|
~~~
|
|
|
|
Pairing establishes device identity. Joining the peer to a library is a second
|
|
step, `sd-cli library sync-setup`, described in
|
|
[Library Sync Setup](/cli/library-sync-setup). `sd-cli devices list` shows the
|
|
current library's members, so it lists the peer only after that step.
|
|
|
|
## Run a command on a paired device
|
|
|
|
The global `--device` flag sends any command to a paired device and prints its
|
|
answer. Name the device by display name, slug, or id:
|
|
|
|
~~~bash
|
|
sd-cli --device james-s-mac-studio sources list
|
|
sd-cli --device "James's MacBook Pro" job list
|
|
sd-cli --device <DEVICE_ID> devices list
|
|
~~~
|
|
|
|
Library commands resolve against the target device's own current library.
|
|
|
|
## File and search operations
|
|
|
|
Explore each domain from the installed binary:
|
|
|
|
~~~bash
|
|
sd-cli file --help
|
|
sd-cli search --help
|
|
sd-cli tag --help
|
|
sd-cli redundancy --help
|
|
sd-cli volume --help
|
|
~~~
|
|
|
|
Use Spacedrive paths when a command needs an address. The same operation can
|
|
route to local files, remote sources, provider objects, or content identity.
|
|
|
|
Compare two indexed folders without reading either drive. The first folder is
|
|
A and the second is B. By path, files pair by their path within each folder. By
|
|
content, a file matches wherever its bytes sit in the other folder, which is
|
|
how to check a backup:
|
|
|
|
~~~bash
|
|
sd-cli file compare ~/Photos /Volumes/Backup/Photos --show different
|
|
sd-cli file compare ~/Photos /Volumes/Backup/Photos --by content --show only-a
|
|
~~~
|
|
|
|
Delete from a folder what a comparison names, here the files the backup
|
|
already holds. The job reads both copies in full before it removes one:
|
|
|
|
~~~bash
|
|
sd-cli file delete ~/Photos --against /Volumes/Backup/Photos --by content --show both
|
|
~~~
|
|
|
|
Copy, move, merge and delete all show their findings and their plan before
|
|
asking. Validation says whether the operation may run and how, the plan says
|
|
what would exist afterward, and the job applies the same decisions leaf by
|
|
leaf against the live filesystem. `--dry-run` stops after the plan:
|
|
|
|
~~~bash
|
|
sd-cli file copy ~/Photos/2024 --destination /Volumes/Backup/Photos --dry-run
|
|
sd-cli file copy ~/Photos/2024 --destination /Volumes/Backup/Photos --move-files
|
|
sd-cli file merge /Volumes/Expansion/Photos --into /Volumes/NAS/Photos --dry-run
|
|
sd-cli file merge /Volumes/Expansion/Photos --into /Volumes/NAS/Photos --on-conflict keep-both --consume
|
|
~~~
|
|
|
|
A delete's plan says which of the files are the last copy of their content
|
|
anywhere in the library, before anything is removed. Dedupe keeps one copy of
|
|
each duplicated content and removes the rest, or the copies you name, after
|
|
reading each pair in full:
|
|
|
|
~~~bash
|
|
sd-cli file dedupe ~/Photos --dry-run
|
|
sd-cli file dedupe --keep ~/Photos/2024/IMG_0041.JPG
|
|
sd-cli file dedupe ~/Downloads --keep-under /Volumes/NAS/Photos
|
|
~~~
|
|
|
|
Rename one path, or many by rules applied to each name in order. Validation
|
|
resolves every new name against the directory and the volume's filesystem, so
|
|
a name NTFS refuses, a name already there, or two files wanting one name is a
|
|
finding before anything moves, and the plan lists each old name beside its
|
|
new one:
|
|
|
|
~~~bash
|
|
sd-cli file rename ~/Photos/2024/IMG_0041.JPG --to "beach.jpg"
|
|
sd-cli file rename ~/Photos/2024/*.JPG --sequence "Trip_{n:03}" --lower-extension --dry-run
|
|
sd-cli file rename ~/Photos/2024/*.JPG --template "{captured:%Y-%m-%d} {name}{ext}"
|
|
~~~
|
|
|
|
Every job that changes the filesystem writes a journal of what it did, and
|
|
undo reverses it from the journal, newest effect first, after showing the
|
|
plan and refusing whatever changed since. A merge with `--remove-extras` is
|
|
a mirror: the destination ends up matching the sources, with the files it
|
|
alone held moved to the trash. The trash lists what Spacedrive put there
|
|
and puts it back through undo:
|
|
|
|
~~~bash
|
|
sd-cli job journal 4f2c…
|
|
sd-cli file undo 4f2c… --dry-run
|
|
sd-cli file merge ~/Projects --into /Volumes/NAS/Projects --on-conflict keep-newer --remove-extras
|
|
sd-cli file trash list
|
|
sd-cli file trash restore 4f2c… 3
|
|
sd-cli file trash empty
|
|
~~~
|
|
|
|
Organize a folder into subfolders by date, kind or extension, or flatten
|
|
one; write an archive or extract one; set attributes; make a link. Each
|
|
shows its plan first:
|
|
|
|
~~~bash
|
|
sd-cli file organize ~/Downloads --by kind --dry-run
|
|
sd-cli file organize ~/Photos/2024 --by date --field captured --granularity year-month --recursive
|
|
sd-cli file flatten ~/Downloads/unpacked --on-conflict keep-both
|
|
sd-cli file archive ~/Projects/site --to ~/Archive/site.tar.zst --remove-sources
|
|
sd-cli file extract ~/Downloads/kit.zip --to ~/Downloads/kit --strip-components 1
|
|
sd-cli file attributes ~/Scripts/*.sh --mode 755
|
|
sd-cli file link ~/Desktop/Photos --target ~/Pictures/Photos
|
|
~~~
|
|
|
|
## Structured output
|
|
|
|
Pass **--format json** before the command to receive structured output:
|
|
|
|
~~~bash
|
|
sd-cli --format json sources list
|
|
sd-cli --format json job list
|
|
~~~
|
|
|
|
Scripts should prefer JSON and stable IDs over parsing human tables.
|
|
|
|
## Call any registered operation
|
|
|
|
The generic passthrough exposes the operation registry:
|
|
|
|
~~~bash
|
|
sd-cli op sources.list --json '{}'
|
|
sd-cli op tags.create --library <LIBRARY_ID> --json '{"name":"Keep"}'
|
|
~~~
|
|
|
|
Operation names and schemas are the contract. The CLI does not translate an
|
|
arbitrary model response into shell commands. Skills and MCP tools use this same
|
|
bounded surface.
|
|
|
|
## Multiple local instances
|
|
|
|
Use a named instance to run isolated daemon state on one machine:
|
|
|
|
~~~bash
|
|
sd-cli --instance lab start
|
|
sd-cli --instance lab status
|
|
~~~
|
|
|
|
An instance changes the data directory and socket identity. It does not create a
|
|
new security boundary from the operating-system account that runs it.
|
|
|
|
## Data directory
|
|
|
|
Override the default state directory explicitly:
|
|
|
|
~~~bash
|
|
sd-cli --data-dir /srv/spacedrive-data start
|
|
~~~
|
|
|
|
Use the same value for every command that addresses that daemon. Back up this
|
|
directory because it contains library state, source stores, assertions, and job
|
|
history that a filesystem scan may not recover.
|
|
|
|
## Discover commands
|
|
|
|
~~~bash
|
|
sd-cli --help
|
|
sd-cli sources --help
|
|
sd-cli network --help
|
|
sd-cli op --help
|
|
~~~
|
|
|
|
The installed help is authoritative for command spelling. See [Get
|
|
Started](/overview/get-started) for the first library flow and [API](/core/api)
|
|
for the operation contract behind the CLI.
|