mirror of
https://github.com/spacedriveapp/spacedrive.git
synced 2026-10-30 21:09:14 -04:00
Jamie Pineandcapy-ai-staging[bot]
f9bdf724b6
* Networking: serve streams on connections a protocol dials
The connection cache is shared between the dial side and the accept side,
so once a protocol dials a peer, the peer's own send path finds that
connection and opens its streams on it. Nothing read those streams: only
connections the event loop accepted, or those the reconnect loop handed
over, had a stream handler. The peer's request then sat unanswered until
its 30 s timeout while requests in the other direction answered in
milliseconds. This is the reverse RegisterDeviceRequest timeout SPAC-12
saw after join-remote.
get_or_create_connection now hands every connection it dials to the
event loop as TrackOutboundConnection, which spawns the same stream
handler an inbound connection gets. Messaging, sync, and job activity
pass their event loop handle; pairing keeps its own connection handling.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Sync: the device that joins a library is the one that backfills
Every sync service started Uninitialized and backfilled from the first
connected partner, whatever state it already held. When the Mac Studio
joined the MacBook's library the MacBook was the one with a partner in
its device table, so it went Uninitialized -> Backfilling{Studio} and
applied the empty copy's snapshot over its own rows (LWW on the
deterministic default-space UUID). The Studio never backfilled because
the reverse registration had timed out and its device table held only
itself.
A copy of a library created from another device's library is the only
thing that may start Uninitialized. create_library_with_id now writes an
initial_backfill_pending marker into sync.db; PeerSync reads it on open
and starts Ready otherwise. A Ready device never requests a snapshot
(SharedChangeRequest.include_snapshot = false on catch-up) and repeated
catch-up failures no longer escalate to a full backfill. A copy still
waiting for its own first backfill refuses to serve as a join target.
The join carries the rule explicitly: RegisterDeviceRequest has
needs_initial_state, and RegisterDeviceResponse returns the responder's
own device record so the joiner's table is complete when the command
returns, replacing the spawned reverse request. sync_setup fails
instead of reporting devices_registered: true when that record is
missing.
Catch-up completion is recorded per peer in sync.db even when no rows
arrived; the Ready loop catches up once with a peer it has no record
for and no longer re-polls every peer whose watermark is 60 s old. A
missing watermark means never synced, not ancient, so the forced full
sync warning only fires for a real watermark past retention.
library_join_test pairs two daemons for real, joins Bob to Alice's
library, and asserts Alice's rows are unchanged, Bob holds Alice's
device row and spaces, and Alice runs no further catch-ups.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Sync: test that catch-up with an empty peer runs once
Alice holds a volume, Bob holds nothing of his own, so Alice never
writes a per-resource watermark for Bob. Before the per-peer catch-up
record, that made Bob a new peer on every loop iteration and Alice
caught up with him forever (2 -> 5 sessions in 16 s on 5e285d3). The
test asserts the session count stops moving and the record exists.
Both new suites join the xtask integration list.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Library info: fleet totals count the same devices as device_count
add_device_summaries added every cached peer summary to a library's
statistics, whatever library the peer belonged to, and the summaries
carried no source count. A joined device therefore reported the member's
3 million files next to Sources 0 and Devices 1.
Summaries now carry the device's own source_count, and only summaries
of devices in the library's device table are added, so files, content,
sources, and capacity describe the members that device_count counts.
The CLI labels the block "Statistics (all library members)".
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Docs: describe the pairing and library join flow as it runs
pairing.mdx and devices.mdx described a confirmation prompt that does
not exist and pointed at `sd devices list` for pairing status, which
reads the library's device table and does not show a peer until it
joins the library. The CLI overview lacked the global `--device` flag,
and the sync setup walkthrough never named `library sync-setup
--action join-remote` or `--node-id`. All four now describe the flow
SPAC-12 ran, and library-sync.mdx gains sections on joining, catch-up,
and fleet totals. One PROJECT_STATUS register row.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Sync: catch up once per connection session, never serve a snapshot while pending
Review of #3110 found two gaps. The per-peer catch-up record was never
cleared, so a peer that changed rows while disconnected was never caught
up with again after reconnecting (the live broadcast only lives in the
in-memory retry queue). The record is now cleared when the sync service
starts and when the peer's connection is lost; ConnectionEstablished is
not used because the event loop emits it per accepted stream.
The SharedChangeRequest handler attached a full snapshot without
checking the first-backfill marker, so two copies that both joined from
the same member could seed each other. It now refuses the snapshot while
pending, and the register handler reads the marker from sync.db directly
when the library's sync service is not up yet.
Also: a failed backfill or catch-up left active_backfill_sessions
incremented; the mock transport no longer lists a peer as a partner
before its sync service is registered; the catch-up test now starts
Alice Ready (marker cleared) and Bob pending, the shape of a real join,
and asserts exactly one catch-up and one backfill.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
* Sync: move past a peer that refuses a snapshot, fail the guard closed, refresh stale catch-ups
A joiner whose chosen peer refused the snapshot kept choosing it: every
partner was scored has_complete_state: true and RTT is only recorded on
success, so the order never changed. A peer whose backfill failed now
scores without complete state until a backfill succeeds, so a connected
member holding the library is tried next.
The snapshot guard read an unreadable marker as "holds state"; it now
refuses, since a wrong refusal only delays a join.
A broadcast whose retries ran out while the QUIC session stayed up had
no recovery under the once-per-session rule. The Ready loop refreshes a
catch-up record older than ten minutes; with per-peer watermarks an
empty refresh is two small requests.
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
---------
Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
173 lines
4.9 KiB
Plaintext
173 lines
4.9 KiB
Plaintext
---
|
|
title: Pair and Connect Libraries
|
|
description: Establish device trust and attach remote library state
|
|
sidebarTitle: Library Sync Setup
|
|
---
|
|
|
|
Pair devices first, then choose which library relationship to establish. Pairing
|
|
creates device trust. Library setup creates scoped shared state.
|
|
|
|
## 1. Check networking
|
|
|
|
Run on both devices:
|
|
|
|
~~~bash
|
|
sd-cli network status
|
|
~~~
|
|
|
|
Each daemon should report a node ID and active networking.
|
|
|
|
## 2. Pair the devices
|
|
|
|
On the device that will show the code:
|
|
|
|
~~~bash
|
|
sd-cli network pair generate
|
|
~~~
|
|
|
|
On the joining device:
|
|
|
|
~~~bash
|
|
sd-cli network pair join "<PAIRING_CODE>"
|
|
~~~
|
|
|
|
The joiner prints "Paired with <name>" and both daemons record trust; there is
|
|
no confirmation prompt. The code expires after five minutes.
|
|
|
|
If the two devices are not on one local network, take the initiator's node ID
|
|
from `sd-cli --format json network pair generate` (field `node_id`) and pass it
|
|
to the joiner:
|
|
|
|
~~~bash
|
|
sd-cli network pair join "<PAIRING_CODE>" --node-id <NODE_ID>
|
|
~~~
|
|
|
|
Check the result on either side:
|
|
|
|
~~~bash
|
|
sd-cli network devices
|
|
sd-cli network status
|
|
~~~
|
|
|
|
Pairing only records network trust. `sd-cli devices list` reads the current
|
|
library's device table and does not show the new peer yet.
|
|
|
|
## 3. Inspect libraries
|
|
|
|
List local libraries:
|
|
|
|
~~~bash
|
|
sd-cli library list
|
|
~~~
|
|
|
|
Discover libraries exposed by the paired device:
|
|
|
|
~~~bash
|
|
sd-cli library sync-setup discover <REMOTE_DEVICE_ID>
|
|
~~~
|
|
|
|
Discovery follows the remote device's current grants. An undisclosed library is
|
|
not evidence that it does not exist.
|
|
|
|
## 4. Join the remote library
|
|
|
|
On the device that should become a member, join the library the other device
|
|
already holds:
|
|
|
|
~~~bash
|
|
sd-cli library sync-setup setup \
|
|
--local-library <ANY_LOCAL_LIBRARY_ID> \
|
|
--remote-device <REMOTE_DEVICE_ID> \
|
|
--remote-library <REMOTE_LIBRARY_ID> \
|
|
--action join-remote \
|
|
--name "<LIBRARY_NAME>"
|
|
~~~
|
|
|
|
`--local-library` names any library already open on this device; the command
|
|
needs one to address the daemon and does not change it. The join:
|
|
|
|
1. creates a new local library with the remote library's UUID (the folder name
|
|
is de-duplicated, for example `My Library 1.sdlibrary`);
|
|
2. registers this device in the remote device's copy of the library and
|
|
receives the remote device's record in return, so both device tables are
|
|
complete when the command returns;
|
|
3. marks the new library as waiting for its first backfill, so sync pulls the
|
|
existing member's state into it. The existing member keeps its data and
|
|
never adopts the empty joiner's snapshot.
|
|
|
|
The other direction, `--action share-local`, runs on the device that holds the
|
|
library and creates the copy on the peer. `--interactive` walks through the same
|
|
choices.
|
|
|
|
Then select the joined library on this device:
|
|
|
|
~~~bash
|
|
sd-cli library switch --library-id <REMOTE_LIBRARY_ID>
|
|
~~~
|
|
|
|
Use stable library and device UUIDs. Do not parse display names as identity.
|
|
|
|
## 5. Confirm state
|
|
|
|
~~~bash
|
|
sd-cli devices list
|
|
sd-cli library info
|
|
sd-cli network devices --connected
|
|
sd-cli sources list
|
|
sd-cli job list
|
|
~~~
|
|
|
|
`devices list` on both devices now shows both members. `library info` reports
|
|
fleet totals: file, content, source, and capacity figures are the sum of every
|
|
library member's own numbers, and the device count is the number of members.
|
|
A joined device that has registered no sources of its own therefore reports the
|
|
same totals as the device that holds the data.
|
|
|
|
Library sync can complete before every remote source is online. Source
|
|
availability and source-generation replication have their own state; a peer's
|
|
source replicas start arriving when the devices connect, independent of the
|
|
join.
|
|
|
|
## What setup does not do
|
|
|
|
Connecting libraries does not:
|
|
|
|
- copy every source generation into the library database;
|
|
- download all file bytes;
|
|
- expose ungranted local paths;
|
|
- turn a remote browse into a backup;
|
|
- make two devices share unrestricted mutation rights.
|
|
|
|
Remote source serving, source-store subscription, caching, and file sync are
|
|
separate choices.
|
|
|
|
## Troubleshooting
|
|
|
|
### The peer is not discovered
|
|
|
|
Use the QR payload or node ID for internet pairing. Check that both daemons have
|
|
networking enabled and that the code has not expired.
|
|
|
|
### Pairing works but discovery returns no libraries
|
|
|
|
Review the remote device's library grant. Pairing proves identity but does not
|
|
grant every library.
|
|
|
|
### `devices list` shows one device after pairing
|
|
|
|
Pairing does not add the peer to a library. Run the join in step 4; the peer
|
|
appears in `devices list` on both sides when it returns.
|
|
|
|
### Metadata appears but files do not open
|
|
|
|
The source store may be cached while its origin is offline, or the grant may
|
|
allow metadata but not bytes. Inspect source availability and subtree scope.
|
|
|
|
### A job remains after reconnect
|
|
|
|
Query the job on the device executing it. Durable jobs remain owned by that
|
|
daemon and can outlive the client connection.
|
|
|
|
See [Device Pairing](/core/pairing), [Library Sync](/core/library-sync), and
|
|
[File Sync](/core/file-sync).
|