spacedrive/docs/core/library-sync.mdx

---
title: Library Sync
sidebarTitle: Library Sync
---

Spacedrive synchronizes library metadata across all your devices using a leaderless peer-to-peer model. Every device is equal. No central server, no single point of failure.

## How Sync Works

Sync uses two protocols based on data ownership:

**Device-owned data** (locations, files): The owning device broadcasts changes in real-time and responds to pull requests for historical data. No conflicts possible since only the owner can modify.

**Shared resources** (tags, collections): Any device can modify. Changes are ordered using Hybrid Logical Clocks (HLC) to ensure consistency across all devices.

<Info>
	Library Sync handles metadata synchronization. For file content
	synchronization between storage locations, see [File
	Sync](/docs/core/file-sync).
</Info>

## Quick Reference

| Data Type     | Ownership    | Sync Method     | Conflict Resolution |
| ------------- | ------------ | --------------- | ------------------- |
| Locations     | Device-owned | State broadcast | None needed         |
| Files/Folders | Device-owned | State broadcast | None needed         |
| Tags          | Shared       | HLC-ordered log | Union merge         |
| Collections   | Shared       | HLC-ordered log | Union merge         |
| User Metadata | Mixed        | Varies by scope | Context-dependent   |

## Data Ownership

Spacedrive recognizes that some data naturally belongs to specific devices.

### Device-Owned Data

Only the device with physical access can modify:

- **Locations**: Filesystem paths like `/Users/alice/Photos`
- **Entries**: Files and folders within those locations
- **Volumes**: Physical drives and mount points

### Shared Resources

Any device can create or modify:

- **Tags**: Labels applied to files
- **Collections**: Groups of files
- **User Metadata**: Notes, ratings, custom fields
- **Extension Data**: Custom models from extensions

This ownership model eliminates most conflicts and simplifies synchronization.

## Sync Protocols

### State-Based Sync (Device-Owned)

<Info>
	This sync protocol is tested in `test_sync_location_device_owned_state_based`
	at core/tests/sync_integration_test.rs:647
</Info>

State-based sync uses two mechanisms depending on the scenario:

**Real-time broadcast**: When Device A creates or modifies a location, it sends a `StateChange` message via unidirectional stream to all connected peers. Peers apply the update immediately.

**Pull-based backfill**: When Device B is new or reconnecting after being offline, it sends a `StateRequest` to Device A. Device A responds with a `StateResponse` containing records in configurable batches. This request/response pattern uses bidirectional streams.

For large datasets, pagination automatically handles multiple batches using cursor-based checkpoints. The `StateRequest` includes both watermark and cursor:

```rust
StateRequest {
    model_types: ["location", "entry"],
    since: Some(last_state_watermark),  // Only records newer than this
    checkpoint: Some("2025-10-21T19:10:00.456Z|uuid"),  // Resume cursor
    batch_size: config.batching.backfill_batch_size,
}
```

No version tracking needed. The owner's state is always authoritative.

### Log-Based Sync (Shared Resources)

<Info>
	This sync protocol is tested in `test_sync_tag_shared_hlc_based` at
	core/tests/sync_integration_test.rs:830
</Info>

Log-based sync uses two mechanisms depending on the scenario:

**Single item sync**: When you create a tag:
```
1. Device A inserts tag in database
2. Device A generates HLC timestamp
3. Device A appends to sync log
4. Device A broadcasts SharedChange message
5. Other devices apply in HLC order
6. After acknowledgment, prune from log
```

**Batch sync**: When creating many items (e.g., 1000 tags during bulk import):
```
1. Device A inserts all tags in database
2. Device A generates HLC for each and appends to sync log
3. Device A broadcasts single SharedChangeBatch message
4. Other devices apply all entries in HLC order
5. After acknowledgment, prune from log
```

The log ensures all devices apply changes in the same order. Batch operations provide 99x reduction in network traffic.

For large datasets, the system uses HLC-based pagination. Each batch request includes the last seen HLC, and the peer responds with the next batch. This scales to millions of shared resources.

## Hybrid Logical Clocks

<Info>
	HLC conflict resolution is tested in `test_concurrent_tag_updates_hlc_conflict_resolution`
	at core/tests/sync_integration_test.rs:1972
</Info>

HLCs provide global ordering without synchronized clocks:

```rust
HLC {
    timestamp: physical_time_ms,  // Physical time (milliseconds)
    counter: 0,                   // Logical counter
    device_id: "device-a-uuid"    // Tie-breaker
}
```

Properties:

- Events maintain causal ordering
- Any two HLCs can be compared
- No clock synchronization required

### Conflict Resolution

When two devices concurrently modify the same record, the change with the higher HLC wins (Last Write Wins):

```
Device A creates tag "Version A" with HLC(timestamp_a, 0, device-a)
Device B creates tag "Version B" with HLC(timestamp_b, 0, device-b)

After sync, both devices converge to "Version B" (higher HLC)
```

The sync system checks the peer log before applying changes to ensure only newer updates are applied.

## Database Architecture

### Main Database (database.db)

Contains all library data from all devices:

```sql
-- Device-owned tables
CREATE TABLE locations (
    id INTEGER PRIMARY KEY,
    uuid TEXT UNIQUE,
    device_id INTEGER,  -- Owner
    path TEXT,
    name TEXT
);

CREATE TABLE entries (
    id INTEGER PRIMARY KEY,
    uuid TEXT UNIQUE,
    location_id INTEGER,  -- Inherits ownership
    name TEXT,
    kind INTEGER,
    size_bytes INTEGER
);

-- Shared resource tables
CREATE TABLE tags (
    id INTEGER PRIMARY KEY,
    uuid TEXT UNIQUE,
    canonical_name TEXT
    -- No device_id (anyone can modify)
);
```

### Sync Database (sync.db)

Contains only pending changes for shared resources:

```sql
CREATE TABLE shared_changes (
    hlc TEXT PRIMARY KEY,
    model_type TEXT,
    record_uuid TEXT,
    change_type TEXT,  -- insert/update/delete
    data TEXT         -- JSON payload
);

CREATE TABLE peer_acks (
    peer_device_id TEXT PRIMARY KEY,
    last_acked_hlc TEXT
);
```

<Note>
	The sync database stays small (under 1MB) due to aggressive pruning after
	acknowledgments.
</Note>

## Using the Sync API

The sync API handles all complexity internally:

```rust
// Sync a single item (device-owned)
let location = create_location("/Photos");
library.sync_model_with_db(&location, ChangeType::Insert, db).await?;

// Sync a single item (shared)
let tag = create_tag("Vacation");
library.sync_model(&tag, ChangeType::Insert).await?;

// Batch sync - much more efficient for bulk operations
// Works for both device-owned (entries) and shared (tags, content_identities)
library.sync_models_batch(&content_identities, ChangeType::Insert, db).await?;
```

The API automatically:

- Detects ownership type (device-owned vs shared)
- Manages HLC timestamps for shared resources
- Converts between local IDs and UUIDs for foreign keys
- Batches network broadcasts (single message for many items)
- Manages the sync log and pruning

## Implementing Syncable Models

To make a model syncable:

```rust
impl Syncable for YourModel {
    const SYNC_MODEL: &'static str = "your_model";

    fn sync_id(&self) -> Uuid {
        self.uuid
    }

    fn sync_depends_on() -> &'static [&'static str] {
        &["parent_model"]  // Models that must sync first
    }

    fn foreign_key_mappings() -> Vec<FKMapping> {
        vec![
            FKMapping::new("device_id", "devices"),
            FKMapping::new("parent_id", "your_models"),
        ]
    }
}
```

### Dependency Resolution Algorithm

To prevent foreign key violations, the sync system must process models in a specific order (e.g., `Device` records must exist before the `Location` records that depend on them). Spacedrive determines this order automatically at startup using a deterministic algorithm.

The process works as follows:

1.  **Dependency Declaration**: Each syncable model declares its parent models using the `sync_depends_on()` function. This creates a dependency graph where an edge from `Location` to `Device` means `Location` depends on `Device`.

2.  **Topological Sort**: The `SyncRegistry` takes the full list of models and their dependencies and performs a **topological sort** using Kahn's algorithm. This algorithm produces a linear ordering of the models where every parent model comes before its children. It also detects impossible sync scenarios by reporting any circular dependencies (e.g., A depends on B, and B depends on A).

3.  **Ordered Execution**: The `BackfillManager` receives this ordered list (e.g., `["device", "tag", "location", "entry"]`) and uses it to sync data in the correct sequence, guaranteeing that no foreign key violations can occur.

### Dependency Management

The sync system respects model dependencies and enforces ordering:

```
Sync Order During Backfill:
1. Shared resources (tags, collections, content_identities)
2. Devices
3. Locations (needs devices)
4. Volumes (needs devices)
5. Entries (needs locations and content_identities)
```

Shared resources sync first because entries reference content identities via foreign key. This prevents NULL foreign key references during backfill.

### Foreign Key Translation

The sync system must ensure that relationships between models are preserved across devices. Since each device uses local, auto-incrementing integer IDs for performance, these IDs cannot be used for cross-device references.

This is where foreign key translation comes in, a process orchestrated by the `foreign_key_mappings()` function on the `Syncable` trait.

**The Process:**

1.  **Outgoing**: When a record is being prepared for sync, the system uses the `foreign_key_mappings()` definition to find all integer foreign key fields (e.g., `parent_id: 42`). It looks up the corresponding UUID for each of these IDs in the local database and sends the UUIDs over the network (e.g., `parent_uuid: "abc-123..."`).

2.  **Incoming**: When a device receives a record, it does the reverse. It uses `foreign_key_mappings()` to identify the incoming UUID foreign keys, looks up the corresponding local integer ID for each UUID, and replaces them before inserting the record into its own database (e.g., `parent_uuid: "abc-123..."` → `parent_id: 15`).

This entire translation process is automatic and transparent.

<Info>
	**Separation of Concerns:** - `sync_depends_on()`: Determines the **order** of
	model synchronization at a high level. - `foreign_key_mappings()`: Handles the
	**translation** of specific foreign key fields within a model during the
	actual data transfer.
</Info>

## Sync Flows

<Info>
	The complete sync infrastructure is validated in `test_sync_infrastructure_summary`
	at core/tests/sync_integration_test.rs:1093
</Info>

### Creating a Location

<Info>
	Location sync with entries is tested in `test_sync_entry_with_location` at
	core/tests/sync_integration_test.rs:939
</Info>

<Steps>
<Step title="Device A Creates Location">
User adds `/Users/alice/Documents`:
- Insert into local database
- Call `library.sync_model(&location)`
- Send `StateChange` message to connected peers via unidirectional stream
</Step>

<Step title="Device B Receives Update">
	Receives `StateChange` message:
	- Map device UUID to local ID
	- Insert location (read-only view)
	- Update UI instantly
</Step>

<Step title="Complete">
Total time: ~100ms
No conflicts possible
</Step>
</Steps>

### Creating a Tag

<Steps>
<Step title="Device A Creates Tag">
User creates "Important" tag:
- Insert into local database
- Generate HLC timestamp
- Append to sync log
- Broadcast to peers
</Step>

<Step title="Device B Applies Change">
	Receives tag creation: - Update local HLC - Apply change in order - Send
	acknowledgment
</Step>

<Step title="Log Cleanup">
After all acknowledgments:
- Remove from sync log
- Log stays small
</Step>
</Steps>

### New Device Joins

<Steps>
<Step title="Pull Shared Resources First">
New device sends `SharedChangeRequest`:
- Peer responds with recent changes from sync log
- If log was pruned, includes current state snapshot
- For larger datasets, paginate using HLC cursors
- Apply tags, collections, content identities in HLC order
- Shared resources sync first to satisfy foreign key dependencies (entries reference content identities)
</Step>

<Step title="Pull Device-Owned Data">
New device sends `StateRequest` to each peer:
- Request locations, entries, volumes owned by peer
- Peer responds with `StateResponse` containing records in batches
- For large datasets, automatically paginates using `timestamp|uuid` cursors
- Apply in dependency order (devices, then locations, then entries)
</Step>

<Step title="Catch Up and Go Live">
Process any changes that occurred during backfill from the buffer queue.
Transition to Ready state.
Begin receiving real-time broadcasts.
</Step>
</Steps>

## Advanced Features

### Transitive Sync

<Info>
	Transitive sync is tested in `test_sync_transitive_three_devices`
	at core/tests/sync_integration_test.rs:1304
</Info>

Spacedrive does not require a direct connection between all devices to keep them in sync. Changes can propagate transitively through intermediaries, ensuring the entire library eventually reaches a consistent state.

This is made possible by two core architectural principles:

1.  **Complete State Replication**: Every device maintains a full and independent copy of the entire library's shared state (like tags, collections, etc.). When Device A syncs a new tag to Device B, that tag becomes a permanent part of Device B's database, not just a temporary message.

2.  **State-Based Backfill**: When a new or offline device (Device C) connects to any peer in the library (Device B), it initiates a backfill process. As part of this process, Device C requests the complete current state of all shared resources from Device B.

**How it Works in Practice:**

<Steps>
	<Step title="1. Device A syncs to B">
		Device A creates a new tag. It connects to Device B and syncs the tag. The
		tag is now stored in the database on both A and B. Device A then goes
		offline.
	</Step>
	<Step title="2. Device C connects to B">
		Device C comes online and connects only to Device B. It has never
		communicated with Device A.
	</Step>
	<Step title="3. Device C Backfills from B">
		Device C requests the complete state of all shared resources from Device B.
		Since Device B has a full copy of the library state (including the tag from
		Device A), it sends that tag to Device C.
	</Step>
	<Step title="4. Library is Consistent">
		Device C now has the tag created by Device A, even though they never
		connected directly. The change has propagated transitively.
	</Step>
</Steps>

This architecture provides significant redundancy and resilience, as the library can stay in sync as long as there is any path of connectivity between peers.

### Delete Handling

<Info>
	Delete operations are tested in `test_sync_deletion_with_cascading_tombstones`
	at core/tests/sync_integration_test.rs
</Info>

**Device-owned deletions** use tombstones that sync via `StateResponse`. When you delete a location or folder with thousands of files, only the root UUID is tombstoned. Receiving devices cascade the deletion through their local tree automatically.

**Shared resource deletions** use HLC-ordered log entries with `ChangeType::Delete`. All devices process deletions in the same order for consistency.

**Pruning:** Both deletion mechanisms use acknowledgment-based pruning. Tombstones and peer log entries are removed after all devices have synced past them. A 7-day safety limit prevents offline devices from blocking pruning indefinitely.

The system tracks deletions in a `device_state_tombstones` table. Each tombstone contains just the root UUID of what was deleted. When syncing entries for a device, the `StateResponse` includes both updated records and a list of deleted UUIDs since your last sync.

```rust
StateResponse {
    records: [...],           // New and updated entries
    deleted_uuids: [uuid1],   // Root UUID only (cascade handles children)
}
```

Receiving devices look up each deleted UUID and call the same deletion logic used locally. For entries, this triggers `delete_subtree()` which removes all descendants via the `entry_closure` table. A folder with thousands of files requires only one tombstone and one network message.

**Race condition protection:** Models check tombstones before applying state changes during backfill. If a deletion arrives before the record itself, the system skips creating it. For entries, the system also checks if the parent is tombstoned to prevent orphaned children.

### Pre-Sync Data

<Info>
	Pre-sync data backfill is tested in
	`test_sync_backfill_includes_pre_sync_data` at
	core/tests/sync_integration_test.rs:1142
</Info>

Data created before enabling sync is included during backfill. When the peer log has been pruned or contains fewer items than expected, the response includes a current state snapshot:

```rust
SharedChangeResponse {
    entries: [...],              // Recent changes from peer log
    current_state: {
        tags: [...],             // Complete snapshot
        content_identities: [...],
        collections: [...],
    },
    has_more: bool,              // True if snapshot exceeds batch limit
}
```

The receiving device applies both the incremental changes and the current state snapshot, ensuring all shared resources sync correctly even if created before sync was enabled.

### Watermark-Based Incremental Sync

<Info>
	Watermark-based reconnection sync is tested in `test_watermark_reconnection_sync`
	at core/tests/sync_integration_test.rs:1744
</Info>

When devices reconnect after being offline, they use watermarks to avoid full re-sync.

**State Watermark**: Timestamp of the last device-owned state update received (stored in `last_state_watermark`).

**Shared Watermark**: HLC of the last shared resource change seen (stored in `last_shared_watermark`).

The sync service automatically monitors connectivity. When devices reconnect or fall behind, the system checks watermarks at configured intervals. If a device has fallen behind, incremental catch-up begins automatically.

During catch-up, the device sends a `StateRequest` with the `since` parameter set to its watermark. The peer responds with only records modified after that timestamp. This is a **pull request**, not a broadcast.

Example flow when Device B reconnects:

```
1. Device B checks watermark: last_state_watermark = 2025-10-20 14:30:00
2. Device B sends StateRequest(since: 2025-10-20 14:30:00) to Device A
3. Device A queries: SELECT * FROM entries WHERE updated_at >= '2025-10-20 14:30:00'
4. Device A responds with StateResponse containing 3 new entries
5. Device B applies changes and updates watermark
```

This typically syncs a small number of changed records instead of re-syncing the entire dataset, completing in milliseconds.

### Pagination for Large Datasets

<Info>
	Pagination ensures backfill works reliably for libraries with millions of records.
</Info>

Both device-owned and shared resources use cursor-based pagination for large datasets. Batch size is configurable via `SyncConfig`.

**Device-owned pagination** uses a `timestamp|uuid` cursor format:

```
checkpoint: "2025-10-21T19:10:00.456Z|abc-123-uuid"
```

Query logic handles identical timestamps from batch inserts:

```sql
WHERE (updated_at > cursor_timestamp)
   OR (updated_at = cursor_timestamp AND uuid > cursor_uuid)
ORDER BY updated_at, uuid
LIMIT {configured_batch_size}
```

**Shared resource pagination** uses HLC cursors:

```rust
SharedChangeRequest {
    since_hlc: Some(last_hlc),  // Resume from this HLC
    limit: config.batching.backfill_batch_size,
}
```

The peer log query returns the next batch starting after the provided HLC, maintaining total ordering.

Both pagination strategies ensure all records are fetched exactly once, no records are skipped even with identical timestamps, and backfill is resumable from checkpoint if interrupted.

### Connection State Tracking

<Info>
	Connection state tracking is tested in `test_connection_state_tracking`
	at core/tests/sync_integration_test.rs:1562
</Info>

The sync system uses the Iroh networking layer as the source of truth for device connectivity. When checking if a peer is online, the system queries Iroh's active connections directly rather than relying on cached state.

A background monitor updates the devices table at configured intervals for UI purposes:

```sql
UPDATE devices SET
    is_online = true,
    last_seen_at = NOW()
WHERE uuid = 'peer-device-id';
```

All sync decisions use real-time Iroh connectivity checks, ensuring messages only send to reachable peers.

### Derived Tables

Some data is computed locally and never syncs:

- **directory_paths**: A lookup table for the full paths of directories.
- **entry_closure**: Parent-child relationships
- **tag_closure**: Tag hierarchies

These rebuild automatically from synced base data.

## Portable Volumes & Ownership Changes

A key feature of Spacedrive is the ability to move external drives between devices without losing track of the data. This is handled through a special sync process that allows the "ownership" of a `Location` to change.

### Changing Device Ownership

When you move a volume from one device to another, the `Location` associated with that volume must be assigned a new owner. This process is designed to be extremely efficient, avoiding the need for costly re-indexing or bulk data updates.

It is handled using a **Hybrid Ownership Sync** model:

<Steps>
	<Step title="Ownership Change is Requested">
		When a device detects a known volume that it does not own, it broadcasts a
		special `RequestLocationOwnership` event. Unlike normal device-owned data,
		this event is sent to the HLC-ordered log, treating it like a shared
		resource update.
	</Step>
	<Step title="Peers Process the Change">
		Every device in the library processes this event in the same, deterministic
		order. Upon processing, each peer performs a single, atomic update on its
		local database: `UPDATE locations SET device_id = 'new_owner_id' WHERE uuid
		= 'location_uuid'`
	</Step>
	<Step title="Ownership is Transferred Instantly">
		This single-row update is all that is required. Because an `Entry`'s
		ownership is inherited from its parent `Location` at runtime, this change
		instantly transfers ownership of millions of files. No bulk updates are
		needed on the `entries` or `directory_paths` tables. The new owner then
		takes over state-based sync for that `Location`.
	</Step>
</Steps>

### Handling Mount Point Changes

A simpler scenario is when a volume's mount point changes on the same device (e.g., from `D:\` to `E:\` on Windows).

1.  **Location Update**: The owning device updates the `path` field on its `Location` record.
2.  **Path Table Migration**: This change requires a bulk update on the `directory_paths` table to replace the old path prefix with the new one (e.g., `REPLACE(path, 'D:\', 'E:\')`).
3.  **No Entry Update**: Crucially, the main `entries` table, which is the largest, is completely untouched. This makes the operation much faster than a full re-index.

## Performance

### Sync Characteristics

| Aspect    | Device-Owned  | Shared Resources  |
| --------- | ------------- | ----------------- |
| Latency   | ~100ms        | ~150ms            |
| Storage   | No log        | Less than 1MB log |
| Conflicts | Impossible    | HLC-resolved      |
| Offline   | Full function | Queues changes    |

### Optimizations

**Batching**: The sync system batches both device-owned and shared resource operations. Batch sizes are configurable via `SyncConfig`.

Device-owned data syncs in batches during file indexing. One `StateBatch` message replaces many individual `StateChange` messages, providing significant performance improvement.

Shared resources send batch messages instead of individual changes. For example, linking thousands of files to content identities during indexing sends a small number of network messages instead of one per file, providing substantial reduction in network traffic.

Both batch types still write individual entries to the sync log for proper HLC ordering and conflict resolution. The optimization is purely in network broadcast efficiency.

**Pruning**: The sync log automatically removes entries after all peers acknowledge receipt, keeping the sync database under 1MB.

**Compression**: Network messages use compression to reduce bandwidth usage.

**Caching**: Backfill responses cache for 15 minutes to improve performance when multiple devices join simultaneously.

## Troubleshooting

### Changes Not Syncing

Check:

1. Devices are paired and online
2. Both devices joined the library
3. Network connectivity between devices
4. Sync service is running

Debug commands:

```bash
# Check pending changes
sqlite3 sync.db "SELECT COUNT(*) FROM shared_changes"

# Verify peer connections
sd sync status

# Monitor sync activity
RUST_LOG=sd_core::sync=debug cargo run
```

### Common Issues

**Large sync.db**: Peers not acknowledging. Check network connectivity.

**Missing data**: Verify dependency order. Parents must sync before children.

**Conflicts**: Check HLC implementation maintains ordering.

## Implementation Status

<Info>
	All 10 sync integration tests passing as of October 2025.
	See `core/tests/sync_integration_test.rs` for full test suite.
</Info>

### Production Ready

- One-line sync API
- HLC implementation (thread-safe)
- Syncable trait infrastructure
- Foreign key mapping
- Dependency ordering
- Network transport (Iroh/QUIC)
- Backfill orchestration
- State snapshots
- HLC conflict resolution
- Watermark-based incremental sync
- Connection state tracking
- Transitive sync
- Cascading tombstones for device-owned deletions
- Unified acknowledgment-based pruning

### Currently Syncing

**Device-Owned Models:**
- **Device** - Device records and metadata
- **Location** - Filesystem paths and mount points
- **Entry** - Files and folders within locations
- **Volume** - Physical drives and volumes

**Shared Models:**
- **Tag** - User-created labels
- **Collection** - File groupings
- **ContentIdentity** - Content-based file identification
- **UserMetadata** - User notes, ratings, and custom fields
- **CollectionEntry** - Many-to-many collection relationships
- **UserMetadataTag** - Many-to-many tag relationships
- **TagRelationship** - Tag hierarchy relationships

All models sync automatically during creation, updates, and deletions. File indexing uses batch sync for both device-owned entries (`StateBatch`) and shared content identities (`SharedChangeBatch`), providing 99x reduction in network overhead.

**Deletion sync:** Device-owned models (locations, entries, volumes) use cascading tombstones for efficient deletion propagation. The `device_state_tombstones` table tracks root UUIDs of deleted trees. Shared models use standard `ChangeType::Delete` in the peer log. Both mechanisms prune automatically once all devices have synced.

## Extension Sync

<Note>
	Extension sync framework is ready. SDK integration pending.
</Note>

Extensions can define syncable models using the same infrastructure as core models. The registry pattern automatically handles new model types without code changes to the sync system.

Extensions will declare models with sync metadata:

```rust
#[model(
    table_name = "album",
    sync_strategy = "shared"
)]
struct Album {
    #[primary_key]
    id: Uuid,
    title: String,
    #[metadata]
    metadata_id: i32,
}
```

The sync system will detect and register extension models at runtime, applying the same HLC-based conflict resolution and dependency ordering used for core models.

## Configuration

Sync behavior is controlled through a unified configuration system. All timing, batching, and retention parameters are configurable per library.

### Default Configuration

The system uses sensible defaults tuned for typical usage across LAN and internet connections:

```rust
SyncConfig {
    batching: {
        backfill_batch_size: 10_000,
        state_broadcast_batch_size: 1_000,
        shared_broadcast_batch_size: 100,
    },
    retention: {
        strategy: AcknowledgmentBased,
        tombstone_max_retention_days: 7,
        force_full_sync_threshold_days: 25,
    },
    network: {
        message_timeout_secs: 30,
        backfill_request_timeout_secs: 60,
        sync_loop_interval_secs: 5,
    },
}
```

**Batching** controls how many records are processed at once. Larger batches improve throughput but increase memory usage.

**Retention** controls how long sync coordination data is kept. The acknowledgment-based strategy prunes tombstones and peer log entries as soon as all devices have synced past them. A 7-day safety limit prevents offline devices from blocking pruning indefinitely.

**Network** controls timeouts and polling intervals. Shorter intervals provide faster sync but increase network traffic and CPU usage.

### Presets

**Aggressive** is optimized for fast local networks with always-online devices. Small batches and frequent pruning minimize storage and latency.

**Conservative** handles unreliable networks and frequently offline devices. Large batches improve efficiency, and extended retention accommodates longer offline periods.

**Mobile** optimizes for battery life and bandwidth. Less frequent sync checks and longer retention reduce power consumption.

### Configuring Sync

```bash
# Use a preset
sd sync config set --preset aggressive

# Customize individual settings
sd sync config set --batch-size 5000 --retention-days 14

# Per-library configuration
sd library "Photos" sync config set --preset mobile
```

Configuration can also be set via environment variables or a TOML file. The loading priority is: environment variables, config file, database, then defaults.

## Summary

The sync system combines state-based and log-based protocols to provide reliable peer-to-peer synchronization:

**State-based sync** for device-owned data eliminates conflicts by enforcing single ownership. Changes propagate via real-time broadcasts (`StateChange` messages) to connected peers. Historical data transfers via pull requests (`StateRequest`/`StateResponse`) when devices join or reconnect.

**Log-based sync** for shared resources uses Hybrid Logical Clocks to maintain causal ordering without clock synchronization. All devices converge to the same state regardless of network topology.

**Automatic recovery** handles offline periods through watermark-based incremental sync. Reconnecting devices send pull requests with watermarks, receiving only changes since their last sync. This typically transfers a small number of changed records instead of re-syncing the entire dataset, completing in milliseconds.

The system is production-ready with all core models syncing automatically. Extensions can use the same infrastructure to sync custom models.

## Related Documentation

- [Devices](/docs/core/devices) - Device pairing and management
- [Networking](/docs/core/networking) - Network transport layer
- [Libraries](/docs/core/libraries) - Library structure and management