mirror of
https://github.com/spacedriveapp/spacedrive.git
synced 2026-10-31 05:20:17 -04:00
Jamie Pine
fbfb4ebd3e
A partition was keyed by source id and routed by longest prefix, so a source registered inside another forked the index rather than narrowing it: paths under the inner root resolved to a second partition, the outer one kept an unreachable copy of them, and each ledger minted its own uuid for the same file. A tag applied through one root was invisible through the other. MediumSlot is keyed by MediumKey now: the volume where one is tracked, the outermost registered root otherwise. The medium is asked of the registry rather than remembered on the record, so registering an outer source later moves the inner one onto its medium instead of stranding it. A network share and a source inside it share one map for the same reason a drive and a folder on it do. Stores move off the slot onto the cache, keyed by source id, since one medium now hosts several. store_for resolves the innermost source, which is what makes a nested source persist its own subtree without the outer one writing it twice. The snapshot and thumbs.pvcache follow the arena into mediums/<id>/, leaving sources/<id>/ for the durable store and the block cache. Both wanted the same directory for the same drive's map otherwise. A source now reports what its store holds rather than what the partition around it does, since the arena maps the whole drive. A store holding nothing has not been written yet, so the partition's figure stands until it has one of its own, which keeps a source over an already-mapped drive from showing empty. a_nested_source_shares_the_drive_it_sits_on asserts one arena and one snapshot across two registrations, and two stores. Reverting the medium lookup fails it on the first assertion.
Spacedrive Core
Rust library implementing the Virtual Distributed File System (VDFS) architecture for local-first, AI-native file management.
Architecture
Structure
src/
├── domain/ # Core data models (Entry, Library, Device)
├── ops/ # Operations (actions and queries, CQRS pattern)
├── infra/ # Infrastructure (database, events, wire protocol)
├── service/ # High-level services (network, jobs, sessions)
├── location/ # Location management and indexing
├── library/ # Library lifecycle and operations
├── device/ # Device identity and management
├── volume/ # Volume detection and fingerprinting
├── config/ # Application configuration
├── crypto/ # Cryptographic primitives
└── bin/ # Binaries (cli, daemon)
Core Components
Core Manager (Core struct in lib.rs)
- Coordinates all subsystems
- Manages application lifecycle
- Provides unified access to capabilities
Library (library/)
- File-based storage (
.sdlibrarydirectories) - SQLite database with SeaORM
- Job management and thumbnail generation
- Device registration and sync coordination
Entry-Centric Model (domain/entry.rs)
- Unified representation for files and directories
- Conditional UUIDs (directories immediate, files after content ID)
- On-demand UserMetadata creation
- Relative paths from location roots
Device Management (device/)
- Single device identity per installation
- Persistent across restarts (
device.json) - Sync leadership model per library
- Network address tracking for P2P
Operations Layer (ops/)
- CQRS pattern: Actions (mutations) and Queries (reads)
- Wire protocol with automatic type generation
- Registry system using
inventorycrate - Shared across daemon RPC, iOS FFI, and WASM extensions
Infrastructure (infra/)
api/: Wire protocol dispatcher and RPC serverevent/: Event bus for state changesaction/: Transactional action system with preview-commit-verifyquery/: Read-optimized query handlers
Networking (service/network/)
- Iroh-based P2P networking
- Device pairing protocol
- File transfer (Spacedrop)
- mDNS local discovery
Jobs (service/jobs/)
- Durable, resumable operations
- MessagePack serialization for state
- Progress reporting and cancellation
- Per-library job managers
Indexing (location/indexer/)
- Five-phase pipeline (discover, classify, extract, thumbnail, cleanup)
- File system watcher integration
- Rules engine with glob patterns
- Resumable with checkpoints
Volume Management (volume/)
- Cross-platform volume detection
- Fingerprinting for identity
- Mount point tracking
Communication Patterns
Daemon-Client (Desktop/CLI)
- Unix socket JSON-RPC 2.0
- Wire method strings (e.g.,
query:vdfs.list_entries) - Operations auto-register at compile time
Embedded FFI (iOS/Mobile)
- Direct Rust library integration
- Same JSON-RPC protocol over FFI
- Swift client with Specta-generated types
Extensions (WASM)
- Sandboxed WASM modules
- Minimal host functions (log, register_job)
- SDK with procedural macros (
crates/sdk/) - Models, jobs, actions, agents, UI manifests
Key Technologies
- Async runtime: tokio
- Database: SQLite via SeaORM
- Serialization: serde, Specta for type generation
- Networking: Iroh (P2P), mdns-sd (local discovery)
- WASM: wasmer runtime, spacedrive-sdk
- Jobs: inventory for registration, rmp-serde for state
- Crypto: blake3 (content addressing), ed25519 (signing)
- Indexing: notify (fs watcher), globset (rules)
Building
# Full build
cargo build --release
# With optional features
cargo build --features ffmpeg,ai,heif
# Specific binary
cargo build --bin spacedrive
cargo build --bin daemon
# Run CLI
cargo run --bin spacedrive -- --help
Binaries
spacedrive: CLI interfacedaemon: Background daemon process
Development
- Uses CQRS and DDD patterns
- Operations register via
inventorycrate macros - Resumable jobs with MessagePack state serialization
- Type-safe Wire protocol with Specta generation
- Event-driven architecture with EventBus
- No layered architecture (direct Rust patterns)
Testing
# All tests
cargo test
# Specific module
cargo test --lib location::indexer
# Integration tests
cargo test --test indexer_test
See /docs/core/ for detailed architecture documentation.