* ci: install rustfmt and clippy components on every runner
Self-hosted runners have rustc preinstalled, so setup-rust skipped the
dtolnay/rust-toolchain step that added the components and the Rust
Formatting job died with 'cargo-fmt is not installed'. Add the components
to rust-toolchain.toml and run rustup component add unconditionally.
* setup: install FFmpeg development libraries on Linux and macOS
crates/ffmpeg is a default workspace member that links libav* through
ffmpeg-sys-next, so a fresh machine fails the first cargo build with
'libavutil.pc not found'. Install the libav dev packages and pkg-config
from scripts/setup.sh for every supported package manager, warn from
xtask setup when pkg-config cannot find them, and document the requirement
in the README build section.
* test: call the trash directly in delete_strategy_test
bcc7337 gave LocalDeleteStrategy::move_to_trash a JobContext parameter so
it can record where the item went; the integration test still passed one
argument and stopped every cargo test target from compiling. The two trash
cases now call ops::files::trash::trash with the same arguments the
strategy forwards, which is what they were asserting on.
* interface: clear the 30 Tauri typecheck errors
Remove the 23 unused imports and locals, apply the rename rules' serde
defaults (regex and whole_name false, start and step 1) where the modal
renders optional fields, drop the size prop ToggleGroup never accepted,
and type SourceCard on ts-client's SourceInfo so a filesystem source with
no adapter renders its data type instead. Register the /redundancy/duplicates
route the dashboard already navigates to, which is what the DuplicatesView
import was for.
* clippy: make cargo clippy --workspace -D warnings pass on Linux
sd-store names its sqlx row tuples with type aliases. sd-ffmpeg converts
with try_from and f32::from where the cast was lossless, takes raw
borrows for FFI out-pointers, resamples from an integer position, and
allows the pointer-alignment and s32-to-f32 precision casts with the
reason. sd-mobile-core sets DaemonRequest.device (None, local execution)
on the two envelopes it builds. file-opening-macos and file-opening-windows
pull their platform bindings only on that target and gate their sources,
matching sd-desktop-macos. The Tauri build script drops externalBin from
the config when no daemon sidecar exists, so a check or clippy run that
never built the daemon still passes, and the desktop binary's macOS-only
commands return on other hosts without unreachable code. Workspace
rust-version moves to 1.95, the floor gpui already imposes, so the MSRV
lint stops flagging APIs the pinned 1.97.1 toolchain uses.
* docs: make the stale doctests compile
sd-media-metadata examples still imported from the old image module and
asserted a plus code that belonged to the other example's coordinates (the
value is verified against the reference Open Location Code encoder);
MediaLocation has no Display, so the parse example compares coordinates.
The sd-task-system example unwraps dispatch and matches the (TaskId,
TaskOutput) pair Done now carries. The spacedrive-sdk sketch names types
that do not exist yet and the job-derive expansion only resolves inside
sd-core, so both are marked ignore with the reason beside them.
* build: pin bun to 1.3.4 everywhere
package.json said bun@1.3.0 while CI installed 1.3.4, and any newer bun
rewrites bun.lockb on install, so the lockfile churned on every machine.
Pin packageManager to the CI version, document it, and install with
--frozen-lockfile in the TypeScript job so a drifting lockfile fails
instead of being silently rewritten.
* status: register the CI green on sources work and the libav requirement
* docs: make the sd-core doctests compile too
Five sd-core examples referenced the crate by its old name, used an arrow
glyph in a rust fence, or called methods without the trait in scope; the
copy strategy sketch needs a running JobContext so it is marked ignore.
* ci: stop shrinking the runner's root disk before installing a toolchain
easimon/maximize-build-space moves the root filesystem's free space into an
LVM volume mounted at the workspace and leaves root with the reserve, which
on the Blacksmith runner came out at 100 MB after the swap file. rustup
installs the pinned 1.97.1 toolchain under /home/runner/.rustup on root, so
both Rust jobs died with 'No space left on device'. The runner has 82 GB
free on root, which is plenty for a clippy target directory, so the step
goes.
* ci: run clippy on the Ubuntu 24.04 Blacksmith runner; let sd-server check without bun
The Blacksmith migration wrote the runner label into the matrix's target
field and left runs-on at GitHub's ubuntu-22.04, whose FFmpeg 4.4 predates
the libav 6 API sd-ffmpeg uses (ch_layout, av_dict_iterate), so the job
could never compile sd-ffmpeg. It now runs on blacksmith-4vcpu-ubuntu-2404
with FFmpeg 6.1, the same as a developer machine on 24.04, and the target
field holds a target triple again.
sd-server's build script skipped the web build when bun was absent and
RustEmbed then failed on the missing apps/web/dist. A debug build now gets
an empty dist with a warning so check and clippy pass; a release build
without bun and without a prebuilt dist still refuses, so a server never
ships without its UI by accident.
* build: let the desktop crate compile without a built frontend
tauri::generate_context! panics when frontendDist is missing, which is how
every check or clippy run looks before bun run build. The build script now
creates an empty apps/tauri/dist for debug builds with a warning; a release
build without one still refuses.
* status: record the CI result for the sources green-up
Spacebot ships separately. This drops its UI, the voice overlay and the
Alt+Space shortcut that opened it, the spacebot app config and window, the
@spacebot/api-client and @spacedrive/ai wiring, and the Spacebot and
Console design docs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The first client to connect starts automatic discovery: it tracks the
system volume, adds the home folder as a source and walks every attached
drive. For a library meant to hold only what you add, like one whose data
directory lives on the drive it catalogs, that indexes the Mac into it.
sd start --no-default-sources, and the same flag on restart and on
sd-daemon, now leaves those three out. The pass still restores snapshots,
heals coverage for sources already tracked and hashes them, so a source
you added keeps working across restarts. sd-server keeps the defaults.
From a parallel session. thumbs.generate accepts a file or an indexed
folder with missing, stale, and force modes, runs in resumable batches,
and versions video tiles against host FFmpeg availability so a tile
regenerated with better tooling replaces its predecessor in place. File
and folder context menus and the new sd thumbs CLI domain call the same
operation. paths.context now reports whether the exact path has a
committed source record, and the path bar surfaces it with a refresh
affordance. Coverage and resume semantics are documented in
docs/core/thumbnail-generation.mdx.
A machine-scoped registry discovers host FFmpeg and companion tools,
surfaced in Services settings. The thumbnail service uses them for
video posters and on-demand scrub sheets: thumbstrips bake per request
into the volume hot tier and serve over the local HTTP servers with
immutable content-versioned URLs, wired to the timeline scrubber.
Deleting locations.triggerJob cut the last live dispatcher for every media
job. What was left was six jobs, six processors, six registered actions and
an 853-line SidecarManager, all keyed on entry rows nothing writes, reachable
from nothing. The UI buttons that fired them were the only thing keeping the
registrations honest, and they were firing into a pipeline that had no input.
The hot tier is unaffected. ThumbService over sources/<id>/thumbs.pvcache is
the live thumbnail path, it is source-scoped already, and it shares no code
with any of this.
Two things survive because they outlive their writers. SidecarKind, variant,
format and status move to domain::sidecar: they are the vocabulary a derived
artifact is addressed by, they are what SdPath::Sidecar parses, and
content_sidecar will be keyed on the same three. And File.sidecars keeps its
shape, empty until something fills it, so the subtitle and thumbstrip lookups
in the client keep compiling.
The EXIF and ffmpeg extraction went with the rest. It read into
image_media_data and video_media_data, which are entity rows on their way
out, and its field mapping is a straight match for the facets declared in
df965b851. When the facet writer gets built it wants rewriting against those
columns rather than resurrecting; git show df965b851~1:core/src/ops/media/
metadata_extractor.rs has the EXIF handling worth copying.
sd-server's sidecar route now builds its path through sd-sidecar-path, the
same way the tauri one already did. Both still serve from <library>/sidecars,
both are equally empty, and L5 re-roots them together.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sidebar lists indexed drives including detached ones, volume rows and overview
cards carry persisted entry counts, and the explorer gains a list view over
daemon ops with event-driven refresh.
- Add SHARE-000..011 task specs and docs/design/shares.md defining
the sd.app public sharing architecture and wire protocols
- Rework INDEX-010 to scope ephemeral UUID reconciliation per-library
instead of a single global map, since core can have multiple
libraries loaded at once
- Add sidecar serving and configurable bind host to apps/server, and
wire the web platform's daemon status to the server's own origin
- Add quick preview button to the file inspector, with a null-safe
useOptionalExplorer for use outside the explorer context
* feat: add redundancy awareness & cross-volume file comparison
Surface content redundancy data so users can answer "if this drive dies,
what do I lose?" — builds on existing content identity and volume systems.
Backend:
- New `redundancy.summary` library query with per-volume at-risk vs
redundant byte/file counts and a library-wide replication score
- Extend `SearchFilters` with `at_risk`, `on_volumes`, `not_on_volumes`,
`min_volume_count`, `max_volume_count` filters
- Add composite index migration on entries(content_id, volume_id)
Frontend:
- `/redundancy` dashboard with replication score, volume bars, at-risk callout
- `/redundancy/at-risk` paginated file list sorted by size
- `/redundancy/compare` two-volume comparison (unique/shared toggle)
- Sidebar ShieldCheck button linking to redundancy view
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* redundancy UI improvements + ZFS volume detection fix
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix ZFS pool capacity reporting and stats filtering
- ZFS: override total_capacity for pool-root volumes using zfs list
used+available. df under-reports pool-root Size because it only
counts the root dataset's own used bytes plus avail — on a 60 TB
raidz2 pool this shows as ~15 TB instead of ~62 TB. The pool root's
own used property includes descendants, so used+available is the
real usable capacity.
- Library stats: drop volumes where is_user_visible=false AND
re-apply should_hide_by_mount_path retroactively so stale DB rows
(detected before the Linux visibility filters existed) don't
inflate reported capacity.
- Extract should_hide_by_mount_path into volume/utils as a shared
helper used by both the list query and the stats calculation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* show capacity and visibility in sd volume list
Makes it possible to verify library-level capacity aggregation from
the CLI — previously the list only showed mount, fingerprint, and
tracked/mounted state, which meant debugging the ZFS pool capacity
issue required querying the library DB directly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* document filesystem support matrix and detection
New docs/core/filesystems.mdx covering per-filesystem capabilities
(CoW, pool-awareness, visibility filtering, capacity correction),
platform detection strategies, the FilesystemHandler trait, Linux/
macOS/ZFS visibility rules, the ZFS pool-root capacity problem and
fix, copy strategy selection, and known limitations.
Registered under File Management in both mint.json and docs.json.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* WIP: redundancy filter wiring across search, CLI, and UI
- core/src/ops/search: wire redundancy filters (at_risk, on_volumes,
not_on_volumes, min/max volume_count) through the search query;
fix UUID-to-SQLite BLOB literal so volume UUID comparisons actually
match (volumes.uuid is stored as a 16-byte BLOB, quoted-string
comparison silently returned zero rows).
- apps/cli: new redundancy subcommand + populate the new
SearchFilters fields from search args.
- packages/interface: redundancy at-risk and compare pages reworked
to consume the new filter surface; explorer context/hook updates
to support redundancy-scoped views.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* add web context menu renderer and UI polish
- New WebContextMenuProvider + Radix DropdownMenu-based renderer anchored
at cursor via a 1x1 virtual trigger. Handles separators, submenus,
disabled, and the danger variant via text-status-error.
- useContextMenu now routes web clicks through the provider instead of
parking data in unused local state, and trims leading/trailing/adjacent
separators so condition-filtered menus don't render orphaned lines.
- Drop app-frame corner rounding on the web build.
- Add shrink-0 to the sidebar space switcher so the scrollable sibling
can't compress it vertically.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* sd-server dev workflow: auto web build, shutdown watchdog, stable data dir
- build.rs runs `bun run build` in apps/web so `just dev-server` always
embeds the latest UI. rerun-if-changed covers apps/web/src,
packages/interface/src, and packages/ts-client/src so Rust-only edits
skip the rebuild. Skips gracefully when bun isn't on PATH or
SD_SKIP_WEB_BUILD is set; Dockerfile sets the latter since dist is
pre-built and bun isn't in the Rust stage.
- Graceful shutdown was hanging because the browser holds the /events
SSE stream open forever and axum waits for all connections to drain.
After the first signal, arm a background force-exit on second Ctrl+C
or 5s timeout so the process can't stick.
- Debug builds were starting from a fresh tempfile::tempdir() on every
run (the TempDir handle dropped at end of the closure, deleting the
dir we just took a path to). Default to ~/.spacedrive in debug so data
persists and `just dev-server` shares a data dir with the Tauri app.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* add Sources space item alongside Redundancy in default library layout
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* add TrueNAS native build script
Uses zig cc as C/C++ compiler on TrueNAS Scale where /usr is
read-only and no system gcc exists. Dev tools live at
/mnt/pool/dev-tools/ (zig, cmake, make, extracted deb headers).
Builds sd-server + sd-cli in ~4 min on a 12-core NAS. AI feature
disabled (whisper.cpp C11 atomics incompatible with zig clang-18).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* never block RPC on synchronous statistics calculation
On first load (fresh library, all stats zero), libraries.info used
to calculate statistics synchronously before responding. On large
libraries during active indexing this hangs indefinitely — the
closure-table walk in calculate_file_statistics loads every
descendant ID into a Vec then issues a WHERE IN(...) with millions
of entries, which SQLite can't finish while the indexer is writing.
Now always return cached (possibly zero) stats and let the
background recalculate_statistics task fill them in. The UI
refreshes via the ResourceChanged event when the calculation
completes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* self-heal protocol handler registration on re-init
Core::new() registers default protocol handlers after starting networking,
but swallows any failure (error is only logged). If the initial registration
fails — e.g. on a host where start_networking hasn't fully set up the event
loop command sender by the time register_default_protocol_handlers runs —
the registry is left empty. A subsequent call to Core::init_networking()
would see `services.networking().is_some()` and skip re-registration,
permanently leaving protocols unregistered for the life of the process.
sd-server calls init_networking() right after Core::new(), so it's the
client most exposed to this. Symptom: pairing over the web UI returns
"Pairing protocol not registered" while the same library works fine
from Tauri and mobile.
Fix: init_networking now queries the registry directly for the pairing
handler and re-registers the default set if it's missing, independent of
whether networking is already initialized.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fall back to pkarr+DNS discovery when mDNS port is unavailable
Iroh's endpoint.bind() fails wholesale if any configured discovery service
fails to initialize. MdnsDiscovery requires binding UDP :5353, which on
most Linux systems (including TrueNAS) is already owned by avahi-daemon.
Result: endpoint creation errors out with "Service 'mdns' error", the
event loop never starts, command_sender stays None, and protocol
registration fails — so sd-server has no working networking at all.
Make mDNS best-effort: on any error whose message mentions "mdns",
retry endpoint creation with only pkarr + DNS discovery. Local-network
auto-discovery is lost but remote pairing via node ID (which uses n0's
DNS infrastructure, not mDNS) continues to work normally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* succeed pairing if either mDNS or relay discovery wins
The dual-path discovery in start_pairing_as_joiner_with_code used
tokio::select! to race mDNS and relay. select! resolves on the first
branch to complete — including errors — so a host that can't bind
mDNS (e.g. a Linux box where avahi already owns UDP :5353) would fail
pairing wholesale: mDNS discovery errors out in <1ms with
"Failed to create mDNS discovery: Service 'mdns' error", that Err
wins the race, and relay discovery gets cancelled before it can even
begin.
Switch to futures::select_ok so we only return the error if EVERY
discovery path has failed. mDNS failing immediately now leaves relay
running to completion, which is the common case for remote pairing
into a NAS.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- cargo fmt across all modified files
- Add Vite client types and Window.__SPACEDRIVE__ declaration
- Fix @sd/interface/platform import to @sd/interface
- Align @types/react versions between tauri and interface packages
- Remove unused imports/vars in useDropZone, DragOverlay, ContextMenuWindow
- Fix WebviewWindow.location references to use globalThis
- Exclude updater.example.ts from typecheck
- Changed versioning from "2.0.0-pre.1" to "2.0.0-alpha.1" in various Cargo.toml files and package.json to reflect a new alpha release.
- Added support for ARM64 builds in the GitHub Actions workflow, including a new host configuration and installation of VA-API libraries for the ARM target.
- Introduced a new `bump` command in the xtask to facilitate version updates across all relevant files, improving version management.
Introduces a proper feature flag hierarchy for AI-related dependencies,
allowing lite builds without heavy speech-to-text dependencies.
Feature hierarchy:
- `ai` - umbrella feature for all AI capabilities
- `speech-to-text` - transcription capability (requires ffmpeg + whisper)
- `whisper` - Whisper speech recognition engine (internal)
- `ffmpeg` - audio/video processing (existing)
Changes:
- Make whisper-rs, hound, rubato optional deps behind `whisper` feature
- Add `speech-to-text` feature that requires both ffmpeg and whisper
- Add `ai` umbrella feature that enables all AI features
- Update all cfg guards to use semantic `#[cfg(feature = "speech-to-text")]`
- Expose new features in apps/server and apps/cli
- Update release workflow to include `ai` feature (maintains existing behavior)
This enables:
- Mobile builds without STT (smaller binary, already using default-features=false)
- Desktop "lite" builds for users who want minimal size
- Future AI features can be added under the `ai` umbrella
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit removes the outdated Dockerfile, docker-compose.yml, and related documentation for the Spacedrive server, consolidating the deployment process into a single Dockerfile located in the apps/server directory. The new setup supports multi-architecture builds and includes enhanced media processing capabilities. Additionally, a self-hosting guide is introduced to assist users in deploying the server on their infrastructure, ensuring a more efficient and user-friendly experience.
This commit updates the VSCode settings to include new configurations for newline handling and formatting. In the Cargo.toml file for the server, the default features have been cleared to streamline dependencies. Additionally, the TagsGroup and TagSelector components have been modified to handle both TagSearchResult and raw Tag objects, improving tag extraction and selection logic. These changes enhance the overall development experience and ensure better tag management in the application.
- Changed the CI workflow to run tests on self-hosted runners for better performance and flexibility.
- Updated the test execution condition to trigger on push events and specific pull requests.
- Consolidated test execution to run all tests in the workspace instead of a single test.
- Modified the default features in Cargo.toml for the server and core applications to include "heif" and "ffmpeg" for enhanced media support.
- Added metrics collection capabilities to the SyncProtocolHandler for improved monitoring of sync operations.
- Refactored sync tests to streamline setup and improve clarity in test scenarios.
- Simplified the timeout handling in `is_daemon_running` for better readability.
- Updated type definitions in the macOS Tauri library for clarity.
- Reformatted JSON configuration for better structure and readability.
- Changed function signatures in `files.rs` and `server.rs` to use `Path` instead of `PathBuf` for consistency.
- Enhanced error handling and argument passing in various functions for improved clarity.
- Added `#[allow(dead_code)]` annotations to unused functions in several modules to suppress warnings.
- Improved the display string methods in `volume.rs` and `pairing/types.rs` for better performance and clarity.
- Added a step to set up native dependencies using `cargo xtask setup` in both CI and release workflows.
- Updated the build commands for CLI binaries to include `heif` and `ffmpeg` features, ensuring the binaries are built with the necessary capabilities.
- Modified `Cargo.toml` files for `sd-cli`, `sd-server`, and `tauri` to define `heif` and `ffmpeg` features separately, improving modularity and clarity in dependency management.
- Add show_paired flag to device listing (CLI arg and input) - Extend
LibraryDeviceInfo with is_paired and is_connected; include Paired
devices in library listing when requested - Add Devices group to UI
(DevicesGroup) and hook into SpaceGroup - Extend device queries/types to
support show_paired and paired devices - Refactor Dockerfile to
multi-stage Bun + Rust builds; reuse web assets - Remove obsolete
core/ops/entries/mod.rs
- Introduce an Axum-based HTTP server with an embedded daemon and a
JSON-RPC proxy to the daemon via a Unix socket - Bundle web UI assets
into the server with an assets feature and a build.rs that builds the
frontend using pnpm - Add multi-stage Dockerfile, docker-compose.yml,
and a Distroless runtime image - Provide TrueNAS deployment support with
a build script and setup guide - Add a new web UI (apps/web) with a
Vite-based dev/build flow and a web platform shim for the frontend -
Implement server logic (apps/server/src/main.rs): health, auth, /rpc
proxy and data-dir/socket-path wiring - Include server-specific
Cargo.toml and a comprehensive server README - Add architecture and
memory-focused docs to guide usage and design - Minor core tweak:
simplify location/resource event emission in
core/src/location/manager.rs to align with new flow - Tauri app: adjust
menus to add an Edit submenu and remove unused items
* Update rspc, prisma-client-rust, axum and tanstack-query
- Deleted some unused examples and fully commented out frontend code
- Implement many changes required due to the updates
- Update most rust dependencies
* Re-enable p2p
* Fix server
* Auto format
* Fix injected script format
- Update some github actions
- Update pnpm lock file
* Fix devtools showing up when app opens
- Fix million complaining about Sparkles component
* Fix sd-server
* Fix and improve thumbnails rendering
- Fix core always saying a new thumbnail was generated even for files that it skiped thumbnail generation
- Rewrite FileThumb and improve related components
* Ignore tmp files when running prettier
* Improve FileThumb component performance
- Rework useExplorerDraggable and useExplorerItemData hooks due to reduce unecessary re-renders
* More fixes for thumb component
- A couple of minor performance improvements to frontend code
* auto format
* Fix Thumbnail and QuickPreview
* Fix logic for when to show 'fail to load original' error message in QuickPreview
- Updated prisma-client-rust, libp2p, tauri, tauri-specta, rspc and hyper
* Fix type checking
- Format scripts
* Add script prettier config
* Fix serde missing feature
- Use rust-libp2p spacedrive fork again
- Update rspc
* Autoformat + fix pnpm lock
* Fix thumbnail first load again
* Autoformat
* autoformat
* Fix rust-libp2p fork url again?
* Remove usePathsInfiniteQuery hook
* Update tauri 2.0.6
* Remove deps-generator app
- It is broken for quite a while and not being used
* Remove unused code and unused deps
* auto format
* Restore incorrectly removed files
- Remove aws deps and test code using it
* Replace winapi-util with windows-rs
- Adjust some deps versions
* Autoformat
* Update rand, remove unused dep
* Fix image crate due to changes in pdf-renderer
- Dont use default features in reqwest
- Use native-tls tauri
* Remove incorrect notice about blake3
* Migrate to Tauri v2 Release Canidate
* Formatting
* Update pnpm
* Update all tauri deps (js and rust) to Release Candidate 2
- A lot of misc fixes due to changes in the new versions of tauri, rspc and specta
* Recreate pnpm-lock
* Fix pnpm-lock
* Formatting
* Use 10 chars SHA for git deps
- Update a couple of rust dependencies
- Add taplo for toml auto formatting
- Adjust some formatting configurations
- Updated prettier and its plugins
* Fix formatter settings
* Minor improvements to Cargo.toml format
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
