bcbcd260d4
* Introducing location online/offline checks and location relink * Some initial drafts for location watchers * Location metadata relink and add library * Many improvements at job system Now using prisma batching at identifier job Removing blocking I/O from extension subcrate Implementing lazy generation of thumbnails New current directory identifier job to be used on light rescans * Some optimizations on identifier and object validator jobs * merge jamie's identifier PR * fully repaired identifier job * properly hooked up object kind * inspector fix * fix video badge * small improvements to libraries settings * identifier and inspector improvements * fix feature flags and hook up context menu location utilities * BETTER CONTEXT MENU x100 * test-files * style tweaks * new icon designs * manifest * fix thumbnails on web * media data * New Location Watcher and some minor fixes * disable broken media_data extractor, wip * wip * function name fix * Fixing pnpm prep and some warnings * Solving a race condition beetween indexer job and FS event handlerSome other minor warnings * Generating thumbnails on watcher * Remove event handler on watcher * Some initial works on modify events and other small fixes * File update event * Trying to be more generic with used events and some tests to validate our assumptions * Turning on location metadata file * Introducing core unit tests on CI pipeline * Submiting new unit test assumptions to validate on windows CI * Fixing unit tests * Fixing unit tests again * Fixing unit tests * Fixing unit tests for macos * Fixing unit tests for macos again * New structure for platform dependent event handling Implementing event handlers for Linux and MacOS * minor fixes + rustfmt + clippy * Windows event handling * Introducing a feature gate to only use location watching on desktop app for now * Putting more stuff behind feature gates to avoid warnings * Adding feature to cargo test on CI * Changing some debug logs to trace logs and removing Jamie specific stuff * Make location removal from manager less async * fix build when "location-watcher" feature disabled * fix types + clippy * make location manager non-static * remove uses of `to_string_lossy` * more invalidate_query calls * Expose `library_ctx` directly to avoid needless clones * New materialized_path handling for directories * Removing cascade delete between file_path and object - Some other minor stuff * remove unused `CurrentDirFileIdentifierJob` Co-authored-by: Jamie Pine <ijamespine@me.com> Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
Spacedrive
A file explorer from the future.
spacedrive.com »
Download for
macOS
·
Windows
·
Linux
·
iOS
·
watchOS
·
Android
~ Links will be added once a release is available. ~
NOTE: Spacedrive is under active development, most of the listed features are still experimental and subject to change.
Organize files across many devices in one place. From cloud services to offline hard drives, Spacedrive combines the storage capacity and processing power of your devices into one personal distributed cloud, that is both secure and intuitive to use.
For independent creatives, hoarders and those that want to own their digital footprint, Spacedrive provides a free file management experience like no other.
What is a VDFS?
A VDFS (virtual distributed filesystem) is a filesystem designed to work across a variety of storage layers. With a uniform API to manipulate and access content across many devices, VDFS is not restricted to a single machine. It achieves this by maintaining a virtual index of all storage locations, synchronizing the database between clients in realtime. This implementation also uses CAS (Content-addressable storage) to uniquely identify files, while keeping record of logical file paths relative to the storage locations.
The first implementation of a VDFS can be found in this UC Berkeley paper by Haoyuan Li. This paper describes its use for cloud computing, however the underlying concepts can be translated to open consumer software.
Motivation
Many of us have multiple cloud accounts, drives that aren’t backed up and data at risk of loss. We depend on cloud services like Google Photos and iCloud, but are locked in with limited capacity and almost zero interoperability between services and operating systems. Photo albums shouldn’t be stuck in a device ecosystem, or harvested for advertising data. They should be OS agnostic, permanent and personally owned. Data we create is our legacy, that will long outlive us—open source technology is the only way to ensure we retain absolute control over the data that defines our lives, at unlimited scale.
Roadmap
View a list of our planned features here: spacedrive.com/roadmap
Developer Guide
Please refer to the contributing guide for how to install Spacedrive from sources.
Architecture
This project is using what I'm calling the "PRRTT" stack (Prisma, Rust, React, TypeScript, Tauri).
- Prisma on the front-end? 🤯 Made possible thanks to prisma-client-rust, developed by Brendonovich. Gives us access to the powerful migration CLI in development, along with the Prisma syntax for our schema. The application bundles with the Prisma query engine and codegen for a beautiful Rust API. Our lightweight migration runner is custom built for a desktop app context.
- Tauri allows us to create a pure Rust native OS webview, without the overhead of your average Electron app. This brings the bundle size and average memory usage down dramatically. It also contributes to a more native feel, especially on macOS due to Safari's close integration with the OS.
- We also use rspc which allows us to define functions in Rust and call them on the Typescript frontend in a completely typesafe manner, so no unnecessary bugs make it into production!
- The core (
sdcore) is written in pure Rust.
Monorepo structure:
Apps:
desktop: A Tauri app.mobile: A React Native app.web: A React webapp.landing: A React app using Vite SSR & Vite pages.
Core:
core: The Rust core, referred to internally assdcore. Contains filesystem, database and networking logic. Can be deployed in a variety of host applications.
Packages:
client: A TypeScript client library to handle dataflow via RPC between UI and the Rust core.ui: A React Shared component library.interface: The complete user interface in React (used by appsdesktop,webandlanding)config:eslintconfigurations (includeseslint-config-next,eslint-config-prettierand alltsconfig.jsonconfigs used throughout the monorepo.macos: A Swift Native binary for MacOS system extensions.ios: A Swift Native binary (planned).windows: A C# Native binary (planned).android: A Kotlin Native binary (planned).