- Introduced `remove_emojis.sh`, a Bash script designed to identify and remove 3D/colorful emojis followed by a space in Rust files while preserving simple symbols.
- The script counts the number of Rust files processed and reports modifications made.
- Updated various Rust files to remove emoji characters from log messages and print statements for a cleaner output.
- Enhanced user experience by providing color-coded output for progress and results during the emoji removal process.
- Added support for handling file conflicts during copy operations by introducing an `on_conflict` field in `FileCopyInput`.
- Updated the `run_copy_with_confirmation` function to prompt users for conflict resolution choices (overwrite, rename, abort).
- Enhanced the `FileCopyAction` to apply conflict resolution strategies based on user input.
- Refactored related validation and execution logic to accommodate new conflict handling features.
Co-authored-by: ijamespine <ijamespine@me.com>
- Added `reset_spacedrive_v2_data` function to safely remove specific Spacedrive v2 files and directories while preserving user data.
- Updated `Stop` and `Restart` commands to include a `reset` option, prompting user confirmation before data deletion.
- Enhanced logging to provide feedback on the reset process, including success and error messages.
- Modified documentation to reflect the new `restart` command usage for testing the CLI.
Co-authored-by: ijamespine <ijamespine@me.com>
- Updated `CopyMethod` enum to include `Atomic` and `Streaming` variants, replacing the previous `AtomicMove` and `StreamingCopy` options for clarity.
- Refactored the `select_strategy` method to respect user preferences for copy methods, improving the logic for same-volume operations.
- Added CLI support for the new copy methods in `args.rs`, allowing users to specify their preferred method during file copy operations.
- Updated relevant tests to reflect changes in copy method naming and functionality.
- Enhanced documentation to include new copy method options and their usage.
Co-authored-by: ijamespine <ijamespine@me.com>
- Updated various print statements across the codebase to remove emoji indicators, ensuring a more uniform logging style.
- Adjusted output messages in CLI, job handling, and event processing to enhance clarity and maintainability.
- Refactored job-related logging to provide clearer status updates without emoji clutter.
Co-authored-by: ijamespine <ijamespine@me.com>
- Deleted `ARCHITECTURE.md`, `Cargo.toml.bak`, and various integration summary files to streamline the codebase.
- Removed outdated demo files and testing guides that are no longer relevant.
- Cleaned up the repository by eliminating unnecessary documentation and binary files, ensuring a more maintainable project structure.
Co-authored-by: ijamespine <ijamespine@me.com>
- Introduced `LibraryInfoArgs` struct for querying library information, allowing optional library ID input.
- Implemented `to_query` method for `LibraryInfoArgs` to facilitate query creation.
- Updated `LibraryCmd` enum to include `Info` variant for handling library information requests.
- Refactored argument handling in `args.rs` for improved readability and consistency.
- Enhanced `run` function in `mod.rs` to process the new `Info` command and display detailed library information.
Co-authored-by: ijamespine <ijamespine@me.com>
- Upgraded `bincode` to version `2.0.0-rc.3` with `serde` feature enabled in `Cargo.toml`.
- Refactored error handling to accommodate changes in `bincode` by updating the error conversion implementation in `error.rs`.
- Modified deserialization logic in macros to align with the new `bincode` API.
- Removed outdated metrics test file from the core examples.
Co-authored-by: ijamespine <ijamespine@me.com>
- Added a comprehensive testing guide for the FS Event Pipeline, detailing metrics collection, logging, and troubleshooting steps.
- Updated the CLI package name from "spacedrive-cli" to "sd-cli" in Cargo.toml for consistency.
- Modified various files to reflect the new package structure and improve logging and metrics handling.
Co-authored-by: ijamespine <ijamespine@me.com>
- Added a new end-to-end search test with real data to validate the complete search workflow, including file indexing and search result display.
- Removed the outdated search integration test to streamline testing efforts.
- Introduced the `chrono` dependency in the CLI for enhanced date handling in search operations.
- Updated the search output structure to include matched content and improved pagination handling.
These changes enhance the robustness of the search feature and ensure better integration with real-world data scenarios.
- Improved formatting and consistency in the search command implementation.
- Enhanced the file search query logic by adding total count retrieval for pagination.
- Refactored the path construction and filtering methods for better clarity and maintainability.
- Updated the output handling to ensure proper display of search results and suggestions.
This refactor sets the stage for future enhancements in the search functionality.
This commit introduces the file search functionality, including core logic, CLI integration, and necessary data structures.
Co-authored-by: ijamespine <ijamespine@me.com>
- Refactored the tag command arguments in the CLI to improve clarity and consistency in the codebase.
- Updated the migration script for the tag system to enhance the database schema, including the creation of new tables and indexes.
- Adjusted foreign key constraints and triggers to maintain data integrity and support full-text search capabilities.
This refactor streamlines the tagging system's command structure and database interactions, setting the foundation for future enhancements.
- Updated `bincode` dependency to version 1.3.3 for improved performance and security.
- Refactored error handling in `error.rs` to use `bincode::Error` instead of `bincode::ErrorKind`, ensuring better error management.
- Enhanced macro definitions in `macros.rs` to utilize the updated error handling structure.
- Improved error reporting in `manager.rs` when creating directories, providing clearer context for IO errors.
- Added a new design document for Sync Conduits, detailing the architecture and implementation plan for file synchronization in Spacedrive.
- Refactored argument handling in the job and library domains to improve clarity and consistency.
- Updated imports to use the correct input types for actions in the library and location modules.
- Enhanced type annotations for better readability and maintainability in the codebase.
- Added confirm_or_abort utility for dangerous operations
- Added confirmation prompts to:
- location remove
- library delete (new command)
- network revoke
- Kept our modular CLI structure with domains/ subdirectories
- Removed old flat .rs files that were moved to modular structure
- Deleted several domain modules including file, index, job, library, location, and network, which were no longer needed in the CLI structure.
- Cleaned up the codebase to improve maintainability and reduce complexity by removing obsolete functionality.
- Updated the module structure in `util/mod.rs` to reflect these changes.
- Added a new documentation file outlining the finalization plan for the Spacedrive refactor, detailing remaining tasks and objectives for migrating to a CQRS architecture.
- Implemented session state management to track the current library context across operations, improving the overall architecture's modularity and maintainability.
- Refactored CLI commands and core components to utilize the new session state service, ensuring a more consistent and error-free user experience.
- Enhanced error handling and logging in various modules to facilitate debugging and improve reliability.
- Introduced a new `ARCHITECTURE.md` file detailing the system overview, core architecture, daemon infrastructure, operations system, CLI client, data flow, key design decisions, implementation details, and future improvements.
- Enhanced CLI commands to validate instance names for security and check if the daemon is already running before starting a new instance.
- Updated the daemon's RPC server to handle requests concurrently and improved error handling with structured error types.
- Refactored action and query execution to support async operations more effectively.
- Introduced new methods for executing actions and queries by method string, improving the flexibility of the core API.
- Refactored the `CoreClient` to utilize a `METHOD` constant instead of `TYPE_ID`, enhancing clarity in action dispatching.
- Removed the legacy `ActionHandler` trait and associated registry, streamlining action execution through the unified `LibraryAction` trait.
- Updated the `RpcServer` to handle action and query requests more efficiently, improving error handling and response management.
- Enhanced the `FileCopyAction` and related tests to support modular action construction, ensuring better validation and type safety.
These changes significantly improve the modularity and maintainability of the action system while ensuring a consistent API surface.
- Added a new `apps/cli` module to encapsulate the CLI application logic, improving organization and modularity.
- Updated `Cargo.toml` to include dependencies for the CLI, such as `clap` for command-line argument parsing.
- Implemented a `Context` struct to manage application state and core client interactions.
- Created command handling for file operations and library management, enhancing user experience and functionality.
- Introduced utility functions for output formatting, supporting both human-readable and JSON outputs.
These changes significantly enhance the structure and maintainability of the CLI application while providing a more robust user interface.
- Introduced a new `client` module to enhance the modularity of the codebase.
- Implemented the `Wire` trait for `FileCopyInput` and `ListLibrariesQuery`, providing type IDs for better integration with the system.
- Updated the `lib.rs` and `mod.rs` files to include the new `client` module, ensuring proper organization and accessibility.
These changes improve the structure and maintainability of the codebase while enhancing type safety in data handling.
* Moving file-path-helper to a sub-crate on core
* Parallel walker tested and working
* Change inner core crate names to sd-core-* naming scheme
* Moving stuff around
* Save and Update tasks for the new indexer job
* Some initial drafts on the new job system
* More drafts on job system
* Changing walker task to a more explicit state machine
Also drafting more of job system
* More drafting on job resume
* Completed the draft on job system inner workings
* New job context abstraction to decouple library stuff from job system
* Properly use composition on task dispatcher
* First draft on indexer job
* Job serialization
* Handling ancestors in the distributed walker for indexer
* Saving computed directories sizes on a location to db
* Enable a way to check if jobs are running in a location
* Progress reporting on indexer job
* Reorganizing modules
* Shallow indexer
* Rust fmt
* Attempting windows CI fix
* Attempting windows CI fix again
* Attempting windows CI fix again
* rebase: `crates/crypto` into current `main`
* refactor: remove `mnemonic` module
* feat: disable secure erase temporarily
* fix: tsc
* fix: tsc due to unused import
* fix: remove `cli` crypto info
* deps: update
* chore: remove dead comment
* refactor: remove `bincode` feature
* refactor: give `keyring` a dedicated feature so it's not reliant on `sys` as well
* fix: remove `aes-gcm` as it's no longer supported
* refactor: remove dead comment
* fix: update `keyring` imports
* refactor: change tests to `aes-256-gcm`
* feat: make `Key` a `Box<>` internally to ensure it's heap allocated (and fix tests)
* chore: clippy
* fix: hashing tests now that `const` keys aren't available
this will be cleaned up with test vectors and `include_bytes!()`
* chore: clippy
* refactor: remove dead code
* test: bring back `encrypt_with_invalid_nonce` test
* fix: secret service keyring
* fix: `zbus` build issues
* doc: update comment for clearer reasoning
* fix: cargo fmt
* fix: use bytes directly
* deps: update lockfile
* fix: secret service keyring
* fix: comment out windows keyring for now
* fix: use session keyring if no keyring backend
* fix: completely remove keyring module if no keyring is available for that OS
* fix: clippy
* fix: move iimport to correct conditional compilation
* fix: fmt
* First draft on image labeling
* Fixing execution providers for other OSs
* Better error handling and shutdown
* Working with shallow media processor
* bruh
* Fix warnings
* Now hooked to media processor job
* Link desktop app with libonnxruntime to avoid TLS error during startup
* Be able to change models on runtime
Revert to use labels table instead of tags
* A bug on a model-less inference
* Show AI labels on Inspector
- Change yolo inference to use half precision
- Add labels api to core
* Remove LD_PRELOAD
* Fix race condition on model executor shutdown
* Don't load all images in memory moron
* Embeed yolo model in prod build
- Change yolo model path to new one relative to executable
* Disable volume watcher on linux, it was crashing the app
- Invalidate labels when they are updated
* Rust fmt
* Minor changes
* Gate onnxruntime linking to the ai-models feature
* Add build script to sd-server to handle onnxruntime linking workaround
* Move AI stuff to its own crate and normalize deps
* Rust fmt
* Don't regenerate labels unless asked to
* Now blazingly fast
* Bad merge
* Fix
* Fix
* Add backend logic to download extra yolo models
* Add models api route
- Add api call to get available model version
- Add api call to change the model version
* Improve new model download logic
- Add frontend to change image labeler model
* Fix new model downloader
* Fix model select width
* invalidate labels count after media_processor generates a new output
* Rename AI crate and first draft on download notifications
* fix types
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
* Normalize PATH and XDG envvars on Linux
* Fix normalize_xdg_environment
* Replace custom Desktop file parsers with Glib
* Fix AppImage env influencing external apps
- Normalize GStream plugin path
* Fix macos pulling linux deps
* Attempt to fix gnome apps failing to launch
- Fix incorrect logic in `normalize_pathlist`
* Ensure empty envvars are not set
* Revert AppImage ovewritten GTK specific variables
- `normalize_pathlist` now prefers entries with less priority when dealing with repeated entries, this is not compatible with the default behavior, but it is a more sane approach IMHO
* Remove 32-bit libs from release CI build host
* Remove 32-bit from github runner 2 attempt
- Remove deprecated vscode config
* Remove libc6-i386
* [ENG-916] Implement `Open With` logic compatible with macOS < 12
* Add some missing gstreamer deps in Linux
* Replace opener with Glib API
* Fix reveal opening file instead of dir
- Improve Open With logic in Windows
- Expose functions to test if app is in a flatpak, snap or appimage
---------
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
* Generalizing filesystem jobs to accept multiple files at once
* Some small changes and fixing front
* Enable rename route to replicate Finder's behavior
* Assign tag to multiple objects at once
* Remove from recents accepting multiple files
* Fixing some warnings
* Adding multiple files feature to Open and Open With
* Conditional stuff for macos
* Generating commands.ts and other minor warnings
* Rust fmt
* TS typecheck
* Rust format and TS typecheck
* Requested changes
* Requested changes
---------
Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
* update crypto MSRV
* rename `keychain` to `keyring`
* make a start on the keymanager unlock refactor/keychain integration
* update routes
* update bindings
* add const identifiers
* add UI/front-end support for unlocking KM with OS keychains
* remove SK from lib creation dialog
* update query name
* add keyring functions
* attempt to update `change_master_password()` to use the keychain
* cleanup, fix master password change ui, better secret key in keyring detection
* cleanup TS a little
* add route for getting secret key from keyring
* update bindings
* update var names + show secret key in keys settings
* add `react-qr-code` and option to view the secret key (if it's in the OS keyring)
* allow copying SK to clipboard
* add `key_type` so we're not reliant on specific UUIDs for root/verification key handling
* clippy
* fix mobile typecheck
* fix typecheck, fix typo and tweak balloon hash parameters
* minor cleanup + typo fix
* use newtype structs
* WIP type refactoring (major readability boost!)
* update `use`
* add tokio `sync` feature
* too many structs? idk
* more cleanup
* add `generate` and `Nonce`
* `Nonce` and `Key` typesafety (beautiful)
* clippy + cleanup
* update code & examples
* fix bug & remove `ProtectedVec` as it looked out of place
* use `Key`
* add a query invalidation to make the UI extremely responsive
* ci pls work
* remove `keyringHasSk` route
