- Marked the `leader` argument in `SetupArgs` as deprecated, clarifying its usage.
- Introduced a `backfill_manager` to the `SyncService`, enabling automatic orchestration of initial sync processes.
- Enhanced the `run_sync_loop` method to manage backfill attempts and periodic maintenance tasks, improving overall sync reliability.
- Updated the `MockTransportPeer` to support request/response handling for backfill operations, ensuring seamless data retrieval during synchronization.
- Added `query_for_sync` method to the tag model to retrieve all tags for initial device synchronization, ensuring new devices receive the complete current state.
- Enhanced the sync registry to support shared models with backfill capabilities, allowing for both device-owned and shared model queries.
- Implemented `get_full_shared_state` in the PeerSync service to fetch all shared resources during initial backfill requests.
- Updated sync protocol handler to include the current state snapshot in shared change responses, improving the synchronization experience for devices.
- Refactored the entry model to extract fields from JSON instead of direct deserialization, allowing for better error handling and validation of incoming data.
- Introduced a helper function to streamline field extraction, ensuring all required fields are present before processing.
- Updated the tag model to similarly extract fields from JSON, enhancing its robustness during synchronization.
- Improved the handling of optional fields in both models, ensuring that missing data is managed gracefully.
- Removed the obsolete ENTRY_DIRECTORY_PATH_SUMMARY.md and ENTRY_PATH_SYNC_ANALYSIS.md files, consolidating documentation for clarity.
- Added a new `foreign_key_mappings` method to the Syncable trait, allowing models to declare their foreign key relationships for automatic UUID conversion during synchronization.
- Implemented the `to_sync_json` method to utilize the new FK mappings, enabling seamless conversion of local integer IDs to UUIDs before syncing.
- Updated the entry and location models to include foreign key mappings, enhancing their synchronization capabilities.
- Enhanced the `apply_state_change` method to leverage the generic mapping logic, simplifying the implementation for models with foreign keys.
- Introduced comprehensive documentation on the new FK mapping system and its usage across models.
- Revised documentation in the NetworkTransport trait to clarify the source of target device UUIDs and connected partners, switching references from the obsolete sync_partners table to the devices table.
- Introduced `create_library_no_sync` method in LibraryManager to allow library creation without automatic sync initialization, facilitating testing with mock transports.
- Enhanced logging in PeerSync to include network transport details for better debugging.
- Updated MockNetworkTransport to support transport name retrieval for improved test clarity.
- Added `sync_enabled` and `last_sync_at` fields to the device model to support synchronization capabilities.
- Created a migration script to update the devices table, consolidating sync-related data.
- Updated various components to ensure sync is enabled by default for new devices.
- Removed the obsolete sync partners table, streamlining the sync architecture.
- Introduced comprehensive sync integration tests to validate the new functionality and ensure robust performance.
- Introduced a new `ExtensionJobRegistry` to manage custom job types for extensions at runtime.
- Added `host_register_job` function to facilitate job registration from WASM extensions.
- Updated `PluginEnv` and `PluginManager` to include job registry functionality.
- Enhanced the `job` macro to support automatic job registration during plugin initialization.
- Updated documentation and tests to reflect the new job registration capabilities.
- Added `README_EXTENSIONS.md` to document the WASM extension system, outlining its features, quick start guide, and future plans.
- Implemented `WasmJob` for executing WASM extension jobs, including methods for job dispatching and state management.
- Enhanced `CoreContext` to include a `plugin_manager`, facilitating the management of extensions.
- Created integration tests for WASM job execution, ensuring the functionality of the new system.
- Removed the obsolete `sync_leadership` column from the database schema, streamlining the data model.
- Updated various modules to support the new extension architecture and improve overall system integration.
- Introduced `IT_WORKS.md` to document the successful execution of the WASM extension system, detailing test results and the steps taken during the process.
- Updated the `wasm_extension_test.rs` to reflect changes in the test extension structure, ensuring accurate loading and verification of the new test extension.
- Refactored the `Cargo.toml` files for both the SDK and test extension to streamline dependencies and improve build configurations.
- Removed the obsolete `test-extension-minimal` directory, consolidating the extension structure for better clarity and maintenance.
- Enhanced the `JobContext` implementation to utilize byte arrays for UUIDs, improving memory handling and consistency across the extension system.
- Documented next steps for further development, including the creation of a WasmJob executor and additional testing for job dispatching.
- Introduced `CURRENT_STATUS.md` to outline the current state of the WASM extension system, detailing what is working and what remains to be addressed.
- Documented the implementation status of core components, including the `PluginManager`, host functions, and test structure.
- Highlighted immediate tasks and timelines for further development, providing a clear path forward for the extension system.
- Updated the `Core` struct to include the `plugin_manager`, ensuring proper initialization during the core setup.
- Created integration tests for loading WASM extensions, validating the functionality of the new system.
- Enhanced `MacOSHandler` to query the database for inode information when handling rename events, allowing detection of renamed files even when the old path no longer exists.
- Implemented `get_inode_from_db()` method to facilitate database lookups for inode retrieval.
- Updated `handle_single_rename_event()` to utilize the new database query logic, ensuring proper event emission and entry identity preservation.
- Registered and unregistered database connections for locations in `LocationWatcher`, improving rename detection accuracy.
- Added integration tests to validate rename functionality and ensure no duplicate entries are created during renames.
- Added event listener for `LocationAdded` events in `LocationWatcher`, allowing dynamic addition of new locations.
- Updated `MacOSHandler` to emit proper `FsRawChange` events for Create, Modify, and Remove actions, improving event handling.
- Modified `handle_move_by_inode()` to update existing entries instead of creating duplicates for same-path, same-inode modifications.
- Introduced comprehensive integration tests for the location watcher, validating real-time filesystem monitoring and event detection.
- Documented test scenarios and helper methods for better clarity and usability in testing.
- Added QR code generation for remote pairing, allowing users to scan codes for easier device connection.
- Updated CLI to include QR code display and manual entry options for pairing.
- Enhanced PairingCoordinator to handle QR code JSON, including NodeId and relay URL for improved connectivity.
- Implemented camera permission handling in iOS for QR code scanning functionality.
- Refactored networking service to support relay-based pairing, ensuring reliable connections across different networks.
- Added tests for relay-only pairing scenarios to validate functionality and reliability.
- Introduced three new markdown files: CLI_LIBRARY_SYNC_COMPLETE.md, IMPLEMENTATION_COMPLETE.md, and LIBRARY_SYNC_SETUP_IMPLEMENTATION.md for comprehensive documentation.
- Updated various CLI domain modules to improve argument handling and output formatting.
- Enhanced device, index, job, library, location, network, and search modules for better integration and user experience.
- Refactored code across multiple domains to improve maintainability and clarity.
- Added a new SwiftUI application to monitor Spacedrive daemon jobs in real-time.
- Implemented job status tracking with visual indicators for running, completed, failed, and paused jobs.
- Included build and run scripts for easy setup and execution.
- Created a README with detailed instructions on building, running, and using the application.
- Added integration with the Spacedrive daemon via Unix domain socket for real-time updates.
- Introduced a translucent window design for a seamless macOS experience.
- Reduced the interruption point for processing events to 10 for faster testing during development.
- Added initial job status checks after core initialization to handle job completion and failure scenarios more effectively.
- Improved logging for job progress events, including phase extraction from generic progress data.
- Increased timeout for unresponsive job detection from 10 to 30 seconds to accommodate less frequent progress events during the Aggregation phase.
- Enhanced logging for job state serialization and deserialization, providing clearer insights during pause and resume processes.
- Updated job handling logic to ensure state is saved effectively, improving robustness in job management.
- Refactored the `ChangeDetector` to include accurate size information for entries, aiding in distinguishing between genuine moves and duplicate content.
- Introduced new methods for checking file content equivalence, enhancing change detection accuracy.
- Removed redundant checkpoint logic in indexing phases, streamlining state management during job execution.
- Introduced new methods for moving entries and updating them within existing transactions, improving consistency and performance.
- Refactored the `move_entry` function to handle transactions more effectively, allowing for rollback on errors.
- Added `bulk_move_entries` method to facilitate moving multiple entries in a single transaction, optimizing batch processing.
- Updated the job handling logic to ensure state is created early and improved interruption handling during processing phases.
- Enhanced logging for better visibility into job states and transaction management.
- Introduced two new benchmark recipes: `desktop_complex.yaml` and `desktop_extreme.yaml`.
- `desktop_complex.yaml` simulates a realistic desktop environment with 500k files and 8 levels of directory nesting.
- `desktop_extreme.yaml` targets power users with 1M files and 12 levels, featuring a comprehensive file type coverage and realistic size distribution.
- Updated documentation to include details about the new benchmark recipes and their intended use cases.
- Introduced a new `core/benchmarks` module to facilitate performance testing and benchmarking.
- Updated `Cargo.toml` to include the new benchmarks module in the workspace.
- Refactored job handling in the `JobManager` and `JobExecutor` to support job resumption, enhancing the ability to recover from interruptions.
- Improved logging throughout the job lifecycle to provide better visibility into job states and transitions.
- Added integration tests for job pause/resume functionality, ensuring robustness in job management.
- Updated the `LibraryCreateAction` to automatically set the newly created library as the current library in the session context.
- Introduced a new function `load_existing_locations` in the `LocationWatcher` to load and add existing library locations from the database to the watcher, improving the initialization process.
- Enhanced error handling and logging for loading locations, ensuring better visibility into potential issues during the loading process.
- Refactored `remove_emojis.sh` to enhance the emoji removal process by using a more efficient Python script for detecting and removing 3D/colorful emojis followed by spaces.
- Updated various Rust files to remove emoji characters from log messages and print statements, resulting in a cleaner output.
- Improved user experience with color-coded progress and results during the emoji removal process.
- Ensured that the script counts processed files and reports modifications accurately.
- Introduced `remove_emojis.sh`, a Bash script designed to identify and remove 3D/colorful emojis followed by a space in Rust files while preserving simple symbols.
- The script counts the number of Rust files processed and reports modifications made.
- Updated various Rust files to remove emoji characters from log messages and print statements for a cleaner output.
- Enhanced user experience by providing color-coded output for progress and results during the emoji removal process.
- Changed status of "Epic: Command-Line Interface" from "To Do" to "In Progress".
- Updated status of "Semantic Tagging Architecture" from "To Do" to "Done".
- Modified status of "Location Watcher Service" from "Done" to "In Progress".
- Set status of "Stale File Detection Algorithm" from "In Progress" to "To Do".
- Changed status of "Epic: Temporal-Semantic Search" from "To Do" to "In Progress".
- Refactored volume detection logic to improve filesystem-aware operations, including enhancements for APFS, Btrfs, NTFS, and ReFS.
- Introduced new platform-specific volume detection methods for Linux and macOS, streamlining the overall detection process.
- Added comprehensive tests for volume detection and filesystem-aware copy strategy selection.
- Updated `CopyMethod` enum to include `Atomic` and `Streaming` variants, replacing the previous `AtomicMove` and `StreamingCopy` options for clarity.
- Refactored the `select_strategy` method to respect user preferences for copy methods, improving the logic for same-volume operations.
- Added CLI support for the new copy methods in `args.rs`, allowing users to specify their preferred method during file copy operations.
- Updated relevant tests to reflect changes in copy method naming and functionality.
- Enhanced documentation to include new copy method options and their usage.
Co-authored-by: ijamespine <ijamespine@me.com>
- Updated various print statements across the codebase to remove emoji indicators, ensuring a more uniform logging style.
- Adjusted output messages in CLI, job handling, and event processing to enhance clarity and maintainability.
- Refactored job-related logging to provide clearer status updates without emoji clutter.
Co-authored-by: ijamespine <ijamespine@me.com>
- Updated the `tag.rs` file to clarify relationships between tags and their associated entities, improving the ORM definitions for better data integrity.
- Refined the initial schema migration to ensure proper column definitions and constraints for user metadata and device tracking.
- Improved the `manager.rs` and `action.rs` files to streamline tag creation and application processes, enhancing overall performance and maintainability.
- Added integration tests to verify the persistence of tagging operations and metadata integrity during move actions.
These changes collectively enhance the robustness and clarity of the tagging system, ensuring better data management and user experience.
- Deleted the `semantic_tag_validation` module to streamline the codebase as part of the transition to a unified tagging system.
- Updated module imports in `mod.rs` and adjusted references in the `semantic_tagging_test` to reflect the removal of the validation module.
- This refactor enhances maintainability and clarity within the tagging architecture, aligning with recent structural changes.
- Renamed the `user_metadata_manager` module to `manager` for consistency and clarity.
- Updated all references to `UserMetadataManager` across the codebase to reflect the new module name.
- Adjusted documentation to point to the new module location.
This refactor enhances the organization of the metadata management system, improving maintainability and readability.
- Renamed and restructured the tagging modules, replacing `semantic_tag` with `tag` for consistency across the codebase.
- Updated all references in the domain, operations, and database layers to reflect the new naming conventions.
- Removed deprecated files and entities related to the old semantic tagging system, including `semantic_tag.rs` and `metadata_tag.rs`.
- Enhanced the organization of the tagging system, improving clarity and maintainability for future development.
This commit streamlines the tagging architecture, paving the way for advanced features and improved usability.
- Reintroduced the `metadata` module in the operations layer for better organization.
- Replaced `UserMetadataService` with `UserMetadataManager` for improved clarity in user metadata handling.
- Updated `SemanticTagService` references to `SemanticTagManager` in tag operations for consistency.
- Enhanced documentation in the `tags` module to clarify the purpose and functionality of the tag operations.
- Removed deprecated services and cleaned up imports to streamline the codebase.
This commit improves the structure and readability of the semantic tagging system, setting the stage for future enhancements.
