Commit Graph

3471 Commits

Author SHA1 Message Date
Cursor Agent
dc66b34de1 feat: Add file and index commands to CLI
Co-authored-by: ijamespine <ijamespine@me.com>
2025-09-11 01:14:10 +00:00
Jamie Pine
050b2a7e8e Merge pull request #5 from jamiepine/cursor/fix-library-action-naming-inconsistency-3036 2025-09-10 20:01:58 -04:00
Cursor Agent
0dc3995f74 Refactor: Pluralize action kinds for consistency
Co-authored-by: ijamespine <ijamespine@me.com>
2025-09-11 00:00:44 +00:00
Jamie Pine
f6edb570c4 fix: update subproject commit and enhance action registration
- Updated the subproject commit to indicate a dirty state.
- Commented out the unused import of `PathBuf` in `daemon.rs` for clarity.
- Added a test module in `registry.rs` to list registered operations and ensure at least one action or query is registered.
- Registered core actions for library creation, deletion, export, renaming, and location management, improving modularity and consistency in action handling.

These changes enhance the reliability of the operation registration process and improve code clarity.
2025-09-10 19:48:11 -04:00
Jamie Pine
13df73bef0 refactor: streamline action and query structures
- Removed unnecessary whitespace in `cqrs.rs` to enhance code cleanliness.
- Reorganized imports in `addressing.rs` for better clarity and consistency.
- Introduced new input types for actions, including `Input` associated types in `CoreAction` and `LibraryAction`, promoting a more modular design.
- Updated action implementations to utilize the new input structures, improving maintainability and reducing redundancy.
- Enhanced event handling in `event.rs` to ensure proper library ID filtering.

These changes improve the overall structure and maintainability of the action and query systems while ensuring a consistent API surface.
2025-09-10 17:34:11 -04:00
Jamie Pine
25db936675 feat: introduce unified op! macro for operation registration
- Added the `op!` macro to streamline the registration of library actions, core actions, and queries, reducing boilerplate and promoting consistency across the codebase.
- Implemented helper macros for generating method strings for actions and queries, enhancing clarity in operation declarations.
- Updated existing file operations (copy, delete, duplicate detection) to utilize the new macro, simplifying their input and action structures.
- Introduced comprehensive documentation for the operations initialization API, outlining usage patterns and conversion mechanisms.

These changes significantly enhance the modularity and maintainability of the operation registration process while ensuring a consistent API surface.
2025-09-10 15:40:13 -04:00
Jamie Pine
a25caddd41 refactor: streamline action manager and input structures
- Refactored the `ActionManager` to improve the dispatching of core and library-scoped actions, enhancing clarity and maintainability.
- Updated action input structures to remove unnecessary library ID fields, promoting a cleaner API and reducing redundancy.
- Enhanced the `dispatch_library` method to accept library ID as a parameter, simplifying the action execution process.
- Improved validation and error handling across various action implementations, ensuring consistent feedback and robustness.

These changes significantly enhance the modularity and maintainability of the action system while ensuring a consistent API surface.
2025-09-10 15:10:29 -04:00
Jamie Pine
5c6735ea28 refactor: rename action registration macros for clarity
- Updated the action registration macros from `register_action_input!` to `register_library_action_input!` to better reflect their purpose and improve clarity in the codebase.
- Adjusted related documentation and input structures to ensure consistency with the new naming convention.
- Enhanced the organization of input types for file operations and library management, promoting better modularity and maintainability.

These changes improve the clarity and consistency of the action registration process across the codebase.
2025-09-10 13:14:30 -04:00
Jamie Pine
c29c93f130 fix: update subproject commit and add tests for operation registration
- Updated the subproject commit to indicate a dirty state.
- Added a suite of tests for verifying the registration of queries and actions, ensuring proper naming conventions and the absence of duplicates.
- Implemented debugging output to list registered operations, enhancing traceability and verification of the inventory system.

These changes improve the reliability of the operation registration process and facilitate easier debugging.
2025-09-10 13:00:02 -04:00
Jamie Pine
3566fa7838 refactor: enhance action and query handling with dynamic registry
- Introduced a dynamic registry for actions and queries, allowing for self-registration and improved modularity.
- Updated the `Core` struct to support new methods for executing actions and queries by method string, enhancing flexibility.
- Refactored the `CoreClient` to utilize a `METHOD` constant for action dispatching, improving clarity.
- Removed legacy action handling code and streamlined the `RpcServer` for better request management.
- Enhanced input structures for file operations and library management, ensuring type safety and consistency.

These changes significantly improve the modularity and maintainability of the action system while ensuring a consistent API surface.
2025-09-10 12:59:55 -04:00
Jamie Pine
940ddead3e refactor: enhance core action handling and modularity
- Introduced new methods for executing actions and queries by method string, improving the flexibility of the core API.
- Refactored the `CoreClient` to utilize a `METHOD` constant instead of `TYPE_ID`, enhancing clarity in action dispatching.
- Removed the legacy `ActionHandler` trait and associated registry, streamlining action execution through the unified `LibraryAction` trait.
- Updated the `RpcServer` to handle action and query requests more efficiently, improving error handling and response management.
- Enhanced the `FileCopyAction` and related tests to support modular action construction, ensuring better validation and type safety.

These changes significantly improve the modularity and maintainability of the action system while ensuring a consistent API surface.
2025-09-10 12:46:37 -04:00
Jamie Pine
1f2c38b440 refactor: introduce CLI application structure and enhance command handling
- Added a new `apps/cli` module to encapsulate the CLI application logic, improving organization and modularity.
- Updated `Cargo.toml` to include dependencies for the CLI, such as `clap` for command-line argument parsing.
- Implemented a `Context` struct to manage application state and core client interactions.
- Created command handling for file operations and library management, enhancing user experience and functionality.
- Introduced utility functions for output formatting, supporting both human-readable and JSON outputs.

These changes significantly enhance the structure and maintainability of the CLI application while providing a more robust user interface.
2025-09-09 20:04:20 -04:00
Jamie Pine
e4b463d78e refactor: add client module and implement Wire trait for input structures
- Introduced a new `client` module to enhance the modularity of the codebase.
- Implemented the `Wire` trait for `FileCopyInput` and `ListLibrariesQuery`, providing type IDs for better integration with the system.
- Updated the `lib.rs` and `mod.rs` files to include the new `client` module, ensuring proper organization and accessibility.

These changes improve the structure and maintainability of the codebase while enhancing type safety in data handling.
2025-09-09 16:53:44 -04:00
Jamie Pine
80c2fff76e refactor: enhance command handling for file operations and indexing
- Updated the `DaemonCommand` structure to support typed input for file copy and indexing operations, improving clarity and type safety.
- Refactored the `FileHandler` to directly execute commands using the new input structures, streamlining the action execution process.
- Removed legacy browse command handling in favor of a unified indexing approach, enhancing maintainability and consistency.
- Improved error handling and validation across command executions to ensure robust feedback and user experience.

These changes significantly enhance the modularity and maintainability of the command handling system while ensuring a consistent API surface.
2025-09-09 15:21:43 -04:00
Jamie Pine
45d24b8c83 refactor: update indexing actions to use unified input structure
- Refactored the `IndexingAction` to utilize a new `IndexInput` structure, enhancing clarity and modularity in action parameters.
- Removed legacy fields from `IndexingAction`, streamlining the job configuration process.
- Updated the `execute` method to handle multiple paths and job configurations more efficiently, improving error handling and validation.
- Adjusted related action handlers to align with the new input structure, ensuring consistency across the action system.

These changes improve the maintainability and extensibility of the indexing actions while ensuring a consistent API surface.
2025-09-09 14:01:46 -04:00
Jamie Pine
fef30be6bd refactor: update action handling to use LibraryAction trait
- Refactored multiple action handlers to implement the new `LibraryAction` trait, enhancing modularity and clarity in action execution.
- Removed legacy action handling code, streamlining the action dispatch process.
- Updated the `FileCopy`, `LocationRescan`, `DeviceRevoke`, and `VolumeTrack` actions to utilize the new structure, improving type safety and reducing redundancy.
- Enhanced error handling and validation across actions to ensure consistent feedback and maintainability.

These changes significantly improve the maintainability and extensibility of the action system while ensuring a consistent API surface.
2025-09-09 14:01:40 -04:00
Jamie Pine
56782b237b refactor: update action system to modular structure with CoreAction and LibraryAction
- Refactored the action system to implement distinct `CoreAction` and `LibraryAction` traits, enhancing modularity and clarity.
- Removed the old `ActionTrait` and associated boilerplate, streamlining action implementations.
- Updated multiple actions to utilize the new traits, ensuring type safety and reducing redundancy in library validation.
- Enhanced the `ActionManager` to support generic dispatch for both core and library actions, preserving existing validation and logging infrastructure.

These changes significantly improve the maintainability and extensibility of the action system while ensuring a consistent API surface.
2025-09-09 11:10:33 -04:00
Jamie Pine
469fc7cbe2 refactor: complete migration to modular action system and remove ActionOutput enum
- Eliminated the centralized `ActionOutput` enum, achieving true modularity in action handling.
- Introduced the `ActionType` trait, allowing each action to define its own output type directly.
- Updated multiple actions to utilize native output types, enhancing type safety and reducing complexity.
- Enhanced the `ActionManager` with a generic `dispatch_action` method for improved action execution.
- Preserved existing infrastructure for validation, audit logging, and error handling throughout the migration.

These changes significantly improve the modularity and maintainability of the action system while ensuring a consistent API surface.
2025-09-09 02:03:17 -04:00
Jamie Pine
478f3136c5 refactor: enhance CQRS implementation with modular command and query traits
- Updated the `Command` trait to support modular output types, eliminating the need for a centralized `ActionOutput` enum.
- Refactored the `execute_command` function for direct command execution, improving type safety and performance by avoiding unnecessary JSON serialization.
- Implemented the new `ListLibrariesQuery` and associated output types for library listing operations, enhancing the query capabilities of the API.
- Updated existing actions like `LibraryCreateAction` and `VolumeTrackAction` to utilize the new modular command structure, simplifying their implementations.

These changes improve the modularity and maintainability of the action system while preserving existing functionality.
2025-09-08 20:08:44 -04:00
Jamie Pine
5f68a0d697 refactor: implement CQRS pattern with Command and Query traits
- Introduced a new `Command` trait to streamline action execution, reducing boilerplate and enhancing type safety.
- Updated the `Core` API to include `execute_command` and `execute_query` methods, providing a unified interface for command and query operations.
- Implemented the `Command` trait for `LibraryCreateAction`, ensuring compatibility with existing ActionManager functionality.
- Enhanced the API design documentation to reflect the new CQRS structure and its benefits.

These changes improve the clarity and maintainability of the action system while preserving existing functionality.
2025-09-08 19:36:37 -04:00
Jamie Pine
e129a90e3a refactor: remove API design document and update cloud submodule
- Deleted the `API_DESIGN_DOC.md` file, which contained the proposed architecture for the Spacedrive CLI and daemon refactoring.
- Updated the cloud submodule to reflect a dirty state, indicating local changes.

These changes streamline the project by removing outdated documentation and ensuring the cloud submodule is accurately represented.
2025-09-08 19:15:49 -04:00
Jamie Pine
d90b486865 refactor: update project dependencies and remove unused components
- Removed the `prisma-client-rust` and `prisma-client-rust-sdk` dependencies from `Cargo.toml` to streamline the project.
- Deleted the `spacedrive-jobs-derive` crate, consolidating job registration functionality into the main job system.
- Updated the `.gitattributes` file to reflect changes in generated code tracking.
- Adjusted the `package.json` scripts for improved build processes and removed unnecessary commands.
- Enhanced the `combine.sh` script to correctly reference documentation paths.

These changes aim to simplify the project structure and improve maintainability.
2025-09-08 17:26:31 -04:00
Jamie Pine
70a69e1bdc refactor: streamline volume tracking and untracking actions
- Removed the VolumeTrackHandler and VolumeUntrackHandler implementations, consolidating their logic into the respective action structs for better maintainability.
- Updated the VolumeTrackAction and VolumeUntrackAction to directly handle their execution logic, improving clarity and reducing redundancy.
- Enhanced error handling during volume tracking and untracking processes to provide more informative feedback.

These changes aim to simplify the action handling for volume operations and improve code organization.
2025-09-07 12:35:46 -04:00
Jamie Pine
aa7ccacea8 sorting 2025-09-07 01:56:50 -04:00
Jamie Pine
ed7e101d71 more 2025-09-07 01:51:35 -04:00
Jamie Pine
1a0a293700 again 2025-09-07 01:44:02 -04:00
Jamie Pine
a8a2c732ef more refactor 2025-09-07 01:43:57 -04:00
Jamie Pine
8f03cc6d4c x 2025-09-06 21:23:39 -04:00
Jamie Pine
41df3c86ee more 2025-09-06 21:13:31 -04:00
Jamie Pine
29a87a91ce huge refactor 2025-09-06 21:00:37 -04:00
Jamie Pine
5001646cb3 refactor: improve indexing rules and library configuration
- Enhanced the LibraryConfig and LibrarySettings structs with additional indexer settings for better file filtering during indexing.
- Implemented a new rules system for the indexer, allowing for customizable file handling based on user-defined criteria.
- Updated the discovery phase to leverage the new rules engine, improving indexing efficiency.

These changes aim to modernize the indexing functionality and enhance project organization.
2025-09-06 19:41:08 -04:00
Jamie Pine
648496de75 refactor: update library configuration and indexing rules
- Removed the outdated combine_tasks.sh script to streamline the project structure.
- Enhanced the LibraryConfig and LibrarySettings structs with new indexer settings for improved file filtering during indexing.
- Introduced a new rules system for the indexer, allowing for more flexible and customizable file handling based on user-defined rules.
- Updated the discovery phase to utilize the new rules engine for filtering entries, improving the efficiency of the indexing process.

These changes aim to modernize the indexing functionality and improve the overall organization of the project.
2025-09-06 18:12:15 -04:00
Jamie Pine
fe0d5acd19 refactor: consolidate documentation and scripts for improved organization
- Removed outdated scripts for combining documentation and Rust files, as well as the combined documentation file, to streamline the project structure.
- Introduced a new generic script for combining files from various directories, enhancing flexibility and usability.
- Updated task tracking documentation to include new filtering and sorting options for better task management.

These changes aim to simplify the documentation process and improve the overall organization of task management within the project.
2025-09-03 14:56:18 -06:00
Jamie Pine
9d6c243139 feat: remove outdated synchronization tasks and documentation
- Deleted several markdown files related to the synchronization epic, including task descriptions for the sync relationship schema, sync engine service, and VDFS diffing logic.
- This cleanup reflects a shift in focus away from the previous synchronization design, streamlining the project structure and preparing for new implementations aligned with the updated architecture.
2025-08-30 23:46:31 -06:00
Jamie Pine
cc6985ec2e feat: add new task files for AI and resource management epics
- Introduced new markdown files for the AI epic (AI-000) and its associated AI agent task (AI-001), outlining features for an intelligent data management system.
- Added resource management epic (RES-000) and its first task on adaptive throttling (RES-001) to ensure efficient resource usage on mobile devices.
- Created security tasks for implementing SQLCipher encryption (SEC-002) and a cryptographic audit log (SEC-003), enhancing data security measures.
- Established synchronization tasks, including the sync relationship schema (SYNC-001) and the sync engine service (SYNC-002), to support VDFS-powered synchronization.

These additions lay the groundwork for critical features in AI, resource management, security, and synchronization within the project.
2025-08-30 22:24:53 -06:00
Jamie Pine
f6a06d7c0b refactor: update task descriptions to future tense and enhance task-validator functionality
- Changed implementation notes in various task markdown files to future tense, indicating planned features and implementations.
- Updated `Cargo.toml` in the task-validator to include new dependencies for enhanced functionality.
- Refactored the `main.rs` file in the task-validator to introduce a command-line interface for listing and validating tasks, improving usability and organization.

These changes aim to clarify the status of tasks and enhance the task validation process within the project.
2025-08-30 22:04:49 -06:00
Jamie Pine
2936541fcf feat: add task-validator and validation script for task files
- Introduced a new `task-validator` package to validate task files against a defined schema.
- Added a validation script that checks for proper YAML front matter in task markdown files, ensuring data integrity before commits.
- Updated `Cargo.toml` to include the new workspace member `task-validator`.
- Created a `crush.json` configuration for service providers and added a combined Rust files document for better organization.

These changes enhance the development workflow by enforcing task file validation and improving project structure.
2025-08-30 21:58:06 -06:00
Jamie Pine
2a40dc2ff5 refactor: use EventBus for job progress monitoring in tests
Replace polling-based job monitoring with event-driven approach using EventBus.
The tests now subscribe to JobProgress and JobCompleted events instead of
repeatedly querying job status, making them more robust and idiomatic.
2025-08-26 13:12:33 -06:00
Jamie Pine
cd9da2e09d Merge remote-tracking branch 'refs/remotes/origin/main' 2025-08-24 18:10:26 -07:00
Jamie Pine
2a974f3bc0 Enhance device connection handling and address discovery in networking service
- Updated the `handle_command` method to retrieve remote device addresses from the active connection map, improving accuracy in device connection updates.
- Implemented logging for cases where active connections are not found, aiding in troubleshooting.
- Enhanced the `get_node_addr` method to wait for address discovery if none are available, improving the reliability of device communication.
- Added comprehensive logging for address discovery attempts, providing better visibility into the networking process.

These changes aim to improve the robustness and clarity of device connection management within the networking service.
2025-08-24 18:09:58 -07:00
Jamie Pine
22bbb2cf61 Enhance networking and pairing protocols for improved error handling and functionality
- Introduced a data directory configuration for persistent pairing sessions, enhancing device communication reliability.
- Improved error messages for pairing code validation, providing clearer guidance for users.
- Enhanced the handling of device connections by including address information, improving the accuracy of device state updates.
- Added robust logging for connection and pairing processes, aiding in troubleshooting and monitoring.

These changes aim to streamline the networking setup and enhance the reliability of the pairing protocol within the system.
2025-08-24 18:09:51 -07:00
Jamie Pine
a5da8ebf09 feat(sync): Add refactored sync design and update original doc
- Introduces a new, dated sync design document (`SYNC_DESIGN_2025_08_19.md`) to reflect a more robust and refined architecture.
- The new design incorporates several key improvements based on a detailed analysis of the V2 codebase and data models.

Key changes in the new design include:
- A clear architectural distinction between finite "Jobs" (e.g., Backfill) and long-running "Services" (e.g., LiveSync).
- Explicit clarification that derived data, such as closure tables, will not be synced and must be rebuilt locally on each device.
- A new requirement for log redaction and compaction on the leader to ensure data privacy and efficient storage.

- Also updates the original `SYNC_DESIGN.md` to include the clarification about not syncing closure tables.
2025-08-20 01:25:11 -04:00
Jamie Pine
386109b765 Refactor networking and pairing protocol for improved clarity and functionality
- Removed the `start_networking` method call from the daemon initialization process, as it was redundant after `init_networking`.
- Commented out the volume detection initialization code for future review, improving code clarity.
- Enhanced the pairing protocol by establishing a persistent connection immediately after successful pairing, ensuring better device communication.
- Updated the handling of pairing messages to maintain a continuous stream for message exchanges, improving responsiveness.
- Improved logging throughout the networking and pairing processes for better visibility into operations and error handling.

These changes aim to streamline the networking setup and enhance the reliability of the pairing protocol within the system.
2025-08-12 18:59:46 -07:00
Jamie Pine
8620913633 Refactor database migration test and update whitepaper metrics
- Refactored the `test_database_creation_and_migration` function in `database_migration_test.rs` for improved readability and structure.
- Updated the whitepaper to replace the outdated indexing duration summary with a new performance benchmarks description, including testing details on hardware specifications.
- Removed deprecated CSV inputs from the LaTeX document and adjusted related figures to enhance clarity in performance reporting.

These changes aim to improve the maintainability of the test code and enhance the accuracy of performance insights in the whitepaper.
2025-08-12 12:29:18 -07:00
Jamie Pine
d77ffa26de Add custom data directory support for DeviceManager and DeviceKeyManager
- Introduced a `data_dir` field in `DeviceManager` to allow specification of a custom data directory for device configurations and keys.
- Updated `DeviceKeyManager` to support fallback file paths for master keys, ensuring consistent key management across different data directories.
- Enhanced the `DevicePersistence` and `DeviceRegistry` implementations to utilize the new data directory structure, improving the reliability of device connection persistence.
- Added comprehensive logging and retry mechanisms in the networking service to improve connection stability and visibility during device interactions.

These changes aim to enhance the flexibility and robustness of device management and key handling within the system.
2025-08-12 10:54:06 -07:00
Jamie Pine
88809dc3c1 Enhance pairing protocol by marking initiator device as connected after pairing
- Added functionality to immediately mark the initiator device as connected upon successful pairing completion, ensuring accurate device status even if the completion message fails.
- Implemented logging for both successful and failed attempts to mark the device as connected, improving visibility into the pairing process.
- Updated file transfer tests to display full checksums instead of truncated versions, enhancing clarity in verification outputs.

These changes aim to improve the reliability of device connection status during pairing and enhance the clarity of file transfer verification.
2025-08-12 00:19:15 -07:00
Jamie Pine
820c3fcb23 Add LocationAdded variant to ActionOutput and update location command handling
- Introduced a new `LocationAdded` variant in the `ActionOutput` enum to encapsulate the location ID and optional job ID upon successful location addition.
- Updated the `handle_location_command` function to handle the new response format from the daemon, including improved error handling for unexpected responses.
- Modified the `LocationHandler` to extract and return the location ID and job ID from the action output, ensuring proper response structure.
- Refactored the `ActionHandler` for `LocationAddHandler` to utilize the new `LocationAdded` output format, enhancing clarity in action results.

These changes aim to improve the handling of location addition actions and provide clearer feedback in the system's output.
2025-08-11 22:52:15 -07:00
Jamie Pine
cd913aa79f Add build script and update dependencies for enhanced functionality
- Introduced a new `build.rs` file to automate the generation of build metadata, including Git SHA, commit timestamp, and branch information.
- Updated `Cargo.toml` to include new dependencies such as `comfy-table`, `dialoguer`, `indicatif`, `owo-colors`, `supports-color`, `console`, and `colored` for improved CLI output and user interaction.
- Added a new shell script `update_spacedrive.sh` for streamlined updates of the Spacedrive CLI, including Git pull and project build processes.
- Refactored benchmark scenarios by renaming and consolidating related functionality, enhancing clarity and maintainability.

These changes aim to improve the build process and enhance the user experience within the Spacedrive system.
2025-08-11 21:37:03 -07:00
Jamie Pine
fc3c9c7107 Add benchmark results for content identification scenarios
- Introduced new JSON files for shape_large, shape_medium, and shape_small content identification benchmarks, detailing performance metrics and hardware specifications.
- Updated the whitepaper metrics CSV to include the new benchmark results, enhancing the reporting structure for better analysis.
- Enhanced the LaTeX document to incorporate new tables and figures summarizing indexing durations and performance comparisons across different hardware setups.

These changes aim to improve the clarity and comprehensiveness of performance insights within the Spacedrive system.
2025-08-10 04:05:07 -07:00
Jamie Pine
ae1e51dbca Enhance benchmark reporting by adding duration metrics
- Updated the CSV structure for whitepaper metrics to include a new 'Duration_s' column, providing insights into the time taken for each phase of the benchmarking process.
- Modified the CSV reporting logic in the Rust code to calculate and include phase-specific durations, improving the clarity and usability of performance data.
- Adjusted the LaTeX document to reflect the updated CSV format, ensuring accurate representation of benchmark results in the whitepaper.

These changes aim to improve the analysis of performance metrics within the Spacedrive system.
2025-08-10 02:47:47 -07:00