- Added a new `file_count` column to the `content_kinds` table to track the number of content identities for each kind, improving statistics calculation efficiency.
- Implemented a migration script to add the `file_count` column and ensure backward compatibility.
- Updated the `Library` struct to include a method for updating content kind counts based on existing content identities.
- Introduced a new query for retrieving content kind statistics, including file counts, to facilitate better data analysis and reporting.
- Refactored related components to support the new statistics functionality, enhancing overall data integrity and performance.
- Updated the CI workflow to run all tests for the `sd-core` package, ensuring comprehensive coverage.
- Modified test execution commands to include specific tests for better granularity and clarity.
- Improved device management by regenerating slugs upon name changes and updating global device slugs.
- Enhanced session key generation for improved security during device pairing.
- Updated progress monitoring tests to ensure robust performance tracking for large file operations.
- Updated the CI workflow to run tests specifically for the package for clarity.
- Deleted outdated example test files for event serialization, location watcher, and simple metrics to reduce clutter.
- Enhanced error handling in for better robustness.
- Improved device data handling in by adding additional fields for comprehensive device information.
- Adjusted test assertions in to align with updated change detection logic.
- Adjusted the CI workflow to run tests with updated command syntax for improved clarity.
- Marked subproject commits in ios, landing, macos, and workbench as dirty to reflect uncommitted changes.
- Modified conflict resolution options in copy action tests to use `None` for better handling of file operations.
- Enhanced metadata handling in various tests to improve context and accuracy during execution.
- Removed the setup step for native dependencies in the CI workflow to streamline the build process.
- Updated the `copy_action_test` to include conflict resolution options for better handling of file conflicts.
- Enhanced metadata handling in `entry_move_integrity_test` to include job policies.
- Added metadata to event handling in `file_structure_test` for improved context during file operations.
- Updated service configuration in `file_sync_simple_test` and `file_sync_test` to use `fs_watcher_enabled` and included default logging settings.
- Deleted the outdated README.md from the tests directory to reduce clutter and improve documentation focus.
- Changed the CI workflow to run tests on self-hosted runners for better performance and flexibility.
- Updated the test execution condition to trigger on push events and specific pull requests.
- Consolidated test execution to run all tests in the workspace instead of a single test.
- Modified the default features in Cargo.toml for the server and core applications to include "heif" and "ffmpeg" for enhanced media support.
- Added metrics collection capabilities to the SyncProtocolHandler for improved monitoring of sync operations.
- Refactored sync tests to streamline setup and improve clarity in test scenarios.
- Refactored the upsert logic for Space, SpaceItem, and SpaceGroup entities to utilize atomic operations, preventing race conditions during synchronization.
- Enhanced database interactions with `on_conflict` handling based on UUID for improved data integrity.
- Introduced a comprehensive sync setup test to validate the functionality and ensure no UNIQUE constraint errors occur during device pairing and library sharing.
- Marked subproject commits in ios, landing, macos, and workbench as dirty to reflect uncommitted changes.
- Added detailed logging in the volume listing and sync processes to improve traceability and debugging capabilities, including information on fetched volumes and device pairing statuses.
- Introduced a new test for bidirectional volume sync to ensure both devices correctly receive each other's volumes, enhancing test coverage for sync functionality.
- Introduced AppResetContext to manage app reset operations across components.
- Updated App and RootLayout to utilize the reset context, allowing for a clean reset of the app state.
- Enhanced SettingsScreen with a reset data option, prompting users for confirmation before clearing all data and refreshing the app.
- Added core reset action in the backend to handle data deletion and cleanup operations.
- Improved user feedback during reset operations with alerts and status messages.
- Improved the `emit_change_event` method in `DatabaseAdapter` to handle resource events more effectively, including detailed logging for event emissions.
- Updated the `PersistentEventHandler` to ensure paths are registered with the filesystem watcher upon startup, enhancing real-time monitoring capabilities.
- Introduced new event collectors for better diagnostics and logging of filesystem and core events during tests.
- Enhanced integration tests to cover various scenarios, including batch file operations and delete/restore patterns, ensuring accurate event handling and preventing duplicates.
- Updated the `delete` method in `MemoryAdapter` to emit a `ResourceDeleted` event, allowing the frontend to update its cache accordingly.
- Modified `CollectedCoreEvent` structure to include an `event_type` field for better event categorization.
- Enhanced event collection logic in `CoreEventCollector` to handle both `ResourceChanged` and `ResourceDeleted` events, improving event summary reporting.
- Updated `remove_entry` and `remove_directory_tree` methods in `EphemeralIndex` to ensure proper removal of entries from parent nodes' children lists.
- Introduced `FsEventCollector` and `CoreEventCollector` for improved event collection from the filesystem watcher and core event bus, respectively.
- Enhanced test harness to utilize the new collectors, ensuring accurate event logging and summary reporting.
- Added scenarios to test delete and restore patterns, verifying correct entry counts and preventing duplicate entries.
- Implemented methods for batch creation and deletion of files and directories in the test harness.
- Added scenarios to verify the creation and deletion of multiple files and directories, ensuring proper event handling.
- Enhanced logging for better visibility during test execution.
- Updated the macOS event handler to improve directory deduplication and manage recent directory creations effectively.
- Updated `connect` methods in `EphemeralEventHandler` and `PersistentEventHandler` to be asynchronous, allowing for non-blocking connections to the `FsWatcherService`.
- Modified `init_handlers` in `FsWatcherService` to support async initialization of event handlers.
- Adjusted service startup methods to await handler initialization, ensuring proper setup before starting the watcher.
- Enhanced test harness to utilize the new async capabilities, improving event collection and verification processes.
- Introduced a new File Operation Modal for interactive file copy/move operations with conflict resolution options.
- Integrated sound effects for file operations, including copy and startup sounds.
- Updated the Explorer component to utilize the new modal for drag-and-drop file operations.
- Refactored job management hooks to unify job data handling and improve performance.
- Enhanced documentation for the File Operation Modal and sound effects integration.
- Add a complete ephemeral indexing subsystem
- core/src/ops/core/ephemeral_status with input/output and query types
- core/src/ops/indexing/ephemeral with arena, cache, registry,
index_cache, types
- expose EphemeralIndexCache and EphemeralIndex through core modules
- EphemeralIndexCache supports
get/insert/create_for_indexing/mark_indexing_complete eviction and
stats
- Implement EphemeralIndex data structures for memory-efficient storage
- NodeArena, NameCache, NameRegistry, and related types
- Add EphemeralIndex status API
- EphemeralCacheStatusInput and EphemeralCacheStatusQuery
- EphemeralCacheStatus with per-index details
- Wire ephemeral indexing into the indexing flow
- Change default Ephemeral Indexer behavior to shallow mode
- Align code to EphemeralIndex usage across the codebase
- Enhance content kind detection in UI
- Add getContentKind(file) helper (prefers content_identity.kind, then
content_kind)
- Use getContentKind in Explorer utilities and UI components
- Invalidate directory listings when location index_mode changes
- Add useLocationChangeInvalidation to trigger refetches for ephemeral
vs persistent indexing transitions
- Misc refactors and formatting to accommodate the new modules and APIs
- Revert to the November 16 baseline for realtime sync - Use a single
state watermark and drop per-resource watermarks, counts, and hashes
from watermark messages - Revert DataAvailableNotification handling and
the updated FK filtering approach in favor of NULL-based handling - Add
registry helper get_model_type_by_table to map tables to models for FK
resolution - Update mocks and tests to reflect the baseline (linkage
stats in summary) - Fix test infrastructure: remove block_in_place usage
and minor imports - Remove obsolete test fixture file
1. Watermark Messages: - Nov 16: Simple - single my_state_watermark,
single state_watermark in response - Main: Complex - per-resource
my_resource_watermarks HashMap, resource counts, content hashes 2.
DataAvailableNotification: - Nov 16: Doesn't exist - Main: New message
type that triggers watermark exchanges 3. FK Mapper: - Nov 16: Sets
missing FKs to NULL, applies all records - Main: Marks records with
missing FKs as failed, filters them out 4. Watermark Storage (internal,
not in messages): - Nov 16: Single last_watermark per resource - Main:
Dual watermarks - cursor_watermark and
- Scope the closure query to the indexing path by resolving its entry_id
from directory_paths and use that for descendants; fall back to the
location root if not found. - Seed all ancestors between the location
root and the indexing path into the in-memory cache during the
processing phase to ensure correct parent links. - Seed ancestor
directories into the create path cache to prevent ghost folders on
subpath indexing. - Add test core/tests/indexing_move_folder_bug_test.rs
reproducing the ghost-folder bug when moving a folder into a managed
location and asserting correct parent linkage.
- Added `include_descendants` option to event filters, allowing recursive path matching for resource events.
- Updated `affects_path` method to support descendant matching, improving event handling accuracy.
- Refactored subscription logic to utilize the new filtering capabilities, ensuring only relevant events are processed.
- Introduced tests for event filtering to validate exact vs. recursive matching, enhancing reliability of event-driven updates.
- Updated related components to leverage the new filtering options, improving overall performance and user experience.
- Introduced a new constructor for the HLC struct to allow explicit initialization with timestamp, counter, and device ID.
- Updated indexing tests to improve error handling for entry IDs and added a comprehensive test for reindexing after offline changes, ensuring all original and new files are detected correctly.
- Introduced new modules and actions for generating media proxies and thumbstrips, improving media handling capabilities.
- Added configuration, error handling, and processing logic for both proxy and thumbstrip generation.
- Updated existing media operations to integrate new functionalities, ensuring seamless media processing and user experience.
- Enhanced tests to cover new features and ensure reliability in media operations.
- Introduced new functionality to validate resource counts during watermark exchanges, improving synchronization accuracy.
- Added methods to retrieve device-owned resource counts for gap detection, ensuring that discrepancies are identified and addressed.
- Updated the `SyncMessage` structure to include actual resource counts, facilitating better tracking of synchronization state.
- Enhanced the `PeerSync` implementation to clear watermarks for mismatched resources, allowing for surgical recovery in case of count mismatches.
- Improved logging for better visibility into synchronization processes and potential issues.
- Introduced a new `DataAvailableNotification` message type to notify peers of new data availability, triggering immediate watermark exchanges for synchronization.
- Enhanced the `PeerSync` implementation to handle notifications and manage real-time activity tracking per peer, improving synchronization efficiency.
- Updated the `SyncProtocolHandler` to process the new notification type and initiate catch-up logic accordingly.
- Implemented periodic watermark checks to ensure timely synchronization and prevent missed events during idle periods.
- Added support for per-resource watermarks in the synchronization process, allowing for fine-grained comparison and catch-up logic.
- Implemented new configuration options for real-time batching, including maximum entries before flush and flush interval in milliseconds.
- Enhanced the `apply_state_change` function to check for tombstoned records, preventing the resurrection of deleted entries.
- Updated various components to utilize the new watermark and batching features, improving synchronization efficiency and robustness.
- Added clipboard permissions for reading and writing text in the default capabilities configuration.
- Enhanced the sync test harness to capture job lifecycle events, improving tracking of indexing completion and orphaned file management.
- Implemented checks for orphaned files during synchronization, ensuring robust handling of incomplete sync scenarios.
- Updated the submodule reference in `workbench`.
- Added debug logging in `ResourceManager` to track resource event emissions and virtual resource mappings.
- Improved the `PeerSync` implementation to emit resource events in a non-blocking manner, enhancing UI reactivity and ensuring proper resource formatting.
- Updated integration tests to include new resource types and refined synchronization checks for stability.
