mirror of
https://github.com/spacedriveapp/spacedrive.git
synced 2026-10-30 04:49:32 -04:00
Capture time and place in listings and search, read from the image facet on both backends (#3151)
* Capture time and place in listings and search, read from the image facet on both backends search.files sorts by CapturedAt and filters by a CapturedAt range and has_location; files.directory_listing sorts by datetaken; the media listing's datetaken reads the facet. The store joins facet_image into every entry read, guarded by the row's content hash, and the arena path reads the facet from the store ahead of its sort and filter, so both backends hand the same File to one predicate and one order (R8 row 19, new row 29). File.image_media_data is filled from the facet on every path; the inspector shows the capture time and the coordinates; Date Taken is in every explorer view's sort menu. Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com> * search.media judges the capture filters on its rows and leaves replicas out under them; a directory listing reads facets ahead of the sort only for a capture order Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com> * search.media judges a replica's rows under a capture filter as search.files does instead of dropping the partition Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai-staging[bot] <305915985+capy-ai-staging[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
15d7df1808
commit
e4b4cb71f5
28 files changed
+1365
-137
No files matched your search
@@ -215,6 +215,7 @@ Recent landed.
|
||||
| Extension host functions, phase 1 | in progress 2026-10-07 (Fable, SPAC-30); one PR against `main`, one commit per gap | Fable | Product sequence step 5, second half, in SPAC-28's gap order. Every guest request that returns data now goes through one import, `spacedrive_op(name, payload)`, answered on the job's async side with permission checked against a strict manifest (`read_records` with an optional glob, `read_sidecars`, `write_sidecars` per kind, `use_models` per category; unknown fields are refused at load). Landed: task execution (`#[task]` attempts bracketed by `task.begin`/`task.end` with the SDK's deadline and an attempt line in the job log), entropy and clock (`spacedrive_random` backs `getrandom` so `Uuid::new_v4` works in the guest; `spacedrive_now_ms` backs `clock::now`), records over source stores (`Record` replaces the SDK's `Entry`; `records.get/query/read` resolve through the library's sources and read bytes from the store root; tag and metadata filters are refused), sidecars as one JSON document per extension kind under `SidecarKind::Extension { extension_id, kind }`, models (`#[model]` emits a definition `register_model` collects; the registry builds a `DataTypeSchema` and opens an sd-store db at `<library>/extensions/<id>/data.db`; `models.put/get/list` keyed by uuid or `content:<uuid>`), `config.get` for `config.json` beside the manifest, and `ai.infer`, which checks the `use_models` grant and answers `not_available` because the core has no inference provider. The photos extension handles that with one warning and a skip, so `analyze_photos` completes over a fixture library of twelve JPEGs in `core/tests/wasm_extension_test.rs` (the test extension's new `catalog` job covers records, sidecars, models and tasks in the same suite). Still unbacked: inference itself (so scenes, places and moments have nothing to run on), tags, custom fields, `dispatch_jobs`, agents, actions, queries, and file-kind or preview registration (gap 9, untouched). |
|
||||
| Extension host functions, phase 2 | in progress 2026-10-08 (Fable, SPAC-44); one PR against `main`, one commit per gap | Fable | Product sequence step 5, second half, continued from phase 1 and SPAC-28 gaps 6 and 7. Landed: `tags.add` and `tags.remove` (grant `write_tags`) over the library's own `tags.create`, `tags.apply` and `tags.unapply`, so an extension's tags are ordinary tags in the source store, created by path when nothing defines them; `records.query` honors a tag filter. `records.set_field` and `records.get_field` (grant `write_custom_fields` per namespace) keep `namespace.name` values as JSON text in a built-in `custom_field` model every extension store carries, keyed by record uuid, because the source store holds no per-record extension metadata and its schema stays untouched. `jobs.dispatch` (grant `dispatch_jobs`) queues one of the calling extension's own jobs by registered name with a starting state; the name resolves under the caller's id. Photos: nothing on `main` writes EXIF onto records (`facet_image` is declared, no ingest fills it), so a `records.exif` op reads the facet when present and otherwise parses the file through `sd-media-metadata`; `create_moments` now runs end to end with no inference (time-gap and distance clustering into `Moment` models, `Moments/<title>` tags, `photos.moment_id` field as the re-run guard, titles from dates), `identify_places` clusters by GPS into `Place` models with tags and fields and leaves a place `Unknown Location` when the reverse geocode's language model answers `not_available`, `analyze_scenes` skips with one warning without a classifier. `core/tests/wasm_extension_test.rs` gained the test extension's `tag` job (tags, fields, refusal of an ungranted namespace, a dispatched counter, and the same module refused under a grant-less manifest) and a fixture of twelve hand-built EXIF JPEGs yielding three moments and four places. sd-store fix: a bare string in a JSON column reads back as that string instead of null. Still unbacked: inference itself (face detection, scene classification, language models for place and moment names), dispatch priority and placement, agents, actions, queries, model downloads. |
|
||||
| EXIF enrichment behind identification | in progress 2026-10-08 (Fable, SPAC-47); one PR against `main` | Fable | Follow-up to SPAC-44's `records.exif`, which parsed every photo on demand. `facet_image` is now written by an `exif_enrich` job (`core/src/ops/indexing/exif_enrich.rs`) that every content identification pass dispatches as it finishes, at low priority and silent when the pass was a background one: it claims image records (built-in and extension kinds whose parent is `Image`) whose facet row is missing or names other bytes, parses four at a time on the blocking pool through `sd-media-metadata` (raw and TIFF containers are read whole), and writes one row per content hash to every record holding those bytes (capture time, GPS, camera and lens, focal length, shutter, ISO, orientation, color, geometry, artist, copyright, description; aperture stays empty because the reader has no FNumber). The row carries `content_hash` (new facet column, added to existing stores on open), so a changed file re-queues itself once identification gives it a new hash and nothing else ever re-reads it; a photo without EXIF gets an empty hashed row. A source with `identify_content` off is never enriched. The launch identification sweep backfills stores from before this pass. `records.exif` answers from any hashed row, `null` included, and logs when it has to parse. Tests: mapping unit tests, `crates/store/tests/image.rs`, `core/tests/exif_enrich_acceptance_test.rs` (acceptance group), and the WASM photos suite asserts `create_moments` never parses on demand. Not done: a capture-time sort or filter in `files.list` and search, because no query path joins a facet today (`File.image_media_data` is always `None`); that is a query-layer change of its own. |
|
||||
| Capture time and place in listings and search | in progress 2026-10-09 (Fable, SPAC-51); one PR against `main` | Fable | The query-layer half SPAC-47 left. `search.files` sorts by `CapturedAt` and filters by a `CapturedAt` date range and `has_location`; `files.directory_listing` sorts by `datetaken`; `files.media_listing`'s `datetaken` now reads the facet (birth time stands in for an undated photo). The arena carries nothing of it. Rather than a capture slot in the arena and its snapshot, the arena path reads the facet from the store ahead of the sort and the filter (`core/src/ops/search/capture.rs`, the row 23 pattern a tag filter already uses), and the store path joins `facet_image` into every entry read in `sd_store::read`, guarded by the row's content hash so a rewritten photo reads as unread until the pass catches up; both hand the same `File` to one predicate and one order, which keeps R8 row 19 true. `File.image_media_data` is filled from the facet on every path (width and height optional now), so the inspector shows the capture time and the coordinates. Explorer: Date Taken in every view's sort menu, an order kept across view changes when the view offers it. Tests: store read test for the guarded join, capture predicate and order unit tests, row 19 extended with facet rows, a new R8 row 29 (`capture_facts_read_the_same_through_arena_and_store`) on the SPAC-47 EXIF fixture through both backends, `bun test` for the sort menu state. |
|
||||
| Extension file kinds and previews | K1 and K2 merged 2026-10-08 (Fable, SPAC-35, 0e53403); K3 to K5 in PR 2026-10-08 (Fable, SPAC-36), one PR against `main` | Fable | Plan: `docs/plans/2026-10-07-extension-kinds-and-previews.md`. Product sequence step 5, second half, and SPAC-28 gap 9. Core (K1, K2): `manifest.json` `kinds` (name, built-in parent, extensions, MIME, magic, preview; `core/src/filetype/kinds.rs`), validated at load and refused when a claim would redefine a built-in extension; `FileTypeRegistry::current()` is the built-in table plus every loaded extension's kinds in load order, swapped by the plugin manager on load and unload; conflicts resolve by sorted load order and `extensions.list` reports `kinds` and `conflicts`; store schema v2 adds `content.kind_name` and `idx_facet_file_extension`; the content identity phase writes `kind` and `kind_name` and checks extension-kind magic bytes; rows identified earlier are named when an extension loads and when a store opens; `File.content_kind_name` reads from the row. Client and route (K3 to K5): `renderers.ts` is the preview registry (built-ins register at import; `resolvePreview` matches `content_kind_name` against `useExtensionKinds`, fed by `extensions.list`, then the parent kind; an unloaded extension's name falls through to the parent's renderer), `ContentRenderer` is that lookup, `getContentKindName` labels the inspector and the file kinds page (kind stats now group by `kind_name`); `ui_manifest.json` `file_viewers` (`id`, `bundle`) is read at load by `sd-extension-ui` (new crate, shared with both HTTP servers), a kind's `preview.viewer` must name one, `extensions.list` reports `viewers`, and `/extension/:id/*path` on sd-server and the desktop app's local server serves a declared bundle and nothing else; `BundleRenderer` imports it and calls `mount(el, ctx)`, falling back to the parent renderer with one warning when the bundle is missing or throws. Fixtures: `extensions/test-extension/ui/fake-viewer.js` behind the `fake` kind; photos' `ui_manifest.json` names `photo_viewer` and `ui/photo_viewer.js`, `raw` stays on the image renderer. Jamie decides: kinds one level deep; conflicts by load order, no picker yet; whether `raw` should move to the photos viewer bundle. |
|
||||
| MCP server over the registry | PR open 2026-10-07 (Fable, SPAC-33); one PR against `main` | Fable | Product sequence step 2, second half. `sd mcp` speaks MCP over stdio with rmcp and exposes every registered query, action, validate and preview as a tool named after the op, with the input schema walked from the op's specta input type (`core/src/infra/wire/json_schema.rs`), `library_id` on library ops defaulting to the CLI's selection when the daemon still lists it, else the first library, read-only annotations on queries, daemon errors as tool errors, and a synthetic `jobs.wait` that polls `jobs.info` to a terminal status. The server holds no state and no auth beyond the daemon socket. `apps/cli/tests/mcp_test.rs` (acceptance group) starts a core and RPC server in-process, spawns `sd-cli mcp`, handshakes, lists tools, calls `libraries.list` and `jobs.list`, tracks a source and waits for its walk. Docs: `docs/cli/mcp.mdx` (Claude Code, Cursor, Capy configs) and the CLI skill. Next: try it from Claude Code against the MacBook daemon; decide whether actions should require a confirmed preview. |
|
||||
| Repository release gates in CI | in progress 2026-10-07 (Fable, SPAC-26); one PR against `main` | Fable | `docs/core/releases.mdx` lists five repository gates; only formatting, clippy, typecheck and tests had jobs. `ci.yml` gains four more on every PR to `main`: Docs (bun 1.3.4, `docs/` frozen lockfile, `types:check` and `next build`), Generated types (`generate_typescript_types` plus `cargo check -p spacedrive` for the Tauri schemas, fails on a diff under `packages/ts-client/src/generated` or `apps/tauri/src-tauri/gen/schemas`, own rust-cache key), Secrets scan (pinned gitleaks over the PR's commits and the tree, allowlist in `.gitleaks.toml` including the AWS docs example key in `cloud_credentials.rs`), and License and notices (LICENSE is Apache-2.0, NOTICE exists and carries the same AGPL-3.0 cut-over date as README). Found and fixed on the way: the docs build failed on an unescaped `<name>` in `cli/library-sync-setup.mdx`, and the committed Tauri schemas predated the updater plugin. The gates in `releases.mdx` now name their job. Next: confirm the four jobs green on the PR and the warm wall time under 10 min. |
|
||||
|
||||
@@ -92,9 +92,10 @@ pub async fn run(ctx: &Context, cmd: FileCmd) -> Result<()> {
|
||||
"modified" => sd_core::ops::files::query::DirectorySortBy::Modified,
|
||||
"size" => sd_core::ops::files::query::DirectorySortBy::Size,
|
||||
"type" => sd_core::ops::files::query::DirectorySortBy::Type,
|
||||
"datetaken" => sd_core::ops::files::query::DirectorySortBy::DateTaken,
|
||||
_ => {
|
||||
anyhow::bail!(
|
||||
"Invalid sort option: {}. Valid options are: name, modified, size, type",
|
||||
"Invalid sort option: {}. Valid options are: name, modified, size, type, datetaken",
|
||||
args.sort_by
|
||||
);
|
||||
}
|
||||
|
||||
@@ -80,6 +80,10 @@ pub struct FileSearchArgs {
|
||||
/// Include archived files
|
||||
#[arg(long)]
|
||||
pub include_archived: bool,
|
||||
|
||||
/// Only photos whose EXIF places them (true) or does not (false)
|
||||
#[arg(long)]
|
||||
pub has_location: Option<bool>,
|
||||
}
|
||||
|
||||
#[derive(clap::ValueEnum, Debug, Clone)]
|
||||
@@ -94,6 +98,8 @@ pub enum DateFieldArg {
|
||||
Created,
|
||||
Modified,
|
||||
Accessed,
|
||||
/// When the photo was taken, from its EXIF
|
||||
Captured,
|
||||
}
|
||||
|
||||
#[derive(clap::ValueEnum, Debug, Clone)]
|
||||
@@ -124,6 +130,8 @@ pub enum SortFieldArg {
|
||||
Size,
|
||||
Modified,
|
||||
Created,
|
||||
/// When the photo was taken, from its EXIF
|
||||
Captured,
|
||||
}
|
||||
|
||||
#[derive(clap::ValueEnum, Debug, Clone)]
|
||||
@@ -172,6 +180,7 @@ impl From<FileSearchArgs> for FileSearchInput {
|
||||
DateFieldArg::Created => DateField::CreatedAt,
|
||||
DateFieldArg::Modified => DateField::ModifiedAt,
|
||||
DateFieldArg::Accessed => DateField::AccessedAt,
|
||||
DateFieldArg::Captured => DateField::CapturedAt,
|
||||
},
|
||||
start: args.date_start,
|
||||
end: args.date_end,
|
||||
@@ -213,6 +222,7 @@ impl From<FileSearchArgs> for FileSearchInput {
|
||||
}),
|
||||
include_hidden: Some(args.include_hidden),
|
||||
include_archived: Some(args.include_archived),
|
||||
has_location: args.has_location,
|
||||
at_risk: None,
|
||||
on_volumes: None,
|
||||
not_on_volumes: None,
|
||||
@@ -227,6 +237,7 @@ impl From<FileSearchArgs> for FileSearchInput {
|
||||
SortFieldArg::Size => SortField::Size,
|
||||
SortFieldArg::Modified => SortField::ModifiedAt,
|
||||
SortFieldArg::Created => SortField::CreatedAt,
|
||||
SortFieldArg::Captured => SortField::CapturedAt,
|
||||
},
|
||||
direction: match args.sort_direction {
|
||||
SortDirectionArg::Asc => SortDirection::Asc,
|
||||
|
||||
+19
-1
@@ -143,6 +143,20 @@ impl crate::domain::resource::Identifiable for File {
|
||||
}
|
||||
|
||||
impl File {
|
||||
/// When the photo was taken, as its EXIF says. Only the image facet
|
||||
/// knows it, so a file listed from an arena has it once the listing
|
||||
/// decorates from the store, and a file without EXIF never does.
|
||||
pub fn captured_at(&self) -> Option<DateTime<Utc>> {
|
||||
self.image_media_data.as_ref()?.date_taken
|
||||
}
|
||||
|
||||
/// Whether the photo's EXIF places it: both coordinates present.
|
||||
pub fn has_location(&self) -> bool {
|
||||
self.image_media_data
|
||||
.as_ref()
|
||||
.is_some_and(|image| image.latitude.is_some() && image.longitude.is_some())
|
||||
}
|
||||
|
||||
/// The file at this path, as the volume index has it.
|
||||
///
|
||||
/// The one place a `File` comes from now: resolve the partition, read the
|
||||
@@ -323,7 +337,10 @@ impl File {
|
||||
alternate_paths: Vec::new(),
|
||||
tags: Vec::new(),
|
||||
sidecars: Vec::new(),
|
||||
image_media_data: None,
|
||||
image_media_data: entry
|
||||
.image
|
||||
.as_ref()
|
||||
.and_then(|facet| ImageMediaData::from_facet(entry.uuid, facet)),
|
||||
video_media_data: None,
|
||||
audio_media_data: None,
|
||||
created_at: from_ms(entry.created_ms).unwrap_or_else(Utc::now),
|
||||
@@ -537,6 +554,7 @@ mod tests {
|
||||
content_kind,
|
||||
content_kind_name: None,
|
||||
content_error: None,
|
||||
image: None,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -11,8 +11,8 @@ use crate::infra::db::entities::*;
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, Type)]
|
||||
pub struct ImageMediaData {
|
||||
pub uuid: Uuid,
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
pub width: Option<u32>,
|
||||
pub height: Option<u32>,
|
||||
pub blurhash: Option<String>,
|
||||
pub date_taken: Option<DateTime<Utc>>,
|
||||
pub latitude: Option<f64>,
|
||||
@@ -33,6 +33,45 @@ pub struct ImageMediaData {
|
||||
pub description: Option<String>,
|
||||
}
|
||||
|
||||
impl ImageMediaData {
|
||||
/// What a record's image facet says, under the record's uuid. `None`
|
||||
/// for the empty row the EXIF pass writes when the bytes carried no
|
||||
/// EXIF, so a file without metadata shows none rather than a blank card.
|
||||
pub fn from_facet(record: Uuid, facet: &sd_store::ImageFacet) -> Option<Self> {
|
||||
if *facet == sd_store::ImageFacet::default() {
|
||||
return None;
|
||||
}
|
||||
let as_u32 = |value: Option<i64>| value.and_then(|v| u32::try_from(v).ok());
|
||||
Some(Self {
|
||||
uuid: record,
|
||||
width: as_u32(facet.width),
|
||||
height: as_u32(facet.height),
|
||||
blurhash: None,
|
||||
date_taken: facet
|
||||
.date_taken
|
||||
.as_deref()
|
||||
.and_then(|date| DateTime::parse_from_rfc3339(date).ok())
|
||||
.map(|date| date.with_timezone(&Utc)),
|
||||
latitude: facet.latitude,
|
||||
longitude: facet.longitude,
|
||||
camera_make: facet.camera_make.clone(),
|
||||
camera_model: facet.camera_model.clone(),
|
||||
lens_model: facet.lens_model.clone(),
|
||||
focal_length: facet.focal_length.clone(),
|
||||
aperture: facet.aperture.clone(),
|
||||
shutter_speed: facet.shutter_speed.clone(),
|
||||
iso: as_u32(facet.iso),
|
||||
orientation: facet.orientation.and_then(|o| u8::try_from(o).ok()),
|
||||
color_space: facet.color_space.clone(),
|
||||
color_profile: facet.color_profile.clone(),
|
||||
bit_depth: facet.bit_depth.clone(),
|
||||
artist: facet.artist.clone(),
|
||||
copyright: facet.copyright.clone(),
|
||||
description: facet.description.clone(),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Video metadata extracted from FFmpeg
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, Type)]
|
||||
pub struct VideoMediaData {
|
||||
|
||||
@@ -55,6 +55,9 @@ pub enum DirectorySortBy {
|
||||
Size,
|
||||
/// Sort by type (directories first, then files)
|
||||
Type,
|
||||
/// Sort by capture time (newest first), from the image facet. Files
|
||||
/// without one come last.
|
||||
DateTaken,
|
||||
}
|
||||
|
||||
/// Output containing directory contents
|
||||
@@ -263,10 +266,20 @@ impl DirectoryListingQuery {
|
||||
SdPath::Physical { device_slug, .. } => device_slug.clone(),
|
||||
_ => String::new(),
|
||||
};
|
||||
let files = self.files_from_index(&index, children, &device_slug).await;
|
||||
// Decorate after the cap: assertions live only in
|
||||
// SQLite, so the arena's page still reads the store.
|
||||
let mut files = self.files_from_index(&index, children, &device_slug).await;
|
||||
// Decorate after the cap: assertions and facets live
|
||||
// only in SQLite, so the arena's page still reads the
|
||||
// store. Only a capture order needs the facet ahead of
|
||||
// the sort, at the directory's cost rather than the page's.
|
||||
let by_capture = matches!(self.input.sort_by, DirectorySortBy::DateTaken);
|
||||
if by_capture {
|
||||
crate::ops::search::capture::decorate_files(&cache, &mut files).await;
|
||||
}
|
||||
let mut listing = self.finalize_listing(files);
|
||||
if !by_capture {
|
||||
crate::ops::search::capture::decorate_files(&cache, &mut listing.files)
|
||||
.await;
|
||||
}
|
||||
crate::ops::tags::decorate::decorate_files(&cache, &mut listing.files).await;
|
||||
crate::ops::indexing::kinds::decorate_kinds(&cache, &mut listing.files).await;
|
||||
return Ok(listing);
|
||||
@@ -719,6 +732,13 @@ impl DirectoryListingQuery {
|
||||
DirectorySortBy::Name => a.name.to_lowercase().cmp(&b.name.to_lowercase()),
|
||||
DirectorySortBy::Modified => b.modified_at.cmp(&a.modified_at),
|
||||
DirectorySortBy::Size => b.size.cmp(&a.size),
|
||||
// Dated files newest first, then the undated ones by name.
|
||||
DirectorySortBy::DateTaken => match (a.captured_at(), b.captured_at()) {
|
||||
(Some(a), Some(b)) => b.cmp(&a),
|
||||
(Some(_), None) => std::cmp::Ordering::Less,
|
||||
(None, Some(_)) => std::cmp::Ordering::Greater,
|
||||
(None, None) => a.name.to_lowercase().cmp(&b.name.to_lowercase()),
|
||||
},
|
||||
DirectorySortBy::Type => {
|
||||
// Sort by kind (directories first), then name
|
||||
if !folders_first {
|
||||
|
||||
@@ -174,6 +174,9 @@ impl LibraryQuery for MediaListingQuery {
|
||||
}
|
||||
drop(index);
|
||||
|
||||
if matches!(self.input.sort_by, MediaSortBy::DateTaken) {
|
||||
crate::ops::search::capture::decorate_files(cache, &mut files).await;
|
||||
}
|
||||
sort_media(&mut files, &self.input.sort_by);
|
||||
|
||||
let total_count = files.len() as u32;
|
||||
@@ -193,11 +196,14 @@ impl LibraryQuery for MediaListingQuery {
|
||||
fn sort_media(files: &mut [File], sort_by: &MediaSortBy) {
|
||||
match sort_by {
|
||||
MediaSortBy::Modified => files.sort_by(|a, b| b.modified_at.cmp(&a.modified_at)),
|
||||
// The arena carries no capture time; birth time is the closest thing
|
||||
// it knows, and it is the one a photo library sorts by.
|
||||
MediaSortBy::Created | MediaSortBy::DateTaken => {
|
||||
files.sort_by(|a, b| b.created_at.cmp(&a.created_at))
|
||||
}
|
||||
MediaSortBy::Created => files.sort_by(|a, b| b.created_at.cmp(&a.created_at)),
|
||||
// A photo without a capture time still belongs in the roll, so its
|
||||
// birth time stands in rather than sending it to the end.
|
||||
MediaSortBy::DateTaken => files.sort_by(|a, b| {
|
||||
b.captured_at()
|
||||
.unwrap_or(b.created_at)
|
||||
.cmp(&a.captured_at().unwrap_or(a.created_at))
|
||||
}),
|
||||
MediaSortBy::Name => {
|
||||
files.sort_by(|a, b| a.name.to_lowercase().cmp(&b.name.to_lowercase()))
|
||||
}
|
||||
|
||||
@@ -79,7 +79,7 @@ pub async fn search_arena(
|
||||
let index = share.index.read().await;
|
||||
matches_in(&index, query, Some(local_path), filters)
|
||||
};
|
||||
let results = collect_results(
|
||||
let mut results = collect_results(
|
||||
&share.index,
|
||||
matching_paths,
|
||||
query,
|
||||
@@ -88,6 +88,9 @@ pub async fn search_arena(
|
||||
file_type_registry,
|
||||
)
|
||||
.await?;
|
||||
// A replica has no store here, so its hits read as undated and
|
||||
// unplaced rather than passing a capture filter unjudged.
|
||||
crate::ops::search::capture::retain_matching(&mut results, filters);
|
||||
return Ok(SearchPage::single_partition(results, sort, pagination));
|
||||
}
|
||||
|
||||
@@ -137,7 +140,7 @@ pub async fn search_arena(
|
||||
matching_paths.retain(|path| scope.admits(path));
|
||||
}
|
||||
|
||||
let results = collect_results(
|
||||
let mut results = collect_results(
|
||||
&index_arc,
|
||||
matching_paths,
|
||||
query,
|
||||
@@ -146,6 +149,7 @@ pub async fn search_arena(
|
||||
file_type_registry,
|
||||
)
|
||||
.await?;
|
||||
crate::ops::search::capture::apply_to_arena_results(cache, &mut results, filters, sort).await;
|
||||
Ok(SearchPage::single_partition(results, sort, pagination))
|
||||
}
|
||||
|
||||
@@ -185,6 +189,7 @@ async fn store_scoped_page(
|
||||
)
|
||||
.await?;
|
||||
retain_tagged(&mut partition.results, tag_scope);
|
||||
crate::ops::search::capture::retain_matching(&mut partition.results, filters);
|
||||
|
||||
Ok(Some(SearchPage::single_partition_with(
|
||||
partition.results,
|
||||
@@ -278,6 +283,8 @@ pub async fn search_every_index(
|
||||
file_type_registry,
|
||||
)
|
||||
.await?;
|
||||
crate::ops::search::capture::apply_to_arena_results(cache, &mut partition, filters, sort)
|
||||
.await;
|
||||
total += partition.len() as u64;
|
||||
facets.absorb(&partition);
|
||||
seen.extend(
|
||||
@@ -320,6 +327,7 @@ pub async fn search_every_index(
|
||||
file_type_registry,
|
||||
)
|
||||
.await?;
|
||||
crate::ops::search::capture::retain_matching(&mut partition, filters);
|
||||
total += partition.len() as u64;
|
||||
facets.absorb(&partition);
|
||||
pipeline::narrow(&mut partition, sort, window);
|
||||
@@ -359,6 +367,7 @@ pub async fn search_every_index(
|
||||
.await?;
|
||||
let mut partition = store_partition.results;
|
||||
retain_tagged(&mut partition, tag_scope.as_ref());
|
||||
crate::ops::search::capture::retain_matching(&mut partition, filters);
|
||||
partition.retain(|result| match result.file.sd_path.as_local_path() {
|
||||
Some(path) => seen.insert(path.to_path_buf()),
|
||||
None => true,
|
||||
@@ -537,15 +546,20 @@ pub(super) fn passes_arena_filters(
|
||||
// it: an entry that cannot prove it is in the range is not in the range.
|
||||
// The index holds no access or indexed-at times, so those fields fail
|
||||
// closed until a backend can answer them; the UI offers only the fields
|
||||
// advertised as answerable.
|
||||
if let Some(ref range) = filters.date_range {
|
||||
// advertised as answerable. A capture range is judged after the facet
|
||||
// is read, by `capture::passes`.
|
||||
if let Some(ref range) = filters
|
||||
.date_range
|
||||
.as_ref()
|
||||
.filter(|range| range.field != DateField::CapturedAt)
|
||||
{
|
||||
use chrono::{DateTime, Utc};
|
||||
|
||||
let system_time_opt = match range.field {
|
||||
DateField::ModifiedAt => metadata.modified,
|
||||
DateField::CreatedAt => metadata.created,
|
||||
DateField::AccessedAt => metadata.accessed,
|
||||
DateField::IndexedAt => None,
|
||||
DateField::IndexedAt | DateField::CapturedAt => None,
|
||||
};
|
||||
|
||||
let Some(system_time) = system_time_opt else {
|
||||
@@ -576,8 +590,9 @@ pub(super) fn passes_arena_filters(
|
||||
}
|
||||
}
|
||||
|
||||
// Tags have no arena representation, so a tag filter is ignored rather
|
||||
// than silently excluding everything.
|
||||
// Tags and capture facts have no arena representation; a tag filter
|
||||
// resolves its paths from the stores and a capture filter runs on the
|
||||
// decorated results, so neither is judged here.
|
||||
|
||||
true
|
||||
}
|
||||
@@ -735,33 +750,6 @@ mod tests {
|
||||
];
|
||||
let mtime_secs = 1_700_000_000u64;
|
||||
|
||||
// The arena's copy.
|
||||
let mut index = crate::ops::indexing::Arena::new().expect("index");
|
||||
for (name, size, hidden) in fixture {
|
||||
let path = root.join(name);
|
||||
index
|
||||
.add_entry(
|
||||
path.clone(),
|
||||
Uuid::now_v7(),
|
||||
EntryMetadata {
|
||||
path,
|
||||
kind: EntryKind::File,
|
||||
size: *size,
|
||||
modified: Some(UNIX_EPOCH + Duration::from_secs(mtime_secs)),
|
||||
accessed: None,
|
||||
created: None,
|
||||
inode: None,
|
||||
permissions: None,
|
||||
uid: None,
|
||||
gid: None,
|
||||
link_target: None,
|
||||
is_hidden: *hidden,
|
||||
},
|
||||
)
|
||||
.expect("entry");
|
||||
}
|
||||
let index_arc = std::sync::Arc::new(tokio::sync::RwLock::new(index));
|
||||
|
||||
// The store's copy of the same capture.
|
||||
let dir = tempfile::tempdir().expect("tempdir");
|
||||
let manager = sd_store::SourceManager::new(dir.path().to_path_buf());
|
||||
@@ -803,20 +791,98 @@ mod tests {
|
||||
db.apply_files(&writes, &[], &[], None)
|
||||
.await
|
||||
.expect("apply");
|
||||
|
||||
// The EXIF pass has read two of the clips: one dated and placed, one
|
||||
// dated only. The facet is keyed by content, so each gets a hash.
|
||||
for (name, hash, date, place) in [
|
||||
(
|
||||
"Clip One.MOV",
|
||||
"clip-one",
|
||||
"2024-03-12T10:00:00+00:00",
|
||||
Some((35.6812, 139.7671)),
|
||||
),
|
||||
("ÉLITE.mov", "elite", "2024-03-19T09:00:00+00:00", None),
|
||||
] {
|
||||
let record = sd_store::read::entry_by_path(db.pool(), name)
|
||||
.await
|
||||
.expect("lookup")
|
||||
.expect("written")
|
||||
.uuid;
|
||||
db.set_content_identity(
|
||||
record,
|
||||
&sd_store::ContentIdentity {
|
||||
sampled_hash: Some(hash.to_string()),
|
||||
..Default::default()
|
||||
},
|
||||
)
|
||||
.await
|
||||
.expect("identity");
|
||||
sd_store::set_image_facets(
|
||||
db.pool(),
|
||||
&[(
|
||||
hash.to_string(),
|
||||
sd_store::ImageFacet {
|
||||
date_taken: Some(date.to_string()),
|
||||
latitude: place.map(|(lat, _)| lat),
|
||||
longitude: place.map(|(_, lon)| lon),
|
||||
..Default::default()
|
||||
},
|
||||
)],
|
||||
)
|
||||
.await
|
||||
.expect("facet");
|
||||
}
|
||||
|
||||
// The arena's copy, under the identities the store assigned, which
|
||||
// is what the ledger gives a walked source.
|
||||
let mut index = crate::ops::indexing::Arena::new().expect("index");
|
||||
for (name, size, hidden) in fixture {
|
||||
let path = root.join(name);
|
||||
let uuid = sd_store::read::entry_by_path(db.pool(), name)
|
||||
.await
|
||||
.expect("lookup")
|
||||
.expect("written")
|
||||
.uuid;
|
||||
index
|
||||
.add_entry(
|
||||
path.clone(),
|
||||
uuid,
|
||||
EntryMetadata {
|
||||
path,
|
||||
kind: EntryKind::File,
|
||||
size: *size,
|
||||
modified: Some(UNIX_EPOCH + Duration::from_secs(mtime_secs)),
|
||||
accessed: None,
|
||||
created: None,
|
||||
inode: None,
|
||||
permissions: None,
|
||||
uid: None,
|
||||
gid: None,
|
||||
link_target: None,
|
||||
is_hidden: *hidden,
|
||||
},
|
||||
)
|
||||
.expect("entry");
|
||||
}
|
||||
let index_arc = std::sync::Arc::new(tokio::sync::RwLock::new(index));
|
||||
drop(db);
|
||||
let db = manager.open_read_only("src-1").await.expect("read-only");
|
||||
|
||||
let registry = FileTypeRegistry::new();
|
||||
type Seen = (String, u64, f32, Option<String>, bool);
|
||||
let seen = |r: &FileSearchResult| -> Seen {
|
||||
(
|
||||
r.file.name.clone(),
|
||||
r.file.size,
|
||||
r.score,
|
||||
r.file.captured_at().map(|t| t.to_rfc3339()),
|
||||
r.file.has_location(),
|
||||
)
|
||||
};
|
||||
let compare = |arena: Vec<FileSearchResult>,
|
||||
store: Vec<crate::ops::search::output::FileSearchResult>| {
|
||||
let mut arena: Vec<(String, u64, f32)> = arena
|
||||
.into_iter()
|
||||
.map(|r| (r.file.name.clone(), r.file.size, r.score))
|
||||
.collect();
|
||||
let mut store: Vec<(String, u64, f32)> = store
|
||||
.into_iter()
|
||||
.map(|r| (r.file.name.clone(), r.file.size, r.score))
|
||||
.collect();
|
||||
let mut arena: Vec<Seen> = arena.iter().map(seen).collect();
|
||||
let mut store: Vec<Seen> = store.iter().map(seen).collect();
|
||||
arena.sort_by(|a, b| a.0.cmp(&b.0));
|
||||
store.sort_by(|a, b| a.0.cmp(&b.0));
|
||||
assert_eq!(arena, store);
|
||||
@@ -833,16 +899,29 @@ mod tests {
|
||||
let index = index_arc.read().await;
|
||||
matches_in(&index, query, None, &filters)
|
||||
};
|
||||
let arena = collect_results(&index_arc, matching, query, "dev", &filters, registry)
|
||||
let mut arena =
|
||||
collect_results(&index_arc, matching, query, "dev", &filters, registry)
|
||||
.await
|
||||
.expect("arena results");
|
||||
// What search_arena does with a cache: read the facet from
|
||||
// the store, then judge the capture filters on both alike.
|
||||
let mut files: Vec<File> = arena.iter().map(|r| r.file.clone()).collect();
|
||||
let positions: Vec<usize> = (0..files.len()).collect();
|
||||
crate::ops::search::capture::decorate_from_db(db, &mut files, &positions)
|
||||
.await
|
||||
.expect("arena results");
|
||||
.expect("decorated");
|
||||
for (result, file) in arena.iter_mut().zip(files) {
|
||||
result.file.image_media_data = file.image_media_data;
|
||||
}
|
||||
crate::ops::search::capture::retain_matching(&mut arena, &filters);
|
||||
|
||||
let store = crate::ops::search::store_search::search_source_store(
|
||||
let mut store = crate::ops::search::store_search::search_source_store(
|
||||
db, &root, "dev", query, None, &filters, registry,
|
||||
)
|
||||
.await
|
||||
.expect("store results");
|
||||
assert!(!store.truncated);
|
||||
crate::ops::search::capture::retain_matching(&mut store.results, &filters);
|
||||
(arena, store.results)
|
||||
}
|
||||
};
|
||||
@@ -893,6 +972,63 @@ mod tests {
|
||||
// Without one, an empty query matches nothing on either.
|
||||
let (arena, store) = run("", SearchFilters::default()).await;
|
||||
assert!(compare(arena, store).is_empty());
|
||||
|
||||
// Capture facts come from the facet on both backends: the arena
|
||||
// reads them from the store before it judges, the store joins them.
|
||||
let (arena, store) = run("mov", SearchFilters::default()).await;
|
||||
let matched = compare(arena, store);
|
||||
assert_eq!(
|
||||
matched
|
||||
.iter()
|
||||
.map(|(name, _, _, taken, placed)| (name.as_str(), taken.is_some(), *placed))
|
||||
.collect::<Vec<_>>(),
|
||||
vec![("Clip One", true, true), ("ÉLITE", true, false)]
|
||||
);
|
||||
|
||||
let placed = SearchFilters {
|
||||
has_location: Some(true),
|
||||
..Default::default()
|
||||
};
|
||||
let (arena, store) = run("mov", placed).await;
|
||||
assert_eq!(compare(arena, store).len(), 1);
|
||||
|
||||
let late_march = SearchFilters {
|
||||
date_range: Some(DateRangeFilter {
|
||||
field: DateField::CapturedAt,
|
||||
start: Some(chrono::DateTime::from_timestamp(1_710_720_000, 0).unwrap()),
|
||||
end: None,
|
||||
}),
|
||||
..Default::default()
|
||||
};
|
||||
let (arena, store) = run("mov", late_march).await;
|
||||
let matched = compare(arena, store);
|
||||
assert_eq!(matched.len(), 1);
|
||||
assert_eq!(matched[0].0, "ÉLITE");
|
||||
|
||||
// The order is shared too: the same candidates, sorted by capture
|
||||
// time through the one pipeline, page identically.
|
||||
let by_capture = SortOptions {
|
||||
field: crate::ops::search::input::SortField::CapturedAt,
|
||||
direction: crate::ops::search::input::SortDirection::Desc,
|
||||
};
|
||||
let (mut arena, mut store) = run(
|
||||
"",
|
||||
SearchFilters {
|
||||
file_types: Some(vec!["mov".to_string(), "txt".to_string()]),
|
||||
..Default::default()
|
||||
},
|
||||
)
|
||||
.await;
|
||||
pipeline::order(&mut arena, &by_capture);
|
||||
pipeline::order(&mut store, &by_capture);
|
||||
let order = |results: &[FileSearchResult]| {
|
||||
results
|
||||
.iter()
|
||||
.map(|r| r.file.name.clone())
|
||||
.collect::<Vec<_>>()
|
||||
};
|
||||
assert_eq!(order(&arena), ["ÉLITE", "Clip One", "notes"]);
|
||||
assert_eq!(order(&arena), order(&store));
|
||||
}
|
||||
|
||||
/// A filter-only search in a folder reaches every entry beneath it, and
|
||||
|
||||
@@ -0,0 +1,257 @@
|
||||
//! Capture time and place, which only the image facet knows.
|
||||
//!
|
||||
//! The arena holds what a walk sees: names, sizes and filesystem times. When
|
||||
//! a photo was taken and where sit in `facet_image`, written by the EXIF
|
||||
//! pass after identification, so a listing or search served from an arena
|
||||
//! reads them from the store before it sorts, filters or pages, the way a
|
||||
//! tag filter resolves its paths from the stores first. A store-served row
|
||||
//! carries its facet from the join in `sd_store::read`, so both backends
|
||||
//! hand the same `File` to the one predicate and the one order below.
|
||||
//!
|
||||
//! A file with no facet has no capture time and no place: it fails a
|
||||
//! capture-date range and `has_location: true`, and trails every dated file
|
||||
//! in a capture order. A replica of another device's source has no store
|
||||
//! here and reads the same way.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::Arc;
|
||||
|
||||
use uuid::Uuid;
|
||||
|
||||
use super::input::{DateField, SearchFilters, SortField, SortOptions};
|
||||
use super::output::FileSearchResult;
|
||||
use crate::domain::{EntryKind, File, ImageMediaData};
|
||||
use crate::ops::indexing::{SourceStore, VolumeIndex};
|
||||
|
||||
/// Whether the request sorts or filters by something only the facet holds,
|
||||
/// which is what makes the decoration a step before the sort rather than a
|
||||
/// finish on the page.
|
||||
pub fn wanted(filters: &SearchFilters, sort: &SortOptions) -> bool {
|
||||
sort.field == SortField::CapturedAt || filtered(filters)
|
||||
}
|
||||
|
||||
/// Whether a filter only the facet can judge is set.
|
||||
pub fn filtered(filters: &SearchFilters) -> bool {
|
||||
filters.has_location.is_some()
|
||||
|| filters
|
||||
.date_range
|
||||
.as_ref()
|
||||
.is_some_and(|range| range.field == DateField::CapturedAt)
|
||||
}
|
||||
|
||||
/// The capture half of the filter set, judged from the `File` whichever
|
||||
/// backend built it. The per-row predicates leave `CapturedAt` and
|
||||
/// `has_location` to this one.
|
||||
pub fn passes(file: &File, filters: &SearchFilters) -> bool {
|
||||
if let Some(wanted) = filters.has_location {
|
||||
if file.has_location() != wanted {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if let Some(range) = &filters.date_range {
|
||||
if range.field == DateField::CapturedAt {
|
||||
let Some(taken) = file.captured_at() else {
|
||||
return false;
|
||||
};
|
||||
if range.start.is_some_and(|start| taken < start) {
|
||||
return false;
|
||||
}
|
||||
if range.end.is_some_and(|end| taken > end) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
/// Keep the results the capture filters admit.
|
||||
pub fn retain_matching(results: &mut Vec<FileSearchResult>, filters: &SearchFilters) {
|
||||
if filtered(filters) {
|
||||
results.retain(|result| passes(&result.file, filters));
|
||||
}
|
||||
}
|
||||
|
||||
/// Enrich arena-served results from their stores when the request needs
|
||||
/// it, then apply the capture filters. Runs on a partition's full match set,
|
||||
/// ahead of the page window, so the sort and the total see every row.
|
||||
pub async fn apply_to_arena_results(
|
||||
cache: &VolumeIndex,
|
||||
results: &mut Vec<FileSearchResult>,
|
||||
filters: &SearchFilters,
|
||||
sort: &SortOptions,
|
||||
) {
|
||||
if wanted(filters, sort) {
|
||||
decorate_results(cache, results).await;
|
||||
}
|
||||
retain_matching(results, filters);
|
||||
}
|
||||
|
||||
pub async fn decorate_results(cache: &VolumeIndex, results: &mut [FileSearchResult]) {
|
||||
let mut files: Vec<File> = results.iter().map(|result| result.file.clone()).collect();
|
||||
decorate_files(cache, &mut files).await;
|
||||
for (result, file) in results.iter_mut().zip(files) {
|
||||
result.file.image_media_data = file.image_media_data;
|
||||
}
|
||||
}
|
||||
|
||||
/// Give files listed from an arena the image facet their store rows carry.
|
||||
/// A file that already has one, because its store built it, is left alone.
|
||||
pub async fn decorate_files(cache: &VolumeIndex, files: &mut [File]) {
|
||||
let mut by_store: HashMap<Uuid, (Arc<SourceStore>, Vec<usize>)> = HashMap::new();
|
||||
for (position, file) in files.iter().enumerate() {
|
||||
if file.kind != EntryKind::File || file.image_media_data.is_some() {
|
||||
continue;
|
||||
}
|
||||
let Some(path) = file.sd_path.as_local_path() else {
|
||||
continue;
|
||||
};
|
||||
let Some(store) = cache.store_for(path).await else {
|
||||
continue;
|
||||
};
|
||||
by_store
|
||||
.entry(store.id())
|
||||
.or_insert_with(|| (store.clone(), Vec::new()))
|
||||
.1
|
||||
.push(position);
|
||||
}
|
||||
|
||||
for (_, (store, positions)) in by_store {
|
||||
if let Err(error) = decorate_from_db(store.db(), files, &positions).await {
|
||||
tracing::warn!(source = %store.id(), %error, "image facets unavailable for listing");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Decorate the files at `positions` from one store, by record uuid.
|
||||
pub async fn decorate_from_db(
|
||||
db: &sd_store::SourceDb,
|
||||
files: &mut [File],
|
||||
positions: &[usize],
|
||||
) -> Result<(), sd_store::Error> {
|
||||
let ids: Vec<Uuid> = positions.iter().map(|&p| files[p].id).collect();
|
||||
let by_record: HashMap<Uuid, sd_store::ImageFacet> =
|
||||
sd_store::image_facets_for_records(db.pool(), &ids)
|
||||
.await?
|
||||
.into_iter()
|
||||
.collect();
|
||||
for &position in positions {
|
||||
let file = &mut files[position];
|
||||
if let Some(facet) = by_record.get(&file.id) {
|
||||
file.image_media_data = ImageMediaData::from_facet(file.id, facet);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::domain::SdPath;
|
||||
use crate::ops::search::input::DateRangeFilter;
|
||||
use chrono::{DateTime, Utc};
|
||||
|
||||
fn photo(taken: Option<&str>, place: Option<(f64, f64)>) -> File {
|
||||
let mut file = File::planned(SdPath::local("/vol/p.jpg"), 1, false);
|
||||
if taken.is_some() || place.is_some() {
|
||||
let facet = sd_store::ImageFacet {
|
||||
date_taken: taken.map(String::from),
|
||||
latitude: place.map(|(lat, _)| lat),
|
||||
longitude: place.map(|(_, lon)| lon),
|
||||
..Default::default()
|
||||
};
|
||||
file.image_media_data = ImageMediaData::from_facet(file.id, &facet);
|
||||
}
|
||||
file
|
||||
}
|
||||
|
||||
fn at(rfc3339: &str) -> DateTime<Utc> {
|
||||
DateTime::parse_from_rfc3339(rfc3339)
|
||||
.unwrap()
|
||||
.with_timezone(&Utc)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_file_without_a_facet_has_no_place_and_no_capture_time() {
|
||||
let placed = photo(Some("2024-03-12T10:00:00+00:00"), Some((35.68, 139.77)));
|
||||
let unplaced = photo(Some("2024-03-19T09:00:00+00:00"), None);
|
||||
let bare = photo(None, None);
|
||||
|
||||
let placed_only = SearchFilters {
|
||||
has_location: Some(true),
|
||||
..Default::default()
|
||||
};
|
||||
assert!(passes(&placed, &placed_only));
|
||||
assert!(!passes(&unplaced, &placed_only));
|
||||
assert!(!passes(&bare, &placed_only));
|
||||
|
||||
let unplaced_only = SearchFilters {
|
||||
has_location: Some(false),
|
||||
..Default::default()
|
||||
};
|
||||
assert!(!passes(&placed, &unplaced_only));
|
||||
assert!(passes(&unplaced, &unplaced_only));
|
||||
assert!(passes(&bare, &unplaced_only));
|
||||
}
|
||||
|
||||
/// A capture range is inclusive at both ends and fails closed, like the
|
||||
/// filesystem date filters.
|
||||
#[test]
|
||||
fn a_capture_range_fails_closed_on_undated_files() {
|
||||
let march = SearchFilters {
|
||||
date_range: Some(DateRangeFilter {
|
||||
field: DateField::CapturedAt,
|
||||
start: Some(at("2024-03-12T10:00:00+00:00")),
|
||||
end: Some(at("2024-03-31T00:00:00+00:00")),
|
||||
}),
|
||||
..Default::default()
|
||||
};
|
||||
assert!(passes(
|
||||
&photo(Some("2024-03-12T10:00:00+00:00"), None),
|
||||
&march
|
||||
));
|
||||
assert!(passes(
|
||||
&photo(Some("2024-03-19T09:00:00+00:00"), None),
|
||||
&march
|
||||
));
|
||||
assert!(!passes(
|
||||
&photo(Some("2024-04-01T00:00:00+00:00"), None),
|
||||
&march
|
||||
));
|
||||
assert!(!passes(&photo(None, None), &march));
|
||||
|
||||
// A range on another field is not this predicate's to judge.
|
||||
let modified = SearchFilters {
|
||||
date_range: Some(DateRangeFilter {
|
||||
field: DateField::ModifiedAt,
|
||||
start: Some(at("2030-01-01T00:00:00+00:00")),
|
||||
end: None,
|
||||
}),
|
||||
..Default::default()
|
||||
};
|
||||
assert!(passes(&photo(None, None), &modified));
|
||||
}
|
||||
|
||||
/// Only a capture sort or filter pulls the facet ahead of the sort.
|
||||
#[test]
|
||||
fn the_facet_is_wanted_before_the_sort_only_when_the_request_reads_it() {
|
||||
let by_name = SortOptions {
|
||||
field: SortField::Name,
|
||||
direction: super::super::input::SortDirection::Asc,
|
||||
};
|
||||
assert!(!wanted(&SearchFilters::default(), &by_name));
|
||||
assert!(wanted(
|
||||
&SearchFilters {
|
||||
has_location: Some(false),
|
||||
..Default::default()
|
||||
},
|
||||
&by_name
|
||||
));
|
||||
assert!(wanted(
|
||||
&SearchFilters::default(),
|
||||
&SortOptions {
|
||||
field: SortField::CapturedAt,
|
||||
..by_name
|
||||
}
|
||||
));
|
||||
}
|
||||
}
|
||||
@@ -59,6 +59,10 @@ pub struct SearchFilters {
|
||||
pub content_types: Option<Vec<ContentKind>>,
|
||||
pub include_hidden: Option<bool>,
|
||||
pub include_archived: Option<bool>,
|
||||
/// Only photos whose EXIF places them (true) or does not (false). A
|
||||
/// file with no image facet has no place and fails `true`.
|
||||
#[serde(default)]
|
||||
pub has_location: Option<bool>,
|
||||
|
||||
// Redundancy filters
|
||||
/// Only return files that are at risk (true) or redundant (false).
|
||||
@@ -83,6 +87,7 @@ impl SearchFilters {
|
||||
|| self.content_types.is_some()
|
||||
|| self.size_range.is_some()
|
||||
|| self.date_range.is_some()
|
||||
|| self.has_location.is_some()
|
||||
|| self
|
||||
.tags
|
||||
.as_ref()
|
||||
@@ -114,6 +119,8 @@ pub enum DateField {
|
||||
ModifiedAt,
|
||||
AccessedAt,
|
||||
IndexedAt,
|
||||
/// When the photo was taken, from its image facet.
|
||||
CapturedAt,
|
||||
}
|
||||
|
||||
/// Filter for file size in bytes
|
||||
@@ -139,6 +146,9 @@ pub enum SortField {
|
||||
ModifiedAt,
|
||||
CreatedAt,
|
||||
IndexedAt,
|
||||
/// When the photo was taken, from its image facet. Files without one
|
||||
/// sort last whichever way the order runs.
|
||||
CapturedAt,
|
||||
}
|
||||
|
||||
/// Sort direction
|
||||
|
||||
@@ -7,15 +7,18 @@
|
||||
//! store in the order its indexes already keep, directory path and then name,
|
||||
//! so a page reads its own rows and the next resumes from a cursor.
|
||||
//!
|
||||
//! Extension, kind and hidden narrow in SQL. The name, size, date and tag
|
||||
//! filters are judged per row exactly as the store backend of `search.files`
|
||||
//! judges them, so a search and the media in it answer the same question. Names
|
||||
//! match in Rust for the Unicode folding SQLite does not do.
|
||||
//! Extension, kind and hidden narrow in SQL. The name, size, date, tag and
|
||||
//! capture filters are judged per row exactly as the store backend of
|
||||
//! `search.files` judges them, so a search and the media in it answer the same
|
||||
//! question. Names match in Rust for the Unicode folding SQLite does not do;
|
||||
//! the capture filters read the image facet the row carries.
|
||||
//!
|
||||
//! Sources this device replicates from paired devices page after its own. Most
|
||||
//! arrive as index snapshots with no store beside them, so a replica pages from
|
||||
//! the index this device holds of it, in path order, judged as the arena
|
||||
//! backend of `search.files` judges its rows.
|
||||
//! backend of `search.files` judges its rows. A replica holds no facet, so
|
||||
//! its files read as undated and unplaced under a capture filter, as they do
|
||||
//! in `search.files`.
|
||||
|
||||
use std::path::Path;
|
||||
use std::sync::Arc;
|
||||
@@ -25,6 +28,7 @@ use specta::Type;
|
||||
use uuid::Uuid;
|
||||
|
||||
use super::arena_search::passes_arena_filters;
|
||||
use super::capture;
|
||||
use super::input::{SearchFilters, SearchScope};
|
||||
use super::store_search::passes_store_filters;
|
||||
use super::tag_scope::TagScope;
|
||||
@@ -369,6 +373,11 @@ fn replica_files(
|
||||
},
|
||||
);
|
||||
file.content_kind = content_kind;
|
||||
// No facet here, so the file is undated and unplaced: it fails a
|
||||
// capture range and `has_location: true`, and passes `false`.
|
||||
if !capture::passes(&file, judge.filters) {
|
||||
continue;
|
||||
}
|
||||
page.files.push(file);
|
||||
if page.files.len() == wanted {
|
||||
page.full_at = Some(cursor_at(root, path));
|
||||
@@ -427,7 +436,9 @@ impl Judge<'_> {
|
||||
path: absolute,
|
||||
},
|
||||
);
|
||||
matches!(file.content_kind, ContentKind::Image | ContentKind::Video).then_some(file)
|
||||
(matches!(file.content_kind, ContentKind::Image | ContentKind::Video)
|
||||
&& capture::passes(&file, self.filters))
|
||||
.then_some(file)
|
||||
}
|
||||
|
||||
/// Whether an entry of a replica's index is an image or a video the
|
||||
@@ -536,6 +547,7 @@ mod tests {
|
||||
content_kind: None,
|
||||
content_kind_name: None,
|
||||
content_error: None,
|
||||
image: None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -571,6 +583,36 @@ mod tests {
|
||||
assert!(judge.admit(&entry("trip/.élite.mov", true), root).is_none());
|
||||
}
|
||||
|
||||
/// The capture filters read the facet the row carries, so a row without
|
||||
/// one fails `has_location` and a capture range as a search hit would.
|
||||
#[test]
|
||||
fn a_row_is_judged_by_its_facet_under_a_capture_filter() {
|
||||
let registry = FileTypeRegistry::new();
|
||||
let placed_only = SearchFilters {
|
||||
has_location: Some(true),
|
||||
..Default::default()
|
||||
};
|
||||
let extensions = media_extensions(®istry, &placed_only);
|
||||
let judge = Judge {
|
||||
needle: String::new(),
|
||||
filters: &placed_only,
|
||||
tags: None,
|
||||
registry: ®istry,
|
||||
extensions: &extensions,
|
||||
device_slug: "laptop".to_string(),
|
||||
};
|
||||
let root = Path::new("/vol/kept");
|
||||
|
||||
let mut placed = entry("trip/placed.jpg", false);
|
||||
placed.image = Some(sd_store::ImageFacet {
|
||||
latitude: Some(35.68),
|
||||
longitude: Some(139.77),
|
||||
..Default::default()
|
||||
});
|
||||
assert!(judge.admit(&placed, root).is_some());
|
||||
assert!(judge.admit(&entry("trip/bare.jpg", false), root).is_none());
|
||||
}
|
||||
|
||||
/// A replica's index holding these files beneath `root`.
|
||||
fn replica_index(root: &Path, names: &[&str]) -> Arena {
|
||||
let mut index = Arena::new().expect("index");
|
||||
@@ -647,5 +689,28 @@ mod tests {
|
||||
|
||||
let missing = replica_files(&mut index, root, "absent", "titan", &judge, None, 10);
|
||||
assert!(missing.files.is_empty() && missing.full_at.is_none());
|
||||
|
||||
// A replica holds no facet, so its files are unplaced: all of them
|
||||
// under `has_location: false`, none under `true`, as in search.files.
|
||||
let unplaced = SearchFilters {
|
||||
has_location: Some(false),
|
||||
..Default::default()
|
||||
};
|
||||
let judge = Judge {
|
||||
filters: &unplaced,
|
||||
..judge
|
||||
};
|
||||
let every = replica_files(&mut index, root, "", "titan", &judge, None, 10);
|
||||
assert_eq!(every.files.len(), 3);
|
||||
let placed = SearchFilters {
|
||||
has_location: Some(true),
|
||||
..Default::default()
|
||||
};
|
||||
let judge = Judge {
|
||||
filters: &placed,
|
||||
..judge
|
||||
};
|
||||
let none = replica_files(&mut index, root, "", "titan", &judge, None, 10);
|
||||
assert!(none.files.is_empty());
|
||||
}
|
||||
}
|
||||
@@ -4,6 +4,7 @@ use serde::{Deserialize, Serialize};
|
||||
use specta::Type;
|
||||
|
||||
pub mod arena_search;
|
||||
pub mod capture;
|
||||
pub mod input;
|
||||
pub mod media;
|
||||
pub mod output;
|
||||
|
||||
@@ -29,12 +29,28 @@ pub fn order(results: &mut [FileSearchResult], sort: &SortOptions) {
|
||||
SortField::Size => a.file.size.cmp(&b.file.size),
|
||||
SortField::ModifiedAt => a.file.modified_at.cmp(&b.file.modified_at),
|
||||
SortField::CreatedAt => a.file.created_at.cmp(&b.file.created_at),
|
||||
SortField::CapturedAt => match (a.file.captured_at(), b.file.captured_at()) {
|
||||
(Some(a), Some(b)) => a.cmp(&b),
|
||||
_ => Ordering::Equal,
|
||||
},
|
||||
};
|
||||
let primary = match sort.direction {
|
||||
SortDirection::Asc => primary,
|
||||
SortDirection::Desc => primary.reverse(),
|
||||
};
|
||||
primary
|
||||
// A file with no capture time has no place in a capture order, so it
|
||||
// trails the dated ones in both directions rather than leading the
|
||||
// ascending one.
|
||||
let undated_last = match sort.field {
|
||||
SortField::CapturedAt => a
|
||||
.file
|
||||
.captured_at()
|
||||
.is_none()
|
||||
.cmp(&b.file.captured_at().is_none()),
|
||||
_ => Ordering::Equal,
|
||||
};
|
||||
undated_last
|
||||
.then(primary)
|
||||
.then_with(|| a.file.name.to_lowercase().cmp(&b.file.name.to_lowercase()))
|
||||
.then_with(|| a.file.id.cmp(&b.file.id))
|
||||
});
|
||||
@@ -155,6 +171,51 @@ mod tests {
|
||||
assert_eq!(names(&results), ["Alpha", "gamma", "beta"]);
|
||||
}
|
||||
|
||||
/// A capture order places dated files by their EXIF time and sends
|
||||
/// undated ones to the end whichever way it runs, so flipping the
|
||||
/// direction reverses the photos without moving the rest to the front.
|
||||
#[test]
|
||||
fn a_capture_order_keeps_undated_files_last_in_both_directions() {
|
||||
let taken = |name: &str, date: Option<&str>| {
|
||||
let mut result = candidate(name, 1, 1, 0.5);
|
||||
if let Some(date) = date {
|
||||
result.file.image_media_data = crate::domain::ImageMediaData::from_facet(
|
||||
result.file.id,
|
||||
&sd_store::ImageFacet {
|
||||
date_taken: Some(date.to_string()),
|
||||
..Default::default()
|
||||
},
|
||||
);
|
||||
}
|
||||
result
|
||||
};
|
||||
let mut results = vec![
|
||||
taken("undated.jpg", None),
|
||||
taken("newest.jpg", Some("2025-01-01T08:30:00+00:00")),
|
||||
taken("oldest.jpg", Some("2023-06-01T12:00:00+00:00")),
|
||||
taken("middle.jpg", Some("2024-03-12T10:00:00+00:00")),
|
||||
taken("also-undated.jpg", None),
|
||||
];
|
||||
|
||||
order(
|
||||
&mut results,
|
||||
&sort(SortField::CapturedAt, SortDirection::Desc),
|
||||
);
|
||||
assert_eq!(
|
||||
names(&results),
|
||||
["newest", "middle", "oldest", "also-undated", "undated"]
|
||||
);
|
||||
|
||||
order(
|
||||
&mut results,
|
||||
&sort(SortField::CapturedAt, SortDirection::Asc),
|
||||
);
|
||||
assert_eq!(
|
||||
names(&results),
|
||||
["oldest", "middle", "newest", "also-undated", "undated"]
|
||||
);
|
||||
}
|
||||
|
||||
/// Equal primary keys settle on name and then id, so two runs over the
|
||||
/// same candidates produce the same order.
|
||||
#[test]
|
||||
|
||||
@@ -71,7 +71,7 @@ impl FileSearchQuery {
|
||||
let cache = context.volume_index();
|
||||
let registry = context.file_type_registry();
|
||||
|
||||
let page = match &self.input.scope {
|
||||
let mut page = match &self.input.scope {
|
||||
SearchScope::Path { path } => {
|
||||
search_arena(
|
||||
&self.input.query,
|
||||
@@ -99,6 +99,11 @@ impl FileSearchQuery {
|
||||
}
|
||||
};
|
||||
|
||||
// The page shows what the facet knows whatever it was sorted by, so
|
||||
// the inspector reads capture time and place off a search hit as it
|
||||
// does off a listing.
|
||||
crate::ops::search::capture::decorate_results(cache, &mut page.results).await;
|
||||
|
||||
let execution_time = start_time.elapsed().as_millis() as u64;
|
||||
|
||||
Ok(FileSearchOutput::from_page(
|
||||
|
||||
@@ -123,12 +123,18 @@ pub(super) fn passes_store_filters(
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(ref range) = filters.date_range {
|
||||
// A capture range is judged on the built `File` by `capture::passes`,
|
||||
// as it is for arena hits.
|
||||
if let Some(range) = filters
|
||||
.date_range
|
||||
.as_ref()
|
||||
.filter(|range| range.field != DateField::CapturedAt)
|
||||
{
|
||||
let millis = match range.field {
|
||||
DateField::ModifiedAt => entry.mtime_ms,
|
||||
DateField::CreatedAt => entry.created_ms,
|
||||
DateField::AccessedAt => entry.atime_ms,
|
||||
DateField::IndexedAt => None,
|
||||
DateField::IndexedAt | DateField::CapturedAt => None,
|
||||
};
|
||||
let Some(date) = millis.and_then(chrono::DateTime::from_timestamp_millis) else {
|
||||
return false;
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
mod helpers;
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::time::Duration;
|
||||
|
||||
use helpers::*;
|
||||
use sd_core::{
|
||||
@@ -19,8 +20,8 @@ use sd_core::{
|
||||
indexing::{metadata::EntryMetadata, state::EntryKind, IndexScope, VolumeAnchor},
|
||||
search::{
|
||||
input::{
|
||||
FileSearchInput, PaginationOptions, SearchFilters, SearchMode, SearchScope,
|
||||
SortDirection, SortField, SortOptions, TagFilter,
|
||||
DateField, DateRangeFilter, FileSearchInput, PaginationOptions, SearchFilters,
|
||||
SearchMode, SearchScope, SortDirection, SortField, SortOptions, TagFilter,
|
||||
},
|
||||
output::FileSearchOutput,
|
||||
query::FileSearchQuery,
|
||||
@@ -497,6 +498,157 @@ async fn a_file_under_nested_sources_is_one_hit_from_the_stores() -> anyhow::Res
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// R8 "Same capture read through arena and SQLite", the capture half, and
|
||||
/// "Arena candidates need store-only filter or sort fields" for the image
|
||||
/// facet: a capture-time order, a capture-date range and a has-location
|
||||
/// filter answer the same whether the source's arena or its store serves
|
||||
/// the search. The arena holds none of it; it reads the facet from the
|
||||
/// store before it sorts and filters, and the store joins the facet into
|
||||
/// its rows.
|
||||
///
|
||||
/// One thread on purpose: the first search after the arenas drop starts a
|
||||
/// background refill from the store, and on one thread that refill cannot
|
||||
/// land before the search has chosen its backend, so every store-pass
|
||||
/// search is store-served and the arenas are dropped again before each.
|
||||
#[tokio::test]
|
||||
async fn capture_facts_read_the_same_through_arena_and_store() -> anyhow::Result<()> {
|
||||
let harness = IndexingHarnessBuilder::new("r8_capture_facts")
|
||||
.disable_watcher()
|
||||
.build()
|
||||
.await?;
|
||||
let photos = harness.create_test_dir("photos").await?;
|
||||
std::fs::write(
|
||||
photos.path().join("tokyo.jpg"),
|
||||
exif_jpeg(1, Some("2024:03:12 10:00:00"), Some((35.6812, 139.7671))),
|
||||
)?;
|
||||
std::fs::write(
|
||||
photos.path().join("later.jpg"),
|
||||
exif_jpeg(2, Some("2024:03:19 09:00:00"), None),
|
||||
)?;
|
||||
std::fs::write(
|
||||
photos.path().join("kyoto.jpg"),
|
||||
exif_jpeg(3, Some("2023:06:01 12:00:00"), Some((35.0116, 135.7681))),
|
||||
)?;
|
||||
std::fs::write(photos.path().join("bare.jpg"), exif_jpeg(4, None, None))?;
|
||||
photos.write_file("notes.txt", "not a photo").await?;
|
||||
let tracked = photos.track().await?;
|
||||
let cache = harness.core.context.volume_index();
|
||||
assert!(cache.arena_answers(&tracked.root));
|
||||
|
||||
// The EXIF pass runs after identification; wait for it to settle.
|
||||
let store = cache
|
||||
.store_for(&tracked.root.join("tokyo.jpg"))
|
||||
.await
|
||||
.expect("store");
|
||||
let deadline = tokio::time::Instant::now() + Duration::from_secs(60);
|
||||
loop {
|
||||
let rows: i64 =
|
||||
sqlx::query_scalar("SELECT COUNT(*) FROM facet_image WHERE content_hash IS NOT NULL")
|
||||
.fetch_one(store.db().pool())
|
||||
.await?;
|
||||
if rows == 4 && store.files_needing_image_facets_count().await? == 0 {
|
||||
break;
|
||||
}
|
||||
assert!(
|
||||
tokio::time::Instant::now() < deadline,
|
||||
"the EXIF pass did not finish: {rows} rows"
|
||||
);
|
||||
tokio::time::sleep(Duration::from_millis(50)).await;
|
||||
}
|
||||
drop(store);
|
||||
|
||||
let jpegs = SearchFilters {
|
||||
file_types: Some(vec!["jpg".to_string()]),
|
||||
..SearchFilters::default()
|
||||
};
|
||||
let by_capture = |filters: SearchFilters| FileSearchInput {
|
||||
query: String::new(),
|
||||
scope: SearchScope::Path {
|
||||
path: SdPath::local(tracked.root.clone()),
|
||||
},
|
||||
mode: SearchMode::Normal,
|
||||
filters,
|
||||
sort: SortOptions {
|
||||
field: SortField::CapturedAt,
|
||||
direction: SortDirection::Desc,
|
||||
},
|
||||
pagination: PaginationOptions {
|
||||
limit: 10,
|
||||
offset: 0,
|
||||
},
|
||||
};
|
||||
let placed = SearchFilters {
|
||||
has_location: Some(true),
|
||||
..jpegs.clone()
|
||||
};
|
||||
let march_2024 = SearchFilters {
|
||||
date_range: Some(DateRangeFilter {
|
||||
field: DateField::CapturedAt,
|
||||
start: Some(chrono::DateTime::from_timestamp(1_709_251_200, 0).unwrap()),
|
||||
end: Some(chrono::DateTime::from_timestamp(1_711_929_599, 0).unwrap()),
|
||||
}),
|
||||
..jpegs.clone()
|
||||
};
|
||||
let inputs = || {
|
||||
[
|
||||
by_capture(jpegs.clone()),
|
||||
by_capture(placed.clone()),
|
||||
by_capture(march_2024.clone()),
|
||||
]
|
||||
};
|
||||
type Answer = (
|
||||
Vec<String>,
|
||||
u64,
|
||||
Vec<(Option<chrono::DateTime<chrono::Utc>>, bool)>,
|
||||
);
|
||||
let answer = |page: FileSearchOutput| -> Answer {
|
||||
(
|
||||
names(&page),
|
||||
page.total_found,
|
||||
page.results
|
||||
.iter()
|
||||
.map(|r| (r.file.captured_at(), r.file.has_location()))
|
||||
.collect(),
|
||||
)
|
||||
};
|
||||
|
||||
let mut from_arena = Vec::new();
|
||||
for input in inputs() {
|
||||
assert!(cache.arena_answers(&tracked.root));
|
||||
from_arena.push(answer(search(&harness, input).await?));
|
||||
}
|
||||
assert_eq!(from_arena[0].0, ["later", "tokyo", "kyoto", "bare"]);
|
||||
assert_eq!(from_arena[0].1, 4);
|
||||
assert_eq!(from_arena[1].0, ["tokyo", "kyoto"]);
|
||||
assert_eq!(from_arena[2].0, ["later", "tokyo"]);
|
||||
assert!(from_arena[0].2[0].0.is_some() && !from_arena[0].2[0].1);
|
||||
assert!(from_arena[0].2[3].0.is_none());
|
||||
|
||||
// The daemon restarts with no snapshot, so the source answers from its
|
||||
// store, the way the nested-store row drops its arenas.
|
||||
let mut from_store = Vec::new();
|
||||
for input in inputs() {
|
||||
let ids: Vec<Uuid> = cache
|
||||
.sources_of(harness.library.id())
|
||||
.iter()
|
||||
.map(|source| source.id)
|
||||
.collect();
|
||||
let targets = cache.quiesce_targets(&ids);
|
||||
cache.detach_library(harness.library.id());
|
||||
let (hold, _) = cache.quiesce_stores(&ids, &targets).await;
|
||||
drop(hold);
|
||||
cache
|
||||
.attach_library(harness.library.id(), harness.library.db().clone())
|
||||
.await?;
|
||||
assert!(!cache.arena_answers(&tracked.root), "the store answers");
|
||||
from_store.push(answer(search(&harness, input).await?));
|
||||
}
|
||||
assert_eq!(from_store, from_arena);
|
||||
|
||||
harness.shutdown().await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Track `root` as a second, nested source and wait for its walk.
|
||||
async fn track_nested(harness: &IndexingHarness, root: PathBuf) -> anyhow::Result<()> {
|
||||
use sd_core::infra::job::types::JobId;
|
||||
|
||||
@@ -72,7 +72,7 @@ pub use image::{
|
||||
PendingImage,
|
||||
};
|
||||
pub use migrate::SCHEMA_VERSION;
|
||||
pub use read::{FsEntry, TitleMatches};
|
||||
pub use read::{image_facets_for_records, FsEntry, TitleMatches};
|
||||
pub use record::{ContentIdentity, Record, RECORD_SCHEMA};
|
||||
pub use revision::Revision;
|
||||
pub use schema::{DataTypeSchema, FieldType, ModelDef};
|
||||
|
||||
+150
-13
@@ -16,6 +16,7 @@ use std::collections::HashMap;
|
||||
|
||||
use crate::error::Result;
|
||||
use crate::file::FileKind;
|
||||
use crate::image::ImageFacet;
|
||||
use sqlx::{FromRow, SqlitePool};
|
||||
use uuid::Uuid;
|
||||
|
||||
@@ -57,6 +58,10 @@ pub struct FsEntry {
|
||||
/// not; `content_kind` is then its parent.
|
||||
pub content_kind_name: Option<String>,
|
||||
pub content_error: Option<String>,
|
||||
/// The image facet, when the EXIF pass has read these bytes. A row
|
||||
/// keyed by other bytes than the record's current content is not
|
||||
/// here: it describes an image the file no longer is.
|
||||
pub image: Option<ImageFacet>,
|
||||
}
|
||||
|
||||
impl FsEntry {
|
||||
@@ -70,21 +75,51 @@ impl FsEntry {
|
||||
}
|
||||
}
|
||||
|
||||
/// The image facet's columns, over the alias `i`, in the order
|
||||
/// [`ImageFacet`] is read in.
|
||||
macro_rules! image_columns {
|
||||
() => {
|
||||
"i.record_uuid IS NOT NULL AS has_image, i.width AS image_width, \
|
||||
i.height AS image_height, i.date_taken AS date_taken, i.latitude AS latitude, \
|
||||
i.longitude AS longitude, i.camera_make AS camera_make, i.camera_model AS camera_model, \
|
||||
i.lens_model AS lens_model, i.focal_length AS focal_length, i.aperture AS aperture, \
|
||||
i.shutter_speed AS shutter_speed, i.iso AS iso, i.orientation AS orientation, \
|
||||
i.color_space AS color_space, i.color_profile AS color_profile, \
|
||||
i.bit_depth AS bit_depth, i.artist AS artist, i.copyright AS copyright, \
|
||||
i.description AS description"
|
||||
};
|
||||
}
|
||||
|
||||
/// Every column [`FsEntry`] is built from, over the aliases `r` (the record),
|
||||
/// `own` and `parent` (its own and its parent's `directory_path` rows), `f`
|
||||
/// and `c`. A directory's path comes from its own row; a file's from its
|
||||
/// `own` and `parent` (its own and its parent's `directory_path` rows), `f`,
|
||||
/// `c` and `i`. A directory's path comes from its own row; a file's from its
|
||||
/// parent's row plus its title, and a root-level entry from its title alone.
|
||||
macro_rules! entry_columns {
|
||||
() => {
|
||||
"r.rowid AS rowid, r.uuid AS uuid, r.type AS kind, \
|
||||
COALESCE(r.title, '') AS title, \
|
||||
COALESCE(own.path, parent.path || '/' || r.title, COALESCE(r.title, '')) AS rel_path, \
|
||||
f.size AS size, f.mtime AS mtime_ms, f.atime AS atime_ms, r.created_at AS created_ms, \
|
||||
COALESCE(f.is_hidden, 0) AS is_hidden, f.extension AS extension, \
|
||||
f.link_target AS link_target, f.inode AS inode, f.mode AS mode, f.uid AS uid, f.gid AS gid, \
|
||||
f.content_error AS content_error, c.uuid AS content_uuid, \
|
||||
c.sampled_hash AS sampled_hash, c.integrity_hash AS integrity_hash, c.kind AS content_kind, \
|
||||
c.kind_name AS content_kind_name"
|
||||
concat!(
|
||||
"r.rowid AS rowid, r.uuid AS uuid, r.type AS kind, \
|
||||
COALESCE(r.title, '') AS title, \
|
||||
COALESCE(own.path, parent.path || '/' || r.title, COALESCE(r.title, '')) AS rel_path, \
|
||||
f.size AS size, f.mtime AS mtime_ms, f.atime AS atime_ms, r.created_at AS created_ms, \
|
||||
COALESCE(f.is_hidden, 0) AS is_hidden, f.extension AS extension, \
|
||||
f.link_target AS link_target, f.inode AS inode, f.mode AS mode, f.uid AS uid, f.gid AS gid, \
|
||||
f.content_error AS content_error, c.uuid AS content_uuid, \
|
||||
c.sampled_hash AS sampled_hash, c.integrity_hash AS integrity_hash, c.kind AS content_kind, \
|
||||
c.kind_name AS content_kind_name, ",
|
||||
image_columns!()
|
||||
)
|
||||
};
|
||||
}
|
||||
|
||||
/// The image facet joined to the record whose bytes it was read from. The
|
||||
/// facet is keyed by record but describes content, so a row the EXIF pass
|
||||
/// wrote for bytes the file no longer holds joins as nothing, exactly as a
|
||||
/// missing row does; the pass rewrites it once identification lands.
|
||||
macro_rules! image_join {
|
||||
() => {
|
||||
"LEFT JOIN facet_image i ON i.record_uuid = r.uuid \
|
||||
AND i.content_hash IS NOT NULL \
|
||||
AND i.content_hash = COALESCE(c.sampled_hash, c.integrity_hash)"
|
||||
};
|
||||
}
|
||||
|
||||
@@ -95,7 +130,8 @@ const ENTRY_SELECT: &str = concat!(
|
||||
LEFT JOIN directory_path own ON own.record_uuid = r.uuid \
|
||||
LEFT JOIN directory_path parent ON parent.record_uuid = r.parent_uuid \
|
||||
LEFT JOIN facet_file f ON f.record_uuid = r.uuid \
|
||||
LEFT JOIN content c ON c.id = r.content_id"
|
||||
LEFT JOIN content c ON c.id = r.content_id ",
|
||||
image_join!()
|
||||
);
|
||||
|
||||
/// The same columns, driven from the directories: `CROSS JOIN` holds
|
||||
@@ -108,7 +144,8 @@ const BENEATH_SELECT: &str = concat!(
|
||||
CROSS JOIN record r ON r.parent_uuid = parent.record_uuid \
|
||||
LEFT JOIN directory_path own ON own.record_uuid = r.uuid \
|
||||
LEFT JOIN facet_file f ON f.record_uuid = r.uuid \
|
||||
LEFT JOIN content c ON c.id = r.content_id"
|
||||
LEFT JOIN content c ON c.id = r.content_id ",
|
||||
image_join!()
|
||||
);
|
||||
|
||||
#[derive(FromRow)]
|
||||
@@ -135,6 +172,64 @@ struct EntryRow {
|
||||
integrity_hash: Option<String>,
|
||||
content_kind: Option<i64>,
|
||||
content_kind_name: Option<String>,
|
||||
#[sqlx(flatten)]
|
||||
image: ImageRow,
|
||||
}
|
||||
|
||||
/// [`image_columns!`] as sqlx reads them; `has_image` tells a row with no
|
||||
/// facet from a facet row whose every column is empty, which the pass writes
|
||||
/// for a photo that carried no EXIF.
|
||||
#[derive(FromRow)]
|
||||
struct ImageRow {
|
||||
has_image: bool,
|
||||
image_width: Option<i64>,
|
||||
image_height: Option<i64>,
|
||||
date_taken: Option<String>,
|
||||
latitude: Option<f64>,
|
||||
longitude: Option<f64>,
|
||||
camera_make: Option<String>,
|
||||
camera_model: Option<String>,
|
||||
lens_model: Option<String>,
|
||||
focal_length: Option<String>,
|
||||
aperture: Option<String>,
|
||||
shutter_speed: Option<String>,
|
||||
iso: Option<i64>,
|
||||
orientation: Option<i64>,
|
||||
color_space: Option<String>,
|
||||
color_profile: Option<String>,
|
||||
bit_depth: Option<String>,
|
||||
artist: Option<String>,
|
||||
copyright: Option<String>,
|
||||
description: Option<String>,
|
||||
}
|
||||
|
||||
impl ImageRow {
|
||||
fn into_facet(self) -> Option<ImageFacet> {
|
||||
if !self.has_image {
|
||||
return None;
|
||||
}
|
||||
Some(ImageFacet {
|
||||
width: self.image_width,
|
||||
height: self.image_height,
|
||||
date_taken: self.date_taken,
|
||||
latitude: self.latitude,
|
||||
longitude: self.longitude,
|
||||
camera_make: self.camera_make,
|
||||
camera_model: self.camera_model,
|
||||
lens_model: self.lens_model,
|
||||
focal_length: self.focal_length,
|
||||
aperture: self.aperture,
|
||||
shutter_speed: self.shutter_speed,
|
||||
iso: self.iso,
|
||||
orientation: self.orientation,
|
||||
color_space: self.color_space,
|
||||
color_profile: self.color_profile,
|
||||
bit_depth: self.bit_depth,
|
||||
artist: self.artist,
|
||||
copyright: self.copyright,
|
||||
description: self.description,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
fn entry_from_row(row: EntryRow) -> Option<FsEntry> {
|
||||
@@ -162,9 +257,51 @@ fn entry_from_row(row: EntryRow) -> Option<FsEntry> {
|
||||
content_kind: row.content_kind,
|
||||
content_kind_name: row.content_kind_name,
|
||||
content_error: row.content_error,
|
||||
image: row.image.into_facet(),
|
||||
})
|
||||
}
|
||||
|
||||
/// The image facet of each of `record_uuids` whose row describes its
|
||||
/// current bytes, for a reader holding records from elsewhere (an arena)
|
||||
/// that needs what only the facet knows. Records without a current row are
|
||||
/// absent from the answer.
|
||||
pub async fn image_facets_for_records(
|
||||
pool: &SqlitePool,
|
||||
record_uuids: &[Uuid],
|
||||
) -> Result<Vec<(Uuid, ImageFacet)>> {
|
||||
let mut facets = Vec::new();
|
||||
// Bound each statement's parameter count well under SQLite's limit.
|
||||
for chunk in record_uuids.chunks(2_000) {
|
||||
let sql = format!(
|
||||
"SELECT r.uuid AS uuid, {} FROM record r \
|
||||
LEFT JOIN content c ON c.id = r.content_id {} \
|
||||
WHERE i.record_uuid IS NOT NULL AND r.uuid IN ({})",
|
||||
image_columns!(),
|
||||
image_join!(),
|
||||
vec!["?"; chunk.len()].join(", ")
|
||||
);
|
||||
let mut query = sqlx::query_as::<_, FacetRow>(&sql);
|
||||
for id in chunk {
|
||||
query = query.bind(*id);
|
||||
}
|
||||
facets.extend(
|
||||
query
|
||||
.fetch_all(pool)
|
||||
.await?
|
||||
.into_iter()
|
||||
.filter_map(|row| Some((row.uuid, row.image.into_facet()?))),
|
||||
);
|
||||
}
|
||||
Ok(facets)
|
||||
}
|
||||
|
||||
#[derive(FromRow)]
|
||||
struct FacetRow {
|
||||
uuid: Uuid,
|
||||
#[sqlx(flatten)]
|
||||
image: ImageRow,
|
||||
}
|
||||
|
||||
/// File counts per content kind and kind name, from the content rows records
|
||||
/// point at. Kinds are the store's integers; the caller owns the enum
|
||||
/// mapping. The name is an extension kind's id, `None` for a built-in kind,
|
||||
|
||||
@@ -419,3 +419,139 @@ async fn contents_beneath_a_directory_answer_by_scope() {
|
||||
holders.sort();
|
||||
assert_eq!(holders, ["2019/b.jpg", "2019/trip/a.jpg"]);
|
||||
}
|
||||
|
||||
/// An entry read carries the image facet only while the row describes the
|
||||
/// record's current bytes. A row the EXIF pass wrote for an earlier
|
||||
/// version of the file reads as no row at all, on a listing and on the
|
||||
/// by-record lookup an arena decorates from alike.
|
||||
#[tokio::test]
|
||||
async fn an_entry_carries_the_image_facet_keyed_by_its_current_bytes() {
|
||||
let fixture = Fixture::new().await;
|
||||
populate(
|
||||
&fixture,
|
||||
&[
|
||||
("photos", FileKind::Directory, false),
|
||||
("photos/dated.jpg", FileKind::File, false),
|
||||
("photos/plain.jpg", FileKind::File, false),
|
||||
("photos/notes.txt", FileKind::File, false),
|
||||
],
|
||||
)
|
||||
.await;
|
||||
let db = fixture.manager.open("drive-1").await.expect("open");
|
||||
let record = |path: &'static str| {
|
||||
let db = &db;
|
||||
async move {
|
||||
read::entry_by_path(db.pool(), path)
|
||||
.await
|
||||
.expect("lookup")
|
||||
.expect("file exists")
|
||||
.uuid
|
||||
}
|
||||
};
|
||||
let dated = record("photos/dated.jpg").await;
|
||||
let plain = record("photos/plain.jpg").await;
|
||||
let text = record("photos/notes.txt").await;
|
||||
for (uuid, hash) in [(dated, "dated-v1"), (plain, "plain-v1"), (text, "text")] {
|
||||
db.set_content_identity(
|
||||
uuid,
|
||||
&ContentIdentity {
|
||||
sampled_hash: Some(hash.to_string()),
|
||||
kind: Some(1),
|
||||
..Default::default()
|
||||
},
|
||||
)
|
||||
.await
|
||||
.expect("identity");
|
||||
}
|
||||
sd_store::set_image_facets(
|
||||
db.pool(),
|
||||
&[
|
||||
(
|
||||
"dated-v1".to_string(),
|
||||
sd_store::ImageFacet {
|
||||
date_taken: Some("2024-03-12T10:00:00+00:00".to_string()),
|
||||
latitude: Some(35.68),
|
||||
longitude: Some(139.77),
|
||||
camera_make: Some("Fuji".to_string()),
|
||||
..Default::default()
|
||||
},
|
||||
),
|
||||
("plain-v1".to_string(), sd_store::ImageFacet::default()),
|
||||
],
|
||||
)
|
||||
.await
|
||||
.expect("facets");
|
||||
|
||||
let folder = read::entry_by_path(db.pool(), "photos")
|
||||
.await
|
||||
.expect("lookup")
|
||||
.expect("dir")
|
||||
.uuid;
|
||||
let facet_of = |entries: &[read::FsEntry], name: &str| {
|
||||
entries
|
||||
.iter()
|
||||
.find(|entry| entry.name == name)
|
||||
.expect("listed")
|
||||
.image
|
||||
.clone()
|
||||
};
|
||||
|
||||
let listed = read::children_of(db.pool(), Some(folder), false)
|
||||
.await
|
||||
.expect("children");
|
||||
let facet = facet_of(&listed, "dated.jpg").expect("the dated photo has its facet");
|
||||
assert_eq!(
|
||||
facet.date_taken.as_deref(),
|
||||
Some("2024-03-12T10:00:00+00:00")
|
||||
);
|
||||
assert_eq!(
|
||||
(facet.latitude, facet.longitude),
|
||||
(Some(35.68), Some(139.77))
|
||||
);
|
||||
assert_eq!(facet.camera_make.as_deref(), Some("Fuji"));
|
||||
assert_eq!(
|
||||
facet_of(&listed, "plain.jpg"),
|
||||
Some(sd_store::ImageFacet::default()),
|
||||
"a photo read and found to carry no EXIF has an empty facet, not none"
|
||||
);
|
||||
assert_eq!(facet_of(&listed, "notes.txt"), None);
|
||||
|
||||
let by_record = read::image_facets_for_records(db.pool(), &[dated, plain, text])
|
||||
.await
|
||||
.expect("facets by record");
|
||||
assert_eq!(by_record.len(), 2);
|
||||
assert_eq!(
|
||||
by_record
|
||||
.iter()
|
||||
.find(|(uuid, _)| *uuid == dated)
|
||||
.map(|(_, facet)| facet.date_taken.clone()),
|
||||
Some(Some("2024-03-12T10:00:00+00:00".to_string()))
|
||||
);
|
||||
|
||||
// The bytes change: identification gives the record a new hash and the
|
||||
// old row no longer describes it, so both reads answer nothing until
|
||||
// the pass rewrites the row.
|
||||
db.set_content_identity(
|
||||
dated,
|
||||
&ContentIdentity {
|
||||
sampled_hash: Some("dated-v2".to_string()),
|
||||
kind: Some(1),
|
||||
..Default::default()
|
||||
},
|
||||
)
|
||||
.await
|
||||
.expect("identity");
|
||||
let listed = read::children_of(db.pool(), Some(folder), false)
|
||||
.await
|
||||
.expect("children");
|
||||
assert_eq!(facet_of(&listed, "dated.jpg"), None);
|
||||
assert!(read::image_facets_for_records(db.pool(), &[dated])
|
||||
.await
|
||||
.expect("facets by record")
|
||||
.is_empty());
|
||||
let page = read::files_beneath(db.pool(), "", read::Start::First, None, false, 10)
|
||||
.await
|
||||
.expect("beneath");
|
||||
assert_eq!(facet_of(&page, "dated.jpg"), None);
|
||||
assert!(facet_of(&page, "plain.jpg").is_some());
|
||||
}
|
||||
@@ -59,21 +59,30 @@ cargo test -p sd-core --lib repeated_clear_and_refill_keeps_allocation_bounded -
|
||||
| 16 | Continuous real writes | `core/tests/source_replication_test.rs` `test_source_replication` (Alice writes and re-walks; Bob lands a newer generation holding every record); `crates/store/tests/revision.rs` `file_changes_move_the_revision_and_nothing_else_does` | passing |
|
||||
| 17 | Repeated subtree clear and refill | `core/src/ops/indexing/arena.rs` `repeated_clear_and_refill_keeps_allocation_bounded` (slots bounded by the live tree across 100 cycles, snapshot carries live nodes), `a_shrunken_tree_compacts_once_vacancy_outgrows_it` (the compaction trigger), `a_snapshot_after_churn_carries_only_live_nodes` (round-trip after churn), `a_listing_cursor_taken_before_compaction_still_pages` (path cursors survive renumbering); `core/src/ops/indexing/nodes.rs` `a_vacated_slot_is_reused_and_unreadable_until_then` | fixed (see F4) |
|
||||
| 18 | Cold search across 100 stores | `crates/store/tests/scale.rs` `a_hundred_cold_stores_answer_without_writers` (CI size: 100 stores x 200 records); `a_cold_fan_out_across_many_stores` (the measurement, ignored, sized by `SD_SCALE_*`) | passing |
|
||||
| 19 | Same capture read through arena and SQLite | `core/src/ops/search/arena_search.rs` `the_store_backend_matches_the_arena_for_the_same_capture` (identities, matching, filters, scores); `core/src/ops/search/pipeline.rs` `every_sort_field_orders_and_reverses`, `equal_scores_tiebreak_deterministically`, `a_page_is_a_window_over_the_sorted_whole` (ordering and pagination shared by both backends) | passing |
|
||||
| 19 | Same capture read through arena and SQLite | `core/src/ops/search/arena_search.rs` `the_store_backend_matches_the_arena_for_the_same_capture` (identities, matching, filters, scores, and the capture time and place both backends read from `facet_image`); `core/src/ops/search/pipeline.rs` `every_sort_field_orders_and_reverses`, `a_capture_order_keeps_undated_files_last_in_both_directions`, `equal_scores_tiebreak_deterministically`, `a_page_is_a_window_over_the_sorted_whole` (ordering and pagination shared by both backends) | passing |
|
||||
| 20 | Suitable loaded arena returns no matches | `core/tests/source_runtime_acceptance_test.rs` `an_empty_answer_from_a_suitable_arena_is_final` | passing |
|
||||
| 21 | Loaded arena has insufficient coverage or query support | `core/tests/source_runtime_acceptance_test.rs` `an_unwalked_source_answers_from_its_store` | passing |
|
||||
| 22 | Five suitable arenas and 95 stores | `core/tests/source_runtime_acceptance_test.rs` `five_arenas_and_ninety_five_stores_page_the_same` | passing |
|
||||
| 23 | Arena candidates need store-only filter or sort fields | `core/tests/source_runtime_acceptance_test.rs` `a_store_only_filter_narrows_before_pagination` (tags are the store-only field) | passing |
|
||||
| 23 | Arena candidates need store-only filter or sort fields | `core/tests/source_runtime_acceptance_test.rs` `a_store_only_filter_narrows_before_pagination` (tags are the store-only field); row 29 for the image facet's capture time and place as sort and filter fields | passing |
|
||||
| 24 | Search with requested limit five | `core/tests/source_runtime_acceptance_test.rs` `a_limit_of_five_returns_five_with_an_honest_total` | passing |
|
||||
| 25 | Same record in multiple representations | `core/tests/source_runtime_acceptance_test.rs` `a_file_under_nested_sources_is_one_hit_from_the_arena` (loaded); `a_file_under_nested_sources_is_one_hit_from_the_stores` (store-backed) | passing / fixed (see F5) |
|
||||
| 26 | Replica replacement with local assertions | none | not automatable today: replicas open read-only and carry no receiver-owned assertions (R6 results, "receiver-owned assertions on replica databases" is a registered follow-on gated on FD2), so there is no local assertion to preserve. `crates/store/tests/files.rs` `a_moved_file_carries_its_assertions_with_it` and `a_removal_takes_the_facet_and_leaves_the_assertion` cover the owner-side contract the replica path will have to reuse |
|
||||
| 27 | Mapped volume with no source | `core/src/ops/indexing/volume_index.rs` `a_tracked_drive_maps_without_appearing_as_a_source` (browse and snapshot), `a_mapped_drive_without_a_source_is_watchable` (watcher); `core/src/ops/indexing/store.rs` `a_partition_with_no_store_still_browses` | passing |
|
||||
| 28 | Status query and resource event | `core/tests/source_runtime_acceptance_test.rs` `listing_status_and_store_agree_on_a_sources_count` (`sources.list`, `core.index_status` and the store report one count and one observation time for one source) | passing |
|
||||
| 29 | Capture time and place through arena and SQLite | `core/tests/source_runtime_acceptance_test.rs` `capture_facts_read_the_same_through_arena_and_store` (the EXIF fixture tracked and enriched; a `CapturedAt` order, a `CapturedAt` range and a `has_location` filter give one order, one total and the same capture facts per hit from the walked arena and, after the arenas drop, from the store; `crates/store/tests/read.rs` `an_entry_carries_the_image_facet_keyed_by_its_current_bytes` for the hash-guarded join underneath) | passing |
|
||||
|
||||
Totals: 26 rows fully passing (21 at the commit that landed this file, plus
|
||||
rows 3, 10, 11, 17 and 25 fixed since); row 9 passing except its APFS half,
|
||||
which no Linux runner can produce; row 26 not automatable because the
|
||||
feature it names does not exist yet. No ignored test remains.
|
||||
Totals: 27 rows fully passing (21 at the commit that landed this file, plus
|
||||
rows 3, 10, 11, 17 and 25 fixed since, and row 29 added for the image
|
||||
facet); row 9 passing except its APFS half, which no Linux runner can
|
||||
produce; row 26 not automatable because the feature it names does not exist
|
||||
yet. No ignored test remains.
|
||||
|
||||
Row 29 is not in the plan's table. It extends rows 19 and 23 to a facet:
|
||||
the arena holds no capture time or place, so the arena path reads them from
|
||||
the store before it sorts and filters (`core/src/ops/search/capture.rs`)
|
||||
and the store path joins `facet_image` into every entry read, guarded by
|
||||
the row's content hash. Neither backend needs the other to answer, and
|
||||
both hand the same `File` to one predicate and one order.
|
||||
|
||||
## Failing and fixed rows
|
||||
|
||||
|
||||
@@ -116,6 +116,20 @@ camera, lens, exposure, orientation and pixel geometry live in the store's
|
||||
extension host's `records.exif`, the rename resolver's capture time, and any
|
||||
inspector read the row first.
|
||||
|
||||
Listings and search read it too. A store-served row carries its facet from
|
||||
the join in `sd_store::read`, which takes the row only while its content
|
||||
hash is the record's current one, so an edited photo reads as unread until
|
||||
the pass rewrites it. An arena holds none of it: a listing or search served
|
||||
from an arena reads the facet from the store ahead of its sort and filter
|
||||
(`core/src/ops/search/capture.rs`), the way a tag filter resolves its paths
|
||||
from the stores first. Both backends then hand the same `File` to one
|
||||
predicate and one order. `search.files` sorts by `CapturedAt` and filters by
|
||||
a `CapturedAt` date range and by `has_location`; `files.directory_listing`
|
||||
sorts by `datetaken`. A file without a facet has no capture time and no
|
||||
place: it fails those filters and trails every dated file in a capture
|
||||
order. A replica of another device's source has no store here and reads
|
||||
the same way.
|
||||
|
||||
The pass claims its work from the store. A row carries the hash of the bytes
|
||||
it was read from, so a record is pending when it has no row or its row names
|
||||
other bytes. A changed file loses its content identity in the walk, gains a
|
||||
|
||||
@@ -521,6 +521,13 @@ function MediaMetadataCard({file}: {file: File}) {
|
||||
);
|
||||
}
|
||||
|
||||
/** Decimal degrees to five places, about a metre, signed by hemisphere. */
|
||||
function formatCoordinates(latitude: number, longitude: number) {
|
||||
const lat = `${Math.abs(latitude).toFixed(5)}° ${latitude < 0 ? 'S' : 'N'}`;
|
||||
const lon = `${Math.abs(longitude).toFixed(5)}° ${longitude < 0 ? 'W' : 'E'}`;
|
||||
return `${lat}, ${lon}`;
|
||||
}
|
||||
|
||||
function OverviewTab({file}: {file: File}) {
|
||||
const formatDate = (dateStr: string) => {
|
||||
const date = new Date(dateStr);
|
||||
@@ -530,6 +537,15 @@ function OverviewTab({file}: {file: File}) {
|
||||
year: 'numeric'
|
||||
});
|
||||
};
|
||||
// A capture time is a moment, not a day: the clock tells two frames apart.
|
||||
const formatDateTime = (dateStr: string) =>
|
||||
new Date(dateStr).toLocaleString('en-US', {
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
year: 'numeric',
|
||||
hour: '2-digit',
|
||||
minute: '2-digit'
|
||||
});
|
||||
|
||||
// Tag mutations — refetch queries on success to update the UI
|
||||
const refetchTagQueries = useRefetchTagQueries();
|
||||
@@ -612,7 +628,7 @@ function OverviewTab({file}: {file: File}) {
|
||||
{file.image_media_data?.date_taken && (
|
||||
<InfoRow
|
||||
label="Taken"
|
||||
value={formatDate(file.image_media_data.date_taken)}
|
||||
value={formatDateTime(file.image_media_data.date_taken)}
|
||||
/>
|
||||
)}
|
||||
<InfoRow label="Created" value={formatDate(file.created_at)} />
|
||||
@@ -631,10 +647,22 @@ function OverviewTab({file}: {file: File}) {
|
||||
{/* Image Metadata */}
|
||||
{file.image_media_data && (
|
||||
<Section title="Image Info" icon={Image}>
|
||||
<InfoRow
|
||||
label="Dimensions"
|
||||
value={`${file.image_media_data.width} × ${file.image_media_data.height}`}
|
||||
/>
|
||||
{file.image_media_data.width && file.image_media_data.height && (
|
||||
<InfoRow
|
||||
label="Dimensions"
|
||||
value={`${file.image_media_data.width} × ${file.image_media_data.height}`}
|
||||
/>
|
||||
)}
|
||||
{file.image_media_data.latitude != null &&
|
||||
file.image_media_data.longitude != null && (
|
||||
<InfoRow
|
||||
label="Location"
|
||||
value={formatCoordinates(
|
||||
file.image_media_data.latitude,
|
||||
file.image_media_data.longitude
|
||||
)}
|
||||
/>
|
||||
)}
|
||||
{file.image_media_data.camera_make && (
|
||||
<InfoRow
|
||||
label="Camera"
|
||||
|
||||
@@ -13,28 +13,28 @@ import { motion, AnimatePresence } from "framer-motion";
|
||||
import clsx from "clsx";
|
||||
import { CircleButton } from "@spacedrive/primitives";
|
||||
import type { DirectorySortBy, MediaSortBy } from "@sd/ts-client";
|
||||
import { sortOptionsFor, type ExplorerSortBy, type ExplorerViewMode } from "./sortOptions";
|
||||
|
||||
interface SortMenuPanelProps {
|
||||
sortBy: DirectorySortBy | MediaSortBy;
|
||||
onSortChange: (sort: DirectorySortBy | MediaSortBy) => void;
|
||||
viewMode: "grid" | "list" | "media" | "column";
|
||||
viewMode: ExplorerViewMode;
|
||||
}
|
||||
|
||||
const SORT_ICONS: Record<ExplorerSortBy, typeof TextAa> = {
|
||||
name: TextAa,
|
||||
modified: CalendarBlank,
|
||||
created: CalendarBlank,
|
||||
datetaken: Camera,
|
||||
size: Ruler,
|
||||
type: FileText,
|
||||
};
|
||||
|
||||
export function SortMenuPanel({ sortBy, onSortChange, viewMode }: SortMenuPanelProps) {
|
||||
const sortOptions = viewMode === "media"
|
||||
? [
|
||||
{ value: "datetaken", label: "Date Taken", icon: Camera },
|
||||
{ value: "modified", label: "Date Modified", icon: CalendarBlank },
|
||||
{ value: "created", label: "Date Created", icon: CalendarBlank },
|
||||
{ value: "name", label: "Name", icon: TextAa },
|
||||
{ value: "size", label: "Size", icon: Ruler },
|
||||
]
|
||||
: [
|
||||
{ value: "name", label: "Name", icon: TextAa },
|
||||
{ value: "modified", label: "Date Modified", icon: CalendarBlank },
|
||||
{ value: "size", label: "Size", icon: Ruler },
|
||||
{ value: "type", label: "Type", icon: FileText },
|
||||
];
|
||||
const sortOptions = sortOptionsFor(viewMode).map((option) => ({
|
||||
...option,
|
||||
icon: SORT_ICONS[option.value],
|
||||
}));
|
||||
|
||||
return (
|
||||
<div className="w-56 bg-app-box border border-app-line rounded-lg shadow-lg overflow-hidden">
|
||||
@@ -51,7 +51,7 @@ export function SortMenuPanel({ sortBy, onSortChange, viewMode }: SortMenuPanelP
|
||||
<button
|
||||
key={option.value}
|
||||
onClick={() => {
|
||||
onSortChange(option.value as DirectorySortBy | MediaSortBy);
|
||||
onSortChange(option.value);
|
||||
}}
|
||||
className={clsx(
|
||||
"flex items-center gap-2.5 w-full px-3 py-2 hover:bg-app-hover transition-colors text-sm",
|
||||
@@ -72,7 +72,7 @@ export function SortMenuPanel({ sortBy, onSortChange, viewMode }: SortMenuPanelP
|
||||
interface SortMenuProps {
|
||||
sortBy: DirectorySortBy | MediaSortBy;
|
||||
onSortChange: (sort: DirectorySortBy | MediaSortBy) => void;
|
||||
viewMode: "grid" | "list" | "media" | "column";
|
||||
viewMode: ExplorerViewMode;
|
||||
className?: string;
|
||||
}
|
||||
|
||||
|
||||
@@ -23,6 +23,7 @@ import type {
|
||||
ViewMode as TabViewMode
|
||||
} from '../../components/TabManager/TabManagerContext';
|
||||
import {useTabManager} from '../../components/TabManager/useTabManager';
|
||||
import {coerceSortBy, type ExplorerViewMode} from './sortOptions';
|
||||
import {useNormalizedQuery} from '../../contexts/SpacedriveContext';
|
||||
|
||||
export type SortBy = DirectorySortBy | MediaSortBy;
|
||||
@@ -723,14 +724,12 @@ export function ExplorerProvider({
|
||||
}
|
||||
}, [pathKey, sortPrefs]);
|
||||
|
||||
// "datetaken" only applies to media view; fall back to "modified" elsewhere.
|
||||
// An order the new view does not offer falls back to that view's first.
|
||||
useEffect(() => {
|
||||
if (viewMode === 'media' && uiState.sortBy === 'type') {
|
||||
uiDispatch({type: 'SET_SORT_BY', sort: 'datetaken'});
|
||||
sortPrefs.setPreferences(pathKey, 'datetaken');
|
||||
} else if (viewMode !== 'media' && uiState.sortBy === 'datetaken') {
|
||||
uiDispatch({type: 'SET_SORT_BY', sort: 'modified'});
|
||||
sortPrefs.setPreferences(pathKey, 'modified');
|
||||
const kept = coerceSortBy(viewMode as ExplorerViewMode, uiState.sortBy);
|
||||
if (kept !== uiState.sortBy) {
|
||||
uiDispatch({type: 'SET_SORT_BY', sort: kept});
|
||||
sortPrefs.setPreferences(pathKey, kept);
|
||||
}
|
||||
}, [viewMode, uiState.sortBy, pathKey, sortPrefs]);
|
||||
|
||||
|
||||
@@ -13,6 +13,7 @@ import {
|
||||
useSpacedriveClient,
|
||||
} from "../../../contexts/SpacedriveContext";
|
||||
import { useExplorer } from "../context";
|
||||
import { searchSortField } from "../sortOptions";
|
||||
import { isLapsedHandle, usePlanPreviewStore } from "./usePlanPreview";
|
||||
import { useVirtualListing } from "./useVirtualListing";
|
||||
|
||||
@@ -77,18 +78,6 @@ export function useSearchInput(): FileSearchInput | null {
|
||||
// silently widening to the library.
|
||||
if (scope === "source" && !sourceScopePath) return null;
|
||||
|
||||
// Map explorer sortBy to search SortField
|
||||
const searchSortField = (() => {
|
||||
if (!sortBy) return "Relevance" as const;
|
||||
const sortMap: Record<string, "Relevance" | "Name" | "Size" | "ModifiedAt" | "CreatedAt"> = {
|
||||
name: "Name",
|
||||
size: "Size",
|
||||
modified: "ModifiedAt",
|
||||
type: "Relevance",
|
||||
};
|
||||
return sortMap[sortBy] || "Relevance";
|
||||
})();
|
||||
|
||||
return {
|
||||
query,
|
||||
scope:
|
||||
@@ -100,7 +89,7 @@ export function useSearchInput(): FileSearchInput | null {
|
||||
filters: searchFilters,
|
||||
mode: "Normal",
|
||||
sort: {
|
||||
field: searchSortField,
|
||||
field: searchSortField(sortBy),
|
||||
direction: "Desc",
|
||||
},
|
||||
pagination: {
|
||||
@@ -143,7 +132,7 @@ export function useExplorerFiles(): ExplorerFilesResult {
|
||||
const filteredQueryInput = useMemo<FileSearchInput | null>(() => {
|
||||
if (!isFilteredMode || mode.type !== "filtered") return null;
|
||||
|
||||
const searchSortField = (() => {
|
||||
const filteredSortField = (() => {
|
||||
if (!sortBy) return "Size" as const;
|
||||
const sortMap: Record<
|
||||
string,
|
||||
@@ -163,7 +152,7 @@ export function useExplorerFiles(): ExplorerFilesResult {
|
||||
filters: mode.filters,
|
||||
mode: "Fast",
|
||||
sort: {
|
||||
field: searchSortField,
|
||||
field: filteredSortField,
|
||||
direction: "Desc",
|
||||
},
|
||||
pagination: {
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { coerceSortBy, searchSortField, sortOptionsFor } from "./sortOptions";
|
||||
|
||||
describe("sort menu state", () => {
|
||||
test("every view offers date taken", () => {
|
||||
for (const view of ["grid", "list", "column", "media"] as const) {
|
||||
expect(sortOptionsFor(view).map((o) => o.value)).toContain("datetaken");
|
||||
}
|
||||
});
|
||||
|
||||
test("a folder view keeps type and the media view keeps created", () => {
|
||||
expect(sortOptionsFor("grid").map((o) => o.value)).toEqual([
|
||||
"name",
|
||||
"modified",
|
||||
"datetaken",
|
||||
"size",
|
||||
"type",
|
||||
]);
|
||||
expect(sortOptionsFor("media").map((o) => o.value)).toEqual([
|
||||
"datetaken",
|
||||
"modified",
|
||||
"created",
|
||||
"name",
|
||||
"size",
|
||||
]);
|
||||
});
|
||||
|
||||
test("date taken survives a view change in both directions", () => {
|
||||
expect(coerceSortBy("media", "datetaken")).toBe("datetaken");
|
||||
expect(coerceSortBy("grid", "datetaken")).toBe("datetaken");
|
||||
expect(coerceSortBy("list", "datetaken")).toBe("datetaken");
|
||||
});
|
||||
|
||||
test("an order a view does not offer falls back to the view's first", () => {
|
||||
expect(coerceSortBy("media", "type")).toBe("datetaken");
|
||||
expect(coerceSortBy("grid", "created")).toBe("name");
|
||||
expect(coerceSortBy("column", undefined)).toBe("name");
|
||||
});
|
||||
|
||||
test("search sorts by capture time for date taken", () => {
|
||||
expect(searchSortField("datetaken")).toBe("CapturedAt");
|
||||
expect(searchSortField("modified")).toBe("ModifiedAt");
|
||||
expect(searchSortField("type")).toBe("Relevance");
|
||||
expect(searchSortField(null)).toBe("Relevance");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,62 @@
|
||||
import type { DirectorySortBy, MediaSortBy, SortField } from "@sd/ts-client";
|
||||
|
||||
export type ExplorerSortBy = DirectorySortBy | MediaSortBy;
|
||||
export type ExplorerViewMode = "grid" | "list" | "media" | "column";
|
||||
|
||||
export interface SortOption {
|
||||
value: ExplorerSortBy;
|
||||
label: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* The orders a view offers. Every view sorts by capture time now that the
|
||||
* listing reads it from the image facet; the media view adds birth time and
|
||||
* drops the type order, which means nothing in a roll.
|
||||
*/
|
||||
export function sortOptionsFor(viewMode: ExplorerViewMode): SortOption[] {
|
||||
if (viewMode === "media") {
|
||||
return [
|
||||
{ value: "datetaken", label: "Date Taken" },
|
||||
{ value: "modified", label: "Date Modified" },
|
||||
{ value: "created", label: "Date Created" },
|
||||
{ value: "name", label: "Name" },
|
||||
{ value: "size", label: "Size" },
|
||||
];
|
||||
}
|
||||
return [
|
||||
{ value: "name", label: "Name" },
|
||||
{ value: "modified", label: "Date Modified" },
|
||||
{ value: "datetaken", label: "Date Taken" },
|
||||
{ value: "size", label: "Size" },
|
||||
{ value: "type", label: "Type" },
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* The order to keep when a view changes: the current one when the new view
|
||||
* offers it, otherwise the view's first option, so a roll opens by capture
|
||||
* time and a folder falls back to its name.
|
||||
*/
|
||||
export function coerceSortBy(viewMode: ExplorerViewMode, sortBy: string | null | undefined): ExplorerSortBy {
|
||||
const options = sortOptionsFor(viewMode);
|
||||
const kept = options.find((option) => option.value === sortBy);
|
||||
return kept ? kept.value : options[0]!.value;
|
||||
}
|
||||
|
||||
/** The search sort field an explorer order maps to. */
|
||||
export function searchSortField(sortBy: string | null | undefined): SortField {
|
||||
switch (sortBy) {
|
||||
case "name":
|
||||
return "Name";
|
||||
case "size":
|
||||
return "Size";
|
||||
case "modified":
|
||||
return "ModifiedAt";
|
||||
case "created":
|
||||
return "CreatedAt";
|
||||
case "datetaken":
|
||||
return "CapturedAt";
|
||||
default:
|
||||
return "Relevance";
|
||||
}
|
||||
}
|
||||
@@ -652,7 +652,11 @@ export type DataVolumeSnapshot = { entries_synced: { [key in string]: number };
|
||||
/**
|
||||
* Time-based fields that can be filtered
|
||||
*/
|
||||
export type DateField = "CreatedAt" | "ModifiedAt" | "AccessedAt" | "IndexedAt";
|
||||
export type DateField = "CreatedAt" | "ModifiedAt" | "AccessedAt" | "IndexedAt" |
|
||||
/**
|
||||
* When the photo was taken, from its image facet.
|
||||
*/
|
||||
"CapturedAt";
|
||||
|
||||
/**
|
||||
* Filter for a time-based field
|
||||
@@ -957,7 +961,12 @@ export type DirectorySortBy =
|
||||
/**
|
||||
* Sort by type (directories first, then files)
|
||||
*/
|
||||
"type";
|
||||
"type" |
|
||||
/**
|
||||
* Sort by capture time (newest first), from the image facet. Files
|
||||
* without one come last.
|
||||
*/
|
||||
"datetaken";
|
||||
|
||||
export type DiscoverRemoteLibrariesInput = {
|
||||
/**
|
||||
@@ -1990,7 +1999,7 @@ export type HealthState = "ok" | "failing" | "unknown";
|
||||
/**
|
||||
* Image metadata extracted from EXIF
|
||||
*/
|
||||
export type ImageMediaData = { uuid: string; width: number; height: number; blurhash: string | null; date_taken: string | null; latitude: number | null; longitude: number | null; camera_make: string | null; camera_model: string | null; lens_model: string | null; focal_length: string | null; aperture: string | null; shutter_speed: string | null; iso: number | null; orientation: number | null; color_space: string | null; color_profile: string | null; bit_depth: string | null; artist: string | null; copyright: string | null; description: string | null };
|
||||
export type ImageMediaData = { uuid: string; width: number | null; height: number | null; blurhash: string | null; date_taken: string | null; latitude: number | null; longitude: number | null; camera_make: string | null; camera_model: string | null; lens_model: string | null; focal_length: string | null; aperture: string | null; shutter_speed: string | null; iso: number | null; orientation: number | null; color_space: string | null; color_profile: string | null; bit_depth: string | null; artist: string | null; copyright: string | null; description: string | null };
|
||||
|
||||
/**
|
||||
* Canonical input for indexing requests from any interface (CLI, API, etc.)
|
||||
@@ -4202,6 +4211,11 @@ kinds: { [key in string]: number }; tags: { [key in string]: number }; date_rang
|
||||
* Container for all structured filters
|
||||
*/
|
||||
export type SearchFilters = { file_types: string[] | null; tags: TagFilter | null; date_range: DateRangeFilter | null; size_range: SizeRangeFilter | null; content_types: ContentKind[] | null; include_hidden: boolean | null; include_archived: boolean | null;
|
||||
/**
|
||||
* Only photos whose EXIF places them (true) or does not (false). A
|
||||
* file with no image facet has no place and fails `true`.
|
||||
*/
|
||||
has_location?: boolean | null;
|
||||
/**
|
||||
* Only return files that are at risk (true) or redundant (false).
|
||||
* At risk = content exists on exactly one volume.
|
||||
@@ -4446,7 +4460,12 @@ export type SortDirection = "Asc" | "Desc";
|
||||
/**
|
||||
* Fields that can be used for sorting
|
||||
*/
|
||||
export type SortField = "Relevance" | "Name" | "Size" | "ModifiedAt" | "CreatedAt" | "IndexedAt";
|
||||
export type SortField = "Relevance" | "Name" | "Size" | "ModifiedAt" | "CreatedAt" | "IndexedAt" |
|
||||
/**
|
||||
* When the photo was taken, from its image facet. Files without one
|
||||
* sort last whichever way the order runs.
|
||||
*/
|
||||
"CapturedAt";
|
||||
|
||||
/**
|
||||
* Sorting options for search results
|
||||
|
||||
Reference in new issue
Block a user