Commit Graph
506 Commits
Author SHA1 Message Date
Jörn Friedrich Dreyer 1bafa4bd18 align the handler verifier, update docs
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-09-08 17:11:55 +02:00
Dominik Schmidt be83ed19e3 fix(search): take extraction facets from the main document only
The recursive tika response lists the file first, then its embedded
resources (cover art, thumbnails, the clip appended to a motion photo).
The loop applied getImage/getPhoto/getLocation/getAudio/getLivePhoto to
every part, so an mp3's embedded cover art leaked a 200x200 image facet
onto the track (and an embedded EXIF image would leak photo/location).
Read those facets from metas[0] only, like the video facet already does;
the loop now only concatenates title/content and detects the motion
photo clip.
2026-09-08 14:52:35 +02:00
Dominik Schmidt 5995938c34 chore(search): trim the live photo comments 2026-09-03 03:34:33 +02:00
Dominik Schmidt 0a4f3ed786 fix(search): keep livePhoto.contentId out of the search siblings 2026-09-03 03:34:33 +02:00
Dominik Schmidt a7214b3d73 fix(search): drop the live photo facet on an empty pairing id 2026-09-03 03:34:33 +02:00
Dominik Schmidt 81b0df1ad4 feat(search): live photo facet 2026-09-03 03:34:33 +02:00
Dominik Schmidt 3515594787 fix(search): keep the video facet on the file itself 2026-09-03 01:09:41 +02:00
Dominik Schmidt a8318fe03b refactor(search): take the motion photo video size from the video tika extracted 2026-09-03 00:32:07 +02:00
Dominik Schmidt 706aaf02f2 refactor(search): decide the motion photo from the file xmp and its extracted video 2026-09-03 00:32:06 +02:00
Dominik Schmidt 88f21d804b refactor(search): restore cs3 Retrieve, the download split had only one caller left 2026-09-03 00:31:40 +02:00
Dominik Schmidt 8aebf80124 refactor(search): confirm the motion photo video by type, not by name 2026-09-03 00:31:40 +02:00
Dominik Schmidt ae9fd595d9 chore(search): drop the dead legacy tika key 2026-09-03 00:31:40 +02:00
Dominik Schmidt 709962b616 feat(search): confirm the motion photo video via tika instead of reading bytes 2026-09-03 00:31:40 +02:00
Dominik Schmidt a2050d1fb9 feat(search): index and expose the motion photo facet 2026-09-03 00:31:40 +02:00
Dominik Schmidt 0f4b95b9ae feat(search): add RetrieveRange to the content retriever 2026-09-03 00:27:04 +02:00
Dominik Schmidt 42876ca61c refactor(search): let getFirstValue try multiple metadata keys 2026-09-03 00:27:04 +02:00
Jörn Friedrich Dreyer f5a064395a Merge pull request #3201 from opencloud-eu/feat/search-video-facet
feat(search): video facet
2026-09-03 00:20:02 +02:00
amir 67124a6c37 fix(search): ignore invalid exposure denominators 2026-09-01 20:24:44 +03:30
amir db4cee156b fix(search): ignore invalid photo exposure times 2026-09-01 20:24:19 +03:30
Dominik Schmidt 727c986de8 feat(search): index and expose the video facet 2026-09-01 16:26:01 +02:00
Dominik Schmidt df8d0e8fd3 Merge pull request #3198 from opencloud-eu/feat/search-tika-facets
feat(search): extract more data from tika 4 (if available)
2026-09-01 11:45:11 +02:00
Dominik Schmidt 7d4c6d6524 Merge pull request #3424 from opencloud-eu/feat/search-scope
feat(search): scope searches to a drive via the driveId field
2026-09-01 09:36:20 +02:00
Dominik Schmidt 5e4f124e78 fix(search): find the language endpoint on tika 4, keep the legacy path 2026-09-01 09:09:55 +02:00
Dominik Schmidt a1e8aa783d fix(search): reject NaN coordinates and non-positive bitrates 2026-09-01 09:09:55 +02:00
Dominik Schmidt ae9ec4ce70 fix(search): harden facet value parsing 2026-09-01 09:09:55 +02:00
Dominik Schmidt c03b9c7229 fix(search): gate the audio facet inside getAudio, keep facets across meta entries 2026-09-01 09:09:55 +02:00
Dominik Schmidt 77bb974f9b test(search): run acceptance and CI against the released tika 4.0.0
Both tika pins (local compose and the CI service) point at apache/tika:4.0.0-full; the search README links tika version-less.
2026-09-01 09:09:55 +02:00
Dominik Schmidt 1be01ae50c fix(search): read the tika 4 content key, keep the legacy one
Tika 4.0.0 renamed the meta prefix from X-TIKA: to tk:.
2026-09-01 09:09:55 +02:00
Dominik Schmidt 2b4a6fd739 feat(search): populate audio and gps-altitude facets from tika, drop the audio-only gate 2026-09-01 09:09:55 +02:00
Dominik Schmidt 7e0f536b83 refactor(search): split the tika extractor and its tests into per-facet files 2026-09-01 09:09:55 +02:00
Jörn Friedrich Dreyer 8028e3e310 Merge pull request #3435 from fschade/test-search-cjk-parity
test(search): cover chinese search in the parity suite
2026-09-01 08:24:34 +02:00
Dominik Schmidt 7d75b98c69 Merge pull request #3423 from opencloud-eu/test/parity-readme-staleness
test(search): fail the parity suite when the committed matrix is stale
2026-09-01 08:24:15 +02:00
Florian Schade 9e8403818c test(search): cover chinese search in the parity suite 2026-09-01 07:46:03 +02:00
Dominik Schmidt ef41429838 test(search): rename the update flag to UPDATE_SEARCH_PARITY_MATRIX 2026-09-01 07:35:15 +02:00
Dominik Schmidt 2041fdc259 test(search): comment the root-id assumption, pin the OR fan-out 2026-08-31 21:14:58 +02:00
Dominik Schmidt 8e68a34b8f fix(search): keep wildcard drive ids out of completion and pinning 2026-08-31 21:14:04 +02:00
Dominik Schmidt 774c732f52 test(search): pin the driveId scope in the parity suite
SCOPE-06..09 cover the completed root id, negation and composition; the cross-space fixture root follows the space-root convention (opaque id = space id).
2026-08-31 15:23:25 +02:00
Dominik Schmidt 95f175fe10 perf(search): prune the space fan-out for drive-pinned queries
When top-level AND conjuncts pin the query to a single root
(driveId/RootID restrictions), only that space's index is asked; the
restriction itself stays in the query, so this is purely an
optimization. Conservative by design: any top-level OR, negated or
group-nested restriction leaves the fan-out untouched, searching a
space too many is wasted work while skipping one would be wrong.
Mountpoints are kept for result path mapping.

Costs one extra parse of the query in the service; parsing once and
handing the AST to the engines (which currently re-parse per space) is
a follow-up that changes the engine interface.
2026-08-31 15:23:25 +02:00
Dominik Schmidt 094baab7be feat(search): driveId as a query field
scope: takes an opaque resource id, which is hostile to hand-written
queries. Accept driveId:"<storage$space>" as a regular KQL field
instead: it resolves to the indexed RootID, and a bare drive id is
completed to the root resource id (a space root's opaque id is its
space id). Full root ids pass through untouched.

Combined with path: this gives a readable location scope without any
token stripping: both are plain fields, so they compose with groups,
OR and NOT like everything else.
2026-08-31 15:23:25 +02:00
Dominik Schmidt feb13d426b test(search): name the update flag in the generated header 2026-08-31 15:16:15 +02:00
Dominik Schmidt b1533a3135 test(search): write the matrix only with UPDATE_MATRIX
A plain run compares and fails on a stale README without touching the working tree; UPDATE_MATRIX=true regenerates it.
2026-08-31 15:16:15 +02:00
Dominik Schmidt 874d8fdac2 test(search): fail the parity suite when the committed matrix is stale
The suite already regenerates README.md on every run; now a run whose content differs from the committed file also fails, so CI catches a stale matrix. The fresh content is on disk after the run, committing it is the fix.
2026-08-31 15:16:15 +02:00
Dominik Schmidt 0e2e2876f4 test(search): a golden diff says whether it needs a SchemaVersion bump
The failure runs the classifier on golden vs generated: additive means regenerate only, breaking means bump too.
2026-08-31 15:15:18 +02:00
Dominik Schmidt e11ce95d95 test(search): golden for the generated OpenSearch index definition
Pins the shipped schema as a reviewable diff; regenerate with UPDATE_GOLDEN=1.
2026-08-31 15:12:54 +02:00
Dominik Schmidt 7493f189a0 chore(search): adapt tests and golden to the v4 base
The refuse specs use registered analyzers (fulltext is gone), the golden regenerates via UPDATE_GOLDEN, MappingGetResp grew an accessor, and the parity suite passes the new NewBackend signature.
2026-08-31 15:12:54 +02:00
Dominik Schmidt 53fa66f120 chore(search): follow the rebased base
The opensearch-go bump renamed the mapping-get accessor, the schema check
takes a context and a logger now, and the golden bleve mapping carries the
word-broken Name and Title.
2026-08-31 15:12:54 +02:00
Dominik Schmidt 519dd112ea review: note SchemaVersion bump on bleve marshaling drift in golden test 2026-08-31 15:12:54 +02:00
Dominik Schmidt 414247d6c4 review: use any over interface{}, clone slice in AddBreaking 2026-08-31 15:12:54 +02:00
Dominik Schmidt 7ff2643d23 refactor(search): share SortedUnionKeys + Classification.AddBreaking
- export mapping.SortedUnionKeys and reuse it in bleve.compareKeysExcept
  instead of a copied union-of-keys block
- add Classification.AddBreaking to fold engine-specific breaking reasons and
  force the verdict, replacing the identical block in the bleve and opensearch
  Classify paths
2026-08-31 15:12:54 +02:00
Dominik Schmidt a0521f0260 review: trim verbose comments, cover Reconcile, table-driven Classify tests
- shorten the multi-line doc comments flagged as too verbose
- add reconcile_test.go: direct unit tests for Reconcile incl. the
  persisted-but-errored and classify-error branches (previously only
  reached indirectly through the engine integration tests)
- convert the 11 near-identical Classify It blocks to a DescribeTable
2026-08-31 15:12:54 +02:00