Commit Graph
61 Commits
Author SHA1 Message Date
Dominik Schmidt be83ed19e3 fix(search): take extraction facets from the main document only
The recursive tika response lists the file first, then its embedded
resources (cover art, thumbnails, the clip appended to a motion photo).
The loop applied getImage/getPhoto/getLocation/getAudio/getLivePhoto to
every part, so an mp3's embedded cover art leaked a 200x200 image facet
onto the track (and an embedded EXIF image would leak photo/location).
Read those facets from metas[0] only, like the video facet already does;
the loop now only concatenates title/content and detects the motion
photo clip.
2026-09-08 14:52:35 +02:00
Dominik Schmidt 5995938c34 chore(search): trim the live photo comments 2026-09-03 03:34:33 +02:00
Dominik Schmidt a7214b3d73 fix(search): drop the live photo facet on an empty pairing id 2026-09-03 03:34:33 +02:00
Dominik Schmidt 81b0df1ad4 feat(search): live photo facet 2026-09-03 03:34:33 +02:00
Dominik Schmidt 3515594787 fix(search): keep the video facet on the file itself 2026-09-03 01:09:41 +02:00
Dominik Schmidt a8318fe03b refactor(search): take the motion photo video size from the video tika extracted 2026-09-03 00:32:07 +02:00
Dominik Schmidt 706aaf02f2 refactor(search): decide the motion photo from the file xmp and its extracted video 2026-09-03 00:32:06 +02:00
Dominik Schmidt 88f21d804b refactor(search): restore cs3 Retrieve, the download split had only one caller left 2026-09-03 00:31:40 +02:00
Dominik Schmidt 8aebf80124 refactor(search): confirm the motion photo video by type, not by name 2026-09-03 00:31:40 +02:00
Dominik Schmidt ae9fd595d9 chore(search): drop the dead legacy tika key 2026-09-03 00:31:40 +02:00
Dominik Schmidt 709962b616 feat(search): confirm the motion photo video via tika instead of reading bytes 2026-09-03 00:31:40 +02:00
Dominik Schmidt a2050d1fb9 feat(search): index and expose the motion photo facet 2026-09-03 00:31:40 +02:00
Dominik Schmidt 0f4b95b9ae feat(search): add RetrieveRange to the content retriever 2026-09-03 00:27:04 +02:00
Dominik Schmidt 42876ca61c refactor(search): let getFirstValue try multiple metadata keys 2026-09-03 00:27:04 +02:00
Jörn Friedrich Dreyer f5a064395a Merge pull request #3201 from opencloud-eu/feat/search-video-facet
feat(search): video facet
2026-09-03 00:20:02 +02:00
amir 67124a6c37 fix(search): ignore invalid exposure denominators 2026-09-01 20:24:44 +03:30
amir db4cee156b fix(search): ignore invalid photo exposure times 2026-09-01 20:24:19 +03:30
Dominik Schmidt 727c986de8 feat(search): index and expose the video facet 2026-09-01 16:26:01 +02:00
Dominik Schmidt 5e4f124e78 fix(search): find the language endpoint on tika 4, keep the legacy path 2026-09-01 09:09:55 +02:00
Dominik Schmidt a1e8aa783d fix(search): reject NaN coordinates and non-positive bitrates 2026-09-01 09:09:55 +02:00
Dominik Schmidt ae9ec4ce70 fix(search): harden facet value parsing 2026-09-01 09:09:55 +02:00
Dominik Schmidt c03b9c7229 fix(search): gate the audio facet inside getAudio, keep facets across meta entries 2026-09-01 09:09:55 +02:00
Dominik Schmidt 1be01ae50c fix(search): read the tika 4 content key, keep the legacy one
Tika 4.0.0 renamed the meta prefix from X-TIKA: to tk:.
2026-09-01 09:09:55 +02:00
Dominik Schmidt 2b4a6fd739 feat(search): populate audio and gps-altitude facets from tika, drop the audio-only gate 2026-09-01 09:09:55 +02:00
Dominik Schmidt 7e0f536b83 refactor(search): split the tika extractor and its tests into per-facet files 2026-09-01 09:09:55 +02:00
Dominik Schmidt 450f380198 fix(search): type mtime as a date 2026-08-31 13:40:42 +02:00
Dominik Schmidt e78cad1cbe refactor: reflection-based search mapping
Build the bleve and OpenSearch index mappings from the Go struct via
reflection (json tags + per-field overrides) instead of hand-rolled
mappings and hit deserializers. New mapping package: BleveBuildMapping,
OpenSearchBuildMapping, Deserialize[T], PrepareForIndex; field decoding is
fail-soft. Mtime is typed as a date so mtime ranges are chronological on
both backends. Route CS3 facet parsing through mapping.DeserializeStringMap.

The any-valued (bleve hit) and string-valued (CS3 metadata) deserializers
share one generic fillStruct walker with a per-value setLeaf callback.
2026-08-31 13:40:42 +02:00
Florian Schade bd4287762e fix(search): answer the same on both engines
both engines now agree on names, titles, tags, paths, types, sizes, dates,
hidden flags, facet values and wildcards. quotes only delimit phrases and
the equals operator matches the whole field value, following the kql spec.
the index name carries a generation so a changed mapping starts on a fresh
index, MIGRATION.md says how to fill it.
2026-08-31 10:44:03 +02:00
Florian Schade d0e3f14539 chore: remove loop var references 2026-04-23 17:11:55 +02:00
Dominik Schmidt 2fc33d6e60 refactor(search): round xmpDM:duration to the nearest millisecond
Address review feedback: a straight int64 cast truncates toward zero,
so Tika values that produce results like 1234.999... millisecond would
land at 1234 ms instead of 1235 ms. Round before casting so durations
are as accurate as float64 allows.
2026-04-21 15:16:57 +02:00
Dominik Schmidt 3c59935012 fix(search): parse tika xmpDM:duration as a float
Tika emits xmpDM:duration as seconds in floating-point form (for
example "154.57379150390625"), so strconv.ParseInt rejected every
value and the field was silently dropped — every indexed audio item
ended up without a duration.

Parse the value with strconv.ParseFloat and convert to milliseconds
ourselves. Adjust the existing extractor test to cover the fractional
case.
2026-04-21 15:16:57 +02:00
André Duffeck cd0831aa10 We no longer manage favorites via arbitrary metadata 2026-03-13 09:38:28 +01:00
André Duffeck ce5ec1b3dc Add support for favorites to the search service 2026-03-13 09:38:28 +01:00
06kellyjac 2a3a20693a build: Modify mockery config to work with v3
Maintaining the positioning of the files from v2 to reduce cognitive
load.
Indentation of yaml files now matches `.editorconfig`.
All mock files regenerated.
Added empty `{}` following convention from `mockery init` etc.
Removed directory specification where it would already match.
2025-06-19 16:25:39 +01:00
Ralf Haferkamp 3dc9c0cb9a Switch to opencloud-eu fork of libre-graph-api-go 2025-05-15 14:11:35 +02:00
Jörn Friedrich Dreyer 3c38a3e3ed fix mockery warnings
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-04-29 11:01:17 +02:00
Jörn Friedrich Dreyer f0e62323cb update generated code
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-03-27 10:06:45 +01:00
Ralf Haferkamp 2678d0bb89 Bump mockery to 2.53.0
Should fix build on go1.24. Also fix ci-go-generate step in multiarch
Dockerfile.

Closes: #289
2025-03-03 16:02:59 +01:00
Klaas Freitag 669ad7c769 Rename demo users
See https://github.com/opencloud-eu/internal/issues/42
2025-01-28 18:03:12 +01:00
André Duffeck e8d35e1280 Use the opencloud reva from now on 2025-01-21 11:16:38 +01:00
André Duffeck 6bab922c7c Rebrand search 2025-01-17 09:25:36 +01:00
Jörn Friedrich Dreyer b07b5a1149 use plain pkg module
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-01-13 16:42:19 +01:00
Ralf Haferkamp 01cde87af0 Adjust mockery configs for new name 2025-01-13 15:05:38 +01:00
Jörn Friedrich Dreyer 8e028f17e9 change module name
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-01-13 09:58:18 +01:00
Benedikt Kulmann 21a9992980 feat: add image and photo metadata to search (#8504)
* feat: add image and photo metadata to search
2024-05-21 10:21:15 +02:00
Ralf Haferkamp 3d3f8949f5 Bump mockery to 2.40.2
to address issues when building with go1.22:
https://github.com/vektra/mockery/pull/753
2024-02-09 11:48:45 +01:00
Jörn Friedrich Dreyer fad94d2038 bump mockery, add test stub for oidc_auth.go, align mock generation (#8321)
* bump mockery, add test stub for oidc_auth.go

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* use .mockery.yaml for all mocks

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* drop legacy go:generate mockery

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* align mock placement

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2024-02-01 10:07:44 +01:00
Dominik Schmidt 7f83b5020d feat(location): add tika test 2023-12-06 15:13:09 +01:00
Dominik Schmidt 4c27b365fa feat(graph): add location facet to driveItems 2023-12-06 15:13:09 +01:00
Dominik Schmidt 07edd1e23a chore(search): move audio metadata extraction in tika to dedicated method 2023-12-06 15:13:09 +01:00