Commit Graph
8 Commits
Author SHA1 Message Date
Dominik Schmidt 903ef4521b feat(search): search facets by word too, whole values only by opt-out
SharePoint's default for a text property is word breaking, so ours is too:
every keyword field gets the _words sibling unless it opts out with
NoWordBreaker, which now carries SharePoint's polarity as well. Artist,
album, camera model and the other facets match by word like name and title;
tags and favorites stay one label, ids, paths and the mime type one value.
2026-08-31 13:40:42 +02:00
Dominik Schmidt 6b07ea745c feat(search): split names and titles into words on both engines
A single word finds the names and titles that contain it, `report` finds
Report.txt, on bleve as well as on OpenSearch, which so far only did it by
accident of its dynamic mapping. Modelled like SharePoint's NoWordBreaker:
a keyword field is one whole value unless the override switches that off,
which adds a search-only _words sibling next to _lowercase, analyzed into
lowercased words (a dot is a word boundary, no stemming). The base stays the
whole value for returning and aggregating, wildcards and whole values keep
using _lowercase. Quotes do not change the meaning, a phrase is a phrase
either way, and there is no exact-match operator yet.
2026-08-31 13:40:42 +02:00
Dominik Schmidt fc41995a47 fix(search): validate CaseInsensitive against the effective field type
The guard only rejected CaseInsensitive when a non-keyword/path Type was set explicitly. With no Type, isCasedType treated the field as cased, so CaseInsensitive on an inferred numeric/bool/datetime field passed validation but produced no _lowercase sibling, and the query would silently match nothing. Validate now falls back to the inferred Go type.
2026-08-31 13:40:42 +02:00
Dominik Schmidt c680a90e49 fix(search): reject CaseInsensitive on non-keyword/path fields
CaseInsensitive routes queries to a <field>_lowercase sibling that is only generated for keyword/path fields, so marking any other type CaseInsensitive would silently match nothing. Validate now rejects it up front.
2026-08-31 13:40:42 +02:00
Dominik Schmidt ec58861e4e feat(search): per-field case-insensitive search via _lowercase siblings
Keyword and path fields always index their case-preserved base and, when CaseInsensitive is set, an additional <field>_lowercase sibling used only for matching. The KQL lowering marks a restriction case-insensitive; each backend searches the sibling and lowercases the query value the same way the sibling is precomputed at index time (Go strings.ToLower on both sides, so non-ASCII stays consistent).

Search always returns the case-preserved base, so the sibling never has to be read back. In bleve it is indexed but not stored, kept out of _all, and without doc values. In OpenSearch it deliberately stays in _source: excluding it would make every update-by-query script rebuild all siblings from the document via painless toLowerCase, which lowercases differently than Go and would drift from the query side. Keeping it in _source avoids that, and a lowercased copy of a name or path is negligible disk in a cluster.

The OpenSearch move script keeps the base and its sibling in sync by swapping the moved prefix in Path_lowercase and setting Name_lowercase from Go-lowercased params, so case-insensitive search still finds a file after it moves (previously the sibling went stale). bleve re-indexes the whole document on move/delete/restore, so its siblings stay fresh for free.

This also repairs OpenSearch path search (the query value was no longer folded to lowercase, so path:<Foo> returned nothing) and makes bleve path queries match a folder and its descendants like OpenSearch's path_hierarchy. The Path base stays case-preserved so the move/delete descendant update (an exact TermQuery on Path) matches mixed-case folders.
2026-08-31 13:40:42 +02:00
Dominik Schmidt 6695f99c90 test(search): convert mapping package tests to ginkgo
New package, so use the repo's standard test framework.
2026-08-31 13:40:42 +02:00
Dominik Schmidt 29cfdb01a0 feat(search): index Location as a geopoint on both backends
Add a TypeGeopoint field type. The libregraph Location facet is kept as an
object (retrieval / numeric queries) and a sibling <name>_geopoint field
carries the {lat,lon} form for geo-distance / bbox / polygon queries,
uniform across bleve and OpenSearch via the shared mapping. PrepareForIndex
splices the sibling in at write time.
2026-08-31 13:40:42 +02:00
Dominik Schmidt e78cad1cbe refactor: reflection-based search mapping
Build the bleve and OpenSearch index mappings from the Go struct via
reflection (json tags + per-field overrides) instead of hand-rolled
mappings and hit deserializers. New mapping package: BleveBuildMapping,
OpenSearchBuildMapping, Deserialize[T], PrepareForIndex; field decoding is
fail-soft. Mtime is typed as a date so mtime ranges are chronological on
both backends. Route CS3 facet parsing through mapping.DeserializeStringMap.

The any-valued (bleve hit) and string-valued (CS3 metadata) deserializers
share one generic fillStruct walker with a per-value setLeaf callback.
2026-08-31 13:40:42 +02:00