Commit Graph
10 Commits
Author SHA1 Message Date
Dominik Schmidt fc41995a47 fix(search): validate CaseInsensitive against the effective field type
The guard only rejected CaseInsensitive when a non-keyword/path Type was set explicitly. With no Type, isCasedType treated the field as cased, so CaseInsensitive on an inferred numeric/bool/datetime field passed validation but produced no _lowercase sibling, and the query would silently match nothing. Validate now falls back to the inferred Go type.
2026-08-31 13:40:42 +02:00
Dominik Schmidt 69c517a8b2 fix(search): nest json-tagged embedded structs instead of flattening them
resolveField marked every anonymous field embedded, so walkFields (mapping, field index, validate) and fillStruct (deserializer) flattened a json-tagged embedded struct, while conversions.To/encoding/json on the write path nests it under the tag, mapping and deserializing it at the wrong path. An anonymous field is now embedded only without a json tag name, matching encoding/json; fillStruct also recurses into a value nested struct. No current type has a tagged embedded struct, so runtime behavior is unchanged; this hardens the reflection walker.
2026-08-31 13:40:42 +02:00
Dominik Schmidt c680a90e49 fix(search): reject CaseInsensitive on non-keyword/path fields
CaseInsensitive routes queries to a <field>_lowercase sibling that is only generated for keyword/path fields, so marking any other type CaseInsensitive would silently match nothing. Validate now rejects it up front.
2026-08-31 13:40:42 +02:00
Dominik Schmidt 1bf15599f7 fix(search): write a consistent empty _lowercase sibling for empty arrays
The []any branch skipped the sibling for an empty array while the []string branch wrote an empty one; both now write it, matching the base field.
2026-08-31 13:40:42 +02:00
Dominik Schmidt e3c45dd1d0 fix(search): analyze OpenSearch full-text queries, stem Content like bleve
Single-term `content:` built an unanalyzed term query, so once this branch dropped the blanket query-value lowercasing, `content:Foo` missed on OpenSearch (bleve was unaffected, its query analyzes). Fielded full-text queries now use a match query. OpenSearch `Content` also gets a porter stemming analyzer (it used the default standard analyzer and never stemmed), so full-text search matches bleve on both case and stemming.
2026-08-31 13:40:42 +02:00
Dominik Schmidt ec58861e4e feat(search): per-field case-insensitive search via _lowercase siblings
Keyword and path fields always index their case-preserved base and, when CaseInsensitive is set, an additional <field>_lowercase sibling used only for matching. The KQL lowering marks a restriction case-insensitive; each backend searches the sibling and lowercases the query value the same way the sibling is precomputed at index time (Go strings.ToLower on both sides, so non-ASCII stays consistent).

Search always returns the case-preserved base, so the sibling never has to be read back. In bleve it is indexed but not stored, kept out of _all, and without doc values. In OpenSearch it deliberately stays in _source: excluding it would make every update-by-query script rebuild all siblings from the document via painless toLowerCase, which lowercases differently than Go and would drift from the query side. Keeping it in _source avoids that, and a lowercased copy of a name or path is negligible disk in a cluster.

The OpenSearch move script keeps the base and its sibling in sync by swapping the moved prefix in Path_lowercase and setting Name_lowercase from Go-lowercased params, so case-insensitive search still finds a file after it moves (previously the sibling went stale). bleve re-indexes the whole document on move/delete/restore, so its siblings stay fresh for free.

This also repairs OpenSearch path search (the query value was no longer folded to lowercase, so path:<Foo> returned nothing) and makes bleve path queries match a folder and its descendants like OpenSearch's path_hierarchy. The Path base stays case-preserved so the move/delete descendant update (an exact TermQuery on Path) matches mixed-case folders.
2026-08-31 13:40:42 +02:00
Dominik Schmidt 45a8096617 feat(search): derive a case-insensitive field-name index from the resource struct
mapping.FieldNameIndex walks the struct and maps a lowercased field path to the
real field name, including nested facet sub-fields. Backend-neutral.
2026-08-31 13:40:42 +02:00
Dominik Schmidt 6695f99c90 test(search): convert mapping package tests to ginkgo
New package, so use the repo's standard test framework.
2026-08-31 13:40:42 +02:00
Dominik Schmidt 29cfdb01a0 feat(search): index Location as a geopoint on both backends
Add a TypeGeopoint field type. The libregraph Location facet is kept as an
object (retrieval / numeric queries) and a sibling <name>_geopoint field
carries the {lat,lon} form for geo-distance / bbox / polygon queries,
uniform across bleve and OpenSearch via the shared mapping. PrepareForIndex
splices the sibling in at write time.
2026-08-31 13:40:42 +02:00
Dominik Schmidt e78cad1cbe refactor: reflection-based search mapping
Build the bleve and OpenSearch index mappings from the Go struct via
reflection (json tags + per-field overrides) instead of hand-rolled
mappings and hit deserializers. New mapping package: BleveBuildMapping,
OpenSearchBuildMapping, Deserialize[T], PrepareForIndex; field decoding is
fail-soft. Mtime is typed as a date so mtime ranges are chronological on
both backends. Route CS3 facet parsing through mapping.DeserializeStringMap.

The any-valued (bleve hit) and string-valued (CS3 metadata) deserializers
share one generic fillStruct walker with a per-value setLeaf callback.
2026-08-31 13:40:42 +02:00