The suite already regenerates README.md on every run; now a run whose content differs from the committed file also fails, so CI catches a stale matrix. The fresh content is on disk after the run, committing it is the fix.
The refuse specs use registered analyzers (fulltext is gone), the golden regenerates via UPDATE_GOLDEN, MappingGetResp grew an accessor, and the parity suite passes the new NewBackend signature.
The opensearch-go bump renamed the mapping-get accessor, the schema check
takes a context and a logger now, and the golden bleve mapping carries the
word-broken Name and Title.
- export mapping.SortedUnionKeys and reuse it in bleve.compareKeysExcept
instead of a copied union-of-keys block
- add Classification.AddBreaking to fold engine-specific breaking reasons and
force the verdict, replacing the identical block in the bleve and opensearch
Classify paths
- shorten the multi-line doc comments flagged as too verbose
- add reconcile_test.go: direct unit tests for Reconcile incl. the
persisted-but-errored and classify-error branches (previously only
reached indirectly through the engine integration tests)
- convert the 11 near-identical Classify It blocks to a DescribeTable
- the additive warnings advertise --all-spaces --force-rescan; a plain
walk skips unchanged documents and never backfills the new fields
- Apply checks index existence first again, so a pre-provisioned index
needs no create privilege and odd create-error shapes (string error
bodies, cluster blocks) cannot fail a healthy startup; Create on 404
keeps the typed already-exists swallow as the creation-race backstop
- number_of_replicas drift is not breaking, it is runtime-tunable and
needs no rebuild
- bleve returns the classification alongside post-persist errors and
the server warns before the error check, so the one-time additive
warning is not lost when close or reopen fails
- a golden fixture pins the marshaled bleve mapping so a dependency
bump that changes marshaling fails in CI instead of refusing every
installation in the field
Addresses the two Copilot review comments on the PR: the additive
opensearch log now matches the bleve warning (level and re-index hint),
and the refuse message spells out how to delete the index per engine
(DELETE /<name> vs removing the bleve directory).
Both engines now diff the stored/live index schema against the schema
generated from code when the service starts. A shared recursive
classifier in the mapping package is the single oracle:
- equal: start normally.
- additive (new fields without any indexed data): applied in place.
OpenSearch gets a PUT _mapping with the full code properties, bleve
persists the code mapping into the index (SetInternal + reopen) so
the new fields are properly typed immediately and later startups
classify equal. A startup warning lists the new fields because
documents indexed before the upgrade lack them until re-indexed.
- breaking (changed definitions or analyzers, removed or renamed
fields, or new fields that already contain data of unknown form):
refuse to start with an error describing the rebuild procedure
(delete the index, start, run "opencloud search index --all-spaces")
and the OC_EXCLUDE_RUN_SERVICES=search escape hatch.
PUT _mapping is deliberately only the apply mechanism, never the
judge: its merge semantics cannot see removals or renames and it
accepts in-place updatable param changes with an ack. bleve
additionally checks idx.Fields() so previously dynamically indexed
data (which leaves no schema trace in bleve) is caught, matching by
exact name and by path prefix.
While at it: the OpenSearch startup check runs with a real,
minute-bounded context instead of context.TODO(), bleve indexes are
opened with a 5s bolt_timeout so a second process fails fast instead
of hanging on the file lock, and the reversed errors.Is arguments in
bleve.NewIndex were fixed.
https://github.com/opencloud-eu/opencloud/issues/3092
* add the ability to disable the gRPC API handler
(POLICIES_GRPC_DISABLED)
* add the ability to disable the Events API handler
(POLICIES_EVENTS_DISABLED)
* add metrics
* add support for specifying rego files via the environment varirable
POLICIES_ENGINE_FILES
* for file paths specified in yaml or in POLICIES_ENGINE_FILES, support
'config:' and 'data:' path prefixes
* fix typos in the documentation, and try to make it more clear
* add metrics to the documentation
* add a section for testing in the documentation
* introduces a new top-level package pkg/metrics/ with utilities for
metrics that are backported from the groupware branch, with unit
tests
Jörn Friedrich Dreyer