The command no longer times out after 10 minutes when the processing
hasn't been finished yet. Instead it shows progress and logs successful
and failed spaces. The process can also be aborted using ctrl+c.
Example output:
```
$ bin/opencloud search index --all-spaces --insecure --force-rescan
[1/602] indexed space a9033d65-6c13-4556-b923-f321ab33a9ca$eb6a8a0e-2e62-4b40-bce9-360063236676!eb6a8a0e-2e62-4b40-bce9-360063236676 in 5.501190908s
[2/602] indexed space a9033d65-6c13-4556-b923-f321ab33a9ca$34b7c565-da3b-4f53-8559-825e87db4011!34b7c565-da3b-4f53-8559-825e87db4011 in 5.935075697s
[3/602] indexed space a9033d65-6c13-4556-b923-f321ab33a9ca$e08c6026-24f6-4a9e-a805-ed335b34b8da!e08c6026-24f6-4a9e-a805-ed335b34b8da in 6.016350958s
^Caborted, indexing has been stopped
```
Fixes#2592
This speeds up reindexing all spaces which happend sequentially space
after space until now. The level of concurrency can be configured using
the SEARCH_REINDEX_CONCURRENCY env var (3 by default).
The announcement is runtime-managed and injected into config.json; it is not a
static yaml config option. Exclude it from yaml (yaml:"-") so it is not
presented as a configurable knob, keeping the json tag only for the config.json
output.
Replace reva's cache.NewNatsKeyValue (which retries with backoff and can hang
the web service startup) with a direct nats.Connect + jetstream setup like the
graph service, so an unreachable or misconfigured store surfaces as a clear
startup error instead of a hang.
A statically configured announcement is not supported: it cannot be removed via
the runtime API and disappears from the running session once the admin settings
load (they read the store, not config.json). Make the store the single source
of truth, config.json always reflects it and a statically configured
options.announcement is ignored.
Replace the go-micro store with a NATS JetStream key-value bucket, created via
reva's cache.NewNatsKeyValue, so the web service no longer depends on go-micro
stores. The store methods now take a context, threaded through the config.json
render path and the management handlers. Tests use a mockery mock of
jetstream.KeyValue.
Add a logger to the announcement service and log server-side errors before
returning a 5xx, tagged with the request id so they correlate with the
request log line from the logging middleware.
Distinguish the http.MaxBytesReader limit from a malformed body: an
oversized payload now returns 413 Request Entity Too Large instead of a
generic 400.
The permission gates reading the full announcement state (including
disabled ones) as well as writing it, so name it ReadWrite to match the
other management permissions (Accounts.ReadWrite, Settings.ReadWrite, ...)
and use the READWRITE operation.
Make the store authoritative only once it manages an announcement: an
enabled record is exposed, a disabled record clears any static config, and
an empty store falls back to a statically configured
web.config.options.announcement. This matches the review request and stops
a static announcement from being silently dropped.
Always reflect the stored announcement in config.json: expose it while live
and clear it otherwise. A disabled announcement now clears the config output
instead of falling back to a statically configured one, which removes the
precedence ambiguity raised in review.
When walking the tree for creating the search index, ignore "not found"
errors. Search indexing happens asynchronously so the underlying space,
file or folder might just not exist anymore when the indexing starts.
With the recently added new file type icons in Web, the colorPalette
option in the theme.json becomes obsolete. Hence remove it.
Also see https://github.com/opencloud-eu/web/pull/2892
Per review discussion: don't document accidental behavior. The contract is
simply "percent-encode each path segment, as MS Graph requires; encode ':'
as %3A" - OpenCloud allows ':' in names (OneDrive forbids it), so it's one
more character in the mandatory encode set, not a special case.
The parser is unchanged (split on ":/", decode once). This only rewrites the
docs (code comment, acceptance feature, PR description) to state the encode
contract, and drops the tests that relied on a raw, unencoded ':' in a file
name - keeping the "%3A" test that reflects the actual contract.
Review feedback: split the anchor/path and path/suffix on the structural
delimiter ":/" instead of a bare ":". Since the path and suffix always
start with "/", ":/" is the real delimiter, and a ":" *inside* a file or
directory name (which OpenCloud allows but MS Graph/OneDrive forbid) is
kept as part of the path instead of being mistaken for a separator.
A ":" sitting at a segment boundary (e.g. a name ending in ":") stays
ambiguous and must be percent-encoded as "%3A": the split works on the
literal ":/", so "%3A" is never a delimiter and decodes back to ":". This
is now documented in the code and the acceptance feature.
Tests: colon inside a name (with and without a suffix), the Stat path
carrying the colon, and the "%3A" boundary escape.
The colon-syntax parsing is plain string operations now, but a few
comments still referred to "the previous regex" / "captures". Update them
to match: parseColonPath extracts the itemID from the path (driveID comes
from the route param), and the shape checks stand on their own.
parseColonPath split the anchor asymmetrically (root trimmed the delimiter
colon as part of a literal "/root:" prefix, item cut on it), and the root
branch's comment talked about the driveID which isn't this function's
concern. Split once at the first colon into anchor + rest, then classify
the anchor (exactly "/root", or "/items/{id}" with a single-segment id).
Same behavior, easier to follow.
Also add explicit coverage for the rule that a suffix requires a second
colon: "/root:/Documents/children" (no second colon) is the path
"/Documents/children", not the path "/Documents" with a "/children"
suffix. Two tests pin it end-to-end - it must route to the bare item (not
/items/{id}/children) and Stat must receive the full path.
Review feedback: the two colon-syntax regexes were hard to read. Since the
middleware is scoped to /drives/{driveID}, the RoutePath it inspects is just
the sub-path (/root:/... or /items/{id}:/...), so a regex buys nothing.
Replace rootColonRe/itemColonRe (and the matchInto/extract helpers) with a
single parseColonPath that uses plain string operations
(HasPrefix/TrimPrefix/TrimSuffix/Cut), one commented step at a time.
Behavior is unchanged; the existing table tests (trailing colon, no suffix,
deep paths, multi-segment suffixes, item-anchored, encoded paths) still pass
and pin it.
André Duffeck