The command no longer times out after 10 minutes when the processing
hasn't been finished yet. Instead it shows progress and logs successful
and failed spaces. The process can also be aborted using ctrl+c.
Example output:
```
$ bin/opencloud search index --all-spaces --insecure --force-rescan
[1/602] indexed space a9033d65-6c13-4556-b923-f321ab33a9ca$eb6a8a0e-2e62-4b40-bce9-360063236676!eb6a8a0e-2e62-4b40-bce9-360063236676 in 5.501190908s
[2/602] indexed space a9033d65-6c13-4556-b923-f321ab33a9ca$34b7c565-da3b-4f53-8559-825e87db4011!34b7c565-da3b-4f53-8559-825e87db4011 in 5.935075697s
[3/602] indexed space a9033d65-6c13-4556-b923-f321ab33a9ca$e08c6026-24f6-4a9e-a805-ed335b34b8da!e08c6026-24f6-4a9e-a805-ed335b34b8da in 6.016350958s
^Caborted, indexing has been stopped
```
Fixes#2592
This speeds up reindexing all spaces which happend sequentially space
after space until now. The level of concurrency can be configured using
the SEARCH_REINDEX_CONCURRENCY env var (3 by default).
That can be helpful when the search service configuration has changed,
e.g. by enabling TIKA. Previously files that had already been indexed
were not indexed again and thus were no part of the fulltext index.
Fixes#2285Fixes#2578
This allows more direct control of the nats events. As a first step
we'll now acknowledge events manually, but in the future we'll add
metrics etc. to gain more insight into the queues.
Go 1.24 does some stricter checks now on arguments to functions
accepting format strings.
This commit fixes a couple of "non-constant format string in call
to ..." errors when running the unit tests.
* enhancement: use kql as default search query language
* enhancement: add support for unicode search queries
* fix: escape bleve field query whitespace
* fix: search related acceptance tests
* enhancement: remove legacy search query language
* enhancement: add support for kql dateTime restriction node types
* chore: bump web to v8.0.0-alpha.2
* fix: failing search api test
* enhancement: search bleve query compiler use DateRangeQuery as DateTimeNode counterpart
* enhancement: support for colon operators in dateTime kql queries
* feat(search): introduce search query package
With the increasing complexity of how we organize our resources, the search must also be able to find them using entity properties.
The query package provides the necessary functionality to do this.
This makes it possible to search for resources via KQL, the microsoft spec is largely covered and can be used for this.
In the current state, the legacy query language is still used, in a future update this will be deprecated and KQL will become the standard
* unmarshal token to filter spaces for current user
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* Fix tests
---------
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: André Duffeck <andre.duffeck@firondu.de>
* enhancement: use reva client pool selectors
register mock service to registry and pass tests
* enhancement: bump reva
* Fix a couple of linter issues
---------
Co-authored-by: Ralf Haferkamp <rhaferkamp@owncloud.com>
* experimental search backport
fix basic extractor resource name
move escapeQuery regex into global variable
minor pr review changes
rename DebounceDuration env variable
add document title and content when rebuilding bleve resource
Co-authored-by: David Christofas <dchristofas@owncloud.com>
* use min tls 1.2
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* add changelog
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Florian Schade