Commit Graph

44 Commits

Author SHA1 Message Date
Jörn Friedrich Dreyer
f54582ddc4 fix event consumers
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-09-15 13:49:41 +02:00
Jörn Friedrich Dreyer
99dee5ae77 allow disabling search grpc/event servers
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-09-15 12:42:56 +02:00
Juan Pablo Villafáñez
c0b4a5daa0 chore: change constant name to camelcase 2025-09-08 17:32:36 +02:00
Juan Pablo Villafáñez
ca2dc823ef feat: use names for connections to the nats event bus 2025-09-08 17:32:35 +02:00
fschade
f615ccc896 enhancement(search): implement batch api 2025-09-02 13:06:08 +02:00
fschade
82e75e19c1 enhancement(search): move bleve engine into its own package and clean up the search batch processing implementation 2025-08-28 15:53:29 +02:00
fschade
42b794e01a refactor(search): cleanup for review 2025-08-28 09:32:05 +02:00
fschade
a9d21bbb15 enhancement(search): allow to set the resource index name 2025-08-28 09:32:05 +02:00
fschade
85e436b2bb fix(search): open-search engine interface compatibility 2025-08-28 09:31:38 +02:00
fschade
63e71b5bc4 enhancement(search): allow to configure open-search as search backend 2025-08-28 09:30:59 +02:00
André Duffeck
3c8e2dacfd Improve indexing performance using batches 2025-08-04 15:15:54 +02:00
André Duffeck
791b4df173 Collect metrics about search and index durations 2025-07-17 15:23:57 +02:00
André Duffeck
400b9a5d30 Expose nats metrics of the search service 2025-07-17 14:33:23 +02:00
André Duffeck
4998e1e09f Inject a stream into HandleEvents to increase testability 2025-07-16 10:29:34 +02:00
André Duffeck
608af81913 Switch to the raw nats consumer instead of the go-micro events
This allows more direct control of the nats events. As a first step
we'll now acknowledge events manually, but in the future we'll add
metrics etc. to gain more insight into the queues.
2025-07-15 09:57:19 +02:00
Ralf Haferkamp
c46e0c4b6b Fix build with go 1.24
Go 1.24 does some stricter checks now on arguments to functions
accepting format strings.
This commit fixes a couple of "non-constant format string in call
to ..." errors when running the unit tests.
2025-03-10 10:43:34 +01:00
André Duffeck
e8d35e1280 Use the opencloud reva from now on 2025-01-21 11:16:38 +01:00
André Duffeck
8c8cdd9310 Adapt protobuf naming 2025-01-15 11:04:32 +01:00
Jörn Friedrich Dreyer
b07b5a1149 use plain pkg module
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-01-13 16:42:19 +01:00
Jörn Friedrich Dreyer
8e028f17e9 change module name
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-01-13 09:58:18 +01:00
jkoberg
4e28fc45dc feat(search): allow indexing all spaces
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2024-06-25 10:01:37 +02:00
jkoberg
63686d6cd7 feat(search): remove userid from indexspace methods
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2024-06-25 10:01:37 +02:00
jkoberg
3e55223f2d allow authentication for events
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-12-19 10:44:05 +01:00
kobergj
377a23ee7d Non-durable SSE streams (#7986)
* make sse streams non-durable

Signed-off-by: jkoberg <jkoberg@owncloud.com>

* bump reva

Signed-off-by: jkoberg <jkoberg@owncloud.com>

---------

Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-12-18 11:47:25 +01:00
Florian Schade
844783b6f9 [full-ci] - use KQL as default search query language (#7212)
* enhancement: use kql as default search query language

* enhancement: add support for unicode search queries

* fix: escape bleve field query whitespace

* fix: search related acceptance tests

* enhancement: remove legacy search query language

* enhancement: add support for kql dateTime restriction node types

* chore: bump web to v8.0.0-alpha.2

* fix: failing search api test

* enhancement: search bleve query compiler use DateRangeQuery as DateTimeNode counterpart

* enhancement: support for colon operators in dateTime kql queries
2023-09-07 11:13:33 +02:00
Florian Schade
ed0dbce978 enhancement: Keyword Query Language (KQL) search syntax support (#7043)
* feat(search): introduce search query package

With the increasing complexity of how we organize our resources, the search must also be able to find them using entity properties.

The query package provides the necessary functionality to do this.

This makes it possible to search for resources via KQL, the microsoft spec is largely covered and can be used for this.

In the current state, the legacy query language is still used, in a future update this will be deprecated and KQL will become the standard
2023-08-28 16:41:36 +02:00
Daniel Swärd
49fc22d532 Add missing tracing references to multiple services. (#7110) 2023-08-23 15:17:17 +02:00
jkoberg
49cdcad129 unify eventstream creation
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-08-16 10:14:12 +02:00
Jörn Friedrich Dreyer
2ea3b8c400 unmarshal token to filter spaces for current user (#6596)
* unmarshal token to filter spaces for current user

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Fix tests

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: André Duffeck <andre.duffeck@firondu.de>
2023-06-22 16:15:17 +02:00
Florian Schade
4f26424db6 [full-ci] enhancement: use reva client pool selectors (#6452)
* enhancement: use reva client pool selectors

register mock service to registry and pass tests

* enhancement: bump reva

* Fix a couple of linter issues

---------

Co-authored-by: Ralf Haferkamp <rhaferkamp@owncloud.com>
2023-06-08 12:41:04 +02:00
Jörn Friedrich Dreyer
3aa864aecc Remove Staticcheck warnings (#5394)
* remove deprecated ioutil imports

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* remove duplicate imports

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* remove unused values

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* remove some unused types

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* remove deprecated ioutil import

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-01-13 15:29:14 +01:00
jkoberg
b29ffeb3a9 use stream pkg instead server pkg
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2022-12-20 09:53:11 +01:00
Florian Schade
2404eff48e [full-ci] experimental search backport (#5221)
* experimental search backport
fix basic extractor resource name
move escapeQuery regex into global variable
minor pr review changes
rename DebounceDuration env variable
add document title and content when rebuilding bleve resource

Co-authored-by: David Christofas <dchristofas@owncloud.com>
2022-12-13 14:22:41 +01:00
Jörn Friedrich Dreyer
0f7dba53fb use min tls 1.2 (#4969)
* use min tls 1.2

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add changelog

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2022-11-03 16:08:56 +01:00
André Duffeck
12bdf29ac4 Change the default reindex debounce duration to 1s.
Also make it configurable using an env var.
2022-11-03 13:15:54 +01:00
Ralf Haferkamp
3d57f5cc21 Introduce TLS Settings for all reva grpc services and clients 2022-10-25 11:50:08 +02:00
David Christofas
45b92fc56f add config option to enable or disable TLS for nats 2022-10-21 12:40:21 +02:00
David Christofas
4623b6c8e7 Nats tls (#4781)
* use tls for nats connections

* add config options for nats client tls config

* add nats tls config to CI

* add function to create a certpool

* add option to provide a rootCA to validate the server's TLS certificate

* add option to provide a rootCA to validate the server's TLS certificate

* add option to provide a rootCA to validate the server's TLS certificate

* add option to provide a rootCA to validate the server's TLS certificate

* configure nats clients in reva to use tls
2022-10-12 14:56:47 +02:00
Andre Duffeck
68b17eb68d Search dav spaces (#4661)
* Add a Ref option to SearchRequest, e.g. for limiting the search to a space

* Enable REPORT requests for /dav/spaces URLs

* Limit the search to the according space in case of /dav/spaces searches

* Add changelog

* Adapt expected failures

* Comment exported functions
2022-09-27 10:38:15 +02:00
André Duffeck
0fb8721da6 Return the number of total matches 2022-07-13 10:42:50 +02:00
André Duffeck
b6d374f683 Support specifying a search limit 2022-07-04 16:35:44 +02:00
André Duffeck
57c4f056ef Return proper error codes, e.g. when providing an empty query 2022-07-04 16:35:44 +02:00
Christian Richter
f8f1320501 refactor extensions -> services
Signed-off-by: Christian Richter <crichter@owncloud.com>
2022-06-27 14:05:36 +02:00
Christian Richter
78064e6bab rename folder extensions -> services
Signed-off-by: Christian Richter <crichter@owncloud.com>
2022-06-27 14:05:36 +02:00