Commit Graph

56 Commits

Author SHA1 Message Date
André Duffeck
2b9e193584 Rebrand storage-users 2025-01-17 11:44:18 +01:00
Jörn Friedrich Dreyer
318ca21645 metrics, log and supervisor
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-01-14 16:09:54 +01:00
Jörn Friedrich Dreyer
8e028f17e9 change module name
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-01-13 09:58:18 +01:00
jkoberg
c80254c4f1 feat(ocis): remove ocm store
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2024-10-15 17:24:51 +02:00
jkoberg
4a911ac047 feat(storage-users): remove metadata backend envvar
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2024-09-19 12:29:41 +02:00
André Duffeck
d199e79c71 Set a default debounce delay 2024-09-17 10:33:32 +02:00
Jörn Friedrich Dreyer
99c6b6648e no need to introduce dedicated roles
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2024-08-13 16:10:19 +02:00
André Duffeck
f5de0a09cf Enable caching for the xattrs backend 2024-07-12 08:39:30 +02:00
Andre Duffeck
b8e7160880 Merge pull request #9460 from aduffeck/posixfs-events
Fix posixfs events
2024-07-08 08:35:05 +02:00
jkoberg
dcac0cf201 feat(storage-provider): add option to disable versioning
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2024-06-27 15:48:03 +02:00
André Duffeck
0028013794 Enable async uploads for posixfs 2024-06-25 08:40:50 +02:00
André Duffeck
b51cae14ba Onle enable fs-watching for the storageprovider, not the dataprovider
We need to make sure that the storageprovider tree is used for watching
the fs and not the dataprovider, because the dataprovider doesn't have a
mount_id set which is needed to emit proper events.
2024-06-25 08:40:50 +02:00
André Duffeck
5781723432 Add missing events config 2024-06-25 08:40:50 +02:00
André Duffeck
126b083eaf Add config var for using the space groups 2024-05-29 12:14:54 +02:00
André Duffeck
8cd39754c7 Add config vars for space path templates 2024-05-29 12:14:54 +02:00
André Duffeck
4517c5ee19 Make the list of gpfs watchfolder kafka brokers configurable 2024-05-29 12:14:54 +02:00
André Duffeck
2cf823157f Expose the idcache configuration 2024-05-29 12:14:54 +02:00
André Duffeck
69c84e0ee6 Expose the fs watcher configuration 2024-05-29 12:14:53 +02:00
André Duffeck
96645322e7 Configure project spaces layout 2024-05-29 12:14:53 +02:00
André Duffeck
b231457231 Enable treetime and treesize accounting 2024-05-29 12:14:53 +02:00
André Duffeck
aab219a38f Initial support for the posixfs 2024-05-29 12:14:53 +02:00
Jörn Friedrich Dreyer
5ed57cc09a Bump reva deps (#8412)
* bump dependencies

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* bump reva and add config options

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2024-02-21 10:20:36 +01:00
jkoberg
cec6ea50eb remove unused stat cache
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2024-01-24 10:39:32 +01:00
jkoberg
13bb2785d4 fix cache authentication
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2024-01-24 10:39:30 +01:00
jkoberg
10628f399d fix events authentication
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2024-01-24 10:39:09 +01:00
jkoberg
0d1669b3bd allowing disabling persistence of caches
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-12-15 13:25:10 +01:00
jkoberg
49cdcad129 unify eventstream creation
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-08-16 10:14:12 +02:00
André Duffeck
fade832a3e Make the async propagation delay configurable 2023-07-31 10:26:19 +02:00
André Duffeck
5aa9e618f1 Make the storage-users propagator configurable 2023-07-31 10:26:19 +02:00
Daniël Franke
063b05140f Convert storage-users to use service tracing.
This converts the storage users service to use the service tracing provider.
2023-07-28 11:25:43 +02:00
Ralf Haferkamp
e32aef17cd storage-users: Add config option for graceful shutdown timeout
Add STORAGE_USERS_GRACEFUL_SHUTDOWN_TIMEOUT setting to allow a graceful
shutdown of the storage-users service. This currently only applicable
when running storage-user as a sepearate service.

Setting STORAGE_USERS_GRACEFUL_SHUTDOWN_TIMEOUT to a non-zero value
gives the storage-users service a chance to cleanly shutdown and finish
any in progess tasks (e.g. metadata propagation) before exiting.

Partial-Fix: #6602
2023-07-18 16:47:26 +02:00
Jörn Friedrich Dreyer
b8bdd4573a introduce otlp tracing (#5132)
* introduce otel tracing

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* use new trace provider initialization

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* work

* bump reva

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* remove commented code

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add vendor changes

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-05-26 22:21:03 +02:00
jkoberg
2a5caf74e8 idcache configuration
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-05-22 09:51:37 +02:00
Andre Duffeck
129489203b Cleanup and improve the caching config (#6148)
* Cleanup and improve the caching config

* bump reva

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* disable stat cache

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Bump reva

* Linter fixes

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-04-27 12:35:30 +02:00
Andre Duffeck
77bb3d8bcd [full-ci] Refactor stores (#6019)
* Streamline the store implementation with and into reva

* Adapt to the cache/store refactoring in reva

* Streamline config options and their env vars

* Apply suggestions from code review

Co-authored-by: Martin <github@diemattels.at>

* Use the same database for all stores

* Bump reva

* Configure stat and filemetadata cache separately

* Fix default config

---------

Co-authored-by: Martin <github@diemattels.at>
2023-04-24 15:13:35 +02:00
Andre Duffeck
0bc471007e Make the number of concurrent goroutines in decomposedfs configurable (#5887)
* Make the number of concurrent goroutines in decomposedfs configurable

* Update services/storage-users/pkg/config/config.go

Co-authored-by: Martin <github@diemattels.at>

* Update services/storage-users/pkg/config/config.go

Co-authored-by: Martin <github@diemattels.at>

---------

Co-authored-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Martin <github@diemattels.at>
2023-03-24 10:49:51 +01:00
Andre Duffeck
8b68830201 [full-ci] Improve ini backend, support redis sentinel caches (#5737)
* Configure the file metadata cache for storage-users

* Also configure the cache for the system storage

* Update services/storage-system/pkg/config/config.go

Co-authored-by: Martin <github@diemattels.at>

* Improve documentation

* Bump reva to pull in the latest ini backend changes

* Fix missing comment

* Update services/storage-users/README.md

Co-authored-by: Martin <github@diemattels.at>

* Update services/storage-system/README.md

Co-authored-by: Martin <github@diemattels.at>

* Tweak docs

* Apply suggestions from code review

Co-authored-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Martin <github@diemattels.at>

---------

Co-authored-by: Martin <github@diemattels.at>
Co-authored-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-03-06 15:04:19 +01:00
Jörn Friedrich Dreyer
a5fb068650 [full-ci] Bump reva to include ini metadata backend (#5613)
* bump reva

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* default to ini backend

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* bump reva, add metadata backend config

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update changelog

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* fix tag

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Update services/storage-system/pkg/config/config.go

Co-authored-by: kobergj <jkoberg@owncloud.com>

* fix storage user metadata config desc and env var

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: kobergj <jkoberg@owncloud.com>
2023-02-23 10:54:50 +01:00
David Christofas
17b5706cc9 add config options for the max_quota and add the capability value 2023-02-22 16:22:51 +01:00
jkoberg
2f2e93e148 don't configure events for storage provider
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-02-02 16:14:01 +01:00
jkoberg
daad94a332 config and tls
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2022-12-08 11:41:31 +01:00
Jörn Friedrich Dreyer
3ac7701a52 decomposedfs increase filelock duration factor
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2022-11-25 13:17:11 +00:00
Michael Barz
6c2a1999c4 add requestID middleware (#4974)
* add requestID middleware

* Update services/ocdav/pkg/config/config.go

* remove unnecessary config structs

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Jörn Friedrich Dreyer <jfd@owncloud.com>
Co-authored-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2022-11-04 13:01:18 +01:00
Florian Schade
51376b584b add option to configure max_acquire_lock_cycles 2022-11-03 14:22:46 +01:00
Ralf Haferkamp
ee974afebf [full-ci] Introduce TLS Settings for go-micro based grpc services and clients (#4901)
* Introduce TLS Settings for go-micro based grpc services and clients

TLS for the services can be configure by setting the OCIS_MICRO_GRPC_TLS_ENABLED"
"OCIS_MICRO_GRPC_TLS_CERTIFICATE" and "OCIS_MICRO_GRPC_TLS_KEY"
enviroment variables.

TLS for the clients can configured by setting the "OCIS_MICRO_GRPC_CLIENT_TLS_MODE"
and "OCIS_MICRO_GRPC_CLIENT_TLS_CACERT" variables.

By default TLS is disabled.

Co-authored-by: Martin <github@diemattels.at>

* Unify TLS configuration for all grpc services

All grpc service (whether they're based on reva) or go-micro use the
same set of config vars now.

TLS for the services can be configure by setting the OCIS_GRPC_TLS_ENABLED,
OCIS_GRPC_TLS_CERTIFICATE and OCIS_GRPC_TLS_KEY enviroment variables.

TLS for the clients can configured by setting the OCIS_GRPC_CLIENT_TLS_MODE
and OCIS_MICRO_GRPC_CLIENT_TLS_CACERT variables.

There are no individual per service config vars currently. If really
needed, per service tls configurations can be specified via config file.

Co-authored-by: Martin <github@diemattels.at>

Co-authored-by: Martin <github@diemattels.at>
2022-11-03 10:17:08 +01:00
Ralf Haferkamp
8cead4557f Explicitly disable TLS for the permissions client
This is needed as the permission server (provided by the settings service)
is not TLS enabled yet.
2022-10-25 11:50:08 +02:00
Ralf Haferkamp
3d57f5cc21 Introduce TLS Settings for all reva grpc services and clients 2022-10-25 11:50:08 +02:00
David Christofas
45b92fc56f add config option to enable or disable TLS for nats 2022-10-21 12:40:21 +02:00
Jörn Friedrich Dreyer
2197489c26 enable request tracking prometheus middleware in reva (#4571)
* enable request tracking prometheus middleware in reva

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* enable metrics for ocdav

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2022-10-13 13:08:34 +02:00
David Christofas
4623b6c8e7 Nats tls (#4781)
* use tls for nats connections

* add config options for nats client tls config

* add nats tls config to CI

* add function to create a certpool

* add option to provide a rootCA to validate the server's TLS certificate

* add option to provide a rootCA to validate the server's TLS certificate

* add option to provide a rootCA to validate the server's TLS certificate

* add option to provide a rootCA to validate the server's TLS certificate

* configure nats clients in reva to use tls
2022-10-12 14:56:47 +02:00