Commit Graph

64 Commits

Author SHA1 Message Date
Jörn Friedrich Dreyer
2751cfee2c directly connect to frontend
The STORAGE_USERS_DATA_GATEWAY_URL env var is used in the `tokens` `datagateway_endpoint` reva configuration. That `DataGatewayEndpoint` is used by the decomposedfs driver to create urls:
```go
// URL returns a url to download an upload
func (session *DecomposedFsSession) URL(_ context.Context) (string, error) {
       // [ ... ]
	return joinurl(session.store.tknopts.DataGatewayEndpoint, tkn), nil
}
```
As the comment points out this URL is internally used when emitting events. Either in:
```go
func (session *DecomposedFsSession) FinishUploadDecomposed(ctx context.Context) error {
                // [ ... ]
		s, err := session.URL(ctx)
		if err != nil {
			return err
		}

		var iu *userpb.User
		if utils.ExistsInOpaque(u.Opaque, "impersonating-user") {
			iu = &userpb.User{}
			if err := utils.ReadJSONFromOpaque(u.Opaque, "impersonating-user", iu); err != nil {
				return err
			}
		}

		if err := events.Publish(ctx, session.store.pub, events.BytesReceived{
			UploadID:          session.ID(),
			URL:               s,
```

or in
```go

// Postprocessing starts the postprocessing result collector
func (fs *Decomposedfs) Postprocessing(ch <-chan events.Event) {
                        // [ ... ]
			s, err := session.URL(ctx)
			if err != nil {
				sublog.Error().Err(err).Msg("could not create url")
				continue
			}

			metrics.UploadSessionsRestarted.Inc()

			// restart postprocessing
			if err := events.Publish(ctx, fs.stream, events.BytesReceived{
				UploadID:      session.ID(),
				URL:           s,
```

So, we do not need to go throught the proxy here.
2025-08-19 14:58:30 +02:00
Michael Barz
2d24c27329 change: adjust default values for the S3 Uploads (#1224) 2025-07-14 16:43:43 +02:00
Michael Barz
db5ac0a4e2 change: set better decomposedS3 defaults for multipart upload (#1200) 2025-07-10 09:20:39 +02:00
Florian Schade
900d596b65 fix(decomposeds3): enable async-uploads by default (#686) 2025-04-17 09:17:53 +02:00
André Duffeck
91fd396c86 Add env var to make the inotify stats frequency configurable 2025-04-15 09:24:55 +02:00
Jörn Friedrich Dreyer
15cb8680ef change storage users default to posixfs
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-03-19 12:21:58 +01:00
Ralf Haferkamp
90328a7ed1 posixfs: Use userid as the foldername for personal space
This avoids loosing the user's personal space after renaming the user.

Closes: #192
2025-03-17 10:54:52 +01:00
André Duffeck
916c79e856 Make space aliases configure for the posix driver 2025-03-13 15:35:45 +01:00
André Duffeck
d42034202d Add option to disable the posix fs watcher 2025-02-13 13:11:49 +01:00
André Duffeck
5b85029813 Add EnableFSRevisions config option 2025-02-13 10:07:17 +01:00
André Duffeck
57fa8eb43c Switch from ocis/s3ng to decomposed/decomposed_s3 storage drivers
ocis/s3ng are still supported for backwards compatibility reasons, but
they need to be configured using the decomposed/decomposed_s3 options.
2025-01-24 11:04:23 +01:00
Jörn Friedrich Dreyer
243ffe81c7 nats cluster name
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-01-16 16:14:22 +01:00
Jörn Friedrich Dreyer
db39b8ed3b use opencloudurl
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-01-16 11:13:08 +01:00
Ralf Haferkamp
213e9663a9 Rename go-micro services 2025-01-16 09:45:46 +01:00
Jörn Friedrich Dreyer
b07b5a1149 use plain pkg module
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-01-13 16:42:19 +01:00
Jörn Friedrich Dreyer
8e028f17e9 change module name
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-01-13 09:58:18 +01:00
jkoberg
4a911ac047 feat(storage-users): remove metadata backend envvar
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2024-09-19 12:29:41 +02:00
André Duffeck
ffe1946686 Fix default posix root and the description of the according config var
Fixes https://github.com/owncloud/ocis/issues/9910
2024-09-17 10:33:32 +02:00
André Duffeck
d199e79c71 Set a default debounce delay 2024-09-17 10:33:32 +02:00
André Duffeck
0028013794 Enable async uploads for posixfs 2024-06-25 08:40:50 +02:00
André Duffeck
126b083eaf Add config var for using the space groups 2024-05-29 12:14:54 +02:00
André Duffeck
8cd39754c7 Add config vars for space path templates 2024-05-29 12:14:54 +02:00
André Duffeck
96645322e7 Configure project spaces layout 2024-05-29 12:14:53 +02:00
André Duffeck
aab219a38f Initial support for the posixfs 2024-05-29 12:14:53 +02:00
Christian Richter
fe4cd7a901 make code more readable
Signed-off-by: Christian Richter <crichter@owncloud.com>
2024-03-19 14:21:21 +01:00
Christian Richter
b5ca297f54 prevent nil pointer
Signed-off-by: Christian Richter <crichter@owncloud.com>
2024-03-19 13:56:10 +01:00
Christian Richter
72916dc53c adapt cors headers
Signed-off-by: Christian Richter <crichter@owncloud.com>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2024-03-19 10:54:58 +01:00
jkoberg
f1a6472a6d disable multiparts uploads
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2024-03-15 15:59:57 +01:00
Jörn Friedrich Dreyer
5ed57cc09a Bump reva deps (#8412)
* bump dependencies

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* bump reva and add config options

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2024-02-21 10:20:36 +01:00
Christian Richter
2269d2cf27 change default for MaxConcurrency
Signed-off-by: Christian Richter <crichter@owncloud.com>
2024-01-29 14:22:26 +01:00
jkoberg
cec6ea50eb remove unused stat cache
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2024-01-24 10:39:32 +01:00
Roman Perekhod
367f1ff9e8 trash-bin cli has been exteneded by the list and restore commands (#7917)
* trash-bin cli has been exteneded by the list and restore commands

* v4 to v5 changes

---------

Co-authored-by: Roman Perekhod <rperekhod@owncloud.com>
2023-12-19 15:44:28 +01:00
kobergj
dbc50a655b use different dbs for filemetadata caches
Co-authored-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-12-14 11:08:10 +01:00
jkoberg
d62cb9d34b use inmemory cache by default
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-12-13 12:49:54 +01:00
jkoberg
ad87ac955a configure services individually
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-12-13 12:49:54 +01:00
jkoberg
bbc4f83f7d bump dependencies
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-12-13 12:49:53 +01:00
jkoberg
003310a2b0 use nats-js cache
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-12-13 12:48:27 +01:00
jkoberg
f189b11ba9 default async uploads
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-10-05 15:33:42 +02:00
André Duffeck
e8a7872d29 Add default propagator config 2023-07-31 10:26:19 +02:00
Ralf Haferkamp
e32aef17cd storage-users: Add config option for graceful shutdown timeout
Add STORAGE_USERS_GRACEFUL_SHUTDOWN_TIMEOUT setting to allow a graceful
shutdown of the storage-users service. This currently only applicable
when running storage-user as a sepearate service.

Setting STORAGE_USERS_GRACEFUL_SHUTDOWN_TIMEOUT to a non-zero value
gives the storage-users service a chance to cleanly shutdown and finish
any in progess tasks (e.g. metadata propagation) before exiting.

Partial-Fix: #6602
2023-07-18 16:47:26 +02:00
Florian Schade
4f26424db6 [full-ci] enhancement: use reva client pool selectors (#6452)
* enhancement: use reva client pool selectors

register mock service to registry and pass tests

* enhancement: bump reva

* Fix a couple of linter issues

---------

Co-authored-by: Ralf Haferkamp <rhaferkamp@owncloud.com>
2023-06-08 12:41:04 +02:00
mmattel
e5746d8176 [docs-only] Fix wrong default metadata backend 2023-05-12 13:34:13 +02:00
mmattel
037422ba1f [docs-only] Set messagepack as default metadata backend 2023-05-03 12:22:09 +02:00
Andre Duffeck
129489203b Cleanup and improve the caching config (#6148)
* Cleanup and improve the caching config

* bump reva

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* disable stat cache

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Bump reva

* Linter fixes

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-04-27 12:35:30 +02:00
Andre Duffeck
77bb3d8bcd [full-ci] Refactor stores (#6019)
* Streamline the store implementation with and into reva

* Adapt to the cache/store refactoring in reva

* Streamline config options and their env vars

* Apply suggestions from code review

Co-authored-by: Martin <github@diemattels.at>

* Use the same database for all stores

* Bump reva

* Configure stat and filemetadata cache separately

* Fix default config

---------

Co-authored-by: Martin <github@diemattels.at>
2023-04-24 15:13:35 +02:00
David Christofas
924c2ca346 add missing comments 2023-03-06 19:01:16 +01:00
David Christofas
63fa35fa50 reduce duplication in configuration code 2023-03-06 16:52:44 +01:00
Jörn Friedrich Dreyer
a5fb068650 [full-ci] Bump reva to include ini metadata backend (#5613)
* bump reva

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* default to ini backend

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* bump reva, add metadata backend config

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update changelog

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* fix tag

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Update services/storage-system/pkg/config/config.go

Co-authored-by: kobergj <jkoberg@owncloud.com>

* fix storage user metadata config desc and env var

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: kobergj <jkoberg@owncloud.com>
2023-02-23 10:54:50 +01:00
Florian Schade
5da3df8ffe Space Trash-bin expiration cli (#5500)
* add storage-users trash-bin cli
add task to clean up outdated trash-bin resources
add trash-bin cli purge-expired command to purge expired trash-bin resources
add purge-expired task tests
2023-02-10 12:04:47 +01:00
jkoberg
daad94a332 config and tls
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2022-12-08 11:41:31 +01:00