Commit Graph
282 Commits
Author SHA1 Message Date
Jörn Friedrich Dreyer 04ad5b84bb fix(storage-users): keep uploads sessions CLI out of the dcfs event queue
The uploads sessions command builds a posix driver with async file
uploads enabled, which makes it join the "dcfs" JetStream queue group.
Clean/restart/resume events published by the command itself can then be
routed to the CLI's own short-lived consumer; the nats client auto-acks
them and they are lost when the process exits, leaving stale upload
sessions behind.

Disable asyncfileuploads for the CLI driver instance so it only lists
sessions and publishes events. Also emit [] instead of null when the
session list is empty so the JSON output stays parseable.
2026-09-25 15:47:47 +02:00
Jörn Friedrich Dreyer 27c42ad135 fix(storage-users): publish CleanUpload event instead of purging directly
Ported from owncloud/ocis#11693. The uploads clean command now emits an events.CleanUpload event via the event stream rather than calling Purge directly. Requires the corresponding reva bump (vendor changes pulled in separately).
2026-09-24 11:21:28 +02:00
Alex Ababii a419485391 upd trash bin cli commands flags 2026-09-16 16:44:51 +02:00
Alex Ababii 43385ba6dc document params for thrash bin cli commands 2026-09-16 16:44:51 +02:00
Alex Ababii 983c836653 cli: restore single item from trash bin, listing on empty thrash bin 2026-09-01 10:03:34 +02:00
Jörn Friedrich Dreyer f065e8acdc expect only the tus related cors allow header values
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-08-21 07:34:47 +02:00
Jörn Friedrich Dreyer 72c0ee9e54 do not set datagateway url for storage drivers
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-08-21 07:34:46 +02:00
Jörn Friedrich Dreyer fff84b844f drop STORAGE_USERS_EXPOSE_DATA_SERVER env var
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-08-21 07:34:46 +02:00
Jörn Friedrich Dreyer a74661618e refactor datagateway into proxy middleware
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-08-21 07:34:46 +02:00
André Duffeck 3d632fb90e Expose upload status and message 2026-08-19 14:08:12 +02:00
Viktor Scharf 5accb6056b set 7.3.0 version placeholder (#3132) 2026-07-14 18:12:24 +02:00
Jörn Friedrich Dreyer 77fd4fca69 add tls support for all nats connections
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-06-10 17:04:18 +02:00
Viktor Scharf cf0ec50da9 [full-ci] chore: bump reva to v2.44.0 update opencloud version 6.2.0 (#2734) 2026-05-11 17:08:39 +02:00
Andre Duffeck efb73c8a79 Merge pull request #2702 from aduffeck/fix-nats-ttl
Set new defaults for caches and stores
2026-05-11 11:04:10 +02:00
André Duffeck e60d86cf2c Use time.Hour where appropriate 2026-05-07 21:41:26 +02:00
André Duffeck e246d6d613 Make the ScanFS option configurable via env var 2026-05-07 08:29:48 +02:00
Christian Richter bf5c95ef1a replace deprecationNotice by deprecationInfo
Signed-off-by: Christian Richter <c.richter@opencloud.eu>
2026-05-06 10:08:28 +02:00
André Duffeck 544968a4de Set new defaults for caches and stores
See https://github.com/opencloud-eu/opencloud/issues/2681 for more
details.
2026-04-30 11:00:12 +02:00
Florian Schade 288e67cc39 chore: replace interface with any 2026-04-23 09:31:11 +02:00
André Duffeck e2f6a68810 Do not ever set a TTL for the ID cache. It's not supposed to expire. 2026-01-28 12:52:24 +01:00
André Duffeck 80601fc0dc Do not try to enable multi tenancy for the system storage 2026-01-20 15:49:13 +01:00
Christian Richter 9b3fde49fe consolidate log config in storage-users
Signed-off-by: Christian Richter <c.richter@opencloud.eu>
2026-01-08 13:36:12 +01:00
Jörn Friedrich Dreyer 9dd03b6c90 small fixes
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-12-15 16:40:27 +01:00
Jörn Friedrich Dreyer 032c218789 replace .Value.String() with .GetString(...)
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-12-15 16:40:27 +01:00
Jörn Friedrich Dreyer 9a597922f2 fix storage-users trash-bin flags
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-12-15 16:40:27 +01:00
Jörn Friedrich Dreyer 93601752a9 fix storage-users uploads sessions flags
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-12-15 16:40:27 +01:00
Christian Richter 7be33b0607 refactor interims DefaultAppCobra to DefaultApp
Signed-off-by: Christian Richter <c.richter@opencloud.eu>
2025-12-15 16:40:26 +01:00
Christian Richter 5869f8c19c migrate storage-users from urfave/cli to spf13/cobra
Signed-off-by: Christian Richter <c.richter@opencloud.eu>
2025-12-15 16:40:26 +01:00
Jörn Friedrich Dreyer 56817b7de7 introduce OC_EVENTS_TLS_INSECURE
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-11-28 11:17:39 +01:00
Jörn Friedrich Dreyer 10913ca00a Merge pull request #1918 from opencloud-eu/otlp-tracing
update otlp tracing
2025-11-27 12:57:26 +01:00
Jörn Friedrich Dreyer a3ef7f6d79 update otlp tracing
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-11-27 12:28:15 +01:00
fschade 60501659c5 chore: bump %%NEXT%% 2025-11-27 10:53:59 +01:00
Ralf Haferkamp 7b5c59e827 feat(reva): Pass the multi-tenancy flag to reva services
This allows to pass the multi-tenant enablement flag as a shared config
option to the reva service. This needs to be done for all reva services
since it ends up in a global variable in reva that is only initialized
once, by the service that is the first to parse its config.
See https://github.com/opencloud-eu/opencloud/issues/1563 for details.
2025-10-06 11:21:57 +02:00
Roman Perekhod d76afadd4d clean up 2025-09-12 12:18:47 +02:00
Roman Perekhod 9a3fc08dd4 to separate controll ower the http and grpc driven services 2025-09-12 12:18:47 +02:00
Roman Perekhod 65d05bbd5c feat: fix the graceful shutdown using the new ocis and reva runners
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2025-09-12 12:18:47 +02:00
Juan Pablo Villafáñez c0b4a5daa0 chore: change constant name to camelcase 2025-09-08 17:32:36 +02:00
Juan Pablo Villafáñez 925444a0b3 feat: change the names of the connections to the registry 2025-09-08 17:32:35 +02:00
Juan Pablo Villafáñez ca2dc823ef feat: use names for connections to the nats event bus 2025-09-08 17:32:35 +02:00
Ralf Haferkamp 88dd36b636 Drop the unsupported and unused S3 driver
It was dropped from reva in https://github.com/opencloud-eu/reva/pull/309
because of unmaintained dependencies. Remember: We have the decomposeds3
driver.
2025-08-21 11:15:35 +02:00
Jörn Friedrich Dreyer 2751cfee2c directly connect to frontend
The STORAGE_USERS_DATA_GATEWAY_URL env var is used in the `tokens` `datagateway_endpoint` reva configuration. That `DataGatewayEndpoint` is used by the decomposedfs driver to create urls:
```go
// URL returns a url to download an upload
func (session *DecomposedFsSession) URL(_ context.Context) (string, error) {
       // [ ... ]
	return joinurl(session.store.tknopts.DataGatewayEndpoint, tkn), nil
}
```
As the comment points out this URL is internally used when emitting events. Either in:
```go
func (session *DecomposedFsSession) FinishUploadDecomposed(ctx context.Context) error {
                // [ ... ]
		s, err := session.URL(ctx)
		if err != nil {
			return err
		}

		var iu *userpb.User
		if utils.ExistsInOpaque(u.Opaque, "impersonating-user") {
			iu = &userpb.User{}
			if err := utils.ReadJSONFromOpaque(u.Opaque, "impersonating-user", iu); err != nil {
				return err
			}
		}

		if err := events.Publish(ctx, session.store.pub, events.BytesReceived{
			UploadID:          session.ID(),
			URL:               s,
```

or in
```go

// Postprocessing starts the postprocessing result collector
func (fs *Decomposedfs) Postprocessing(ch <-chan events.Event) {
                        // [ ... ]
			s, err := session.URL(ctx)
			if err != nil {
				sublog.Error().Err(err).Msg("could not create url")
				continue
			}

			metrics.UploadSessionsRestarted.Inc()

			// restart postprocessing
			if err := events.Publish(ctx, fs.stream, events.BytesReceived{
				UploadID:      session.ID(),
				URL:           s,
```

So, we do not need to go throught the proxy here.
2025-08-19 14:58:30 +02:00
Michael Stingl 9659e97056 Fix posix driver documentation in STORAGE_USERS_DRIVER description (#1305)
- Add 'posix' to the list of supported values
- Correct the default value from 'decomposed' to 'posix'
- Add brief description of the posix driver

Fixes: #1304
2025-08-06 17:46:44 +02:00
Michael Barz 2d24c27329 change: adjust default values for the S3 Uploads (#1224) 2025-07-14 16:43:43 +02:00
Michael Barz db5ac0a4e2 change: set better decomposedS3 defaults for multipart upload (#1200) 2025-07-10 09:20:39 +02:00
Ralf Haferkamp 96684df32d Adjust to new tablewriter release 2025-05-19 19:26:38 +02:00
Florian Schade 900d596b65 fix(decomposeds3): enable async-uploads by default (#686) 2025-04-17 09:17:53 +02:00
André Duffeck f8c89b1c3d Add deprecation notice for STORAGE_USERS_POSIX_WATCH_FOLDER_KAFKA_BROKERS 2025-04-16 11:09:30 +02:00
André Duffeck 8c9e05d1f0 Expose more config vars for the posix fs watchers 2025-04-16 11:08:03 +02:00
André Duffeck 91fd396c86 Add env var to make the inotify stats frequency configurable 2025-04-15 09:24:55 +02:00
André Duffeck 09818a9d7e Adapt command to changed signature 2025-04-01 15:04:53 +02:00