Compare commits

..
Author SHA1 Message Date
Jörn Friedrich Dreyer 1bafa4bd18 align the handler verifier, update docs
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-09-08 17:11:55 +02:00
Dominik Schmidt b18618ab4a Merge pull request #3484 from opencloud-eu/fix/tika-embedded-facet-leak
fix(search): extract facets from the main tika document only
2026-09-08 15:52:21 +02:00
Dominik Schmidt 65f19b5d71 fix(thumbnails): close rejected vips ref, map oversized convert error to forbidden 2026-09-08 15:28:37 +02:00
Dominik Schmidt 7f687ea288 fix(thumbnails): bound declared image dimensions before decoding
The imaging build decodes the full pixel buffer from the header-declared
dimensions before the existing MaxInputWidth/MaxInputHeight guard runs, so a
tiny crafted file whose header declares huge dimensions forces a multi-GB
allocation and can OOM the worker. Read the header with DecodeConfig and reject
oversized sources before the decode allocates, in both the imaging and vips
builds, and thread the limit through the audio cover-art and geogebra decoders
that decode a second attacker-controlled image.
2026-09-08 15:28:37 +02:00
Dominik Schmidt be83ed19e3 fix(search): take extraction facets from the main document only
The recursive tika response lists the file first, then its embedded
resources (cover art, thumbnails, the clip appended to a motion photo).
The loop applied getImage/getPhoto/getLocation/getAudio/getLivePhoto to
every part, so an mp3's embedded cover art leaked a 200x200 image facet
onto the track (and an embedded EXIF image would leak photo/location).
Read those facets from metas[0] only, like the video facet already does;
the loop now only concatenates title/content and detects the motion
photo clip.
2026-09-08 14:52:35 +02:00
Dominik Schmidt b5cb1cd83b test(search): re-search until the expected files are in the result (#3488)
Indexing of newly uploaded resources is asynchronous; the search step
slept a fixed 10s and searched once, so under load a just-uploaded file
was missing from the result and the following assertion failed. The
search step now remembers its query, and new eventually-contain steps
re-run the search until the expected entries are present (or the
WaitHelper timeout elapses), then assert as before. contentSearch
scenarios use them for their positive assertions; the exact-count check
runs after the expected entries have appeared.
2026-09-08 14:50:49 +02:00
zerox80 9797ee903f fix(config): correct pending version annotations
Use the supported %%NEXT%% marker for seven graph and policies settings so the environment annotation check passes.
2026-09-08 14:40:40 +02:00
Jörn Friedrich Dreyer 973dbbdf1a Merge pull request #3241 from opencloud-eu/activitylog-event-handler-split
Activitylog event handler split
2026-09-08 14:13:28 +02:00
Dominik Schmidt 1ce7f07523 Merge pull request #3486 from opencloud-eu/test/extractedprops-wait-for-values
test(search): wait for expected properties and documents
2026-09-08 14:05:12 +02:00
Dominik Schmidt 3b83221a87 test(search): retry the driveItem GET until the extracted properties match
Property extraction reaches the driveItem asynchronously, so the GET
step previously waited only until any extraction facet key appeared and
a separate step then validated the full schema, which could run on a
partial payload and fail intermittently. The GET and the status check
stay plain steps; a new explicit Then step re-fetches the item until
the response satisfies the expected schema (a partial never matches) or
the WaitHelper timeout elapses.
2026-09-08 13:39:00 +02:00
Jörn Friedrich Dreyer 391705b017 Merge pull request #3441 from opencloud-eu/dependabot/go_modules/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp-0.71.0
build(deps): bump go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp from 0.70.0 to 0.71.0
2026-09-08 10:47:59 +02:00
Alex Ababii 8875d062b0 upd tests for auth requests middelware 2026-09-08 09:37:51 +02:00
Alex Ababii d1d20edbd2 log jwt expired on debug level instead of error 2026-09-08 09:37:51 +02:00
dependabot[bot] b50c5e54d6 build(deps): bump go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
Bumps [go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp](https://github.com/open-telemetry/opentelemetry-go-contrib) from 0.70.0 to 0.71.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go-contrib/compare/zpages/v0.70.0...zpages/v0.71.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
  dependency-version: 0.71.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-09-08 07:26:51 +00:00
opencloudeu c2a34e8c45 [tx] updated from transifex 2026-09-07 23:17:03 +00:00
Dominik Schmidt d6c6b6fd0d Merge pull request #3481 from opencloud-eu/fix/proxy-signed-url-methods
fix(proxy): restrict JWT signed urls to the allowed HTTP methods
2026-09-07 16:13:40 +02:00
Dominik Schmidt 7f7a938c3f fix(proxy): restrict JWT signed urls to the allowed HTTP methods
The legacy OC-Signature path already rejects methods outside
PRE_SIGNED_URL_ALLOWED_HTTP_METHODS, the JWT path did not. A leaked
signed download url could be used for PUT, DELETE, MOVE or PROPFIND
as the signing user for the lifetime of the signature.
2026-09-07 13:00:09 +02:00
Jörn Friedrich Dreyer cfefb83d26 revert unrelated changes 2026-09-07 12:52:33 +02:00
Jörn Friedrich Dreyer 9357124a91 add tests to extracted services
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-09-07 12:52:33 +02:00
Jörn Friedrich Dreyer c36af05d14 simplify ConnectNatsKV
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-09-07 12:52:33 +02:00
Jörn Friedrich Dreyer 7947c1c98a reduce packages
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-09-07 12:52:33 +02:00
Jörn Friedrich Dreyer d58c1e3dab use json encode
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-09-07 12:52:33 +02:00
Jörn Friedrich Dreyer 5a609c98dc handle error types
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-09-07 12:52:33 +02:00
Jörn Friedrich Dreyer 29913e1d7f revert to legacy autoack events
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-09-07 12:52:32 +02:00
Jörn Friedrich Dreyer 5028364e2c add tests to cover more ack scenarios
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-09-07 12:52:32 +02:00
Jörn Friedrich Dreyer b2b15d44f7 rename test files
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-09-07 12:52:32 +02:00
Jörn Friedrich Dreyer 6ea363b0b9 add debouncer tests
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-09-07 12:52:32 +02:00
Jörn Friedrich Dreyer 9594313243 add explicit ack
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-09-07 12:52:32 +02:00
Jörn Friedrich Dreyer 62d815abe9 split handlers, fix bugs
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-09-07 12:52:32 +02:00
Jörn Friedrich Dreyer 5923aa5e5e move event + http handler code to the events package
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2026-09-07 12:52:32 +02:00
Dominik Schmidt 81536bbd0e Merge pull request #3471 from opencloud-eu/feat/graph-expand-thumbnails
feat(graph): expand thumbnails on driveItems
2026-09-07 12:50:53 +02:00
Jörn Friedrich Dreyer be226ebb4c Merge pull request #3478 from opencloud-eu/reva-bump-main-20260907
[full-ci] chore: bump reva to latest main
2026-09-07 12:48:12 +02:00
Dominik Schmidt c2b37668e4 chore: bump reva to latest main 2026-09-07 09:33:59 +02:00
Dominik Schmidt 6231a870d5 feat(graph): expand thumbnails on driveItems
$expand=thumbnails was only honored by sharedByMe and sharedWithMe. The
driveItem stat, the children listing and the root children listing now
honor it as well, so a client that lists a folder learns which items have
a preview instead of guessing from the mime type.

The thumbnails are set from the resource info the listing already has, so
a later preview check that needs more than the mime type has a single
place to sit. The two share listings carry driveItems only, they keep
matching on the mime type but share the url building.
2026-09-06 23:51:35 +02:00
167 changed files with 3131 additions and 12830 deletions

No files matched your search

+2 -3
View File
@@ -63,7 +63,7 @@ require (
github.com/open-policy-agent/opa v1.19.1
github.com/opencloud-eu/icap-client v0.0.0-20250930132611-28a2afe62d89
github.com/opencloud-eu/libre-graph-api-go v1.0.8-0.20260902170011-45af3945a067
github.com/opencloud-eu/reva/v2 v2.49.0
github.com/opencloud-eu/reva/v2 v2.49.1-0.20260903122659-26f34ec05774
github.com/opensearch-project/opensearch-go/v4 v4.7.3
github.com/orcaman/concurrent-map v1.0.0
github.com/pkg/errors v0.9.1
@@ -95,7 +95,7 @@ require (
go-micro.dev/v4 v4.11.0
go.etcd.io/bbolt v1.5.0
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.70.0
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.70.0
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.71.0
go.opentelemetry.io/contrib/zpages v0.71.0
go.opentelemetry.io/otel v1.46.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.45.0
@@ -324,7 +324,6 @@ require (
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 // indirect
github.com/pquerna/cachecontrol v0.2.0 // indirect
github.com/prometheus/alertmanager v0.33.1 // indirect
github.com/prometheus/common v0.70.1 // indirect
github.com/prometheus/procfs v0.21.1 // indirect
github.com/prometheus/statsd_exporter v0.22.8 // indirect
+4 -6
View File
@@ -942,8 +942,8 @@ github.com/opencloud-eu/icap-client v0.0.0-20250930132611-28a2afe62d89 h1:W1ms+l
github.com/opencloud-eu/icap-client v0.0.0-20250930132611-28a2afe62d89/go.mod h1:vigJkNss1N2QEceCuNw/ullDehncuJNFB6mEnzfq9UI=
github.com/opencloud-eu/libre-graph-api-go v1.0.8-0.20260902170011-45af3945a067 h1:UkNMKauyJAzY6RE6mmthz9bQZLYkbvBuApm7ZDCparE=
github.com/opencloud-eu/libre-graph-api-go v1.0.8-0.20260902170011-45af3945a067/go.mod h1:lTM8JeGblNpoMySTW7Lui2+c5TTLI95mwxtdUIHHrhU=
github.com/opencloud-eu/reva/v2 v2.49.0 h1:AwECMDDth3NUaihZRf9bI9HNpWutvaRijOCvQyQfTT4=
github.com/opencloud-eu/reva/v2 v2.49.0/go.mod h1:Frg+UWnVcSy+412UB3l2LcD0KY8ZNu1samimwkNywbg=
github.com/opencloud-eu/reva/v2 v2.49.1-0.20260903122659-26f34ec05774 h1:8N42Pwzyafdxq66zYP6kJ1CH/QBRpxovQXq/0Kt6bJY=
github.com/opencloud-eu/reva/v2 v2.49.1-0.20260903122659-26f34ec05774/go.mod h1:Frg+UWnVcSy+412UB3l2LcD0KY8ZNu1samimwkNywbg=
github.com/opencloud-eu/secure v0.0.0-20260312082735-b6f5cb2244e4 h1:l2oB/RctH+t8r7QBj5p8thfEHCM/jF35aAY3WQ3hADI=
github.com/opencloud-eu/secure v0.0.0-20260312082735-b6f5cb2244e4/go.mod h1:BmF5hyM6tXczk3MpQkFf1hpKSRqCyhqcbiQtiAF7+40=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
@@ -1001,8 +1001,6 @@ github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55/go.mod h1:Om
github.com/pquerna/cachecontrol v0.2.0 h1:vBXSNuE5MYP9IJ5kjsdo8uq+w41jSPgvba2DEnkRx9k=
github.com/pquerna/cachecontrol v0.2.0/go.mod h1:NrUG3Z7Rdu85UNR3vm7SOsl1nFIeSiQnrHV5K9mBcUI=
github.com/pquerna/otp v1.3.0/go.mod h1:dkJfzwRKNiegxyNb54X/3fLwhCynbMspSyWKnvi1AEg=
github.com/prometheus/alertmanager v0.33.1 h1:PJHGvTdb8Q0ZEpJnWff120WxB2kxIScLa707AaIudTo=
github.com/prometheus/alertmanager v0.33.1/go.mod h1:V06Uc8EZ5X5wLOJRGhtXx+EE2LgrinFIADbKWMVm1RY=
github.com/prometheus/client_golang v0.8.0/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v0.9.3-0.20190127221311-3c4408c8b829/go.mod h1:p2iRAGwDERtqlqzRXnrOVns+ignqQo//hLXqYxZYVNs=
@@ -1295,8 +1293,8 @@ go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.70.0 h1:oECp5f+hN7nkwjU/8BxQ/q23bGPb8FIrD839owX222E=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.70.0/go.mod h1:DqEFwLumhzMBDQv9PcWbyoDxHI/4lAk6CM4nJBH39sc=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.70.0 h1:LMuyCAyfalSjDyjdC65nK6N0zoTT63+E/u95X0JovZI=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.70.0/go.mod h1:085m8qbm4hgc8rZWGDEa4vmyyo2c3nPxUslYUKUIU04=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.71.0 h1:3g7B90UzBltIDKq1/5mrTGxTnOFDV0ICOhLoxiZ8jlg=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.71.0/go.mod h1:Ef8SuTh59BT7+ofpDxN9z+yOlc4t2GjLmKDgYNJL/NU=
go.opentelemetry.io/contrib/zpages v0.71.0 h1:OrQSEmdlfknIusEyGj/t30uBR7MnbmGcCRBt5ov8/rA=
go.opentelemetry.io/contrib/zpages v0.71.0/go.mod h1:C3IIpqkQbjswcSREqxQLyEapF0SI9bqwJpxpjwtzFrE=
go.opentelemetry.io/otel v1.46.0 h1:FHt5/CDyVxi/8IM1CH7VE/rRgq3kLHa2mSTVMO8AWyc=
+132 -30
View File
@@ -3,17 +3,18 @@ package command
import (
"context"
"fmt"
"strings"
"github.com/opencloud-eu/opencloud/pkg/log"
"github.com/opencloud-eu/opencloud/pkg/runner"
"github.com/opencloud-eu/reva/v2/pkg/events"
"github.com/opencloud-eu/reva/v2/pkg/events/stream"
"github.com/opencloud-eu/reva/v2/pkg/rgrpc/todo/pool"
"github.com/nats-io/nats.go"
"github.com/olekukonko/errors"
"github.com/spf13/cobra"
"github.com/opencloud-eu/opencloud/pkg/config/configlog"
"github.com/opencloud-eu/opencloud/pkg/generators"
"github.com/opencloud-eu/opencloud/pkg/log"
natspkg "github.com/opencloud-eu/opencloud/pkg/nats"
"github.com/opencloud-eu/opencloud/pkg/registry"
"github.com/opencloud-eu/opencloud/pkg/runner"
ogrpc "github.com/opencloud-eu/opencloud/pkg/service/grpc"
"github.com/opencloud-eu/opencloud/pkg/tracing"
"github.com/opencloud-eu/opencloud/pkg/version"
@@ -24,6 +25,12 @@ import (
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/metrics"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/server/debug"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/server/http"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/service/activitylog"
svcEvents "github.com/opencloud-eu/opencloud/services/activitylog/pkg/service/events"
svcHttp "github.com/opencloud-eu/opencloud/services/activitylog/pkg/service/http"
"github.com/opencloud-eu/reva/v2/pkg/events"
"github.com/opencloud-eu/reva/v2/pkg/events/stream"
"github.com/opencloud-eu/reva/v2/pkg/rgrpc/todo/pool"
)
var _registeredEvents = []events.Unmarshaller{
@@ -62,19 +69,11 @@ func Server(cfg *config.Config) *cobra.Command {
gr := runner.NewGroup()
ctx, cancel := context.WithCancel(cmd.Context())
defer cancel()
mtrcs := metrics.New()
mtrcs.BuildInfo.WithLabelValues(version.GetString()).Set(1)
defer cancel()
connName := generators.GenerateConnectionName(cfg.Service.Name, generators.NTypeBus)
evStream, err := stream.NatsFromConfig(connName, false, stream.NatsConfig(cfg.Events))
if err != nil {
logger.Error().Err(err).Msg("Failed to initialize event stream")
return err
}
tm, err := pool.StringToTLSMode(cfg.GRPCClientTLS.Mode)
if err != nil {
logger.Error().Err(err).Msg("Failed to parse tls mode")
@@ -99,28 +98,101 @@ func Server(cfg *config.Config) *cobra.Command {
return err
}
hClient := ehsvc.NewEventHistoryService("eu.opencloud.api.eventhistory", grpcClient)
vClient := settingssvc.NewValueService("eu.opencloud.api.settings", grpcClient)
kv, err := ConnectNatsKV(cfg.Store)
if err != nil {
return err
}
activityLog, err := activitylog.New(kv,
activitylog.Logger(logger),
activitylog.MaxActivities(cfg.MaxActivities),
activitylog.WriteBufferDuration(cfg.WriteBufferDuration),
)
if err != nil {
logger.Error().Err(err).Msg("Failed to initialize activity log")
return err
}
{
svc, err := http.Server(
http.Logger(logger),
http.Config(cfg),
http.Context(ctx), // NOTE: not passing this "option" leads to a panic in go-micro
http.TraceProvider(tracerProvider),
http.Stream(evStream),
http.GatewaySelector(gatewaySelector),
http.HistoryClient(hClient),
http.ValueClient(vClient),
http.RegisteredEvents(_registeredEvents),
if !cfg.HTTP.Disabled {
hClient := ehsvc.NewEventHistoryService("eu.opencloud.api.eventhistory", grpcClient)
svc, err := svcHttp.New(
activityLog,
svcHttp.Logger(logger),
svcHttp.GatewaySelector(gatewaySelector),
svcHttp.RegisteredEvents(_registeredEvents),
//svcHttp.TraceProvider(tracerProvider),
svcHttp.HistoryClient(hClient),
)
if err != nil {
logger.Error().Err(err).Str("transport", "http").Msg("Failed to initialize server")
logger.Error().Err(err).Msg("handler init")
return err
}
// TODO svc = service.NewInstrument(svc, metrics)
// TODO svc = service.NewLogging(svc, logger) // this logs service specific data
// TODO svc = service.NewTracing(svc, traceProvider)
vClient := settingssvc.NewValueService("eu.opencloud.api.settings", grpcClient)
server, err := http.Server(
http.ValueClient(vClient),
http.Logger(logger),
http.Context(ctx),
http.Config(cfg),
http.Service(svc),
)
if err != nil {
logger.Info().
Err(err).
Str("transport", "http").
Msg("Failed to initialize server")
return err
}
gr.Add(runner.NewGoMicroHttpServerRunner(cfg.Service.Name+".http", svc))
gr.Add(runner.NewGoMicroHttpServerRunner(cfg.Service.Name+".http", server))
} else {
logger.Info().Msg("HTTP server disabled, not starting HTTP service")
}
if !cfg.Events.Disabled {
connName := generators.GenerateConnectionName(cfg.Service.Name, generators.NTypeBus)
evStream, err := stream.NatsFromConfig(connName, false, stream.NatsConfig{
Endpoint: cfg.Events.Endpoint,
Cluster: cfg.Events.Cluster,
EnableTLS: cfg.Events.EnableTLS,
TLSInsecure: cfg.Events.TLSInsecure,
TLSRootCACertificate: cfg.Events.TLSRootCACertificate,
AuthUsername: cfg.Events.AuthUsername,
AuthPassword: cfg.Events.AuthPassword,
})
if err != nil {
logger.Error().Err(err).Msg("Failed to initialize event stream")
return err
}
eventSvc, err := svcEvents.New(
activityLog,
evStream,
svcEvents.Context(ctx),
svcEvents.Logger(logger),
svcEvents.ServiceAccount(cfg.ServiceAccount),
svcEvents.GatewaySelector(gatewaySelector),
svcEvents.RegisteredEvents(_registeredEvents),
svcEvents.NumConsumers(cfg.NumConsumers),
)
if err != nil {
logger.Error().Err(err).Str("transport", "event").Msg("Failed to initialize server")
return err
}
gr.Add(runner.New(cfg.Service.Name+".svc", func() error {
return eventSvc.Run()
}, func() {
eventSvc.Close()
}))
} else {
logger.Info().Msg("event listening disabled, not starting event service")
}
{
@@ -149,3 +221,33 @@ func Server(cfg *config.Config) *cobra.Command {
},
}
}
func ConnectNatsKV(cfg config.Store) (nats.KeyValue, error) {
// Connect to NATS servers
secureOption := natspkg.Secure(cfg.EnableTLS, cfg.TLSInsecure, cfg.TLSRootCACertificate)
conn, err := nats.Connect(strings.Join(cfg.Nodes, ","), secureOption, nats.UserInfo(cfg.AuthUsername, cfg.AuthPassword))
if err != nil {
return nil, err
}
js, err := conn.JetStream()
if err != nil {
return nil, err
}
kv, err := js.KeyValue(cfg.Database)
if err != nil {
if !errors.Is(err, nats.ErrBucketNotFound) {
return nil, errors.Wrapf(err, "Failed to get bucket (%s)", cfg.Database)
}
kv, err = js.CreateKeyValue(&nats.KeyValueConfig{
Bucket: cfg.Database,
})
if err != nil {
return nil, errors.Wrapf(err, "Failed to create bucket (%s)", cfg.Database)
}
}
return kv, nil
}
@@ -35,10 +35,12 @@ type Config struct {
WriteBufferDuration time.Duration `yaml:"write_buffer_duration" env:"ACTIVITYLOG_WRITE_BUFFER_DURATION" desc:"The duration to wait before flushing the write buffer. This is used to reduce the number of writes to the store." introductionVersion:"4.0.0"`
MaxActivities int `yaml:"max_activities" env:"ACTIVITYLOG_MAX_ACTIVITIES" desc:"The maximum number of activities to keep in the store per resource. If the number of activities exceeds this value, the oldest activities will be removed." introductionVersion:"4.0.0"`
NumConsumers int `yaml:"num_consumers" env:"ACTIVITYLOG_NUM_CONSUMERS" desc:"The amount of concurrent event consumers to start. Event consumers are used for updating the list of activities. Multiple consumers increase parallelisation, but will also increase CPU and memory demands." introductionVersion:"%NEXT%"`
}
// Events combines the configuration options for the event bus.
type Events struct {
Disabled bool `yaml:"disabled" env:"ACTIVITYLOG_EVENTS_DISABLED" desc:"Disables listening for events. Set this to true if the service should only handle HTTP requests." introductionVersion:"%NEXT%"`
Endpoint string `yaml:"endpoint" env:"OC_EVENTS_ENDPOINT" desc:"The address of the event system. The event system is the message queuing service. It is used as message broker for the microservice architecture." introductionVersion:"1.0.0"`
Cluster string `yaml:"cluster" env:"OC_EVENTS_CLUSTER" desc:"The clusterID of the event system. The event system is the message queuing service. It is used as message broker for the microservice architecture. Mandatory when using NATS as event system." introductionVersion:"1.0.0"`
TLSInsecure bool `yaml:"tls_insecure" env:"OC_INSECURE;OC_EVENTS_TLS_INSECURE" desc:"Whether to verify the server TLS certificates." introductionVersion:"1.0.0"`
@@ -77,6 +79,7 @@ type CORS struct {
// HTTP defines the available http configuration.
type HTTP struct {
Disabled bool `yaml:"disabled" env:"ACTIVITYLOG_HTTP_DISABLED" desc:"Disables the HTTP service. Set this to true if the service should only handle events." introductionVersion:"1.0.0"`
Addr string `yaml:"addr" env:"ACTIVITYLOG_HTTP_ADDR" desc:"The bind address of the HTTP service." introductionVersion:"1.0.0"`
Namespace string `yaml:"-"`
Root string `yaml:"root" env:"ACTIVITYLOG_HTTP_ROOT" desc:"Subdirectory that serves as the root for this HTTP service." introductionVersion:"1.0.0"`
@@ -52,7 +52,9 @@ func DefaultConfig() *config.Config {
},
},
WriteBufferDuration: 10 * time.Second,
MaxActivities: 6000,
// Nats runs into max payload exceeded errors at around 7k activities. Let's keep a buffer.
MaxActivities: 6000,
NumConsumers: 1,
}
}
@@ -4,6 +4,7 @@ import (
"errors"
occfg "github.com/opencloud-eu/opencloud/pkg/config"
"github.com/opencloud-eu/opencloud/pkg/shared"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/config"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/config/defaults"
@@ -34,5 +35,8 @@ func ParseConfig(cfg *config.Config) error {
// Validate validates the config
func Validate(cfg *config.Config) error {
if cfg.Events.Disabled && cfg.HTTP.Disabled {
return shared.AllComponentsDisabledError(cfg.Service.Name)
}
return nil
}
+23 -71
View File
@@ -3,18 +3,12 @@ package http
import (
"context"
gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1"
"github.com/opencloud-eu/opencloud/pkg/log"
ehsvc "github.com/opencloud-eu/opencloud/protogen/gen/opencloud/services/eventhistory/v0"
settingssvc "github.com/opencloud-eu/opencloud/protogen/gen/opencloud/services/settings/v0"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/config"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/metrics"
"github.com/opencloud-eu/reva/v2/pkg/events"
"github.com/opencloud-eu/reva/v2/pkg/rgrpc/todo/pool"
"github.com/spf13/pflag"
"go-micro.dev/v4/store"
"go.opentelemetry.io/otel/trace"
"go.opentelemetry.io/otel/trace/noop"
)
// Option defines a single option function.
@@ -22,19 +16,15 @@ type Option func(o *Options)
// Options defines the available options for this package.
type Options struct {
Logger log.Logger
Context context.Context
Config *config.Config
Metrics *metrics.Metrics
Flags []pflag.Flag
Namespace string
Store store.Store
Stream events.Stream
GatewaySelector pool.Selectable[gateway.GatewayAPIClient]
TraceProvider trace.TracerProvider
HistoryClient ehsvc.EventHistoryService
ValueClient settingssvc.ValueService
RegisteredEvents []events.Unmarshaller
Name string
Namespace string
Logger log.Logger
Context context.Context
Config *config.Config
Flags []pflag.Flag
Service ActivityLogService
TraceProvider trace.TracerProvider
ValueClient settingssvc.ValueService
}
// newOptions initializes the available default options.
@@ -69,10 +59,10 @@ func Config(val *config.Config) Option {
}
}
// Metrics provides a function to set the metrics option.
func Metrics(val *metrics.Metrics) Option {
// Service provides a function to set the service option.
func Service(val ActivityLogService) Option {
return func(o *Options) {
o.Metrics = val
o.Service = val
}
}
@@ -83,58 +73,20 @@ func Flags(flags ...pflag.Flag) Option {
}
}
// Namespace provides a function to set the Namespace option.
func Namespace(val string) Option {
// TraceProvider provides a function to configure the trace provider
func TraceProvider(traceProvider trace.TracerProvider) Option {
return func(o *Options) {
o.Namespace = val
if traceProvider != nil {
o.TraceProvider = traceProvider
} else {
o.TraceProvider = noop.NewTracerProvider()
}
}
}
// Store provides a function to configure the store
func Store(store store.Store) Option {
// ValueClient adds a grpc client for the value service
func ValueClient(vs settingssvc.ValueService) Option {
return func(o *Options) {
o.Store = store
}
}
// Stream provides a function to configure the stream
func Stream(stream events.Stream) Option {
return func(o *Options) {
o.Stream = stream
}
}
// GatewaySelector provides a function to configure the gateway client selector
func GatewaySelector(gatewaySelector pool.Selectable[gateway.GatewayAPIClient]) Option {
return func(o *Options) {
o.GatewaySelector = gatewaySelector
}
}
// HistoryClient provides a function to configure the event history client
func HistoryClient(h ehsvc.EventHistoryService) Option {
return func(o *Options) {
o.HistoryClient = h
}
}
// RegisteredEvents provides a function to register events
func RegisteredEvents(evs []events.Unmarshaller) Option {
return func(o *Options) {
o.RegisteredEvents = evs
}
}
// TraceProvider provides a function to set the TracerProvider option
func TraceProvider(val trace.TracerProvider) Option {
return func(o *Options) {
o.TraceProvider = val
}
}
// ValueClient provides a function to set the ValueClient options
func ValueClient(val settingssvc.ValueService) Option {
return func(o *Options) {
o.ValueClient = val
o.ValueClient = vs
}
}
+103 -46
View File
@@ -1,49 +1,63 @@
package http
import (
"fmt"
stdhttp "net/http"
"context"
"embed"
"encoding/json"
"errors"
"net/http"
"github.com/go-chi/chi/v5"
chimiddleware "github.com/go-chi/chi/v5/middleware"
libregraph "github.com/opencloud-eu/libre-graph-api-go"
"github.com/opencloud-eu/opencloud/pkg/account"
"github.com/opencloud-eu/opencloud/pkg/cors"
"github.com/opencloud-eu/opencloud/pkg/l10n"
"github.com/opencloud-eu/opencloud/pkg/log"
"github.com/opencloud-eu/opencloud/pkg/middleware"
"github.com/opencloud-eu/opencloud/pkg/service/http"
"github.com/opencloud-eu/opencloud/pkg/tracing"
ohttp "github.com/opencloud-eu/opencloud/pkg/service/http"
"github.com/opencloud-eu/opencloud/pkg/version"
svc "github.com/opencloud-eu/opencloud/services/activitylog/pkg/service"
"github.com/riandyrn/otelchi"
settingssvc "github.com/opencloud-eu/opencloud/protogen/gen/opencloud/services/settings/v0"
activityloghttp "github.com/opencloud-eu/opencloud/services/activitylog/pkg/service/http"
revactx "github.com/opencloud-eu/reva/v2/pkg/ctx"
"go-micro.dev/v4"
"google.golang.org/grpc/metadata"
)
// Service is the service interface
type Service any
var (
//go:embed l10n/locale
_localeFS embed.FS
// subfolder where the translation files are stored
_localeSubPath = "l10n/locale"
// domain of the activitylog service (transifex)
_domain = "activitylog"
)
// Server initializes the http service and server.
func Server(opts ...Option) (http.Service, error) {
func Server(opts ...Option) (ohttp.Service, error) {
options := newOptions(opts...)
service := options.Service
service, err := http.NewService(
http.TLSConfig(options.Config.HTTP.TLS),
http.Logger(options.Logger),
http.Namespace(options.Config.HTTP.Namespace),
http.Name(options.Config.Service.Name),
http.Version(version.GetString()),
http.Address(options.Config.HTTP.Addr),
http.Context(options.Context),
http.Flags(options.Flags...),
http.TraceProvider(options.TraceProvider),
newService, err := ohttp.NewService(
ohttp.TLSConfig(options.Config.HTTP.TLS),
ohttp.Logger(options.Logger),
ohttp.Namespace(options.Config.HTTP.Namespace),
ohttp.Name(options.Config.Service.Name),
ohttp.Version(version.GetString()),
ohttp.Address(options.Config.HTTP.Addr),
ohttp.Context(options.Context),
ohttp.Flags(options.Flags...),
)
if err != nil {
options.Logger.Error().
Err(err).
Msg("Error initializing http service")
return http.Service{}, fmt.Errorf("could not initialize http service: %w", err)
return ohttp.Service{}, err
}
middlewares := []func(stdhttp.Handler) stdhttp.Handler{
middlewares := []func(http.Handler) http.Handler{
chimiddleware.RequestID,
middleware.Version(
options.Config.Service.Name,
@@ -52,6 +66,7 @@ func Server(opts ...Option) (http.Service, error) {
middleware.Logger(
options.Logger,
),
middleware.TraceContext,
middleware.ExtractAccountUUID(
account.Logger(options.Logger),
account.JWTSecret(options.Config.TokenManager.JWTSecret),
@@ -68,33 +83,75 @@ func Server(opts ...Option) (http.Service, error) {
mux := chi.NewMux()
mux.Use(middlewares...)
mux.Use(
otelchi.Middleware(
"actitivylog",
otelchi.WithChiRoutes(mux),
otelchi.WithTracerProvider(options.TraceProvider),
otelchi.WithPropagators(tracing.GetPropagator()),
),
)
t := l10n.NewTranslatorFromCommonConfig(options.Config.DefaultLanguage, _domain, options.Config.TranslationPath, _localeFS, _localeSubPath)
mux.Route(options.Config.HTTP.Root, func(r chi.Router) {
r.Get("/graph/v1beta1/extensions/org.libregraph/activities", GetItemActivitiesHandler(options.Logger, service, options.ValueClient, t))
})
handle, err := svc.New(
svc.Logger(options.Logger),
svc.Stream(options.Stream),
svc.Mux(mux),
svc.Config(options.Config),
svc.GatewaySelector(options.GatewaySelector),
svc.TraceProvider(options.TraceProvider),
svc.HistoryClient(options.HistoryClient),
svc.ValueClient(options.ValueClient),
svc.RegisteredEvents(options.RegisteredEvents),
)
err = micro.RegisterHandler(newService.Server(), mux)
if err != nil {
return http.Service{}, err
options.Logger.Fatal().Err(err).Msg("failed to register the handler")
}
if err := micro.RegisterHandler(service.Server(), handle); err != nil {
return http.Service{}, err
}
newService.Init()
return newService, nil
return service, nil
}
// Service defines the business logic implementations need to provide.
type ActivityLogService interface {
GetItemActivities(ctx context.Context, query, loc string, t l10n.Translator) ([]libregraph.Activity, error)
}
// GetActivitiesResponse is the response on GET activities requests
type GetActivitiesResponse struct {
Activities []libregraph.Activity `json:"value"`
}
func GetItemActivitiesHandler(log log.Logger, s ActivityLogService, vc settingssvc.ValueService, t l10n.Translator) func(w http.ResponseWriter, r *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
ctx = metadata.AppendToOutgoingContext(ctx, revactx.TokenHeader, r.Header.Get(revactx.TokenHeader))
activeUser, ok := revactx.ContextGetUser(ctx)
if !ok {
w.WriteHeader(http.StatusUnauthorized)
return
}
loc := l10n.MustGetUserLocale(ctx, activeUser.GetId().GetOpaqueId(), r.Header.Get(l10n.HeaderAcceptLanguage), vc)
activities, err := s.GetItemActivities(ctx, r.URL.Query().Get("kql"), loc, t)
if err != nil {
switch {
case errors.Is(err, activityloghttp.ErrBadRequest):
log.Debug().Str("query", r.URL.Query().Get("kql")).Err(err).Msg("error getting activities")
w.WriteHeader(http.StatusBadRequest)
return
case errors.Is(err, activityloghttp.ErrForbidden):
log.Debug().Err(err).Msg("error getting activities")
w.WriteHeader(http.StatusForbidden)
return
default:
log.Error().Err(err).Msg("error getting activities")
w.WriteHeader(http.StatusInternalServerError)
return
}
}
res := GetActivitiesResponse{
Activities: activities,
}
w.Header().Set("Content-Type", "application/json; odata.metadata=minimal")
w.Header().Set("OData-Version", "4.0")
if reqID := chimiddleware.GetReqID(ctx); reqID != "" {
w.Header().Set("request-id", reqID)
}
w.Header().Set("Cache-Control", "no-cache")
if err := json.NewEncoder(w).Encode(res); err != nil {
log.Error().Err(err).Msg("error encoding activities")
w.WriteHeader(http.StatusInternalServerError)
return
}
}
}
@@ -0,0 +1,366 @@
package activitylog
import (
"context"
"encoding/base32"
"encoding/json"
"fmt"
"sort"
"strconv"
"strings"
"sync"
"time"
provider "github.com/cs3org/go-cs3apis/cs3/storage/provider/v1beta1"
"github.com/jellydator/ttlcache/v2"
"github.com/nats-io/nats.go"
"github.com/opencloud-eu/opencloud/pkg/log"
"github.com/opencloud-eu/reva/v2/pkg/storagespace"
"github.com/vmihailenco/msgpack/v5"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/trace"
)
var tracer trace.Tracer
func init() {
tracer = otel.Tracer("github.com/opencloud-eu/opencloud/services/activitylog/pkg/service/activitylog")
}
var (
_maxActivitiesDefault = 6000
_writeBufferDuration = 10 * time.Second
)
// Activitylog stores and retrieves activities for resources and their parents from a nats kv
type ActivityLog struct {
log log.Logger
// FIXME the lock does not protect agains concurrent resource activities on multiple instances
// known since https://github.com/owncloud/ocis/pull/9361#pullrequestreview-2135350157
// current ocis discussion in https://github.com/owncloud/ocis/issues/12475
lock sync.RWMutex
debouncer *Debouncer
parentIdCache *ttlcache.Cache
natskv nats.KeyValue
maxActivities int
}
type batchInfo struct {
key string
count int
timestamp time.Time
}
// New creates a new ActivitylogService
func New(kv nats.KeyValue, opts ...Option) (*ActivityLog, error) {
o := &Options{
MaxActivities: _maxActivitiesDefault,
WriteBufferDuration: _writeBufferDuration,
Logger: log.NopLogger(),
}
for _, opt := range opts {
opt(o)
}
cache := ttlcache.NewCache()
err := cache.SetTTL(30 * time.Second)
if err != nil {
return nil, err
}
s := &ActivityLog{
log: o.Logger,
lock: sync.RWMutex{},
parentIdCache: cache,
maxActivities: o.MaxActivities,
natskv: kv,
}
s.debouncer = NewDebouncer(o.WriteBufferDuration, s.StoreActivity)
// run migrations
err = s.runMigrations(context.Background(), kv)
if err != nil {
return nil, err
}
return s, nil
}
// RemoveResource removes the resource from the store
func (a *ActivityLog) RemoveResource(rid *provider.ResourceId) error {
if rid == nil {
return fmt.Errorf("resource id is required")
}
a.lock.Lock()
defer a.lock.Unlock()
err := a.natskv.Delete(storagespace.FormatResourceID(rid))
if err != nil {
return fmt.Errorf("could not delete resource %s: %w", rid.OpaqueId, err)
}
return nil
}
// RawActivity represents an activity as it is stored in the activitylog store
type RawActivity struct {
EventID string `json:"event_id"`
Depth int `json:"depth"`
Timestamp time.Time `json:"timestamp"`
}
func (a *ActivityLog) AddActivity(ctx context.Context, initRef *provider.Reference, parentId *provider.ResourceId, eventID string, timestamp time.Time, getResource func(context.Context, *provider.Reference) (*provider.ResourceInfo, error)) error {
var (
err error
depth int
ref = initRef
)
ctx, span := tracer.Start(ctx, "AddActivity")
defer span.End()
for {
var info *provider.ResourceInfo
id := ref.GetResourceId()
if ref.Path != "" {
// Path based reference, we need to resolve the resource id
ctx, span = tracer.Start(ctx, "AddActivity.getResource")
info, err = getResource(ctx, ref)
span.End()
if err != nil {
// TODO If the resource was deleted should we still log an activity in the parent?
return fmt.Errorf("could not get resource info for reference %v: %w", ref, err)
}
id = info.GetId()
}
if id == nil {
return fmt.Errorf("resource id is required")
}
key := storagespace.FormatResourceID(id)
a.debouncer.Debounce(key, RawActivity{
EventID: eventID,
Depth: depth,
Timestamp: timestamp,
})
if id.OpaqueId == id.SpaceId {
// we are at the root of the space, no need to go further
break
}
// check if parent id is cached
// parent id is cached in the format <storageid>$<spaceid>!<resourceid>
// if it is not cached, get the resource info and cache it
if parentId == nil {
if v, err := a.parentIdCache.Get(key); err != nil {
if info == nil {
ctx, span := tracer.Start(ctx, "AddActivity.getResource parent")
info, err = getResource(ctx, ref)
span.End()
if err != nil || info.GetParentId() == nil || info.GetParentId().GetOpaqueId() == "" {
return fmt.Errorf("could not get parent id: %w", err)
}
}
parentId = info.GetParentId()
a.parentIdCache.Set(key, parentId)
} else {
parentId = v.(*provider.ResourceId)
}
} else {
a.log.Debug().Msg("parent id is cached")
}
depth++
ref = &provider.Reference{ResourceId: parentId}
parentId = nil // reset parent id so it's not reused in the next iteration
}
return nil
}
func (a *ActivityLog) StoreActivity(resourceID string, activities []RawActivity) error {
a.lock.Lock()
defer a.lock.Unlock()
ctx, span := tracer.Start(context.Background(), "storeActivity")
defer span.End()
_, subspan := tracer.Start(ctx, "storeActivity.Marshal")
b, err := msgpack.Marshal(activities)
if err != nil {
return err
}
subspan.End()
_, subspan = tracer.Start(ctx, "storeActivity.natskv.Put")
key := natsKey(resourceID, len(activities))
_, err = a.natskv.Put(key, b)
if err != nil {
return err
}
subspan.End()
ctx, subspan = tracer.Start(ctx, "storeActivity.enforceMaxActivities")
a.enforceMaxActivities(ctx, resourceID)
subspan.End()
return nil
}
func (a *ActivityLog) enforceMaxActivities(ctx context.Context, resourceID string) {
if a.maxActivities <= 0 {
return
}
key := fmt.Sprintf("%s.>", base32.StdEncoding.EncodeToString([]byte(resourceID)))
_, subspan := tracer.Start(ctx, "enforceMaxActivities.watch")
watcher, err := a.natskv.Watch(key, nats.IgnoreDeletes())
if err != nil {
a.log.Error().Err(err).Str("resourceID", resourceID).Msg("could not watch")
return
}
defer watcher.Stop()
var keys []string
for update := range watcher.Updates() {
if update == nil {
break
}
var batchActivities []RawActivity
if err := msgpack.Unmarshal(update.Value(), &batchActivities); err != nil {
a.log.Debug().Err(err).Str("resourceID", resourceID).Msg("could not unmarshal messagepack, trying json")
}
keys = append(keys, update.Key())
}
subspan.End()
_, subspan = tracer.Start(ctx, "enforceMaxActivities.compile")
// Parse keys into batches
batches := make([]batchInfo, 0)
var activitiesCount int
for _, k := range keys {
parts := strings.SplitN(k, ".", 3)
if len(parts) < 3 {
a.log.Warn().Str("key", k).Msg("skipping key, not enough parts")
continue
}
c, err := strconv.Atoi(parts[1])
if err != nil {
a.log.Warn().Str("key", k).Msg("skipping key, can not parse count")
continue
}
// parse timestamp
nano, err := strconv.ParseInt(parts[2], 10, 64)
if err != nil {
a.log.Warn().Str("key", k).Msg("skipping key, can not parse timestamp")
continue
}
batches = append(batches, batchInfo{
key: k,
count: c,
timestamp: time.Unix(0, nano),
})
activitiesCount += c
}
// sort batches by timestamp
sort.Slice(batches, func(i, j int) bool {
return batches[i].timestamp.Before(batches[j].timestamp)
})
subspan.End()
_, subspan = tracer.Start(ctx, "enforceMaxActivities.delete")
// remove oldest keys until we are at max activities
for _, b := range batches {
if activitiesCount-b.count < a.maxActivities {
break
}
activitiesCount -= b.count
err = a.natskv.Delete(b.key)
if err != nil {
a.log.Error().Err(err).Str("key", b.key).Msg("could not delete key")
break
}
}
subspan.End()
}
func (a *ActivityLog) InvalidateCachedParentID(purgeId *provider.ResourceId) {
// The parent id cache is populated lazily and its entries expire, so a
// missing key is the expected case rather than an error.
if err := a.parentIdCache.Remove(storagespace.FormatResourceID(purgeId)); err != nil {
a.log.Debug().Interface("event", purgeId).Err(err).Msg("could not delete parent id cache")
}
}
func natsKey(resourceID string, activitiesCount int) string {
return fmt.Sprintf("%s.%d.%d",
base32.StdEncoding.EncodeToString([]byte(resourceID)),
activitiesCount,
time.Now().UnixNano())
}
func (a *ActivityLog) Activities(rid *provider.ResourceId) ([]RawActivity, error) {
a.lock.RLock()
defer a.lock.RUnlock()
return a.activities(rid)
}
func (a *ActivityLog) activities(rid *provider.ResourceId) ([]RawActivity, error) {
resourceID := storagespace.FormatResourceID(rid)
glob := fmt.Sprintf("%s.>", base32.StdEncoding.EncodeToString([]byte(resourceID)))
watcher, err := a.natskv.Watch(glob, nats.IgnoreDeletes())
if err != nil {
return nil, err
}
defer watcher.Stop()
var activities []RawActivity
for update := range watcher.Updates() {
if update == nil {
break
}
var batchActivities []RawActivity
if err := msgpack.Unmarshal(update.Value(), &batchActivities); err != nil {
a.log.Debug().Err(err).Str("resourceID", resourceID).Msg("could not unmarshal messagepack")
}
activities = append(activities, batchActivities...)
}
return activities, nil
}
// RemoveActivities removes the activities from the given resource
func (a *ActivityLog) RemoveActivities(rid *provider.ResourceId, toDelete map[string]struct{}) error {
a.lock.Lock()
defer a.lock.Unlock()
curActivities, err := a.activities(rid)
if err != nil {
return err
}
var acts []RawActivity
for _, a := range curActivities {
if _, ok := toDelete[a.EventID]; !ok {
acts = append(acts, a)
}
}
b, err := json.Marshal(acts)
if err != nil {
return err
}
_, err = a.natskv.Put(storagespace.FormatResourceID(rid), b)
return err
}
@@ -0,0 +1,13 @@
package activitylog_test
import (
"testing"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
func TestActivitylog(t *testing.T) {
RegisterFailHandler(Fail)
RunSpecs(t, "Activitylog Suite")
}
@@ -1,7 +1,6 @@
package service
package activitylog_test
import (
"bytes"
"context"
"net"
"os"
@@ -9,17 +8,13 @@ import (
"time"
provider "github.com/cs3org/go-cs3apis/cs3/storage/provider/v1beta1"
"github.com/go-chi/chi/v5"
"github.com/google/uuid"
nserver "github.com/nats-io/nats-server/v2/server"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/opencloud-eu/opencloud/pkg/log"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/command"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/config"
eventsmocks "github.com/opencloud-eu/reva/v2/pkg/events/mocks"
"github.com/opencloud-eu/reva/v2/pkg/storagespace"
"github.com/test-go/testify/mock"
"go.opentelemetry.io/otel/trace/noop"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/service/activitylog"
)
var (
@@ -68,31 +63,25 @@ var _ = SynchronizedAfterSuite(func() {
var _ = Describe("ActivitylogService", func() {
var (
alog *ActivitylogService
alog *activitylog.ActivityLog
getResource func(_ context.Context, ref *provider.Reference) (*provider.ResourceInfo, error)
writebufferduration = 100 * time.Millisecond
)
JustBeforeEach(func() {
var err error
stream := &eventsmocks.Stream{}
stream.EXPECT().Consume(mock.Anything, mock.Anything).Return(nil, nil)
alog, err = New(
Config(&config.Config{
Service: config.Service{
Name: "activitylog-test",
},
Store: config.Store{
Store: "nats-js-kv",
Nodes: []string{server.Addr().String()},
Database: "activitylog-test-" + uuid.New().String(),
},
MaxActivities: 4,
WriteBufferDuration: writebufferduration,
}),
Stream(stream),
TraceProvider(noop.NewTracerProvider()),
Mux(chi.NewMux()),
db := "activitylog-test-" + uuid.New().String()
kv, err := command.ConnectNatsKV(config.Store{
Nodes: []string{server.Addr().String()},
Database: db,
})
Expect(err).ToNot(HaveOccurred())
alog, err = activitylog.New(
kv,
activitylog.MaxActivities(4),
activitylog.WriteBufferDuration(writebufferduration),
)
Expect(err).ToNot(HaveOccurred())
})
@@ -107,7 +96,7 @@ var _ = Describe("ActivitylogService", func() {
Name string
Tree map[string]*provider.ResourceInfo
Activities map[string]string
Expected map[string][]RawActivity
Expected map[string][]activitylog.RawActivity
}
testCases := []testCase{
@@ -121,7 +110,7 @@ var _ = Describe("ActivitylogService", func() {
Activities: map[string]string{
"activity": "base",
},
Expected: map[string][]RawActivity{
Expected: map[string][]activitylog.RawActivity{
"base": activitites("activity", 0),
"parent": activitites("activity", 1),
"spaceid": activitites("activity", 2),
@@ -138,7 +127,7 @@ var _ = Describe("ActivitylogService", func() {
"activity1": "base",
"activity2": "base",
},
Expected: map[string][]RawActivity{
Expected: map[string][]activitylog.RawActivity{
"base": activitites("activity1", 0, "activity2", 0),
"parent": activitites("activity1", 1, "activity2", 1),
"spaceid": activitites("activity1", 2, "activity2", 2),
@@ -155,7 +144,7 @@ var _ = Describe("ActivitylogService", func() {
}
for k, v := range tc.Activities {
err := alog.addActivity(context.Background(), reference(v), nil, k, time.Time{}, getResource)
err := alog.AddActivity(context.Background(), reference(v), nil, k, time.Time{}, getResource)
Expect(err).NotTo(HaveOccurred())
}
})
@@ -194,9 +183,9 @@ var _ = Describe("ActivitylogService", func() {
It("debounces activities", func() {
err := alog.addActivity(context.Background(), reference("base"), nil, "activity1", time.Time{}, getResource)
err := alog.AddActivity(context.Background(), reference("base"), nil, "activity1", time.Time{}, getResource)
Expect(err).NotTo(HaveOccurred())
err = alog.addActivity(context.Background(), reference("base"), nil, "activity2", time.Time{}, getResource)
err = alog.AddActivity(context.Background(), reference("base"), nil, "activity2", time.Time{}, getResource)
Expect(err).NotTo(HaveOccurred())
Eventually(func(g Gomega) {
@@ -207,7 +196,7 @@ var _ = Describe("ActivitylogService", func() {
})
It("adheres to the MaxActivities setting", func() {
err := alog.addActivity(context.Background(), reference("base"), nil, "activity1", time.Time{}, getResource)
err := alog.AddActivity(context.Background(), reference("base"), nil, "activity1", time.Time{}, getResource)
Expect(err).NotTo(HaveOccurred())
Eventually(func(g Gomega) {
activities, err := alog.Activities(resourceID("base"))
@@ -215,7 +204,7 @@ var _ = Describe("ActivitylogService", func() {
g.Expect(len(activities)).To(Equal(1))
}).Should(Succeed())
err = alog.addActivity(context.Background(), reference("base"), nil, "activity2", time.Time{}, getResource)
err = alog.AddActivity(context.Background(), reference("base"), nil, "activity2", time.Time{}, getResource)
Expect(err).NotTo(HaveOccurred())
Eventually(func(g Gomega) {
activities, err := alog.Activities(resourceID("base"))
@@ -223,11 +212,11 @@ var _ = Describe("ActivitylogService", func() {
g.Expect(len(activities)).To(Equal(2))
}).Should(Succeed())
err = alog.addActivity(context.Background(), reference("base"), nil, "activity3", time.Time{}, getResource)
err = alog.AddActivity(context.Background(), reference("base"), nil, "activity3", time.Time{}, getResource)
Expect(err).NotTo(HaveOccurred())
err = alog.addActivity(context.Background(), reference("base"), nil, "activity4", time.Time{}, getResource)
err = alog.AddActivity(context.Background(), reference("base"), nil, "activity4", time.Time{}, getResource)
Expect(err).NotTo(HaveOccurred())
err = alog.addActivity(context.Background(), reference("base"), nil, "activity5", time.Time{}, getResource)
err = alog.AddActivity(context.Background(), reference("base"), nil, "activity5", time.Time{}, getResource)
Expect(err).NotTo(HaveOccurred())
Eventually(func(g Gomega) {
@@ -244,9 +233,9 @@ var _ = Describe("ActivitylogService", func() {
return tree[ref.GetResourceId().GetOpaqueId()], nil
}
err := alog.addActivity(context.Background(), reference("base"), nil, "activity1", time.Time{}, getResource)
err := alog.AddActivity(context.Background(), reference("base"), nil, "activity1", time.Time{}, getResource)
Expect(err).NotTo(HaveOccurred())
err = alog.addActivity(context.Background(), reference("base"), nil, "activity2", time.Time{}, getResource)
err = alog.AddActivity(context.Background(), reference("base"), nil, "activity2", time.Time{}, getResource)
Expect(err).NotTo(HaveOccurred())
Eventually(func(g Gomega) {
@@ -255,9 +244,9 @@ var _ = Describe("ActivitylogService", func() {
g.Expect(activities).To(ConsistOf(activitites("activity1", 0, "activity2", 0)))
}).Should(Succeed())
err = alog.addActivity(context.Background(), reference("base"), nil, "activity3", time.Time{}, getResource)
err = alog.AddActivity(context.Background(), reference("base"), nil, "activity3", time.Time{}, getResource)
Expect(err).NotTo(HaveOccurred())
err = alog.addActivity(context.Background(), reference("base"), nil, "activity4", time.Time{}, getResource)
err = alog.AddActivity(context.Background(), reference("base"), nil, "activity4", time.Time{}, getResource)
Expect(err).NotTo(HaveOccurred())
Eventually(func(g Gomega) {
@@ -268,47 +257,11 @@ var _ = Describe("ActivitylogService", func() {
})
})
})
Describe("removeCachedParentID", func() {
var logBuffer *bytes.Buffer
newLoggerAtLevel := func(level string) log.Logger {
logBuffer = &bytes.Buffer{}
return log.Logger{Logger: log.NewLogger(log.Level(level)).Output(logBuffer)}
}
It("does not log an error when the entry was never cached", func() {
alog.log = newLoggerAtLevel("error")
alog.removeCachedParentID(reference("never-cached"))
Expect(logBuffer.String()).To(BeEmpty())
})
It("logs a missing entry at debug level", func() {
alog.log = newLoggerAtLevel("debug")
alog.removeCachedParentID(reference("never-cached"))
Expect(logBuffer.String()).To(ContainSubstring("could not delete parent id cache"))
Expect(logBuffer.String()).To(ContainSubstring(`"level":"debug"`))
})
It("does not log at all when the entry was cached", func() {
alog.log = newLoggerAtLevel("debug")
ref := reference("cached")
Expect(alog.parentIdCache.Set(storagespace.FormatResourceID(ref.GetResourceId()), resourceID("parent"))).To(Succeed())
alog.removeCachedParentID(ref)
Expect(logBuffer.String()).To(BeEmpty())
})
})
})
func activitites(acts ...any) []RawActivity {
var activities []RawActivity
act := RawActivity{}
func activitites(acts ...any) []activitylog.RawActivity {
var activities []activitylog.RawActivity
act := activitylog.RawActivity{}
for _, a := range acts {
switch v := a.(type) {
case string:
@@ -0,0 +1,76 @@
package activitylog
import (
"sync"
"time"
)
// Debouncer is used to debounce writes to the activity log store.
type Debouncer struct {
after time.Duration
f func(id string, ra []RawActivity) error
pending sync.Map
inProgress sync.Map
mutex sync.Mutex
}
type queueItem struct {
activities []RawActivity
timer *time.Timer
}
// NewDebouncer returns a new Debouncer instance.
func NewDebouncer(d time.Duration, f func(id string, ra []RawActivity) error) *Debouncer {
return &Debouncer{
after: d,
f: f,
pending: sync.Map{},
inProgress: sync.Map{},
}
}
// Debounce restarts the debounce timer for the given space.
func (d *Debouncer) Debounce(id string, ra RawActivity) {
if d.after == 0 {
d.f(id, []RawActivity{ra})
return
}
d.mutex.Lock()
defer d.mutex.Unlock()
item := &queueItem{
activities: []RawActivity{ra},
}
if i, ok := d.pending.Load(id); ok {
// if the item is already in the queue, append the new activities
item, ok = i.(*queueItem)
if ok {
item.activities = append(item.activities, ra)
}
}
if item.timer == nil {
item.timer = time.AfterFunc(d.after, func() {
if _, ok := d.inProgress.Load(id); ok {
// Reschedule this run for when the previous run has finished
d.mutex.Lock()
if i, ok := d.pending.Load(id); ok {
i.(*queueItem).timer.Reset(d.after)
}
d.mutex.Unlock()
return
}
d.pending.Delete(id)
d.inProgress.Store(id, true)
defer d.inProgress.Delete(id)
d.f(id, item.activities)
})
}
d.pending.Store(id, item)
}
@@ -0,0 +1,121 @@
package activitylog_test
import (
"sync"
"time"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/service/activitylog"
)
var _ = Describe("Debouncer", func() {
var (
mu sync.Mutex
callbacks []activitylog.RawActivity
newCallback func(id string, ra []activitylog.RawActivity) error
)
BeforeEach(func() {
mu.Lock()
callbacks = nil
mu.Unlock()
newCallback = func(id string, ra []activitylog.RawActivity) error {
mu.Lock()
defer mu.Unlock()
callbacks = append(callbacks, ra...)
return nil
}
})
Context("with zero duration", func() {
It("calls the callback immediately", func() {
d := activitylog.NewDebouncer(0, newCallback)
d.Debounce("space1", activitylog.RawActivity{EventID: "activity1"})
Expect(callbacks).To(HaveLen(1))
Expect(callbacks[0].EventID).To(Equal("activity1"))
})
It("calls the callback immediately for each event", func() {
d := activitylog.NewDebouncer(0, newCallback)
d.Debounce("space1", activitylog.RawActivity{EventID: "activity1"})
d.Debounce("space2", activitylog.RawActivity{EventID: "activity2"})
Expect(callbacks).To(HaveLen(2))
})
})
Context("with non-zero duration", func() {
It("batches activities with the same id", func() {
d := activitylog.NewDebouncer(10*time.Millisecond, newCallback)
d.Debounce("space1", activitylog.RawActivity{EventID: "activity1"})
d.Debounce("space1", activitylog.RawActivity{EventID: "activity2"})
d.Debounce("space1", activitylog.RawActivity{EventID: "activity3"})
Eventually(func() int {
mu.Lock()
defer mu.Unlock()
return len(callbacks)
}).Should(Equal(3))
})
It("handles different ids independently", func() {
d := activitylog.NewDebouncer(10*time.Millisecond, newCallback)
d.Debounce("space1", activitylog.RawActivity{EventID: "activity1"})
d.Debounce("space2", activitylog.RawActivity{EventID: "activity2"})
Eventually(func() int {
mu.Lock()
defer mu.Unlock()
return len(callbacks)
}).Should(Equal(2))
})
It("batches activities that arrive within the debounce window", func() {
d := activitylog.NewDebouncer(100*time.Millisecond, newCallback)
d.Debounce("space1", activitylog.RawActivity{EventID: "activity1"})
time.Sleep(20 * time.Millisecond)
d.Debounce("space1", activitylog.RawActivity{EventID: "activity2"})
time.Sleep(20 * time.Millisecond)
d.Debounce("space1", activitylog.RawActivity{EventID: "activity3"})
Eventually(func() int {
mu.Lock()
defer mu.Unlock()
return len(callbacks)
}).Should(Equal(3))
})
It("processes new batch after previous completes", func() {
d := activitylog.NewDebouncer(5*time.Millisecond, newCallback)
d.Debounce("space1", activitylog.RawActivity{EventID: "activity1"})
time.Sleep(20 * time.Millisecond) // let first batch complete
d.Debounce("space1", activitylog.RawActivity{EventID: "activity2"})
Eventually(func() int {
mu.Lock()
defer mu.Unlock()
return len(callbacks)
}).Should(Equal(2))
})
It("skips duplicate write when timer fires during in-progress callback", func() {
slowCallback := func(id string, ra []activitylog.RawActivity) error {
time.Sleep(50 * time.Millisecond) // simulate slow write
mu.Lock()
defer mu.Unlock()
callbacks = append(callbacks, ra...)
return nil
}
d := activitylog.NewDebouncer(10*time.Millisecond, slowCallback)
d.Debounce("space1", activitylog.RawActivity{EventID: "activity1"})
time.Sleep(20 * time.Millisecond) // timer fires while callback is running
Eventually(func() int {
mu.Lock()
defer mu.Unlock()
return len(callbacks)
}).Should(Equal(1))
})
})
})
@@ -1,4 +1,4 @@
package service
package activitylog
import (
"context"
@@ -15,7 +15,7 @@ const currentMigrationVersion = "1"
// RunMigrations checks the activitylog data version and runs migrations if necessary.
// It should be called during service startup, after the NATS KeyValue store is initialized.
func (a *ActivitylogService) runMigrations(ctx context.Context, kv nats.KeyValue) error {
func (a *ActivityLog) runMigrations(ctx context.Context, kv nats.KeyValue) error {
entry, err := kv.Get(activitylogVersionKey)
if err == nats.ErrKeyNotFound {
a.log.Info().Msg("activitylog version key not found. Running migration to V1...")
@@ -40,7 +40,7 @@ func (a *ActivitylogService) runMigrations(ctx context.Context, kv nats.KeyValue
// For each such key, it creates a new key in the format "originalKey.count.timestamp"
// and stores the original list of strings (re-marshalled to messagepack) as its value.
// Finally, it sets the activitylog.version key to "1".
func (a *ActivitylogService) migrateToV1(_ context.Context, kv nats.KeyValue) error {
func (a *ActivityLog) migrateToV1(_ context.Context, kv nats.KeyValue) error {
lister, err := kv.ListKeys()
if err != nil {
return fmt.Errorf("migrateToV1: failed to list keys from NATS KV store: %w", err)
@@ -0,0 +1,35 @@
package activitylog
import (
"time"
"github.com/opencloud-eu/opencloud/pkg/log"
)
// Option for the activitylog service
type Option func(*Options)
// Options for the activitylog service
type Options struct {
Logger log.Logger
MaxActivities int
WriteBufferDuration time.Duration
}
// Logger configures a logger for the activitylog service
func Logger(log log.Logger) Option {
return func(o *Options) {
o.Logger = log
}
}
func MaxActivities(max int) Option {
return func(o *Options) {
o.MaxActivities = max
}
}
func WriteBufferDuration(d time.Duration) Option {
return func(o *Options) {
o.WriteBufferDuration = d
}
}
@@ -1,4 +1,4 @@
package service_test
package events_test
import (
"testing"
@@ -7,7 +7,7 @@ import (
. "github.com/onsi/gomega"
)
func TestService(t *testing.T) {
func TestEvents(t *testing.T) {
RegisterFailHandler(Fail)
RunSpecs(t, "Service Suite")
RunSpecs(t, "Events Suite")
}
@@ -1,17 +1,14 @@
package service
package events
import (
"context"
"time"
gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1"
"github.com/go-chi/chi/v5"
"github.com/opencloud-eu/opencloud/pkg/log"
ehsvc "github.com/opencloud-eu/opencloud/protogen/gen/opencloud/services/eventhistory/v0"
settingssvc "github.com/opencloud-eu/opencloud/protogen/gen/opencloud/services/settings/v0"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/config"
"github.com/opencloud-eu/reva/v2/pkg/events"
"github.com/opencloud-eu/reva/v2/pkg/rgrpc/todo/pool"
"go.opentelemetry.io/otel/trace"
)
// Option for the activitylog service
@@ -19,17 +16,20 @@ type Option func(*Options)
// Options for the activitylog service
type Options struct {
Context context.Context
Logger log.Logger
Config *config.Config
TraceProvider trace.TracerProvider
ServiceAccount config.ServiceAccount
Stream events.Stream
RegisteredEvents []events.Unmarshaller
GatewaySelector pool.Selectable[gateway.GatewayAPIClient]
Mux *chi.Mux
HistoryClient ehsvc.EventHistoryService
ValueClient settingssvc.ValueService
WriteBufferDuration time.Duration
MaxActivities int
NumConsumers int
}
func Context(ctx context.Context) Option {
return func(o *Options) {
o.Context = ctx
}
}
// Logger configures a logger for the activitylog service
@@ -39,17 +39,10 @@ func Logger(log log.Logger) Option {
}
}
// Config adds the config for the activitylog service
func Config(c *config.Config) Option {
// ServiceAccount configures a service account for the activitylog service
func ServiceAccount(sa config.ServiceAccount) Option {
return func(o *Options) {
o.Config = c
}
}
// TraceProvider adds a tracer provider for the activitylog service
func TraceProvider(tp trace.TracerProvider) Option {
return func(o *Options) {
o.TraceProvider = tp
o.ServiceAccount = sa
}
}
@@ -74,23 +67,8 @@ func GatewaySelector(gatewaySelector pool.Selectable[gateway.GatewayAPIClient])
}
}
// Mux defines the muxer for the service
func Mux(m *chi.Mux) Option {
func NumConsumers(num int) Option {
return func(o *Options) {
o.Mux = m
}
}
// HistoryClient adds a grpc client for the eventhistory service
func HistoryClient(hc ehsvc.EventHistoryService) Option {
return func(o *Options) {
o.HistoryClient = hc
}
}
// ValueClient adds a grpc client for the value service
func ValueClient(vs settingssvc.ValueService) Option {
return func(o *Options) {
o.ValueClient = vs
o.NumConsumers = num
}
}
@@ -0,0 +1,302 @@
package events
import (
"context"
"fmt"
"path/filepath"
"sync"
"sync/atomic"
"time"
gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1"
provider "github.com/cs3org/go-cs3apis/cs3/storage/provider/v1beta1"
"github.com/opencloud-eu/opencloud/pkg/log"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/config"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/service/activitylog"
"github.com/opencloud-eu/reva/v2/pkg/events"
"github.com/opencloud-eu/reva/v2/pkg/rgrpc/todo/pool"
"github.com/opencloud-eu/reva/v2/pkg/storagespace"
"github.com/opencloud-eu/reva/v2/pkg/utils"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/trace"
)
var tracer trace.Tracer
func init() {
tracer = otel.Tracer("github.com/opencloud-eu/opencloud/services/activitylog/pkg/service/events")
}
var (
_numConsumersDefault = 1
)
// ActivitylogService logs events per resource
type ActivitylogService struct {
ctx context.Context
sa config.ServiceAccount
log log.Logger
stream events.Stream
gws pool.Selectable[gateway.GatewayAPIClient]
al *activitylog.ActivityLog
numConsumers int
events []events.Unmarshaller
stopCh chan struct{}
stopped *atomic.Bool
}
// New creates a new ActivitylogService
func New(al *activitylog.ActivityLog, stream events.Stream, opts ...Option) (*ActivitylogService, error) {
o := &Options{
NumConsumers: _numConsumersDefault,
}
for _, opt := range opts {
opt(o)
}
s := &ActivitylogService{
ctx: o.Context,
log: o.Logger,
sa: o.ServiceAccount,
stream: stream,
gws: o.GatewaySelector,
events: o.RegisteredEvents,
numConsumers: o.NumConsumers,
al: al,
stopCh: make(chan struct{}, 1),
stopped: new(atomic.Bool),
}
return s, nil
}
// Run to fulfil Runner interface
func (s *ActivitylogService) Run() error {
ch, err := events.Consume(s.stream, "activitylog", s.events...)
if err != nil {
return err
}
var wg sync.WaitGroup
ctx, cancel := context.WithCancel(s.ctx)
defer cancel()
s.log.Debug().Int("worker.count", s.numConsumers).
Str("messaging.consumer.group.name", "activitylog").
Str("messaging.system", "nats").
Str("messaging.operation.name", "receive").
Msg("starting event processing workers")
// start workers
for i := 0; i < s.numConsumers; i++ {
wg.Add(1)
go func(workerID int) {
defer wg.Done()
for {
select {
case <-ctx.Done():
return
case e, ok := <-ch:
if !ok {
return
}
if err := s.processEvent(e); err != nil {
s.log.Error().Err(err).
Int("worker", workerID).
Interface("event", e).
Msg("failed to process event")
}
}
}
}(i)
}
// wait for stop signal
<-s.stopCh
cancel() // signal workers to stop
wg.Wait()
return nil
}
// Close will make the service to stop processing, so the `Run`
// method can finish.
func (s *ActivitylogService) Close() {
if s.stopped.CompareAndSwap(false, true) {
close(s.stopCh)
}
}
func (s *ActivitylogService) processEvent(e events.Event) error {
ctx := e.GetTraceContext(s.ctx)
ctx, span := tracer.Start(ctx, "processEvent")
defer span.End()
s.log.Debug().Interface("event", e).Msg("updating activitylog")
switch ev := e.Event.(type) {
case events.UploadReady:
return s.AddActivity(ctx, ev.FileRef, ev.ParentID, e.ID, utils.TSToTime(ev.Timestamp))
case events.FileTouched:
return s.AddActivity(ctx, ev.Ref, ev.ParentID, e.ID, utils.TSToTime(ev.Timestamp))
// Disabled https://github.com/owncloud/ocis/issues/10293
//case events.FileDownloaded:
// we are only interested in public link downloads - so no need to store others.
//if ev.ImpersonatingUser.GetDisplayName() == "Public" {
// err = a.AddActivity(ev.Ref, e.ID, utils.TSToTime(ev.Timestamp))
//}
case events.ContainerCreated:
return s.AddActivity(ctx, ev.Ref, ev.ParentID, e.ID, utils.TSToTime(ev.Timestamp))
case events.ItemTrashed:
return s.AddActivityTrashed(ctx, ev.ID, ev.Ref, nil, e.ID, utils.TSToTime(ev.Timestamp))
case events.ItemPurged:
return s.al.RemoveResource(ev.ID)
case events.ItemMoved:
// remove the cached parent id for this resource
s.removeCachedParentID(ctx, ev.Ref)
return s.AddActivity(ctx, ev.Ref, nil, e.ID, utils.TSToTime(ev.Timestamp))
case events.ShareCreated:
return s.AddActivity(ctx, toRef(ev.ItemID), nil, e.ID, utils.TSToTime(ev.CTime))
case events.ShareUpdated:
if ev.Sharer != nil && ev.ItemID != nil && ev.Sharer.GetOpaqueId() != ev.ItemID.GetSpaceId() {
return s.AddActivity(ctx, toRef(ev.ItemID), nil, e.ID, utils.TSToTime(ev.MTime))
}
case events.ShareRemoved:
return s.AddActivity(ctx, toRef(ev.ItemID), nil, e.ID, ev.Timestamp)
case events.LinkCreated:
return s.AddActivity(ctx, toRef(ev.ItemID), nil, e.ID, utils.TSToTime(ev.CTime))
case events.LinkUpdated:
if ev.Sharer != nil && ev.ItemID != nil && ev.Sharer.GetOpaqueId() != ev.ItemID.GetSpaceId() {
return s.AddActivity(ctx, toRef(ev.ItemID), nil, e.ID, utils.TSToTime(ev.MTime))
}
case events.LinkRemoved:
return s.AddActivity(ctx, toRef(ev.ItemID), nil, e.ID, utils.TSToTime(ev.Timestamp))
case events.SpaceShared:
return s.AddSpaceActivity(ctx, ev.ID, e.ID, ev.Timestamp)
case events.SpaceUnshared:
return s.AddSpaceActivity(ctx, ev.ID, e.ID, ev.Timestamp)
}
return nil
}
// AddActivity adds the activity to the given resource and all its parents
func (a *ActivitylogService) AddActivity(ctx context.Context, initRef *provider.Reference, parentId *provider.ResourceId, eventID string, timestamp time.Time) error {
ctx, span := tracer.Start(ctx, "AddActivity")
defer span.End()
gwc, err := a.gws.Next()
if err != nil {
return fmt.Errorf("cant get gateway client: %w", err)
}
ctx, err = utils.GetServiceUserContextWithContext(ctx, gwc, a.sa.ServiceAccountID, a.sa.ServiceAccountSecret)
if err != nil {
return fmt.Errorf("cant get service user context: %w", err)
}
return a.al.AddActivity(ctx, initRef, parentId, eventID, timestamp, func(ctx context.Context, ref *provider.Reference) (*provider.ResourceInfo, error) {
return utils.GetResource(ctx, ref, gwc)
})
}
// AddActivityTrashed adds the activity to given trashed resource and all its former parents
func (a *ActivitylogService) AddActivityTrashed(ctx context.Context, resourceID *provider.ResourceId, reference *provider.Reference, parentId *provider.ResourceId, eventID string, timestamp time.Time) error {
ctx, span := tracer.Start(ctx, "AddActivityTrashed")
defer span.End()
gwc, err := a.gws.Next()
if err != nil {
return fmt.Errorf("cant get gateway client: %w", err)
}
ctx, err = utils.GetServiceUserContextWithContext(ctx, gwc, a.sa.ServiceAccountID, a.sa.ServiceAccountSecret)
if err != nil {
return fmt.Errorf("cant get service user context: %w", err)
}
// store activity on trashed item
if err := a.al.StoreActivity(storagespace.FormatResourceID(resourceID), []activitylog.RawActivity{
{
EventID: eventID,
Depth: 0,
Timestamp: timestamp,
},
}); err != nil {
return fmt.Errorf("could not store activity: %w", err)
}
// get previous parent
ref := &provider.Reference{
ResourceId: reference.GetResourceId(),
Path: filepath.Dir(reference.GetPath()),
}
return a.al.AddActivity(ctx, ref, parentId, eventID, timestamp, func(ctx context.Context, ref *provider.Reference) (*provider.ResourceInfo, error) {
return utils.GetResource(ctx, ref, gwc)
})
}
// AddSpaceActivity adds the activity to the given spaceroot
func (a *ActivitylogService) AddSpaceActivity(ctx context.Context, spaceID *provider.StorageSpaceId, eventID string, timestamp time.Time) error {
_, span := tracer.Start(ctx, "AddSpaceActivity")
defer span.End()
// spaceID is in format <providerid>$<spaceid>
// activitylog service uses format <providerid>$<spaceid>!<resourceid>
// lets do some converting, shall we?
rid, err := storagespace.ParseID(spaceID.GetOpaqueId())
if err != nil {
return fmt.Errorf("could not parse space id: %w", err)
}
rid.OpaqueId = rid.GetSpaceId()
err = a.al.StoreActivity(storagespace.FormatResourceID(&rid), []activitylog.RawActivity{
{
EventID: eventID,
Depth: 0,
Timestamp: timestamp,
},
})
if err != nil {
return fmt.Errorf("could not store activity: %w", err)
}
return nil
}
func toRef(r *provider.ResourceId) *provider.Reference {
return &provider.Reference{
ResourceId: r,
}
}
func (a *ActivitylogService) removeCachedParentID(ctx context.Context, ref *provider.Reference) {
var span trace.Span
ctx, span = tracer.Start(ctx, "removeCachedParentID")
defer span.End()
purgeId := ref.GetResourceId()
if ref.GetPath() != "" {
gwc, err := a.gws.Next()
if err != nil {
a.log.Error().Err(err).Msg("could not get gateway client")
return
}
ctx, err = utils.GetServiceUserContextWithContext(ctx, gwc, a.sa.ServiceAccountID, a.sa.ServiceAccountSecret)
if err != nil {
a.log.Error().Err(err).Msg("could not get service user context")
return
}
info, err := utils.GetResource(ctx, ref, gwc)
if err != nil {
a.log.Error().Err(err).Msg("could not get resource info")
return
}
purgeId = info.GetId()
}
a.al.InvalidateCachedParentID(purgeId)
}
@@ -0,0 +1,123 @@
package events_test
import (
"context"
gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/opencloud-eu/opencloud/pkg/log"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/config"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/service/activitylog"
eventssvc "github.com/opencloud-eu/opencloud/services/activitylog/pkg/service/events"
"github.com/opencloud-eu/reva/v2/pkg/events"
"github.com/opencloud-eu/reva/v2/pkg/rgrpc/todo/pool"
)
var _ = Describe("ActivitylogService", func() {
Describe("New", func() {
var (
al *activitylog.ActivityLog
stream events.Stream
)
BeforeEach(func() {
al = &activitylog.ActivityLog{}
stream = nil
})
It("creates a service with minimal options", func() {
svc, err := eventssvc.New(al, stream)
Expect(err).ToNot(HaveOccurred())
Expect(svc).ToNot(BeNil())
})
It("creates a service with context option", func() {
ctx := context.Background()
svc, err := eventssvc.New(al, stream, eventssvc.Context(ctx))
Expect(err).ToNot(HaveOccurred())
Expect(svc).ToNot(BeNil())
})
It("creates a service with logger option", func() {
logger := log.NopLogger()
svc, err := eventssvc.New(al, stream, eventssvc.Logger(logger))
Expect(err).ToNot(HaveOccurred())
Expect(svc).ToNot(BeNil())
})
It("creates a service with service account option", func() {
sa := config.ServiceAccount{
ServiceAccountID: "sa-id",
ServiceAccountSecret: "sa-secret",
}
svc, err := eventssvc.New(al, stream, eventssvc.ServiceAccount(sa))
Expect(err).ToNot(HaveOccurred())
Expect(svc).ToNot(BeNil())
})
It("creates a service with registered events", func() {
evts := []events.Unmarshaller{&events.UploadReady{}, &events.FileTouched{}}
svc, err := eventssvc.New(al, stream, eventssvc.RegisteredEvents(evts))
Expect(err).ToNot(HaveOccurred())
Expect(svc).ToNot(BeNil())
})
It("creates a service with gateway selector", func() {
var gs pool.Selectable[gateway.GatewayAPIClient]
svc, err := eventssvc.New(al, stream, eventssvc.GatewaySelector(gs))
Expect(err).ToNot(HaveOccurred())
Expect(svc).ToNot(BeNil())
})
It("creates a service with num consumers option", func() {
svc, err := eventssvc.New(al, stream, eventssvc.NumConsumers(5))
Expect(err).ToNot(HaveOccurred())
Expect(svc).ToNot(BeNil())
})
It("creates a service with all options", func() {
ctx := context.Background()
logger := log.NopLogger()
sa := config.ServiceAccount{
ServiceAccountID: "sa-id",
ServiceAccountSecret: "sa-secret",
}
evts := []events.Unmarshaller{&events.UploadReady{}, &events.ContainerCreated{}}
var gs pool.Selectable[gateway.GatewayAPIClient]
svc, err := eventssvc.New(
al,
stream,
eventssvc.Context(ctx),
eventssvc.Logger(logger),
eventssvc.ServiceAccount(sa),
eventssvc.RegisteredEvents(evts),
eventssvc.GatewaySelector(gs),
eventssvc.NumConsumers(3),
)
Expect(err).ToNot(HaveOccurred())
Expect(svc).ToNot(BeNil())
})
})
Describe("Close", func() {
It("can be called without panic on a new service", func() {
al := &activitylog.ActivityLog{}
svc, err := eventssvc.New(al, nil)
Expect(err).ToNot(HaveOccurred())
Expect(func() { svc.Close() }).ToNot(Panic())
})
It("can be called multiple times without panic", func() {
al := &activitylog.ActivityLog{}
svc, err := eventssvc.New(al, nil)
Expect(err).ToNot(HaveOccurred())
svc.Close()
svc.Close()
svc.Close()
})
})
})
@@ -0,0 +1,13 @@
package http_test
import (
"testing"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
func TestHTTP(t *testing.T) {
RegisterFailHandler(Fail)
RunSpecs(t, "HTTP Suite")
}
@@ -0,0 +1,59 @@
package http
import (
gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1"
"github.com/opencloud-eu/opencloud/pkg/log"
ehsvc "github.com/opencloud-eu/opencloud/protogen/gen/opencloud/services/eventhistory/v0"
"github.com/opencloud-eu/reva/v2/pkg/events"
"github.com/opencloud-eu/reva/v2/pkg/rgrpc/todo/pool"
)
// Option defines a single option function.
type Option func(o *Options)
// Options defines the available options for this package.
type Options struct {
Logger log.Logger
RegisteredEvents []events.Unmarshaller
GatewaySelector pool.Selectable[gateway.GatewayAPIClient]
HistoryClient ehsvc.EventHistoryService
}
// newOptions initializes the available default options.
func newOptions(opts ...Option) Options {
opt := Options{}
for _, o := range opts {
o(&opt)
}
return opt
}
// Logger provides a function to set the logger option.
func Logger(val log.Logger) Option {
return func(o *Options) {
o.Logger = val
}
}
// RegisteredEvents registers the events the service should listen to
func RegisteredEvents(e []events.Unmarshaller) Option {
return func(o *Options) {
o.RegisteredEvents = e
}
}
// GatewaySelector adds a grpc client selector for the gateway service
func GatewaySelector(gatewaySelector pool.Selectable[gateway.GatewayAPIClient]) Option {
return func(o *Options) {
o.GatewaySelector = gatewaySelector
}
}
// HistoryClient adds a grpc client for the eventhistory service
func HistoryClient(hc ehsvc.EventHistoryService) Option {
return func(o *Options) {
o.HistoryClient = hc
}
}
@@ -1,4 +1,4 @@
package service
package http
import (
"context"
@@ -44,11 +44,6 @@ var (
StrDescription = l10n.Template("description")
)
// GetActivitiesResponse is the response on GET activities requests
type GetActivitiesResponse struct {
Activities []libregraph.Activity `json:"value"`
}
// Resource represents an item such as a file or folder
type Resource struct {
ID string `json:"id"`
@@ -311,7 +306,7 @@ func NewActivity(message string, ts time.Time, eventID string, vars map[string]a
}
// GetVars calls other service to gather the required data for the activity variables
func (s *ActivitylogService) GetVars(ctx context.Context, opts ...ActivityOption) (map[string]any, error) {
func (s *svc) GetVars(ctx context.Context, opts ...ActivityOption) (map[string]any, error) {
gwc, err := s.gws.Next()
if err != nil {
return nil, err
@@ -327,6 +322,12 @@ func (s *ActivitylogService) GetVars(ctx context.Context, opts ...ActivityOption
return vars, nil
}
func toSpace(r *provider.Reference) *provider.StorageSpaceId {
return &provider.StorageSpaceId{
OpaqueId: storagespace.FormatStorageID(r.GetResourceId().GetStorageId(), r.GetResourceId().GetSpaceId()),
}
}
func getFolderName(ctx context.Context, gwc gateway.GatewayAPIClient, ref *provider.Reference) string {
n := filepath.Base(filepath.Dir(ref.GetPath()))
if n == "." || n == "/" {
@@ -0,0 +1,150 @@
package http_test
import (
"context"
"time"
gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1"
user "github.com/cs3org/go-cs3apis/cs3/identity/user/v1beta1"
provider "github.com/cs3org/go-cs3apis/cs3/storage/provider/v1beta1"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/opencloud-eu/opencloud/pkg/l10n"
httpsvc "github.com/opencloud-eu/opencloud/services/activitylog/pkg/service/http"
)
var _ = Describe("Response", func() {
Describe("NewActivity", func() {
It("creates an activity with the given parameters", func() {
ts := time.Date(2024, 1, 15, 10, 30, 0, 0, time.UTC)
vars := map[string]any{
"user": "testuser",
"resource": "testfile.txt",
}
act := httpsvc.NewActivity("Test message", ts, "event-123", vars)
Expect(act.Id).To(Equal("event-123"))
Expect(act.Times.RecordedTime).To(Equal(ts))
Expect(act.Template.Message).To(Equal("Test message"))
Expect(act.Template.Variables).To(HaveKeyWithValue("user", "testuser"))
Expect(act.Template.Variables).To(HaveKeyWithValue("resource", "testfile.txt"))
})
It("handles empty variables map", func() {
act := httpsvc.NewActivity("", time.Time{}, "", map[string]any{})
Expect(act.Id).To(BeEmpty())
Expect(act.Times.RecordedTime).To(Equal(time.Time{}))
Expect(act.Template.Message).To(BeEmpty())
Expect(act.Template.Variables).To(BeEmpty())
})
})
Describe("WithOldResource", func() {
It("sets the oldResource variable from reference path", func() {
ref := &provider.Reference{
Path: "/old/path/oldname.txt",
}
vars := make(map[string]any)
opt := httpsvc.WithOldResource(ref)
err := opt(context.Background(), nil, vars)
Expect(err).ToNot(HaveOccurred())
Expect(vars).To(HaveKey("oldResource"))
})
})
Describe("WithUser", func() {
It("returns error when no user is provided", func() {
opt := httpsvc.WithUser(nil, nil, nil)
err := opt(context.Background(), nil, make(map[string]any))
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("no user provided"))
})
It("uses impersonator when provided", func() {
impersonator := &user.User{
Id: &user.UserId{
OpaqueId: "imp-user-id",
},
DisplayName: "Impersonated User",
}
vars := make(map[string]any)
opt := httpsvc.WithUser(nil, nil, impersonator)
err := opt(context.Background(), nil, vars)
Expect(err).ToNot(HaveOccurred())
Expect(vars).To(HaveKey("user"))
})
It("uses executing user when no impersonator", func() {
execUser := &user.User{
Id: &user.UserId{
OpaqueId: "exec-user-id",
},
DisplayName: "Executing User",
}
vars := make(map[string]any)
opt := httpsvc.WithUser(nil, execUser, nil)
err := opt(context.Background(), nil, vars)
Expect(err).ToNot(HaveOccurred())
Expect(vars).To(HaveKey("user"))
})
})
Describe("WithVar", func() {
It("sets a simple key-value variable", func() {
vars := make(map[string]any)
opt := httpsvc.WithVar("token", "id123", "My Token")
err := opt(context.Background(), nil, vars)
Expect(err).ToNot(HaveOccurred())
Expect(vars).To(HaveKey("token"))
})
})
Describe("WithTranslation", func() {
It("sets translated field variable", func() {
var t l10n.Translator
vars := make(map[string]any)
opt := httpsvc.WithTranslation(&t, "en", "field", []string{"permission"})
err := opt(context.Background(), nil, vars)
Expect(err).ToNot(HaveOccurred())
Expect(vars).To(HaveKey("field"))
})
It("handles empty values slice", func() {
var t l10n.Translator
vars := make(map[string]any)
opt := httpsvc.WithTranslation(&t, "en", "field", []string{})
err := opt(context.Background(), nil, vars)
Expect(err).ToNot(HaveOccurred())
Expect(vars).To(HaveKey("field"))
})
})
Describe("ActivityOption type", func() {
It("allows composing multiple options", func() {
vars := make(map[string]any)
ctx := context.Background()
var gwc gateway.GatewayAPIClient
options := []httpsvc.ActivityOption{
httpsvc.WithVar("key1", "id1", "name1"),
httpsvc.WithVar("key2", "id2", "name2"),
}
for _, opt := range options {
err := opt(ctx, gwc, vars)
Expect(err).ToNot(HaveOccurred())
}
Expect(vars).To(HaveKey("key1"))
Expect(vars).To(HaveKey("key2"))
})
})
})
@@ -1,89 +1,97 @@
package service
package http
import (
"embed"
"encoding/json"
"errors"
"net/http"
"context"
"path/filepath"
"reflect"
"slices"
"strconv"
"strings"
"time"
gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1"
provider "github.com/cs3org/go-cs3apis/cs3/storage/provider/v1beta1"
revactx "github.com/opencloud-eu/reva/v2/pkg/ctx"
"github.com/opencloud-eu/reva/v2/pkg/events"
"github.com/opencloud-eu/reva/v2/pkg/storagespace"
"github.com/opencloud-eu/reva/v2/pkg/utils"
"google.golang.org/grpc/metadata"
"github.com/olekukonko/errors"
libregraph "github.com/opencloud-eu/libre-graph-api-go"
"github.com/opencloud-eu/opencloud/pkg/ast"
"github.com/opencloud-eu/opencloud/pkg/kql"
"github.com/opencloud-eu/opencloud/pkg/l10n"
"github.com/opencloud-eu/opencloud/pkg/log"
ehmsg "github.com/opencloud-eu/opencloud/protogen/gen/opencloud/messages/eventhistory/v0"
ehsvc "github.com/opencloud-eu/opencloud/protogen/gen/opencloud/services/eventhistory/v0"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/service/activitylog"
"github.com/opencloud-eu/reva/v2/pkg/events"
"github.com/opencloud-eu/reva/v2/pkg/rgrpc/todo/pool"
"github.com/opencloud-eu/reva/v2/pkg/storagespace"
"github.com/opencloud-eu/reva/v2/pkg/utils"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/trace"
)
var (
//go:embed l10n/locale
_localeFS embed.FS
var tracer trace.Tracer
// subfolder where the translation files are stored
_localeSubPath = "l10n/locale"
// domain of the activitylog service (transifex)
_domain = "activitylog"
)
// ServeHTTP implements the http.Handler interface.
func (s *ActivitylogService) ServeHTTP(w http.ResponseWriter, r *http.Request) {
s.mux.ServeHTTP(w, r)
func init() {
tracer = otel.Tracer("github.com/opencloud-eu/opencloud/services/activitylog/pkg/service/http")
}
// HandleGetItemActivities handles the request to get the activities of an item.
func (s *ActivitylogService) HandleGetItemActivities(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
ctx = metadata.AppendToOutgoingContext(ctx, revactx.TokenHeader, r.Header.Get(revactx.TokenHeader))
// New returns a new instance of Service
func New(al *activitylog.ActivityLog, opts ...Option) (*svc, error) {
o := newOptions(opts...)
activeUser, ok := revactx.ContextGetUser(ctx)
if !ok {
w.WriteHeader(http.StatusUnauthorized)
return
registeredEvents := make(map[string]events.Unmarshaller)
for _, e := range o.RegisteredEvents {
typ := reflect.TypeOf(e)
registeredEvents[typ.String()] = e
}
return &svc{
log: o.Logger,
evHistory: o.HistoryClient,
al: al,
registeredEvents: registeredEvents,
gws: o.GatewaySelector,
}, nil
}
type svc struct {
log log.Logger
evHistory ehsvc.EventHistoryService
gws pool.Selectable[gateway.GatewayAPIClient]
al *activitylog.ActivityLog
registeredEvents map[string]events.Unmarshaller
}
var (
ErrBadRequest = errors.New("bad request")
ErrForbidden = errors.New("forbidden")
)
func (s *svc) GetItemActivities(ctx context.Context, query, loc string, t l10n.Translator) ([]libregraph.Activity, error) {
gwc, err := s.gws.Next()
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
return nil, err
}
rid, limit, rawActivityAccepted, activityAccepted, sort, err := s.getFilters(r.URL.Query().Get("kql"))
rid, limit, rawActivityAccepted, activityAccepted, sort, err := s.getFilters(query)
if err != nil {
s.log.Info().Str("query", r.URL.Query().Get("kql")).Err(err).Msg("error getting filters")
_, _ = w.Write([]byte(err.Error()))
w.WriteHeader(http.StatusBadRequest)
return
s.log.Info().Str("query", query).Err(err).Msg("error getting filters")
return nil, ErrBadRequest
}
info, err := utils.GetResourceByID(ctx, rid, gwc)
if err != nil {
w.WriteHeader(http.StatusForbidden)
return
return nil, ErrForbidden
}
// you need ListGrants to see activities
if !info.GetPermissionSet().GetListGrants() {
w.WriteHeader(http.StatusForbidden)
return
return nil, ErrForbidden
}
raw, err := s.Activities(rid)
raw, err := s.al.Activities(rid)
if err != nil {
s.log.Error().Err(err).Msg("error getting activities")
w.WriteHeader(http.StatusInternalServerError)
return
return nil, err
}
ids := make([]string, 0, len(raw))
@@ -96,21 +104,21 @@ func (s *ActivitylogService) HandleGetItemActivities(w http.ResponseWriter, r *h
toDelete[a.EventID] = struct{}{}
}
evRes, err := s.evHistory.GetEvents(r.Context(), &ehsvc.GetEventsRequest{Ids: ids})
evRes, err := s.evHistory.GetEvents(ctx, &ehsvc.GetEventsRequest{Ids: ids})
if err != nil {
s.log.Error().Err(err).Msg("error getting events")
w.WriteHeader(http.StatusInternalServerError)
return
return nil, err
}
evs := evRes.GetEvents()
sort(evs)
resp := GetActivitiesResponse{Activities: make([]libregraph.Activity, 0, len(evRes.GetEvents()))}
// TODO cut the interface here?
activities := make([]libregraph.Activity, 0, len(evRes.GetEvents()))
for _, e := range evs {
delete(toDelete, e.GetId())
if limit > 0 && limit <= len(resp.Activities) {
if limit > 0 && limit <= len(activities) {
continue
}
@@ -124,9 +132,6 @@ func (s *ActivitylogService) HandleGetItemActivities(w http.ResponseWriter, r *h
vars map[string]any
)
loc := l10n.MustGetUserLocale(r.Context(), activeUser.GetId().GetOpaqueId(), r.Header.Get(l10n.HeaderAcceptLanguage), s.valService)
t := l10n.NewTranslatorFromCommonConfig(s.cfg.DefaultLanguage, _domain, s.cfg.TranslationPath, _localeFS, _localeSubPath)
switch ev := s.unwrapEvent(e).(type) {
case nil:
// error already logged in unwrapEvent
@@ -224,35 +229,29 @@ func (s *ActivitylogService) HandleGetItemActivities(w http.ResponseWriter, r *h
continue
}
resp.Activities = append(resp.Activities, NewActivity(t.Translate(message, loc), ts, e.GetId(), vars))
activities = append(activities, NewActivity(t.Translate(message, loc), ts, e.GetId(), vars))
}
// delete activities in separate go routine
if len(toDelete) > 0 {
go func() {
err := s.RemoveActivities(rid, toDelete)
err := s.al.RemoveActivities(rid, toDelete)
if err != nil {
s.log.Error().Err(err).Msg("error removing activities")
}
}()
}
return activities, nil
b, err := json.Marshal(resp)
if err != nil {
s.log.Error().Err(err).Msg("error marshalling activities")
w.WriteHeader(http.StatusInternalServerError)
return
}
if _, err := w.Write(b); err != nil {
s.log.Error().Err(err).Msg("error writing response")
w.WriteHeader(http.StatusInternalServerError)
return
}
w.WriteHeader(http.StatusOK)
}
func (s *ActivitylogService) unwrapEvent(e *ehmsg.Event) any {
func toRef(r *provider.ResourceId) *provider.Reference {
return &provider.Reference{
ResourceId: r,
}
}
func (s *svc) unwrapEvent(e *ehmsg.Event) any {
etype, ok := s.registeredEvents[e.GetType()]
if !ok {
s.log.Error().Str("eventid", e.GetId()).Str("eventtype", e.GetType()).Msg("event not registered")
@@ -268,13 +267,13 @@ func (s *ActivitylogService) unwrapEvent(e *ehmsg.Event) any {
return einterface
}
func (s *ActivitylogService) getFilters(query string) (*provider.ResourceId, int, func(RawActivity) bool, func(*ehmsg.Event) bool, func([]*ehmsg.Event), error) {
func (s *svc) getFilters(query string) (*provider.ResourceId, int, func(activitylog.RawActivity) bool, func(*ehmsg.Event) bool, func([]*ehmsg.Event), error) {
qast, err := kql.Builder{}.Build(query)
if err != nil {
return nil, 0, nil, nil, nil, err
}
prefilters := make([]func(RawActivity) bool, 0)
prefilters := make([]func(activitylog.RawActivity) bool, 0)
postfilters := make([]func(*ehmsg.Event) bool, 0)
sortby := func(_ []*ehmsg.Event) {}
@@ -299,7 +298,7 @@ func (s *ActivitylogService) getFilters(query string) (*provider.ResourceId, int
break
}
prefilters = append(prefilters, func(a RawActivity) bool {
prefilters = append(prefilters, func(a activitylog.RawActivity) bool {
return a.Depth <= depth
})
case "limit":
@@ -322,11 +321,11 @@ func (s *ActivitylogService) getFilters(query string) (*provider.ResourceId, int
case *ast.DateTimeNode:
switch v.Operator.Value {
case "<", "<=":
prefilters = append(prefilters, func(a RawActivity) bool {
prefilters = append(prefilters, func(a activitylog.RawActivity) bool {
return a.Timestamp.Before(v.Value)
})
case ">", ">=":
prefilters = append(prefilters, func(a RawActivity) bool {
prefilters = append(prefilters, func(a activitylog.RawActivity) bool {
return a.Timestamp.After(v.Value)
})
}
@@ -345,7 +344,7 @@ func (s *ActivitylogService) getFilters(query string) (*provider.ResourceId, int
// space root requested - fix format
rid.OpaqueId = rid.GetSpaceId()
}
pref := func(a RawActivity) bool {
pref := func(a activitylog.RawActivity) bool {
for _, f := range prefilters {
if !f(a) {
return false
@@ -0,0 +1,76 @@
package http_test
import (
gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/opencloud-eu/opencloud/pkg/log"
ehsvc "github.com/opencloud-eu/opencloud/protogen/gen/opencloud/services/eventhistory/v0"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/service/activitylog"
httpsvc "github.com/opencloud-eu/opencloud/services/activitylog/pkg/service/http"
"github.com/opencloud-eu/reva/v2/pkg/events"
"github.com/opencloud-eu/reva/v2/pkg/rgrpc/todo/pool"
)
var _ = Describe("Service", func() {
Describe("New", func() {
var (
al *activitylog.ActivityLog
)
BeforeEach(func() {
al = &activitylog.ActivityLog{}
})
It("creates a service with default options", func() {
svc, err := httpsvc.New(al)
Expect(err).ToNot(HaveOccurred())
Expect(svc).ToNot(BeNil())
})
It("creates a service with logger option", func() {
logger := log.NopLogger()
svc, err := httpsvc.New(al, httpsvc.Logger(logger))
Expect(err).ToNot(HaveOccurred())
Expect(svc).ToNot(BeNil())
})
It("creates a service with registered events", func() {
evts := []events.Unmarshaller{&events.UploadReady{}}
svc, err := httpsvc.New(al, httpsvc.RegisteredEvents(evts))
Expect(err).ToNot(HaveOccurred())
Expect(svc).ToNot(BeNil())
})
It("creates a service with gateway selector", func() {
var gs pool.Selectable[gateway.GatewayAPIClient]
svc, err := httpsvc.New(al, httpsvc.GatewaySelector(gs))
Expect(err).ToNot(HaveOccurred())
Expect(svc).ToNot(BeNil())
})
It("creates a service with history client", func() {
var hc ehsvc.EventHistoryService
svc, err := httpsvc.New(al, httpsvc.HistoryClient(hc))
Expect(err).ToNot(HaveOccurred())
Expect(svc).ToNot(BeNil())
})
It("creates a service with all options", func() {
logger := log.NopLogger()
evts := []events.Unmarshaller{&events.UploadReady{}, &events.FileTouched{}}
var gs pool.Selectable[gateway.GatewayAPIClient]
var hc ehsvc.EventHistoryService
svc, err := httpsvc.New(
al,
httpsvc.Logger(logger),
httpsvc.RegisteredEvents(evts),
httpsvc.GatewaySelector(gs),
httpsvc.HistoryClient(hc),
)
Expect(err).ToNot(HaveOccurred())
Expect(svc).ToNot(BeNil())
})
})
})
-676
View File
@@ -1,676 +0,0 @@
package service
import (
"context"
"crypto/tls"
"encoding/base32"
"encoding/json"
"fmt"
"path/filepath"
"reflect"
"sort"
"strconv"
"strings"
"sync"
"time"
gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1"
provider "github.com/cs3org/go-cs3apis/cs3/storage/provider/v1beta1"
"github.com/go-chi/chi/v5"
"github.com/jellydator/ttlcache/v2"
"github.com/nats-io/nats.go"
"github.com/opencloud-eu/reva/v2/pkg/events"
"github.com/opencloud-eu/reva/v2/pkg/rgrpc/todo/pool"
"github.com/opencloud-eu/reva/v2/pkg/storagespace"
"github.com/opencloud-eu/reva/v2/pkg/utils"
"github.com/pkg/errors"
"github.com/vmihailenco/msgpack/v5"
"go.opentelemetry.io/otel/trace"
"github.com/opencloud-eu/opencloud/pkg/log"
ehsvc "github.com/opencloud-eu/opencloud/protogen/gen/opencloud/services/eventhistory/v0"
settingssvc "github.com/opencloud-eu/opencloud/protogen/gen/opencloud/services/settings/v0"
"github.com/opencloud-eu/opencloud/services/activitylog/pkg/config"
)
// Nats runs into max payload exceeded errors at around 7k activities. Let's keep a buffer.
var _maxActivitiesDefault = 6000
// RawActivity represents an activity as it is stored in the activitylog store
type RawActivity struct {
EventID string `json:"event_id"`
Depth int `json:"depth"`
Timestamp time.Time `json:"timestamp"`
}
// ActivitylogService logs events per resource
type ActivitylogService struct {
cfg *config.Config
log log.Logger
events <-chan events.Event
gws pool.Selectable[gateway.GatewayAPIClient]
mux *chi.Mux
evHistory ehsvc.EventHistoryService
valService settingssvc.ValueService
lock sync.RWMutex
tp trace.TracerProvider
tracer trace.Tracer
debouncer *Debouncer
parentIdCache *ttlcache.Cache
natskv nats.KeyValue
maxActivities int
registeredEvents map[string]events.Unmarshaller
}
type Debouncer struct {
after time.Duration
f func(id string, ra []RawActivity) error
pending sync.Map
inProgress sync.Map
mutex sync.Mutex
}
type queueItem struct {
activities []RawActivity
timer *time.Timer
}
type batchInfo struct {
key string
count int
timestamp time.Time
}
// NewDebouncer returns a new Debouncer instance
func NewDebouncer(d time.Duration, f func(id string, ra []RawActivity) error) *Debouncer {
return &Debouncer{
after: d,
f: f,
pending: sync.Map{},
inProgress: sync.Map{},
}
}
// Debounce restarts the debounce timer for the given space
func (d *Debouncer) Debounce(id string, ra RawActivity) {
if d.after == 0 {
d.f(id, []RawActivity{ra})
return
}
d.mutex.Lock()
defer d.mutex.Unlock()
activities := []RawActivity{ra}
item := &queueItem{
activities: activities,
}
if i, ok := d.pending.Load(id); ok {
// if the item is already in the queue, append the new activities
item, ok = i.(*queueItem)
if ok {
item.activities = append(item.activities, ra)
}
}
if item.timer == nil {
item.timer = time.AfterFunc(d.after, func() {
if _, ok := d.inProgress.Load(id); ok {
// Reschedule this run for when the previous run has finished
d.mutex.Lock()
if i, ok := d.pending.Load(id); ok {
i.(*queueItem).timer.Reset(d.after)
}
d.mutex.Unlock()
return
}
d.pending.Delete(id)
d.inProgress.Store(id, true)
defer d.inProgress.Delete(id)
d.f(id, item.activities)
})
}
d.pending.Store(id, item)
}
// New creates a new ActivitylogService
func New(opts ...Option) (*ActivitylogService, error) {
o := &Options{
MaxActivities: _maxActivitiesDefault,
}
for _, opt := range opts {
opt(o)
}
if o.Stream == nil {
return nil, errors.New("stream is required")
}
ch, err := events.Consume(o.Stream, o.Config.Service.Name, o.RegisteredEvents...)
if err != nil {
return nil, err
}
cache := ttlcache.NewCache()
err = cache.SetTTL(30 * time.Second)
if err != nil {
return nil, err
}
// Connect to NATS servers
natsOptions := nats.Options{
Servers: o.Config.Store.Nodes,
}
if o.Config.Store.EnableTLS {
if o.Config.Store.TLSRootCACertificate != "" {
// when root ca is configured use it. an insecure flag is ignored.
nats.RootCAs(o.Config.Store.TLSRootCACertificate)(&natsOptions)
} else {
// enable tls and use insecure flag
nats.Secure(&tls.Config{MinVersion: tls.VersionTLS12, InsecureSkipVerify: o.Config.Store.TLSInsecure})(&natsOptions)
}
}
if o.Config.Store.AuthUsername != "" && o.Config.Store.AuthPassword != "" {
nats.UserInfo(o.Config.Store.AuthUsername, o.Config.Store.AuthPassword)(&natsOptions)
}
conn, err := natsOptions.Connect()
if err != nil {
return nil, err
}
js, err := conn.JetStream()
if err != nil {
return nil, err
}
kv, err := js.KeyValue(o.Config.Store.Database)
if err != nil {
if !errors.Is(err, nats.ErrBucketNotFound) {
return nil, errors.Wrapf(err, "Failed to get bucket (%s)", o.Config.Store.Database)
}
kv, err = js.CreateKeyValue(&nats.KeyValueConfig{
Bucket: o.Config.Store.Database,
})
if err != nil {
return nil, errors.Wrapf(err, "Failed to create bucket (%s)", o.Config.Store.Database)
}
}
if err != nil {
return nil, err
}
s := &ActivitylogService{
log: o.Logger,
cfg: o.Config,
events: ch,
gws: o.GatewaySelector,
mux: o.Mux,
evHistory: o.HistoryClient,
valService: o.ValueClient,
lock: sync.RWMutex{},
registeredEvents: make(map[string]events.Unmarshaller),
tp: o.TraceProvider,
tracer: o.TraceProvider.Tracer("github.com/opencloud-eu/opencloud/services/activitylog/pkg/service"),
parentIdCache: cache,
maxActivities: o.Config.MaxActivities,
natskv: kv,
}
s.debouncer = NewDebouncer(o.Config.WriteBufferDuration, s.storeActivity)
// run migrations
err = s.runMigrations(context.Background(), kv)
if err != nil {
return nil, err
}
s.mux.Get("/graph/v1beta1/extensions/org.libregraph/activities", s.HandleGetItemActivities)
for _, e := range o.RegisteredEvents {
typ := reflect.TypeOf(e)
s.registeredEvents[typ.String()] = e
}
go s.Run()
return s, nil
}
// Run runs the service
func (a *ActivitylogService) Run() {
for e := range a.events {
var err error
switch ev := e.Event.(type) {
case events.UploadReady:
err = a.AddActivity(ev.FileRef, ev.ParentID, e.ID, utils.TSToTime(ev.Timestamp))
case events.FileTouched:
err = a.AddActivity(ev.Ref, ev.ParentID, e.ID, utils.TSToTime(ev.Timestamp))
// Disabled https://github.com/owncloud/ocis/issues/10293
//case events.FileDownloaded:
// we are only interested in public link downloads - so no need to store others.
//if ev.ImpersonatingUser.GetDisplayName() == "Public" {
// err = a.AddActivity(ev.Ref, e.ID, utils.TSToTime(ev.Timestamp))
//}
case events.ContainerCreated:
err = a.AddActivity(ev.Ref, ev.ParentID, e.ID, utils.TSToTime(ev.Timestamp))
case events.ItemTrashed:
err = a.AddActivityTrashed(ev.ID, ev.Ref, nil, e.ID, utils.TSToTime(ev.Timestamp))
case events.ItemPurged:
err = a.RemoveResource(ev.ID)
case events.ItemMoved:
// remove the cached parent id for this resource
a.removeCachedParentID(ev.Ref)
err = a.AddActivity(ev.Ref, nil, e.ID, utils.TSToTime(ev.Timestamp))
case events.ShareCreated:
err = a.AddActivity(toRef(ev.ItemID), nil, e.ID, utils.TSToTime(ev.CTime))
case events.ShareUpdated:
if ev.Sharer != nil && ev.ItemID != nil && ev.Sharer.GetOpaqueId() != ev.ItemID.GetSpaceId() {
err = a.AddActivity(toRef(ev.ItemID), nil, e.ID, utils.TSToTime(ev.MTime))
}
case events.ShareRemoved:
err = a.AddActivity(toRef(ev.ItemID), nil, e.ID, ev.Timestamp)
case events.LinkCreated:
err = a.AddActivity(toRef(ev.ItemID), nil, e.ID, utils.TSToTime(ev.CTime))
case events.LinkUpdated:
if ev.Sharer != nil && ev.ItemID != nil && ev.Sharer.GetOpaqueId() != ev.ItemID.GetSpaceId() {
err = a.AddActivity(toRef(ev.ItemID), nil, e.ID, utils.TSToTime(ev.MTime))
}
case events.LinkRemoved:
err = a.AddActivity(toRef(ev.ItemID), nil, e.ID, utils.TSToTime(ev.Timestamp))
case events.SpaceShared:
err = a.AddSpaceActivity(ev.ID, e.ID, ev.Timestamp)
case events.SpaceUnshared:
err = a.AddSpaceActivity(ev.ID, e.ID, ev.Timestamp)
}
if err != nil {
a.log.Error().Err(err).Interface("event", e).Msg("could not process event")
}
}
}
// AddActivity adds the activity to the given resource and all its parents
func (a *ActivitylogService) AddActivity(initRef *provider.Reference, parentId *provider.ResourceId, eventID string, timestamp time.Time) error {
gwc, err := a.gws.Next()
if err != nil {
return fmt.Errorf("cant get gateway client: %w", err)
}
ctx, err := utils.GetServiceUserContext(a.cfg.ServiceAccount.ServiceAccountID, gwc, a.cfg.ServiceAccount.ServiceAccountSecret)
if err != nil {
return fmt.Errorf("cant get service user context: %w", err)
}
var span trace.Span
ctx, span = a.tracer.Start(ctx, "AddActivity")
defer span.End()
return a.addActivity(ctx, initRef, parentId, eventID, timestamp, func(ctx context.Context, ref *provider.Reference) (*provider.ResourceInfo, error) {
return utils.GetResource(ctx, ref, gwc)
})
}
// AddActivityTrashed adds the activity to given trashed resource and all its former parents
func (a *ActivitylogService) AddActivityTrashed(resourceID *provider.ResourceId, reference *provider.Reference, parentId *provider.ResourceId, eventID string, timestamp time.Time) error {
gwc, err := a.gws.Next()
if err != nil {
return fmt.Errorf("cant get gateway client: %w", err)
}
ctx, err := utils.GetServiceUserContext(a.cfg.ServiceAccount.ServiceAccountID, gwc, a.cfg.ServiceAccount.ServiceAccountSecret)
if err != nil {
return fmt.Errorf("cant get service user context: %w", err)
}
// store activity on trashed item
if err := a.storeActivity(storagespace.FormatResourceID(resourceID), []RawActivity{
{
EventID: eventID,
Depth: 0,
Timestamp: timestamp,
},
}); err != nil {
return fmt.Errorf("could not store activity: %w", err)
}
// get previous parent
ref := &provider.Reference{
ResourceId: reference.GetResourceId(),
Path: filepath.Dir(reference.GetPath()),
}
var span trace.Span
ctx, span = a.tracer.Start(ctx, "AddActivityTrashed")
defer span.End()
return a.addActivity(ctx, ref, parentId, eventID, timestamp, func(ctx context.Context, ref *provider.Reference) (*provider.ResourceInfo, error) {
return utils.GetResource(ctx, ref, gwc)
})
}
// AddSpaceActivity adds the activity to the given spaceroot
func (a *ActivitylogService) AddSpaceActivity(spaceID *provider.StorageSpaceId, eventID string, timestamp time.Time) error {
// spaceID is in format <providerid>$<spaceid>
// activitylog service uses format <providerid>$<spaceid>!<resourceid>
// lets do some converting, shall we?
rid, err := storagespace.ParseID(spaceID.GetOpaqueId())
if err != nil {
return fmt.Errorf("could not parse space id: %w", err)
}
rid.OpaqueId = rid.GetSpaceId()
return a.storeActivity(storagespace.FormatResourceID(&rid), []RawActivity{
{
EventID: eventID,
Depth: 0,
Timestamp: timestamp,
},
})
}
// Activities returns the activities for the given resource
func (a *ActivitylogService) Activities(rid *provider.ResourceId) ([]RawActivity, error) {
a.lock.RLock()
defer a.lock.RUnlock()
return a.activities(rid)
}
// RemoveActivities removes the activities from the given resource
func (a *ActivitylogService) RemoveActivities(rid *provider.ResourceId, toDelete map[string]struct{}) error {
a.lock.Lock()
defer a.lock.Unlock()
curActivities, err := a.activities(rid)
if err != nil {
return err
}
var acts []RawActivity
for _, a := range curActivities {
if _, ok := toDelete[a.EventID]; !ok {
acts = append(acts, a)
}
}
b, err := json.Marshal(acts)
if err != nil {
return err
}
_, err = a.natskv.Put(storagespace.FormatResourceID(rid), b)
return err
}
// RemoveResource removes the resource from the store
func (a *ActivitylogService) RemoveResource(rid *provider.ResourceId) error {
if rid == nil {
return fmt.Errorf("resource id is required")
}
a.lock.Lock()
defer a.lock.Unlock()
return a.natskv.Delete(storagespace.FormatResourceID(rid))
}
func (a *ActivitylogService) activities(rid *provider.ResourceId) ([]RawActivity, error) {
resourceID := storagespace.FormatResourceID(rid)
glob := fmt.Sprintf("%s.>", base32.StdEncoding.EncodeToString([]byte(resourceID)))
watcher, err := a.natskv.Watch(glob, nats.IgnoreDeletes())
if err != nil {
return nil, err
}
defer watcher.Stop()
var activities []RawActivity
for update := range watcher.Updates() {
if update == nil {
break
}
var batchActivities []RawActivity
if err := msgpack.Unmarshal(update.Value(), &batchActivities); err != nil {
a.log.Debug().Err(err).Str("resourceID", resourceID).Msg("could not unmarshal messagepack, trying json")
}
activities = append(activities, batchActivities...)
}
return activities, nil
}
// note: getResource is abstracted to allow unit testing, in general this will just be utils.GetResource
func (a *ActivitylogService) addActivity(ctx context.Context, initRef *provider.Reference, parentId *provider.ResourceId, eventID string, timestamp time.Time, getResource func(context.Context, *provider.Reference) (*provider.ResourceInfo, error)) error {
var (
err error
depth int
ref = initRef
)
ctx, span := a.tracer.Start(ctx, "addActivity")
defer span.End()
for {
var info *provider.ResourceInfo
id := ref.GetResourceId()
if ref.Path != "" {
// Path based reference, we need to resolve the resource id
ctx, span = a.tracer.Start(ctx, "addActivity.getResource")
info, err = getResource(ctx, ref)
span.End()
if err != nil {
return fmt.Errorf("could not get resource info: %w", err)
}
id = info.GetId()
}
if id == nil {
return fmt.Errorf("resource id is required")
}
key := storagespace.FormatResourceID(id)
a.debouncer.Debounce(key, RawActivity{
EventID: eventID,
Depth: depth,
Timestamp: timestamp,
})
if id.OpaqueId == id.SpaceId {
// we are at the root of the space, no need to go further
break
}
// check if parent id is cached
// parent id is cached in the format <storageid>$<spaceid>!<resourceid>
// if it is not cached, get the resource info and cache it
if parentId == nil {
if v, err := a.parentIdCache.Get(key); err != nil {
if info == nil {
ctx, span := a.tracer.Start(ctx, "addActivity.getResource parent")
info, err = getResource(ctx, ref)
span.End()
if err != nil || info.GetParentId() == nil || info.GetParentId().GetOpaqueId() == "" {
return fmt.Errorf("could not get parent id: %w", err)
}
}
parentId = info.GetParentId()
a.parentIdCache.Set(key, parentId)
} else {
parentId = v.(*provider.ResourceId)
}
} else {
a.log.Debug().Msg("parent id is cached")
}
depth++
ref = &provider.Reference{ResourceId: parentId}
parentId = nil // reset parent id so it's not reused in the next iteration
}
return nil
}
func (a *ActivitylogService) storeActivity(resourceID string, activities []RawActivity) error {
a.lock.Lock()
defer a.lock.Unlock()
ctx, span := a.tracer.Start(context.Background(), "storeActivity")
defer span.End()
_, subspan := a.tracer.Start(ctx, "storeActivity.Marshal")
b, err := msgpack.Marshal(activities)
if err != nil {
return err
}
subspan.End()
_, subspan = a.tracer.Start(ctx, "storeActivity.natskv.Put")
key := natsKey(resourceID, len(activities))
_, err = a.natskv.Put(key, b)
if err != nil {
return err
}
subspan.End()
ctx, subspan = a.tracer.Start(ctx, "storeActivity.enforceMaxActivities")
a.enforceMaxActivities(ctx, resourceID)
subspan.End()
return nil
}
func (a *ActivitylogService) enforceMaxActivities(ctx context.Context, resourceID string) {
if a.maxActivities <= 0 {
return
}
key := fmt.Sprintf("%s.>", base32.StdEncoding.EncodeToString([]byte(resourceID)))
_, subspan := a.tracer.Start(ctx, "enforceMaxActivities.watch")
watcher, err := a.natskv.Watch(key, nats.IgnoreDeletes())
if err != nil {
a.log.Error().Err(err).Str("resourceID", resourceID).Msg("could not watch")
return
}
defer watcher.Stop()
var keys []string
for update := range watcher.Updates() {
if update == nil {
break
}
var batchActivities []RawActivity
if err := msgpack.Unmarshal(update.Value(), &batchActivities); err != nil {
a.log.Debug().Err(err).Str("resourceID", resourceID).Msg("could not unmarshal messagepack, trying json")
}
keys = append(keys, update.Key())
}
subspan.End()
_, subspan = a.tracer.Start(ctx, "enforceMaxActivities.compile")
// Parse keys into batches
batches := make([]batchInfo, 0)
var activitiesCount int
for _, k := range keys {
parts := strings.SplitN(k, ".", 3)
if len(parts) < 3 {
a.log.Warn().Str("key", k).Msg("skipping key, not enough parts")
continue
}
c, err := strconv.Atoi(parts[1])
if err != nil {
a.log.Warn().Str("key", k).Msg("skipping key, can not parse count")
continue
}
// parse timestamp
nano, err := strconv.ParseInt(parts[2], 10, 64)
if err != nil {
a.log.Warn().Str("key", k).Msg("skipping key, can not parse timestamp")
continue
}
batches = append(batches, batchInfo{
key: k,
count: c,
timestamp: time.Unix(0, nano),
})
activitiesCount += c
}
// sort batches by timestamp
sort.Slice(batches, func(i, j int) bool {
return batches[i].timestamp.Before(batches[j].timestamp)
})
subspan.End()
_, subspan = a.tracer.Start(ctx, "enforceMaxActivities.delete")
// remove oldest keys until we are at max activities
for _, b := range batches {
if activitiesCount-b.count < a.maxActivities {
break
}
activitiesCount -= b.count
err = a.natskv.Delete(b.key)
if err != nil {
a.log.Error().Err(err).Str("key", b.key).Msg("could not delete key")
break
}
}
subspan.End()
}
func toRef(r *provider.ResourceId) *provider.Reference {
return &provider.Reference{
ResourceId: r,
}
}
func toSpace(r *provider.Reference) *provider.StorageSpaceId {
return &provider.StorageSpaceId{
OpaqueId: storagespace.FormatStorageID(r.GetResourceId().GetStorageId(), r.GetResourceId().GetSpaceId()),
}
}
func (a *ActivitylogService) removeCachedParentID(ref *provider.Reference) {
purgeId := ref.GetResourceId()
if ref.GetPath() != "" {
gwc, err := a.gws.Next()
if err != nil {
a.log.Error().Err(err).Msg("could not get gateway client")
return
}
ctx, err := utils.GetServiceUserContext(a.cfg.ServiceAccount.ServiceAccountID, gwc, a.cfg.ServiceAccount.ServiceAccountSecret)
if err != nil {
a.log.Error().Err(err).Msg("could not get service user context")
return
}
info, err := utils.GetResource(ctx, ref, gwc)
if err != nil {
a.log.Error().Err(err).Msg("could not get resource info")
return
}
purgeId = info.GetId()
}
// The parent id cache is populated lazily and its entries expire, so a
// missing key is the expected case rather than an error.
if err := a.parentIdCache.Remove(storagespace.FormatResourceID(purgeId)); err != nil {
a.log.Debug().Interface("event", ref).Err(err).Msg("could not delete parent id cache")
}
}
func natsKey(resourceID string, activitiesCount int) string {
return fmt.Sprintf("%s.%d.%d",
base32.StdEncoding.EncodeToString([]byte(resourceID)),
activitiesCount,
time.Now().UnixNano())
}
-4
View File
@@ -63,10 +63,6 @@ Store specific notes:
- When using `nats-js-kv` it is recommended to set `OC_CACHE_STORE_NODES` to the same value as `OC_EVENTS_ENDPOINT`. That way the cache uses the same nats instance as the event bus.
- When using the `nats-js-kv` store, it is possible to set `OC_CACHE_DISABLE_PERSISTENCE` to instruct nats to not persist cache data on disc.
### Auto-Accept Shares
When setting the `SHARING_AUTO_ACCEPT_SHARES` to `true` (sharing service), all incoming shares will be accepted automatically. Users can overwrite this setting individually in their profile. The deprecated `FRONTEND_AUTO_ACCEPT_SHARES` is still supported for backwards compatibility.
## Passwords
### The Password Policy
+3 -3
View File
@@ -61,7 +61,7 @@ type Spaces struct {
}
type LDAPMetrics struct {
Disabled bool `yaml:"disabled" env:"GRAPH_LDAP_METRICS_DISABLE" desc:"Disables the metrics for outbound LDAP operations." introductionVersion:"%NEXT%"`
Disabled bool `yaml:"disabled" env:"GRAPH_LDAP_METRICS_DISABLE" desc:"Disables the metrics for outbound LDAP operations." introductionVersion:"%%NEXT%%"`
}
type LDAP struct {
@@ -121,7 +121,7 @@ type LDAPEducationConfig struct {
}
type IdentityMetrics struct {
Disabled bool `yaml:"disabled" env:"GRAPH_IDENTITY_BACKEND_METRICS_DISABLE" desc:"Disables the metrics for inbound identity backend operations." introductionVersion:"%NEXT%"`
Disabled bool `yaml:"disabled" env:"GRAPH_IDENTITY_BACKEND_METRICS_DISABLE" desc:"Disables the metrics for inbound identity backend operations." introductionVersion:"%%NEXT%%"`
}
type Identity struct {
@@ -141,7 +141,7 @@ type API struct {
// Events combines the configuration options for the event bus.
type Events struct {
DisabledConsumer bool `yaml:"disabled_consumer" env:"GRAPH_EVENTS_DISABLE_CONSUMER" desc:"Disables consuming events. Set this to true if the service should only handle HTTP requests." introductionVersion:"%NEXT%"`
DisabledConsumer bool `yaml:"disabled_consumer" env:"GRAPH_EVENTS_DISABLE_CONSUMER" desc:"Disables consuming events. Set this to true if the service should only handle HTTP requests." introductionVersion:"%%NEXT%%"`
Endpoint string `yaml:"endpoint" env:"OC_EVENTS_ENDPOINT;GRAPH_EVENTS_ENDPOINT" desc:"The address of the event system. The event system is the message queuing service. It is used as message broker for the microservice architecture. Set to a empty string to disable emitting events." introductionVersion:"1.0.0"`
Cluster string `yaml:"cluster" env:"OC_EVENTS_CLUSTER;GRAPH_EVENTS_CLUSTER" desc:"The clusterID of the event system. The event system is the message queuing service. It is used as message broker for the microservice architecture." introductionVersion:"1.0.0"`
TLSInsecure bool `yaml:"tls_insecure" env:"OC_INSECURE;OC_EVENTS_TLS_INSECURE;GRAPH_EVENTS_TLS_INSECURE" desc:"Whether to verify the server TLS certificates." introductionVersion:"1.0.0"`
+2 -2
View File
@@ -3,12 +3,12 @@ package config
import "github.com/opencloud-eu/opencloud/pkg/shared"
type HTTPMetrics struct {
Disabled bool `yaml:"disabled" env:"GRAPH_HTTP_METRICS_DISABLE" desc:"Disables the metrics for the HTTP service." introductionVersion:"%NEXT%"`
Disabled bool `yaml:"disabled" env:"GRAPH_HTTP_METRICS_DISABLE" desc:"Disables the metrics for the HTTP service." introductionVersion:"%%NEXT%%"`
}
// HTTP defines the available http configuration.
type HTTP struct {
Disabled bool `yaml:"disabled" env:"GRAPH_HTTP_DISABLE" desc:"Disables the HTTP service. Set this to true if the service should only consume events." introductionVersion:"%NEXT%"`
Disabled bool `yaml:"disabled" env:"GRAPH_HTTP_DISABLE" desc:"Disables the HTTP service. Set this to true if the service should only consume events." introductionVersion:"%%NEXT%%"`
Addr string `yaml:"addr" env:"GRAPH_HTTP_ADDR" desc:"The bind address of the HTTP service." introductionVersion:"1.0.0"`
Namespace string `yaml:"-"`
Root string `yaml:"root" env:"GRAPH_HTTP_ROOT" desc:"Subdirectory that serves as the root for this HTTP service." introductionVersion:"1.0.0"`
+1 -5
View File
@@ -40,11 +40,7 @@ func ParseConfig(cfg *config.Config) error {
func Validate(cfg *config.Config) error {
if cfg.HTTP.Disabled && cfg.Events.DisabledConsumer {
// might be debatable, but this situation should be treated as an error,
// as the process wouldn't be able to serve either API and would thus be
// completely useless -- in that case, just don't start this service
// in the first place (especially since it's optional)
return shared.AllComponentsDisabledError("graph")
return shared.AllComponentsDisabledError(cfg.Service.Name)
}
if cfg.TokenManager.JWTSecret == "" {
@@ -0,0 +1,157 @@
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
# Translators:
# Roy-Magne Mo, 2026
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: EMAIL\n"
"POT-Creation-Date: 2026-09-07 23:16+0000\n"
"PO-Revision-Date: 2025-01-27 10:17+0000\n"
"Last-Translator: Roy-Magne Mo, 2026\n"
"Language-Team: Norwegian (https://app.transifex.com/opencloud-eu/teams/204053/no/)\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Language: no\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
#. UnifiedRole Editor, Role DisplayName (resolves directly)
#. UnifiedRole Editor, Role DisplayName (resolves directly)
#. UnifiedRole EditorListGrants, Role DisplayName (resolves directly)
#. UnifiedRole SpaseEditor, Role DisplayName (resolves directly)
#. UnifiedRole FileEditor, Role DisplayName (resolves directly)
#. UnifiedRole FileEditorWithVersions, Role DisplayName (resolves directly)
#. UnifiedRole FileEditorListGrants, Role DisplayName (resolves directly)
#: pkg/unifiedrole/roles.go:137 pkg/unifiedrole/roles.go:143
#: pkg/unifiedrole/roles.go:149 pkg/unifiedrole/roles.go:155
#: pkg/unifiedrole/roles.go:167 pkg/unifiedrole/roles.go:173
#: pkg/unifiedrole/roles.go:179
msgid "Can edit"
msgstr "Kan redigere"
#. UnifiedRole SpaseEditorWithoutVersions, Role DisplayName (resolves
#. directly)
#: pkg/unifiedrole/roles.go:161
msgid "Can edit without versions"
msgstr "Kan redigere uten versjoner"
#. UnifiedRole Manager, Role DisplayName (resolves directly)
#: pkg/unifiedrole/roles.go:191
msgid "Can manage"
msgstr "Kan håndtere"
#. UnifiedRole EditorLite, Role DisplayName (resolves directly)
#: pkg/unifiedrole/roles.go:185
msgid "Can upload"
msgstr "Kan laste opp"
#. UnifiedRole Viewer, Role DisplayName (resolves directly)
#. UnifiedRole ViewerWithVersions, Role DisplayName (resolves directly)
#. UnifiedRole Viewer, Role DisplayName (resolves directly)
#. UnifiedRole SpaseViewer, Role DisplayName (resolves directly)
#. UnifiedRole SpaseViewer, Role DisplayName (resolves directly)
#: pkg/unifiedrole/roles.go:107 pkg/unifiedrole/roles.go:113
#: pkg/unifiedrole/roles.go:119 pkg/unifiedrole/roles.go:125
#: pkg/unifiedrole/roles.go:131
msgid "Can view"
msgstr "Kan se"
#. UnifiedRole SecureViewer, Role DisplayName (resolves directly)
#: pkg/unifiedrole/roles.go:197
msgid "Can view (secure)"
msgstr "Kan vise (sikker)"
#. UnifiedRole FullDenial, Role DisplayName (resolves directly)
#: pkg/unifiedrole/roles.go:203
msgid "Cannot access"
msgstr "Kan ikke få tilgang"
#. UnifiedRole FullDenial, Role Description (resolves directly)
#: pkg/unifiedrole/roles.go:200
msgid "Deny all access."
msgstr "Avvis all tilgang."
#. default description for new spaces
#: pkg/service/v0/spacetemplates.go:32
msgid "Here you can add a description for this Space."
msgstr "Her kan du legge til en beskrivelse for dette rommet."
#. UnifiedRole ViewerWithVersions, Role Description (resolves directly)
#. UnifiedRole SpaceViewer, Role Description (resolves directly)
#: pkg/unifiedrole/roles.go:110 pkg/unifiedrole/roles.go:128
msgid "View and download including the history."
msgstr "Vis og last ned, inkludert historikken."
#. UnifiedRole Viewer, Role Description (resolves directly)
#. UnifiedRole SpaceViewer, Role Description (resolves directly)
#: pkg/unifiedrole/roles.go:104 pkg/unifiedrole/roles.go:122
msgid "View and download."
msgstr "Vis og last ned."
#. UnifiedRole SecureViewer, Role Description (resolves directly)
#: pkg/unifiedrole/roles.go:194
msgid "View only documents, images and PDFs. Watermarks will be applied."
msgstr "Vis kun dokumenter, bilder og PDF-er. Vannmerker vil bli brukt."
#. UnifiedRole FileEditorWithVErsions, Role Description (resolves directly)
#: pkg/unifiedrole/roles.go:176
msgid "View, download and edit including the history."
msgstr "Vis, last ned og rediger, inkludert historikken."
#. UnifiedRole FileEditor, Role Description (resolves directly)
#: pkg/unifiedrole/roles.go:164
msgid "View, download and edit."
msgstr "Vis, last ned og rediger."
#. UnifiedRole ViewerListGrants, Role Description (resolves directly)
#: pkg/unifiedrole/roles.go:116
msgid "View, download and show all invited people."
msgstr "Vis, last ned og vis alle inviterte personer."
#. UnifiedRole EditorLite, Role Description (resolves directly)
#: pkg/unifiedrole/roles.go:182
msgid "View, download and upload."
msgstr "Vis, last ned og last opp."
#. UnifiedRole FileEditorListGrants, Role Description (resolves directly)
#: pkg/unifiedrole/roles.go:170
msgid "View, download, edit and show all invited people."
msgstr "Vis, last ned, rediger og vis alle inviterte personer."
#. UnifiedRole Editor, Role Description (resolves directly)
#: pkg/unifiedrole/roles.go:140
msgid "View, download, upload, edit, add and delete including the history."
msgstr ""
"Vis, last ned, last opp, rediger, legg til og slett, inkludert historikken."
#. UnifiedRole Editor, Role Description (resolves directly)
#. UnifiedRole SpaseEditorWithoutVersions, Role Description (resolves
#. directly)
#: pkg/unifiedrole/roles.go:134 pkg/unifiedrole/roles.go:158
msgid "View, download, upload, edit, add and delete."
msgstr "Vis, last ned, last opp, rediger, legg til og slett."
#. UnifiedRole Manager, Role Description (resolves directly)
#: pkg/unifiedrole/roles.go:188
msgid "View, download, upload, edit, add, delete and manage members."
msgstr ""
"Vis, last ned, last opp, rediger, legg til, slett og administrer medlemmer."
#. UnifiedRoleListGrants Editor, Role Description (resolves directly)
#: pkg/unifiedrole/roles.go:146
msgid "View, download, upload, edit, add, delete and show all invited people."
msgstr ""
"Se, last ned, last opp, rediger, legg til, slett og vis alle inviterte "
"personer."
#. UnifiedRole SpaseEditor, Role Description (resolves directly)
#: pkg/unifiedrole/roles.go:152
msgid "View, download, upload, edit, add, delete including the history."
msgstr ""
"Vis, last ned, last opp, rediger, legg til, slett inkludert historikken."
+20 -63
View File
@@ -27,9 +27,8 @@ import (
// and exposes the remainder via chi.RouteContext().RoutePath. parseColonPath
// therefore only needs to handle the part below the drive:
//
// root-anchored: /root:/<path>[:/<suffix>][:]
// item-anchored: /items/{itemID}:/<path>[:/<suffix>][:]
// special-anchored: /special/{specialName}:/<path>[:/<suffix>][:]
// root-anchored: /root:/<path>[:/<suffix>][:]
// item-anchored: /items/{itemID}:/<path>[:/<suffix>][:]
type contextKey string
@@ -37,24 +36,6 @@ type contextKey string
// tracing/logging consumers.
const OriginalPathContextKey contextKey = "graph.original_path"
// specialPathContextKey holds the decoded path below a special folder for the
// special-anchored colon form. Special folders are not part of the item tree,
// so the path cannot be resolved to an item id here; the handler interprets it.
const specialPathContextKey contextKey = "graph.special_path"
// SpecialFolderPath returns the path below the special folder for a request
// that arrived in the special-anchored colon form, e.g. "/key/sub" for
// /special/recyclebin:/key/sub:/children. ok is false for plain requests.
func SpecialFolderPath(ctx context.Context) (string, bool) {
p, ok := ctx.Value(specialPathContextKey).(string)
return p, ok
}
// WithSpecialFolderPath stores the decoded path below a special folder, see SpecialFolderPath.
func WithSpecialFolderPath(ctx context.Context, p string) context.Context {
return context.WithValue(ctx, specialPathContextKey, p)
}
// Sentinels distinguishing the resolution outcomes that map to specific HTTP
// statuses. Anything else surfaces as 500.
//
@@ -83,17 +64,12 @@ var (
// descended into a sub-router, routeHTTP matches against rctx.RoutePath and
// ignores r.URL.Path.)
//
// Three URL shapes are recognized:
// Two URL shapes are recognized:
//
// /drives/{driveID}/root:/<path>[:/<suffix>][:]
// /drives/{driveID}/items/{itemID}:/<path>[:/<suffix>][:]
// /drives/{driveID}/special/{specialName}:/<path>[:/<suffix>][:]
//
// The special-anchored form is rewritten to /special/{specialName}{suffix}
// without a lookup; the decoded path travels in the context (see
// SpecialFolderPath) because special folders live outside the item tree.
//
// Path resolution for the other two runs as the request user via CS3 Stat. NOT_FOUND and
// Path resolution runs as the request user via CS3 Stat. NOT_FOUND and
// PERMISSION_DENIED collapse to 404 (no existence disclosure); operational
// failures (gateway selection, RPC transport, unexpected status) surface
// as 5xx so outages aren't masked.
@@ -113,30 +89,9 @@ func ResolveGraphPath(gws pool.Selectable[gateway.GatewayAPIClient], logger log.
return
}
match, ok := parseColonPath(rctx.RoutePath)
if !ok {
// No colon-syntax match - pass through untouched.
next.ServeHTTP(w, r)
return
}
driveID := chi.URLParam(r, "driveID")
original := r.URL.Path
var rewritten string
var err error
if match.specialName != "" {
var specialPath string
specialPath, err = url.PathUnescape(match.relPath)
if err != nil {
l.Debug().Err(err).Str("relPath", match.relPath).Msg("undecodable path in special colon path")
err = errInvalidRequest
} else {
r = r.WithContext(WithSpecialFolderPath(r.Context(), specialPath))
rewritten = "/special/" + match.specialName + match.suffix
}
} else {
rewritten, err = rewriteColonPath(r.Context(), gws, l, driveID, match)
}
rewritten, err := rewriteColonPath(r.Context(), gws, l, driveID, rctx.RoutePath)
switch {
case errors.Is(err, errPathNotFound):
l.Debug().Str("original", original).Msg("colon-path resolution: not found")
@@ -156,6 +111,10 @@ func ResolveGraphPath(gws pool.Selectable[gateway.GatewayAPIClient], logger log.
w, r, http.StatusInternalServerError, "internal error resolving path",
)
return
case rewritten == "":
// No colon-syntax match - pass through untouched.
next.ServeHTTP(w, r)
return
}
l.Debug().
@@ -182,26 +141,32 @@ func ResolveGraphPath(gws pool.Selectable[gateway.GatewayAPIClient], logger log.
type colonMatch struct {
isItemAnchored bool // item-anchored form: anchor is itemAnchorID, validate against driveID
itemAnchorID string // itemID from the path for the item-anchored form (empty for root-anchored)
specialName string // special-anchored form: the special folder name (empty otherwise)
relPath string // relative path with leading slash
suffix string // suffix with leading slash (e.g. "/children"); may be empty
}
// rewriteColonPath resolves a root- or item-anchored match and returns:
// - rewritten + nil - resolved to a canonical RoutePath
// rewriteColonPath returns:
// - "" + nil - no colon-syntax pattern matched (passthrough)
// - rewritten + nil - matched and resolved to a canonical RoutePath
// - "" + errPathNotFound - path doesn't exist or user lacks permission (404)
// - "" + errInvalidRequest - malformed input (400)
// - "" + errUnauthenticated - gateway said caller isn't authenticated (401)
// - "" + other error - operational / internal failure (5xx)
//
// driveIDParam is the {driveID} route param (raw chi.URLParam value).
// driveIDParam is the {driveID} route param (raw chi.URLParam value); routePath
// is chi.RouteContext().RoutePath (the part below /drives/{driveID}).
func rewriteColonPath(
ctx context.Context,
gws pool.Selectable[gateway.GatewayAPIClient],
logger zerolog.Logger,
driveIDParam string,
match colonMatch,
routePath string,
) (string, error) {
match, ok := parseColonPath(routePath)
if !ok {
return "", nil
}
// RoutePath follows chi's RawPath, i.e. the percent-encoded wire form
// (e.g. "/Documents/My%20File"). A single PathUnescape reproduces exactly
// what net/http put in r.URL.Path; it is NOT a double-decode (a crafted
@@ -268,7 +233,6 @@ func rewriteColonPath(
//
// /root:/<path>[:/<suffix>][:]
// /items/<itemID>:/<path>[:/<suffix>][:]
// /special/<specialName>:/<path>[:/<suffix>][:]
//
// The structural delimiter is ":/" (a colon immediately followed by the
// leading slash of the path or suffix); a trailing ":" is the no-suffix
@@ -315,13 +279,6 @@ func parseColonPath(routePath string) (colonMatch, bool) {
}
m.isItemAnchored = true
m.itemAnchorID = itemID
case strings.HasPrefix(anchor, "/special/"):
// Special-anchored: /special/{specialName} with a single-segment name.
name := strings.TrimPrefix(anchor, "/special/")
if name == "" || strings.Contains(name, "/") {
return m, false
}
m.specialName = name
default:
return m, false
}
@@ -66,10 +66,6 @@ type leafCapture struct {
driveID string // chi.URLParam(driveID)
itemID string // resolved item id, decoded via PathUnescape
original any // OriginalPathContextKey value
specialName string // chi.URLParam(specialName) for the special leaves
specialPath string // middleware.SpecialFolderPath value
specialPathSet bool // whether SpecialFolderPath reported ok
}
// newGraphTestRouter wires ResolveGraphPath into a chi router that mirrors the
@@ -98,8 +94,6 @@ func newGraphTestRouter(t *testing.T, gw *cs3mocks.GatewayAPIClient) (http.Handl
// mirror that here so we assert on the recovered id.
cap.itemID, _ = url.PathUnescape(raw)
cap.original = r.Context().Value(middleware.OriginalPathContextKey)
cap.specialName = chi.URLParam(r, "specialName")
cap.specialPath, cap.specialPathSet = middleware.SpecialFolderPath(r.Context())
w.WriteHeader(http.StatusOK)
}
}
@@ -134,10 +128,6 @@ func newGraphTestRouter(t *testing.T, gw *cs3mocks.GatewayAPIClient) (http.Handl
r.Get("/", leaf("item"))
r.Get("/children", leaf("children"))
})
r.Route("/special/{specialName}", func(r chi.Router) {
r.Get("/", leaf("special"))
r.Get("/children", leaf("specialChildren"))
})
})
})
return m, cap
@@ -479,96 +469,3 @@ func TestResolveGraphPath_OriginalPathContext(t *testing.T) {
assert.Equal(t, original, cap.original, "original URL must be available via OriginalPathContextKey")
assert.Equal(t, original, cap.urlPath, "r.URL.Path must remain the original request path")
}
// TestResolveGraphPath_SpecialFolder pins the special-anchored colon form:
// no CS3 lookup (special folders live outside the item tree), the request is
// re-routed to /special/{specialName}{suffix} and the decoded path below the
// special folder reaches the handler through SpecialFolderPath.
func TestResolveGraphPath_SpecialFolder(t *testing.T) {
tests := []struct {
name string
urlPath string
expectStatus int
expectHit string
expectPath string
expectPathSet bool
expectSpecName string
}{
{
name: "plain special route passes through without a path",
urlPath: "/graph/v1.0/drives/" + testDriveID + "/special/recyclebin/children",
expectStatus: http.StatusOK,
expectHit: "specialChildren",
expectPathSet: false,
expectSpecName: "recyclebin",
},
{
name: "special-anchored with /children rewrites and carries the path",
urlPath: "/graph/v1.0/drives/" + testDriveID + "/special/recyclebin:/key/sub:/children",
expectStatus: http.StatusOK,
expectHit: "specialChildren",
expectPath: "/key/sub",
expectPathSet: true,
expectSpecName: "recyclebin",
},
{
name: "special-anchored without suffix rewrites to the bare special URL",
urlPath: "/graph/v1.0/drives/" + testDriveID + "/special/recyclebin:/key",
expectStatus: http.StatusOK,
expectHit: "special",
expectPath: "/key",
expectPathSet: true,
expectSpecName: "recyclebin",
},
{
name: "special-anchored with trailing colon rewrites to the bare special URL",
urlPath: "/graph/v1.0/drives/" + testDriveID + "/special/recyclebin:/key:",
expectStatus: http.StatusOK,
expectHit: "special",
expectPath: "/key",
expectPathSet: true,
expectSpecName: "recyclebin",
},
{
name: "percent-encoded path is decoded once",
urlPath: "/graph/v1.0/drives/" + testDriveID + "/special/recyclebin:/key/My%20File:/children",
expectStatus: http.StatusOK,
expectHit: "specialChildren",
expectPath: "/key/My File",
expectPathSet: true,
expectSpecName: "recyclebin",
},
{
// The name is a single segment; a slash inside means this is not the
// colon form and chi decides (here: no such route).
name: "multi-segment special name is not colon syntax",
urlPath: "/graph/v1.0/drives/" + testDriveID + "/special/recyclebin/x:/key:/children",
expectStatus: http.StatusNotFound,
expectHit: "",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
gw := &cs3mocks.GatewayAPIClient{}
router, cap := newGraphTestRouter(t, gw)
req := httptest.NewRequest(http.MethodGet, "http://localhost"+tt.urlPath, nil)
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
assert.Equal(t, tt.expectStatus, rr.Code, "status code")
assert.Equal(t, tt.expectHit, cap.hit, "leaf handler reached")
gw.AssertNotCalled(t, "Stat", mock.Anything, mock.Anything)
if tt.expectHit != "" {
assert.Equal(t, testDriveID, cap.driveID, "driveID param")
assert.Equal(t, tt.expectSpecName, cap.specialName, "specialName param")
assert.Equal(t, tt.expectPathSet, cap.specialPathSet, "SpecialFolderPath ok")
assert.Equal(t, tt.expectPath, cap.specialPath, "SpecialFolderPath value")
// r.URL.Path is the decoded form; only chi's RoutePath is rewritten.
decoded, _ := url.PathUnescape(tt.urlPath)
assert.Equal(t, decoded, cap.urlPath, "r.URL.Path must remain the original request path")
}
})
}
}
+11 -1
View File
@@ -45,7 +45,10 @@ const (
var shareTypesFieldMask = &fieldmaskpb.FieldMask{Paths: []string{"share-types"}}
// opt-in driveItem relations, returned only when requested via $expand
const _expandChildren = "children"
const (
_expandChildren = "children"
_expandThumbnails = "thumbnails"
)
func odataListContains(r *http.Request, parameter, value string) bool {
for _, values := range r.URL.Query()[parameter] {
@@ -253,6 +256,7 @@ func (g Graph) GetRootDriveChildren(w http.ResponseWriter, r *http.Request) {
errorcode.GeneralException.Render(w, r, http.StatusInternalServerError, err.Error())
return
}
g.setDriveItemsThumbnails(r, files, lRes.GetInfos())
if driveItemPropertySelected(r, _selectAllowedValues) {
for i, info := range lRes.GetInfos() {
@@ -346,6 +350,10 @@ func (g Graph) GetDriveItem(w http.ResponseWriter, r *http.Request) {
driveItem.LibreGraphShareTypes = shareTypesOf(res.GetInfo(), g.listLinkShares(ctx, infos))
}
if driveItemRelationExpanded(r, _expandThumbnails) {
setDriveItemThumbnails(driveItem, res.GetInfo(), g.config.Commons.OpenCloudURL)
}
render.Status(r, http.StatusOK)
render.JSON(w, r, &driveItem)
}
@@ -432,6 +440,8 @@ func (g Graph) listDriveItemChildren(w http.ResponseWriter, r *http.Request, dri
g.addShareTypes(r.Context(), files, res.GetInfos())
}
g.setDriveItemsThumbnails(r, files, res.GetInfos())
return files, true
}
@@ -1,595 +0,0 @@
package svc_test
import (
"context"
"encoding/json"
"errors"
"net/http"
"net/http/httptest"
"strings"
"time"
gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1"
userpb "github.com/cs3org/go-cs3apis/cs3/identity/user/v1beta1"
cs3rpc "github.com/cs3org/go-cs3apis/cs3/rpc/v1beta1"
provider "github.com/cs3org/go-cs3apis/cs3/storage/provider/v1beta1"
"github.com/go-chi/chi/v5"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
libregraph "github.com/opencloud-eu/libre-graph-api-go"
"github.com/prometheus/client_golang/prometheus"
"github.com/stretchr/testify/mock"
"google.golang.org/grpc"
revactx "github.com/opencloud-eu/reva/v2/pkg/ctx"
"github.com/opencloud-eu/reva/v2/pkg/rgrpc/status"
"github.com/opencloud-eu/reva/v2/pkg/rgrpc/todo/pool"
"github.com/opencloud-eu/reva/v2/pkg/utils"
cs3mocks "github.com/opencloud-eu/reva/v2/tests/cs3mocks/mocks"
"github.com/opencloud-eu/opencloud/pkg/log"
"github.com/opencloud-eu/opencloud/pkg/shared"
"github.com/opencloud-eu/opencloud/services/graph/mocks"
"github.com/opencloud-eu/opencloud/services/graph/pkg/config"
"github.com/opencloud-eu/opencloud/services/graph/pkg/config/defaults"
identitymocks "github.com/opencloud-eu/opencloud/services/graph/pkg/identity/mocks"
"github.com/opencloud-eu/opencloud/services/graph/pkg/metrics"
graphm "github.com/opencloud-eu/opencloud/services/graph/pkg/middleware"
service "github.com/opencloud-eu/opencloud/services/graph/pkg/service/v0"
)
var _ = Describe("Drive recycle bin", func() {
var (
svc service.Service
ctx context.Context
cfg *config.Config
gatewayClient *cs3mocks.GatewayAPIClient
gatewaySelector pool.Selectable[gateway.GatewayAPIClient]
eventsPublisher mocks.Publisher
identityBackend *identitymocks.Backend
rr *httptest.ResponseRecorder
currentUser = &userpb.User{Id: &userpb.UserId{OpaqueId: "user"}}
deletedAt = time.Date(2026, 9, 7, 12, 0, 0, 0, time.UTC)
)
type params map[string]string
specialParams := params{"driveID": "storageid$spaceid", "specialName": "recyclebin"}
itemParams := params{"driveID": "storageid$spaceid", "driveItemID": "storageid$spaceid!nodeid"}
// newRequest binds the given chi params; query goes into the URL, body is sent as JSON
newRequest := func(method string, p params, query, body string) *http.Request {
r := httptest.NewRequest(method, "/graph/v1.0/drives/storageid$spaceid"+query, strings.NewReader(body))
rctx := chi.NewRouteContext()
for k, v := range p {
rctx.URLParams.Add(k, v)
}
return r.WithContext(context.WithValue(revactx.ContextSetUser(ctx, currentUser), chi.RouteCtxKey, rctx))
}
// withSpecialPath mimics what the colon-path middleware puts into the context
withSpecialPath := func(r *http.Request, specialPath string) *http.Request {
return r.WithContext(graphm.WithSpecialFolderPath(r.Context(), specialPath))
}
decodeItem := func() libregraph.DriveItem {
var item libregraph.DriveItem
Expect(json.Unmarshal(rr.Body.Bytes(), &item)).To(Succeed())
return item
}
decodeList := func() []libregraph.DriveItem {
var res struct {
Value []libregraph.DriveItem
}
Expect(json.Unmarshal(rr.Body.Bytes(), &res)).To(Succeed())
return res.Value
}
trashedFile := &provider.RecycleItem{
Type: provider.ResourceType_RESOURCE_TYPE_FILE,
Key: "nodeid",
Size: 42,
DeletionTime: utils.TimeToTS(deletedAt),
Ref: &provider.Reference{Path: "/Documents/notes.txt"},
}
// mockListRecycle answers every ListRecycle with the items and records the requested keys
mockListRecycle := func(items ...*provider.RecycleItem) *[]string {
var keys []string
gatewayClient.On("ListRecycle", mock.Anything, mock.Anything).Run(func(args mock.Arguments) {
keys = append(keys, args.Get(1).(*provider.ListRecycleRequest).GetKey())
}).Return(&provider.ListRecycleResponse{
Status: status.NewOK(ctx),
RecycleItems: items,
}, nil)
return &keys
}
mockRestore := func(st *cs3rpc.Status) **provider.RestoreRecycleItemRequest {
var req *provider.RestoreRecycleItemRequest
gatewayClient.On("RestoreRecycleItem", mock.Anything, mock.Anything).Run(func(args mock.Arguments) {
req = args.Get(1).(*provider.RestoreRecycleItemRequest)
}).Return(&provider.RestoreRecycleItemResponse{Status: st}, nil)
return &req
}
mockStatOK := func() {
gatewayClient.On("Stat", mock.Anything, mock.Anything).Return(&provider.StatResponse{
Status: status.NewOK(ctx),
Info: &provider.ResourceInfo{
Type: provider.ResourceType_RESOURCE_TYPE_FILE,
Id: &provider.ResourceId{StorageId: "storageid", SpaceId: "spaceid", OpaqueId: "nodeid"},
Path: "./Documents/notes.txt",
},
}, nil)
}
mockPurge := func(st *cs3rpc.Status) **provider.PurgeRecycleRequest {
var req *provider.PurgeRecycleRequest
gatewayClient.On("PurgeRecycle", mock.Anything, mock.Anything).Run(func(args mock.Arguments) {
req = args.Get(1).(*provider.PurgeRecycleRequest)
}).Return(&provider.PurgeRecycleResponse{Status: st}, nil)
return &req
}
BeforeEach(func() {
eventsPublisher.On("Publish", mock.Anything, mock.Anything, mock.Anything).Return(nil)
pool.RemoveSelector("GatewaySelector" + "eu.opencloud.api.gateway")
gatewayClient = &cs3mocks.GatewayAPIClient{}
gatewaySelector = pool.GetSelector[gateway.GatewayAPIClient](
"GatewaySelector",
"eu.opencloud.api.gateway",
func(cc grpc.ClientConnInterface) gateway.GatewayAPIClient {
return gatewayClient
},
)
logger := log.NewLogger()
identityBackend = &identitymocks.Backend{}
metrics, _ := metrics.New(prometheus.NewRegistry(), &logger, func([]string) (string, string) { return "", "" })
rr = httptest.NewRecorder()
ctx = context.Background()
cfg = defaults.FullDefaultConfig()
cfg.Identity.LDAP.CACert = ""
cfg.TokenManager.JWTSecret = "loremipsum"
cfg.Commons = &shared.Commons{}
cfg.GRPCClientTLS = &shared.GRPCClientTLS{}
var err error
svc, err = service.NewService(
service.Config(cfg),
service.Metrics(metrics),
service.WithGatewaySelector(gatewaySelector),
service.EventsPublisher(&eventsPublisher),
service.WithIdentityBackend(identityBackend),
)
Expect(err).ToNot(HaveOccurred())
})
Describe("GetDriveSpecial", func() {
It("rejects an unknown special folder", func() {
svc.GetDriveSpecial(rr, newRequest(http.MethodGet, params{"driveID": "storageid$spaceid", "specialName": "nope"}, "", ""))
Expect(rr.Code).To(Equal(http.StatusBadRequest))
gatewayClient.AssertNotCalled(GinkgoT(), "ListRecycle", mock.Anything, mock.Anything)
})
It("returns the synthetic recycle bin folder without touching the storage", func() {
svc.GetDriveSpecial(rr, newRequest(http.MethodGet, specialParams, "", ""))
Expect(rr.Code).To(Equal(http.StatusOK))
gatewayClient.AssertNotCalled(GinkgoT(), "ListRecycle", mock.Anything, mock.Anything)
item := decodeItem()
Expect(item.GetId()).To(Equal("storageid$spaceid!recyclebin"))
Expect(item.GetName()).To(Equal("recyclebin"))
Expect(item.Folder).ToNot(BeNil())
Expect(item.SpecialFolder.GetName()).To(Equal("recyclebin"))
Expect(item.ParentReference.GetDriveId()).To(Equal("storageid$spaceid"))
Expect(item.ParentReference.GetId()).To(Equal("storageid$spaceid!spaceid"))
})
It("returns a top-level trashed item by key", func() {
var req *provider.ListRecycleRequest
gatewayClient.On("ListRecycle", mock.Anything, mock.Anything).Run(func(args mock.Arguments) {
req = args.Get(1).(*provider.ListRecycleRequest)
}).Return(&provider.ListRecycleResponse{
Status: status.NewOK(ctx),
RecycleItems: []*provider.RecycleItem{trashedFile},
}, nil)
svc.GetDriveSpecial(rr, withSpecialPath(newRequest(http.MethodGet, specialParams, "", ""), "/nodeid"))
Expect(rr.Code).To(Equal(http.StatusOK))
Expect(req.GetKey()).To(Equal("nodeid"))
Expect(req.GetRef().GetResourceId().GetStorageId()).To(Equal("storageid"))
Expect(req.GetRef().GetResourceId().GetSpaceId()).To(Equal("spaceid"))
Expect(req.GetRef().GetResourceId().GetOpaqueId()).To(Equal("spaceid"))
item := decodeItem()
Expect(item.GetId()).To(Equal("storageid$spaceid!nodeid"))
Expect(item.GetName()).To(Equal("notes.txt"))
Expect(item.GetSize()).To(Equal(int64(42)))
Expect(item.File.GetMimeType()).To(Equal("text/plain; charset=utf-8"))
Expect(item.Folder).To(BeNil())
Expect(item.Trash.GetTrashedDateTime()).To(BeTemporally("==", deletedAt))
Expect(item.Trash.TrashedBy).To(BeNil())
Expect(item.ParentReference.GetDriveId()).To(Equal("storageid$spaceid"))
Expect(item.ParentReference.GetPath()).To(Equal("/Documents"))
Expect(item.ParentReference.GetName()).To(Equal("Documents"))
})
It("returns a nested trashed item through its parent listing", func() {
keys := mockListRecycle(
&provider.RecycleItem{Type: provider.ResourceType_RESOURCE_TYPE_FILE, Key: "nodeid/sub/other.txt", Ref: &provider.Reference{Path: "/folder/sub/other.txt"}},
&provider.RecycleItem{Type: provider.ResourceType_RESOURCE_TYPE_CONTAINER, Key: "nodeid/sub/inner", Ref: &provider.Reference{Path: "/folder/sub/inner"}},
)
svc.GetDriveSpecial(rr, withSpecialPath(newRequest(http.MethodGet, specialParams, "", ""), "/nodeid/sub/inner"))
Expect(rr.Code).To(Equal(http.StatusOK))
Expect(*keys).To(Equal([]string{"nodeid/sub/"}))
item := decodeItem()
Expect(item.GetId()).To(Equal("storageid$spaceid!nodeid/sub/inner"))
Expect(item.GetName()).To(Equal("inner"))
Expect(item.Folder).ToNot(BeNil())
Expect(item.File).To(BeNil())
})
It("embeds the trash listing with $expand=children", func() {
keys := mockListRecycle(trashedFile)
svc.GetDriveSpecial(rr, newRequest(http.MethodGet, specialParams, "?$expand=children", ""))
Expect(rr.Code).To(Equal(http.StatusOK))
Expect(*keys).To(Equal([]string{""}))
item := decodeItem()
Expect(item.GetId()).To(Equal("storageid$spaceid!recyclebin"))
Expect(item.Children).To(HaveLen(1))
Expect(item.Children[0].GetId()).To(Equal("storageid$spaceid!nodeid"))
Expect(item.Children[0].Trash).ToNot(BeNil())
})
It("embeds the children of a trashed folder with $expand=children", func() {
var keys []string
record := func(args mock.Arguments) {
keys = append(keys, args.Get(1).(*provider.ListRecycleRequest).GetKey())
}
gatewayClient.On("ListRecycle", mock.Anything, mock.Anything).Run(record).Return(&provider.ListRecycleResponse{
Status: status.NewOK(ctx),
RecycleItems: []*provider.RecycleItem{{Type: provider.ResourceType_RESOURCE_TYPE_CONTAINER, Key: "nodeid", Ref: &provider.Reference{Path: "/folder"}}},
}, nil).Once()
gatewayClient.On("ListRecycle", mock.Anything, mock.Anything).Run(record).Return(&provider.ListRecycleResponse{
Status: status.NewOK(ctx),
RecycleItems: []*provider.RecycleItem{{Type: provider.ResourceType_RESOURCE_TYPE_FILE, Key: "nodeid/a.txt", Ref: &provider.Reference{Path: "/folder/a.txt"}}},
}, nil).Once()
svc.GetDriveSpecial(rr, withSpecialPath(newRequest(http.MethodGet, specialParams, "?$expand=children", ""), "/nodeid"))
Expect(rr.Code).To(Equal(http.StatusOK))
Expect(keys).To(Equal([]string{"nodeid", "nodeid/"}))
item := decodeItem()
Expect(item.GetId()).To(Equal("storageid$spaceid!nodeid"))
Expect(item.Children).To(HaveLen(1))
Expect(item.Children[0].GetId()).To(Equal("storageid$spaceid!nodeid/a.txt"))
})
It("does not expand children of a trashed file", func() {
mockListRecycle(trashedFile)
svc.GetDriveSpecial(rr, withSpecialPath(newRequest(http.MethodGet, specialParams, "?$expand=children", ""), "/nodeid"))
Expect(rr.Code).To(Equal(http.StatusOK))
gatewayClient.AssertNumberOfCalls(GinkgoT(), "ListRecycle", 1)
Expect(decodeItem().Children).To(BeNil())
})
It("returns not found when the key is not in the listing", func() {
mockListRecycle(&provider.RecycleItem{Key: "other"})
svc.GetDriveSpecial(rr, withSpecialPath(newRequest(http.MethodGet, specialParams, "", ""), "/nodeid"))
Expect(rr.Code).To(Equal(http.StatusNotFound))
})
})
Describe("ListDriveSpecialChildren", func() {
It("rejects an unknown special folder", func() {
svc.ListDriveSpecialChildren(rr, newRequest(http.MethodGet, params{"driveID": "storageid$spaceid", "specialName": "nope"}, "", ""))
Expect(rr.Code).To(Equal(http.StatusBadRequest))
})
DescribeTable("maps ListRecycle errors",
func(st *cs3rpc.Status, code int) {
gatewayClient.On("ListRecycle", mock.Anything, mock.Anything).Return(&provider.ListRecycleResponse{Status: st}, nil)
svc.ListDriveSpecialChildren(rr, newRequest(http.MethodGet, specialParams, "", ""))
Expect(rr.Code).To(Equal(code))
},
Entry("not found", status.NewNotFound(context.Background(), "not found"), http.StatusNotFound),
Entry("permission denied as not found", status.NewPermissionDenied(context.Background(), errors.New("denied"), "denied"), http.StatusNotFound),
Entry("unauthenticated", status.NewUnauthenticated(context.Background(), errors.New("no"), "no"), http.StatusUnauthorized),
Entry("internal", status.NewInternal(context.Background(), "internal"), http.StatusInternalServerError),
)
It("lists the trash root with the trash facet", func() {
keys := mockListRecycle(
&provider.RecycleItem{
Type: provider.ResourceType_RESOURCE_TYPE_FILE,
Key: "file-node",
Size: 7,
DeletionTime: utils.TimeToTS(deletedAt),
Ref: &provider.Reference{Path: "/photo.jpg"},
},
&provider.RecycleItem{
Type: provider.ResourceType_RESOURCE_TYPE_CONTAINER,
Key: "folder-node",
Size: 100,
DeletionTime: utils.TimeToTS(deletedAt),
Ref: &provider.Reference{Path: "/Projects/old"},
},
)
svc.ListDriveSpecialChildren(rr, newRequest(http.MethodGet, specialParams, "", ""))
Expect(rr.Code).To(Equal(http.StatusOK))
Expect(*keys).To(Equal([]string{""}))
items := decodeList()
Expect(items).To(HaveLen(2))
Expect(items[0].GetId()).To(Equal("storageid$spaceid!file-node"))
Expect(items[0].GetName()).To(Equal("photo.jpg"))
Expect(items[0].File.GetMimeType()).To(Equal("image/jpeg"))
Expect(items[0].Trash.GetTrashedDateTime()).To(BeTemporally("==", deletedAt))
Expect(items[0].ParentReference.GetPath()).To(Equal("/"))
Expect(items[0].ParentReference.Name).To(BeNil())
Expect(items[1].GetId()).To(Equal("storageid$spaceid!folder-node"))
Expect(items[1].GetName()).To(Equal("old"))
Expect(items[1].Folder).ToNot(BeNil())
Expect(items[1].GetSize()).To(Equal(int64(100)))
Expect(items[1].ParentReference.GetPath()).To(Equal("/Projects"))
Expect(items[1].ParentReference.GetName()).To(Equal("Projects"))
})
It("returns an empty list for an empty trash", func() {
mockListRecycle()
svc.ListDriveSpecialChildren(rr, newRequest(http.MethodGet, specialParams, "", ""))
Expect(rr.Code).To(Equal(http.StatusOK))
Expect(rr.Body.String()).To(MatchJSON(`{"value":[]}`))
})
It("lists inside a trashed folder with a trailing slash key", func() {
keys := mockListRecycle(
&provider.RecycleItem{Type: provider.ResourceType_RESOURCE_TYPE_FILE, Key: "folder-node/sub/a.txt", Ref: &provider.Reference{Path: "/Projects/old/sub/a.txt"}},
)
svc.ListDriveSpecialChildren(rr, withSpecialPath(newRequest(http.MethodGet, specialParams, "", ""), "/folder-node/sub"))
Expect(rr.Code).To(Equal(http.StatusOK))
Expect(*keys).To(Equal([]string{"folder-node/sub/"}))
items := decodeList()
Expect(items).To(HaveLen(1))
Expect(items[0].GetId()).To(Equal("storageid$spaceid!folder-node/sub/a.txt"))
Expect(items[0].ParentReference.GetPath()).To(Equal("/Projects/old/sub"))
})
})
Describe("RestoreDriveItem", func() {
It("rejects an item from another drive", func() {
svc.RestoreDriveItem(rr, newRequest(http.MethodPost, params{"driveID": "storageid$other", "driveItemID": "storageid$spaceid!nodeid"}, "", ""))
Expect(rr.Code).To(Equal(http.StatusNotFound))
gatewayClient.AssertNotCalled(GinkgoT(), "ListRecycle", mock.Anything, mock.Anything)
})
It("rejects a malformed body", func() {
svc.RestoreDriveItem(rr, newRequest(http.MethodPost, itemParams, "", `{"nope": 1}`))
Expect(rr.Code).To(Equal(http.StatusBadRequest))
})
It("returns not found for an unknown key", func() {
mockListRecycle()
svc.RestoreDriveItem(rr, newRequest(http.MethodPost, itemParams, "", ""))
Expect(rr.Code).To(Equal(http.StatusNotFound))
gatewayClient.AssertNotCalled(GinkgoT(), "RestoreRecycleItem", mock.Anything, mock.Anything)
})
It("restores to the original location by default and returns the item", func() {
keys := mockListRecycle(trashedFile)
req := mockRestore(status.NewOK(ctx))
mockStatOK()
svc.RestoreDriveItem(rr, newRequest(http.MethodPost, itemParams, "", ""))
Expect(rr.Code).To(Equal(http.StatusOK))
Expect(*keys).To(Equal([]string{"nodeid"}))
Expect((*req).GetKey()).To(Equal("nodeid"))
Expect((*req).GetRef().GetResourceId().GetOpaqueId()).To(Equal("spaceid"))
Expect((*req).GetRestoreRef().GetResourceId().GetOpaqueId()).To(Equal("spaceid"))
Expect((*req).GetRestoreRef().GetPath()).To(Equal("./Documents/notes.txt"))
item := decodeItem()
Expect(item.GetId()).To(Equal("storageid$spaceid!nodeid"))
Expect(item.GetName()).To(Equal("notes.txt"))
})
It("finds a nested key through its parent listing", func() {
keys := mockListRecycle(
&provider.RecycleItem{Type: provider.ResourceType_RESOURCE_TYPE_FILE, Key: "nodeid/sub/a.txt", Ref: &provider.Reference{Path: "/folder/sub/a.txt"}},
)
req := mockRestore(status.NewOK(ctx))
mockStatOK()
svc.RestoreDriveItem(rr, newRequest(http.MethodPost, params{"driveID": "storageid$spaceid", "driveItemID": "storageid$spaceid!nodeid/sub/a.txt"}, "", ""))
Expect(rr.Code).To(Equal(http.StatusOK))
Expect(*keys).To(Equal([]string{"nodeid/sub/"}))
Expect((*req).GetKey()).To(Equal("nodeid/sub/a.txt"))
Expect((*req).GetRestoreRef().GetPath()).To(Equal("./folder/sub/a.txt"))
})
It("accepts the v1beta1 itemID param", func() {
mockListRecycle(trashedFile)
mockRestore(status.NewOK(ctx))
mockStatOK()
svc.RestoreDriveItem(rr, newRequest(http.MethodPost, params{"driveID": "storageid$spaceid", "itemID": "storageid$spaceid!nodeid"}, "", ""))
Expect(rr.Code).To(Equal(http.StatusOK))
})
It("works without a drive id (me/drive)", func() {
mockListRecycle(trashedFile)
mockRestore(status.NewOK(ctx))
mockStatOK()
svc.RestoreDriveItem(rr, newRequest(http.MethodPost, params{"itemID": "storageid$spaceid!nodeid"}, "", ""))
Expect(rr.Code).To(Equal(http.StatusOK))
})
It("restores into the given parent with a new name", func() {
mockListRecycle(trashedFile)
req := mockRestore(status.NewOK(ctx))
mockStatOK()
body := `{"parentReference": {"id": "storageid$spaceid!parentid"}, "name": "renamed.txt"}`
svc.RestoreDriveItem(rr, newRequest(http.MethodPost, itemParams, "", body))
Expect(rr.Code).To(Equal(http.StatusOK))
Expect((*req).GetRestoreRef().GetResourceId().GetOpaqueId()).To(Equal("parentid"))
Expect((*req).GetRestoreRef().GetPath()).To(Equal("./renamed.txt"))
})
It("restores into a parent given by path", func() {
mockListRecycle(trashedFile)
req := mockRestore(status.NewOK(ctx))
mockStatOK()
svc.RestoreDriveItem(rr, newRequest(http.MethodPost, itemParams, "", `{"parentReference": {"path": "/Archive/2025"}}`))
Expect(rr.Code).To(Equal(http.StatusOK))
Expect((*req).GetRestoreRef().GetResourceId().GetOpaqueId()).To(Equal("spaceid"))
Expect((*req).GetRestoreRef().GetPath()).To(Equal("./Archive/2025/notes.txt"))
})
It("only renames when just a name is given", func() {
mockListRecycle(trashedFile)
req := mockRestore(status.NewOK(ctx))
mockStatOK()
svc.RestoreDriveItem(rr, newRequest(http.MethodPost, itemParams, "", `{"name": "renamed.txt"}`))
Expect(rr.Code).To(Equal(http.StatusOK))
Expect((*req).GetRestoreRef().GetPath()).To(Equal("./Documents/renamed.txt"))
})
It("rejects a name with a slash", func() {
mockListRecycle(trashedFile)
svc.RestoreDriveItem(rr, newRequest(http.MethodPost, itemParams, "", `{"name": "a/b.txt"}`))
Expect(rr.Code).To(Equal(http.StatusBadRequest))
gatewayClient.AssertNotCalled(GinkgoT(), "RestoreRecycleItem", mock.Anything, mock.Anything)
})
It("rejects a parent in another drive", func() {
mockListRecycle(trashedFile)
body := `{"parentReference": {"driveId": "storageid$other", "id": "storageid$other!parentid"}}`
svc.RestoreDriveItem(rr, newRequest(http.MethodPost, itemParams, "", body))
Expect(rr.Code).To(Equal(http.StatusBadRequest))
gatewayClient.AssertNotCalled(GinkgoT(), "RestoreRecycleItem", mock.Anything, mock.Anything)
})
DescribeTable("maps restore errors",
func(st *cs3rpc.Status, code int) {
mockListRecycle(trashedFile)
mockRestore(st)
svc.RestoreDriveItem(rr, newRequest(http.MethodPost, itemParams, "", ""))
Expect(rr.Code).To(Equal(code))
gatewayClient.AssertNotCalled(GinkgoT(), "Stat", mock.Anything, mock.Anything)
},
Entry("already exists", status.NewAlreadyExists(context.Background(), errors.New("exists"), "exists"), http.StatusConflict),
Entry("permission denied", status.NewPermissionDenied(context.Background(), errors.New("denied"), "denied"), http.StatusForbidden),
Entry("not found", status.NewNotFound(context.Background(), "gone"), http.StatusNotFound),
Entry("internal", status.NewInternal(context.Background(), "internal"), http.StatusInternalServerError),
)
})
Describe("PermanentDeleteDriveItem", func() {
It("refuses the drive root", func() {
svc.PermanentDeleteDriveItem(rr, newRequest(http.MethodPost, params{"driveID": "storageid$spaceid", "driveItemID": "storageid$spaceid!spaceid"}, "", ""))
Expect(rr.Code).To(Equal(http.StatusBadRequest))
gatewayClient.AssertNotCalled(GinkgoT(), "Delete", mock.Anything, mock.Anything)
})
It("refuses share jail items", func() {
p := params{"driveID": "a0ca6a90-a365-4782-871e-d44447bbc668$a0ca6a90-a365-4782-871e-d44447bbc668", "driveItemID": "a0ca6a90-a365-4782-871e-d44447bbc668$a0ca6a90-a365-4782-871e-d44447bbc668!shareid"}
svc.PermanentDeleteDriveItem(rr, newRequest(http.MethodPost, p, "", ""))
Expect(rr.Code).To(Equal(http.StatusBadRequest))
gatewayClient.AssertNotCalled(GinkgoT(), "Delete", mock.Anything, mock.Anything)
})
It("deletes and purges the trash key", func() {
var delReq *provider.DeleteRequest
gatewayClient.On("Delete", mock.Anything, mock.Anything).Run(func(args mock.Arguments) {
delReq = args.Get(1).(*provider.DeleteRequest)
}).Return(&provider.DeleteResponse{Status: status.NewOK(ctx)}, nil)
purge := mockPurge(status.NewOK(ctx))
svc.PermanentDeleteDriveItem(rr, newRequest(http.MethodPost, itemParams, "", ""))
Expect(rr.Code).To(Equal(http.StatusNoContent))
Expect(delReq.GetRef().GetResourceId().GetOpaqueId()).To(Equal("nodeid"))
Expect((*purge).GetRef().GetResourceId().GetOpaqueId()).To(Equal("spaceid"))
Expect((*purge).GetKey()).To(Equal("nodeid"))
})
It("maps a failed delete and does not purge", func() {
gatewayClient.On("Delete", mock.Anything, mock.Anything).Return(&provider.DeleteResponse{Status: status.NewNotFound(ctx, "gone")}, nil)
svc.PermanentDeleteDriveItem(rr, newRequest(http.MethodPost, itemParams, "", ""))
Expect(rr.Code).To(Equal(http.StatusNotFound))
gatewayClient.AssertNotCalled(GinkgoT(), "PurgeRecycle", mock.Anything, mock.Anything)
})
It("reports a failed purge after a successful delete", func() {
gatewayClient.On("Delete", mock.Anything, mock.Anything).Return(&provider.DeleteResponse{Status: status.NewOK(ctx)}, nil)
mockPurge(status.NewInternal(ctx, "boom"))
svc.PermanentDeleteDriveItem(rr, newRequest(http.MethodPost, itemParams, "", ""))
Expect(rr.Code).To(Equal(http.StatusInternalServerError))
})
})
Describe("DeleteDriveSpecialItem", func() {
It("rejects an unknown special folder", func() {
svc.DeleteDriveSpecialItem(rr, newRequest(http.MethodDelete, params{"driveID": "storageid$spaceid", "specialName": "nope", "itemID": "storageid$spaceid!nodeid"}, "", ""))
Expect(rr.Code).To(Equal(http.StatusBadRequest))
})
It("rejects an item from another drive", func() {
svc.DeleteDriveSpecialItem(rr, newRequest(http.MethodDelete, params{"driveID": "storageid$spaceid", "specialName": "recyclebin", "itemID": "storageid$other!nodeid"}, "", ""))
Expect(rr.Code).To(Equal(http.StatusNotFound))
gatewayClient.AssertNotCalled(GinkgoT(), "PurgeRecycle", mock.Anything, mock.Anything)
})
It("purges the key", func() {
purge := mockPurge(status.NewOK(ctx))
svc.DeleteDriveSpecialItem(rr, newRequest(http.MethodDelete, params{"driveID": "storageid$spaceid", "specialName": "recyclebin", "itemID": "storageid$spaceid!nodeid/sub/file"}, "", ""))
Expect(rr.Code).To(Equal(http.StatusNoContent))
Expect((*purge).GetKey()).To(Equal("nodeid/sub/file"))
Expect((*purge).GetRef().GetResourceId().GetOpaqueId()).To(Equal("spaceid"))
})
It("maps permission denied to forbidden", func() {
mockPurge(status.NewPermissionDenied(ctx, errors.New("denied"), "denied"))
svc.DeleteDriveSpecialItem(rr, newRequest(http.MethodDelete, params{"driveID": "storageid$spaceid", "specialName": "recyclebin", "itemID": "storageid$spaceid!nodeid"}, "", ""))
Expect(rr.Code).To(Equal(http.StatusForbidden))
})
})
Describe("EmptyDriveSpecial", func() {
It("purges the whole trash", func() {
purge := mockPurge(status.NewOK(ctx))
svc.EmptyDriveSpecial(rr, newRequest(http.MethodDelete, specialParams, "", ""))
Expect(rr.Code).To(Equal(http.StatusNoContent))
Expect((*purge).GetKey()).To(Equal(""))
})
It("purges only the addressed key in the colon path form", func() {
purge := mockPurge(status.NewOK(ctx))
svc.EmptyDriveSpecial(rr, withSpecialPath(newRequest(http.MethodDelete, specialParams, "", ""), "/nodeid"))
Expect(rr.Code).To(Equal(http.StatusNoContent))
Expect((*purge).GetKey()).To(Equal("nodeid"))
})
})
})
@@ -1,237 +0,0 @@
package svc
import (
"mime"
"net/http"
"path"
"strings"
storageprovider "github.com/cs3org/go-cs3apis/cs3/storage/provider/v1beta1"
"github.com/go-chi/chi/v5"
"github.com/go-chi/render"
libregraph "github.com/opencloud-eu/libre-graph-api-go"
"github.com/opencloud-eu/reva/v2/pkg/storagespace"
"github.com/opencloud-eu/reva/v2/pkg/utils"
"github.com/opencloud-eu/opencloud/services/graph/pkg/errorcode"
graphm "github.com/opencloud-eu/opencloud/services/graph/pkg/middleware"
)
// RecycleBinSpecialFolderName addresses a drive's trash as a special folder
const RecycleBinSpecialFolderName = "recyclebin"
// GetDriveSpecial returns the driveItem of a special folder. In the colon path
// form (special/recyclebin:/{key}) it returns the trashed item with that key.
// $expand=children embeds the listing for folders.
func (g Graph) GetDriveSpecial(w http.ResponseWriter, r *http.Request) {
g.logger.Debug().Msg("Calling GetDriveSpecial")
driveID, err := parseIDParam(r, "driveID")
if err != nil {
errorcode.RenderError(w, r, err)
return
}
if chi.URLParam(r, "specialName") != RecycleBinSpecialFolderName {
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "unknown special folder")
return
}
key := specialFolderKey(r)
driveItem, ok := g.getSpecialDriveItemForKey(w, r, &driveID, key)
if !ok {
return
}
if driveItem.Folder != nil && driveItemRelationExpanded(r, _expandChildren) {
children, ok := g.listRecycleChildren(w, r, &driveID, key)
if !ok {
return
}
driveItem.Children = children
}
render.Status(r, http.StatusOK)
render.JSON(w, r, driveItem)
}
// getSpecialDriveItemForKey returns the trash root for an empty key, else the trashed item with that key
func (g Graph) getSpecialDriveItemForKey(w http.ResponseWriter, r *http.Request, driveID *storageprovider.ResourceId, key string) (*libregraph.DriveItem, bool) {
if key == "" {
return recycleBinDriveItem(driveID), true
}
item, ok := g.findRecycleItem(w, r, driveID, key)
if !ok {
return nil, false
}
return recycleItemToDriveItem(driveID, item), true
}
// findRecycleItem returns the trashed item with the given key. A top-level key lists itself,
// a nested key (key/rel/path) only shows up in its parent's listing.
func (g Graph) findRecycleItem(w http.ResponseWriter, r *http.Request, driveID *storageprovider.ResourceId, key string) (*storageprovider.RecycleItem, bool) {
listKey := key
if strings.Contains(key, "/") {
listKey = path.Dir(key) + "/"
}
items, ok := g.listRecycle(w, r, driveID, listKey)
if !ok {
return nil, false
}
for _, item := range items {
if item.GetKey() == key {
return item, true
}
}
errorcode.ItemNotFound.Render(w, r, http.StatusNotFound, "item not found")
return nil, false
}
// ListDriveSpecialChildren lists the children of a special folder. For the
// recycle bin this is the trash listing; the colon path form
// (special/recyclebin:/{key}:/children) lists inside a trashed folder.
func (g Graph) ListDriveSpecialChildren(w http.ResponseWriter, r *http.Request) {
g.logger.Debug().Msg("Calling ListDriveSpecialChildren")
driveID, err := parseIDParam(r, "driveID")
if err != nil {
errorcode.RenderError(w, r, err)
return
}
if chi.URLParam(r, "specialName") != RecycleBinSpecialFolderName {
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "unknown special folder")
return
}
files, ok := g.listRecycleChildren(w, r, &driveID, specialFolderKey(r))
if !ok {
return
}
render.Status(r, http.StatusOK)
render.JSON(w, r, &ListResponse{Value: files})
}
// listRecycleChildren lists the trash root for an empty key, else the children of the trashed folder with that key
func (g Graph) listRecycleChildren(w http.ResponseWriter, r *http.Request, driveID *storageprovider.ResourceId, key string) ([]libregraph.DriveItem, bool) {
// the trailing slash asks reva for the children of the key instead of the key itself
listKey := ""
if key != "" {
listKey = key + "/"
}
items, ok := g.listRecycle(w, r, driveID, listKey)
if !ok {
return nil, false
}
files := make([]libregraph.DriveItem, 0, len(items))
for _, item := range items {
files = append(files, *recycleItemToDriveItem(driveID, item))
}
return files, true
}
// specialFolderKey returns the recycle key addressed by the colon path form, "" for the trash root
func specialFolderKey(r *http.Request) string {
p, _ := graphm.SpecialFolderPath(r.Context())
return strings.Trim(p, "/")
}
func (g Graph) listRecycle(w http.ResponseWriter, r *http.Request, driveID *storageprovider.ResourceId, key string) ([]*storageprovider.RecycleItem, bool) {
gatewayClient, err := g.gatewaySelector.Next()
if err != nil {
errorcode.GeneralException.Render(w, r, http.StatusInternalServerError, err.Error())
return nil, false
}
res, err := gatewayClient.ListRecycle(r.Context(), &storageprovider.ListRecycleRequest{
Ref: &storageprovider.Reference{ResourceId: spaceRootID(driveID)},
Key: key,
})
if err != nil {
errorcode.GeneralException.Render(w, r, http.StatusInternalServerError, err.Error())
return nil, false
}
// like listDriveItemChildren, a denied listing does not disclose existence
if !renderTrashStatus(w, r, res.GetStatus(), true) {
return nil, false
}
return res.GetRecycleItems(), true
}
func spaceRootID(driveID *storageprovider.ResourceId) *storageprovider.ResourceId {
return &storageprovider.ResourceId{
StorageId: driveID.GetStorageId(),
SpaceId: driveID.GetSpaceId(),
OpaqueId: driveID.GetSpaceId(),
}
}
// recycleBinDriveItem is the synthetic folder for the trash root. It has no node in the storage.
func recycleBinDriveItem(driveID *storageprovider.ResourceId) *libregraph.DriveItem {
item := libregraph.NewDriveItem()
item.SetId(storagespace.FormatResourceID(&storageprovider.ResourceId{
StorageId: driveID.GetStorageId(),
SpaceId: driveID.GetSpaceId(),
OpaqueId: RecycleBinSpecialFolderName,
}))
item.SetName(RecycleBinSpecialFolderName)
item.SetFolder(libregraph.Folder{})
item.SetSpecialFolder(libregraph.SpecialFolder{Name: libregraph.PtrString(RecycleBinSpecialFolderName)})
parentRef := libregraph.NewItemReference()
parentRef.SetDriveId(storagespace.FormatStorageID(driveID.GetStorageId(), driveID.GetSpaceId()))
parentRef.SetId(storagespace.FormatResourceID(spaceRootID(driveID)))
item.SetParentReference(*parentRef)
return item
}
// recycleItemToDriveItem maps a trashed item. The id carries the recycle key, which is the
// node id for top-level entries and key/relative/path inside a trashed folder.
func recycleItemToDriveItem(driveID *storageprovider.ResourceId, ri *storageprovider.RecycleItem) *libregraph.DriveItem {
origin := ri.GetRef().GetPath()
name := path.Base(origin)
if name == "." || name == "/" {
name = path.Base(ri.GetKey())
}
item := libregraph.NewDriveItem()
item.SetId(storagespace.FormatResourceID(&storageprovider.ResourceId{
StorageId: driveID.GetStorageId(),
SpaceId: driveID.GetSpaceId(),
OpaqueId: ri.GetKey(),
}))
item.SetName(name)
item.SetSize(int64(ri.GetSize())) // TODO lurking overflow, see cs3ResourceToDriveItem
switch ri.GetType() {
case storageprovider.ResourceType_RESOURCE_TYPE_FILE:
mimeType := mime.TypeByExtension(path.Ext(name))
if mimeType == "" {
mimeType = "application/octet-stream"
}
item.SetFile(libregraph.OpenGraphFile{MimeType: &mimeType})
case storageprovider.ResourceType_RESOURCE_TYPE_CONTAINER:
item.SetFolder(libregraph.Folder{})
}
trash := libregraph.NewTrash()
if ri.GetDeletionTime() != nil {
trash.SetTrashedDateTime(utils.TSToTime(ri.GetDeletionTime()).UTC())
}
item.SetTrash(*trash)
if origin != "" {
// the original location; the parent may itself be trashed or gone, so no id
dir := path.Dir(origin)
parentRef := libregraph.NewItemReference()
parentRef.SetDriveId(storagespace.FormatStorageID(driveID.GetStorageId(), driveID.GetSpaceId()))
parentRef.SetPath(dir)
if base := path.Base(dir); base != "/" && base != "." {
parentRef.SetName(base)
}
item.SetParentReference(*parentRef)
}
return item
}
@@ -208,6 +208,40 @@ var _ = Describe("Driveitems", func() {
Expect(res.Value[0].LibreGraphPermissionsActionsAllowedValues).To(BeNil())
})
It("returns the thumbnails when requested via $expand", func() {
cfg.Commons.OpenCloudURL = "https://cloud.test"
gatewayClient.On("ListStorageSpaces", mock.Anything, mock.Anything).Return(&provider.ListStorageSpacesResponse{
Status: status.NewOK(ctx),
StorageSpaces: []*provider.StorageSpace{{Owner: currentUser, Root: &provider.ResourceId{}}},
}, nil)
gatewayClient.On("ListContainer", mock.Anything, mock.Anything).Return(&provider.ListContainerResponse{
Status: status.NewOK(ctx),
Infos: []*provider.ResourceInfo{
{
Type: provider.ResourceType_RESOURCE_TYPE_FILE,
Id: &provider.ResourceId{StorageId: "storageid", SpaceId: "spaceid", OpaqueId: "opaqueid"},
MimeType: "image/jpeg",
Mtime: utils.TimeToTS(time.Now()),
},
},
}, nil)
r := httptest.NewRequest(http.MethodGet, "/graph/v1.0/me/drive/root/children?$expand=thumbnails", nil)
r = r.WithContext(revactx.ContextSetUser(ctx, currentUser))
svc.GetRootDriveChildren(rr, r)
Expect(rr.Code).To(Equal(http.StatusOK))
data, err := io.ReadAll(rr.Body)
Expect(err).ToNot(HaveOccurred())
res := itemsList{}
Expect(json.Unmarshal(data, &res)).To(Succeed())
Expect(len(res.Value)).To(Equal(1))
Expect(res.Value[0].Thumbnails).To(HaveLen(1))
Expect(res.Value[0].Thumbnails[0].Small.GetUrl()).To(Equal(
"https://cloud.test/dav/spaces/storageid$spaceid!opaqueid" +
"?scalingup=0&preview=1&processor=thumbnail&x=36&y=36",
))
})
It("returns the allowed actions when requested via $select", func() {
gatewayClient.On("ListStorageSpaces", mock.Anything, mock.Anything).Return(&provider.ListStorageSpacesResponse{
Status: status.NewOK(ctx),
@@ -248,6 +282,7 @@ var _ = Describe("Driveitems", func() {
Describe("GetDriveItem", func() {
var (
folderInfo *provider.ResourceInfo
childInfo *provider.ResourceInfo
mtime = time.Now()
)
@@ -281,16 +316,15 @@ var _ = Describe("Driveitems", func() {
Status: status.NewOK(ctx),
Info: folderInfo,
}, nil)
childInfo = &provider.ResourceInfo{
Type: provider.ResourceType_RESOURCE_TYPE_FILE,
Id: &provider.ResourceId{StorageId: "storageid", SpaceId: "spaceid", OpaqueId: "opaqueid"},
Etag: "etag",
Mtime: utils.TimeToTS(mtime),
}
gatewayClient.On("ListContainer", mock.Anything, mock.Anything).Return(&provider.ListContainerResponse{
Status: status.NewOK(ctx),
Infos: []*provider.ResourceInfo{
{
Type: provider.ResourceType_RESOURCE_TYPE_FILE,
Id: &provider.ResourceId{StorageId: "storageid", SpaceId: "spaceid", OpaqueId: "opaqueid"},
Etag: "etag",
Mtime: utils.TimeToTS(mtime),
},
},
Infos: []*provider.ResourceInfo{childInfo},
}, nil)
})
@@ -312,6 +346,49 @@ var _ = Describe("Driveitems", func() {
Expect(getItem(newRequest("?$expand=children")).Children).To(BeNil())
gatewayClient.AssertNotCalled(GinkgoT(), "ListContainer", mock.Anything, mock.Anything)
})
Context("$expand=thumbnails", func() {
const previewURL = "https://cloud.test/dav/spaces/storageid$spaceid!nodeid" +
"?scalingup=0&preview=1&processor=thumbnail"
BeforeEach(func() {
cfg.Commons.OpenCloudURL = "https://cloud.test"
folderInfo.Type = provider.ResourceType_RESOURCE_TYPE_FILE
folderInfo.MimeType = "image/jpeg"
})
It("leaves thumbnails unset without $expand", func() {
Expect(getItem(newRequest("")).Thumbnails).To(BeNil())
})
It("returns the thumbnail urls when requested", func() {
thumbnails := getItem(newRequest("?$expand=thumbnails")).Thumbnails
Expect(thumbnails).To(HaveLen(1))
Expect(thumbnails[0].Small.GetUrl()).To(Equal(previewURL + "&x=36&y=36"))
Expect(thumbnails[0].Medium.GetUrl()).To(Equal(previewURL + "&x=48&y=48"))
Expect(thumbnails[0].Large.GetUrl()).To(Equal(previewURL + "&x=96&y=96"))
})
It("leaves thumbnails unset for a mime type the thumbnailer cannot render", func() {
folderInfo.MimeType = "application/zip"
Expect(getItem(newRequest("?$expand=thumbnails")).Thumbnails).To(BeNil())
})
It("adds them to expanded children as well", func() {
folderInfo.Type = provider.ResourceType_RESOURCE_TYPE_CONTAINER
folderInfo.MimeType = ""
childInfo.MimeType = "image/jpeg"
item := getItem(newRequest("?$expand=children,thumbnails"))
// a folder has no preview of its own
Expect(item.Thumbnails).To(BeNil())
Expect(item.Children).To(HaveLen(1))
Expect(item.Children[0].Thumbnails).To(HaveLen(1))
})
})
})
Describe("GetDriveItem $expand=children error", func() {
@@ -1,304 +0,0 @@
package svc
import (
"errors"
"io"
"net/http"
"path"
cs3rpc "github.com/cs3org/go-cs3apis/cs3/rpc/v1beta1"
storageprovider "github.com/cs3org/go-cs3apis/cs3/storage/provider/v1beta1"
"github.com/go-chi/chi/v5"
"github.com/go-chi/render"
libregraph "github.com/opencloud-eu/libre-graph-api-go"
"github.com/opencloud-eu/reva/v2/pkg/storagespace"
"github.com/opencloud-eu/reva/v2/pkg/utils"
"github.com/opencloud-eu/opencloud/services/graph/pkg/errorcode"
)
// restoreRequest is the optional body of driveItem: restore
type restoreRequest struct {
ParentReference *libregraph.ItemReference `json:"parentReference,omitempty"`
Name *string `json:"name,omitempty"`
}
// RestoreDriveItem restores a trashed item, by default to its original location.
// The item id carries the recycle key, see recycleItemToDriveItem.
func (g Graph) RestoreDriveItem(w http.ResponseWriter, r *http.Request) {
g.logger.Debug().Msg("Calling RestoreDriveItem")
ctx := r.Context()
itemID, ok := parseTrashItemID(w, r)
if !ok {
return
}
var body restoreRequest
if err := StrictJSONUnmarshal(r.Body, &body); err != nil && !errors.Is(err, io.EOF) {
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "invalid body: "+err.Error())
return
}
// the listing gives us the original location, needed for the default target and the response
key := itemID.GetOpaqueId()
trashed, ok := g.findRecycleItem(w, r, itemID, key)
if !ok {
return
}
target, ok := restoreTarget(w, r, itemID, trashed, body)
if !ok {
return
}
gatewayClient, ok := g.GetGatewayClient(w, r)
if !ok {
return
}
res, err := gatewayClient.RestoreRecycleItem(ctx, &storageprovider.RestoreRecycleItemRequest{
Ref: &storageprovider.Reference{ResourceId: spaceRootID(itemID)},
Key: key,
RestoreRef: target,
})
if err != nil {
errorcode.GeneralException.Render(w, r, http.StatusInternalServerError, err.Error())
return
}
if !renderTrashStatus(w, r, res.GetStatus(), false) {
return
}
statRes, err := gatewayClient.Stat(ctx, &storageprovider.StatRequest{Ref: target})
switch {
case err != nil:
errorcode.GeneralException.Render(w, r, http.StatusInternalServerError, err.Error())
return
case statRes.GetStatus().GetCode() != cs3rpc.Code_CODE_OK:
errorcode.GeneralException.Render(w, r, http.StatusInternalServerError, "restored, but could not stat the item: "+statRes.GetStatus().GetMessage())
return
}
driveItem, err := cs3ResourceToDriveItem(g.logger, g.publicBaseURL, statRes.GetInfo())
if err != nil {
errorcode.GeneralException.Render(w, r, http.StatusInternalServerError, err.Error())
return
}
render.Status(r, http.StatusOK)
render.JSON(w, r, driveItem)
}
// restoreTarget builds the restore reference: the original location unless the body moves the item
func restoreTarget(w http.ResponseWriter, r *http.Request, itemID *storageprovider.ResourceId, trashed *storageprovider.RecycleItem, body restoreRequest) (*storageprovider.Reference, bool) {
origin := trashed.GetRef().GetPath()
name := path.Base(origin)
if body.Name != nil && *body.Name != "" {
name = *body.Name
}
if name != path.Base(name) || name == "." || name == "/" {
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "invalid name")
return nil, false
}
parent := body.ParentReference
if parent != nil && parent.DriveId != nil {
parentDrive, err := storagespace.ParseID(parent.GetDriveId())
if err != nil || parentDrive.GetStorageId() != itemID.GetStorageId() || parentDrive.GetSpaceId() != itemID.GetSpaceId() {
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "restore into another drive is not supported")
return nil, false
}
}
switch {
case parent != nil && parent.Id != nil:
parentID, err := storagespace.ParseID(parent.GetId())
if err != nil || parentID.GetStorageId() != itemID.GetStorageId() || parentID.GetSpaceId() != itemID.GetSpaceId() {
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "invalid parentReference.id")
return nil, false
}
return &storageprovider.Reference{ResourceId: &parentID, Path: utils.MakeRelativePath(name)}, true
case parent != nil && parent.Path != nil:
return &storageprovider.Reference{
ResourceId: spaceRootID(itemID),
Path: utils.MakeRelativePath(path.Join(parent.GetPath(), name)),
}, true
default:
return &storageprovider.Reference{
ResourceId: spaceRootID(itemID),
Path: utils.MakeRelativePath(path.Join(path.Dir(origin), name)),
}, true
}
}
// PermanentDeleteDriveItem deletes a live item and purges it from the trash right away.
// reva has no delete that bypasses the trash, so this is two steps; the trash key of a
// freshly deleted item is its node id in both decomposedfs and posixfs.
func (g Graph) PermanentDeleteDriveItem(w http.ResponseWriter, r *http.Request) {
g.logger.Debug().Msg("Calling PermanentDeleteDriveItem")
ctx := r.Context()
itemID, ok := parseTrashItemID(w, r)
if !ok {
return
}
if IsSpaceRoot(itemID) {
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "cannot delete the drive root")
return
}
if IsShareJail(itemID) {
// the trash of a shared item lives in the owner's space, which the share jail id does not tell us
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "not supported for shared items, use the item id of the owning drive")
return
}
gatewayClient, ok := g.GetGatewayClient(w, r)
if !ok {
return
}
delRes, err := gatewayClient.Delete(ctx, &storageprovider.DeleteRequest{
Ref: &storageprovider.Reference{ResourceId: itemID},
})
if err != nil {
errorcode.GeneralException.Render(w, r, http.StatusInternalServerError, err.Error())
return
}
if !renderTrashStatus(w, r, delRes.GetStatus(), false) {
return
}
purgeRes, err := gatewayClient.PurgeRecycle(ctx, &storageprovider.PurgeRecycleRequest{
Ref: &storageprovider.Reference{ResourceId: spaceRootID(itemID)},
Key: itemID.GetOpaqueId(),
})
if err != nil {
errorcode.GeneralException.Render(w, r, http.StatusInternalServerError, "deleted, but could not purge from the trash: "+err.Error())
return
}
if purgeRes.GetStatus().GetCode() != cs3rpc.Code_CODE_OK {
errorcode.GeneralException.Render(w, r, http.StatusInternalServerError, "deleted, but could not purge from the trash: "+purgeRes.GetStatus().GetMessage())
return
}
render.Status(r, http.StatusNoContent)
render.NoContent(w, r)
}
// DeleteDriveSpecialItem purges one trashed item
func (g Graph) DeleteDriveSpecialItem(w http.ResponseWriter, r *http.Request) {
g.logger.Debug().Msg("Calling DeleteDriveSpecialItem")
driveID, err := parseIDParam(r, "driveID")
if err != nil {
errorcode.RenderError(w, r, err)
return
}
if chi.URLParam(r, "specialName") != RecycleBinSpecialFolderName {
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "unknown special folder")
return
}
itemID, ok := parseTrashItemID(w, r)
if !ok {
return
}
g.purgeRecycle(w, r, &driveID, itemID.GetOpaqueId())
}
// EmptyDriveSpecial purges the whole trash. In the colon path form
// (special/recyclebin:/{key}) it purges only that item.
func (g Graph) EmptyDriveSpecial(w http.ResponseWriter, r *http.Request) {
g.logger.Debug().Msg("Calling EmptyDriveSpecial")
driveID, err := parseIDParam(r, "driveID")
if err != nil {
errorcode.RenderError(w, r, err)
return
}
if chi.URLParam(r, "specialName") != RecycleBinSpecialFolderName {
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "unknown special folder")
return
}
g.purgeRecycle(w, r, &driveID, specialFolderKey(r))
}
// purgeRecycle purges the key, or the whole trash for an empty key
func (g Graph) purgeRecycle(w http.ResponseWriter, r *http.Request, driveID *storageprovider.ResourceId, key string) {
gatewayClient, ok := g.GetGatewayClient(w, r)
if !ok {
return
}
res, err := gatewayClient.PurgeRecycle(r.Context(), &storageprovider.PurgeRecycleRequest{
Ref: &storageprovider.Reference{ResourceId: spaceRootID(driveID)},
Key: key,
})
if err != nil {
errorcode.GeneralException.Render(w, r, http.StatusInternalServerError, err.Error())
return
}
if !renderTrashStatus(w, r, res.GetStatus(), false) {
return
}
render.Status(r, http.StatusNoContent)
render.NoContent(w, r)
}
// parseTrashItemID reads the item id, which the v1.0 and v1beta1 routes bind under different names,
// and checks it against the drive id when the route has one
func parseTrashItemID(w http.ResponseWriter, r *http.Request) (*storageprovider.ResourceId, bool) {
param := "itemID"
if chi.URLParam(r, param) == "" {
param = "driveItemID"
}
itemID, err := parseIDParam(r, param)
if err != nil {
errorcode.RenderError(w, r, err)
return nil, false
}
if itemID.GetOpaqueId() == "" {
errorcode.InvalidRequest.Render(w, r, http.StatusBadRequest, "invalid itemID")
return nil, false
}
if chi.URLParam(r, "driveID") != "" {
driveID, err := parseIDParam(r, "driveID")
if err != nil {
errorcode.RenderError(w, r, err)
return nil, false
}
if driveID.GetStorageId() != itemID.GetStorageId() || driveID.GetSpaceId() != itemID.GetSpaceId() {
errorcode.ItemNotFound.Render(w, r, http.StatusNotFound, "item not found")
return nil, false
}
}
return &itemID, true
}
// renderTrashStatus maps a trash call's status; true means OK. hideExistence turns
// PERMISSION_DENIED into a 404, for listings; mutations answer 403 because the caller
// could already see the item.
func renderTrashStatus(w http.ResponseWriter, r *http.Request, st *cs3rpc.Status, hideExistence bool) bool {
switch st.GetCode() {
case cs3rpc.Code_CODE_OK:
return true
case cs3rpc.Code_CODE_NOT_FOUND:
errorcode.ItemNotFound.Render(w, r, http.StatusNotFound, st.GetMessage())
case cs3rpc.Code_CODE_PERMISSION_DENIED:
if hideExistence {
errorcode.ItemNotFound.Render(w, r, http.StatusNotFound, st.GetMessage())
} else {
errorcode.AccessDenied.Render(w, r, http.StatusForbidden, st.GetMessage())
}
case cs3rpc.Code_CODE_UNAUTHENTICATED:
errorcode.Unauthenticated.Render(w, r, http.StatusUnauthorized, st.GetMessage())
case cs3rpc.Code_CODE_ALREADY_EXISTS:
errorcode.NameAlreadyExists.Render(w, r, http.StatusConflict, st.GetMessage())
case cs3rpc.Code_CODE_LOCKED, cs3rpc.Code_CODE_FAILED_PRECONDITION:
errorcode.ItemIsLocked.Render(w, r, http.StatusLocked, st.GetMessage())
default:
errorcode.GeneralException.Render(w, r, http.StatusInternalServerError, st.GetMessage())
}
return false
}
-18
View File
@@ -104,12 +104,6 @@ type Service interface { //nolint:interfacebloat
GetRootDriveChildren(w http.ResponseWriter, r *http.Request)
GetDriveItem(w http.ResponseWriter, r *http.Request)
GetDriveItemChildren(w http.ResponseWriter, r *http.Request)
GetDriveSpecial(w http.ResponseWriter, r *http.Request)
ListDriveSpecialChildren(w http.ResponseWriter, r *http.Request)
EmptyDriveSpecial(w http.ResponseWriter, r *http.Request)
DeleteDriveSpecialItem(w http.ResponseWriter, r *http.Request)
RestoreDriveItem(w http.ResponseWriter, r *http.Request)
PermanentDeleteDriveItem(w http.ResponseWriter, r *http.Request)
CreateUploadSession(w http.ResponseWriter, r *http.Request)
@@ -277,8 +271,6 @@ func NewService(opts ...Option) (Graph, error) { //nolint:maintidx
r.Get("/", drivesDriveItemApi.GetDriveItem)
r.Patch("/", drivesDriveItemApi.UpdateDriveItem)
r.Delete("/", drivesDriveItemApi.DeleteDriveItem)
r.Post("/restore", svc.RestoreDriveItem)
r.Post("/permanentDelete", svc.PermanentDeleteDriveItem)
r.Post("/invite", driveItemPermissionsApi.Invite)
r.Post("/createLink", driveItemPermissionsApi.CreateLink)
r.Route("/permissions", func(r chi.Router) {
@@ -314,8 +306,6 @@ func NewService(opts ...Option) (Graph, error) { //nolint:maintidx
r.Get("/", svc.GetUserDrive)
r.Get("/root/children", svc.GetRootDriveChildren)
r.Post("/items/{itemID}/follow", svc.FollowDriveItem)
r.Post("/items/{itemID}/restore", svc.RestoreDriveItem)
r.Post("/items/{itemID}/permanentDelete", svc.PermanentDeleteDriveItem)
r.Delete("/following/{itemID}", svc.UnfollowDriveItem)
})
r.Get("/drives", svc.GetDrives(APIVersion_1))
@@ -380,14 +370,6 @@ func NewService(opts ...Option) (Graph, error) { //nolint:maintidx
r.Get("/", svc.GetDriveItem)
r.Get("/children", svc.GetDriveItemChildren)
r.Post("/createUploadSession", svc.CreateUploadSession)
r.Post("/restore", svc.RestoreDriveItem)
r.Post("/permanentDelete", svc.PermanentDeleteDriveItem)
})
r.Route("/special/{specialName}", func(r chi.Router) {
r.Get("/", svc.GetDriveSpecial)
r.Delete("/", svc.EmptyDriveSpecial)
r.Get("/children", svc.ListDriveSpecialChildren)
r.Delete("/items/{itemID}", svc.DeleteDriveSpecialItem)
})
})
})
+2 -26
View File
@@ -1,14 +1,12 @@
package svc
import (
"fmt"
"net/http"
"strings"
rpc "github.com/cs3org/go-cs3apis/cs3/rpc/v1beta1"
"github.com/go-chi/render"
libregraph "github.com/opencloud-eu/libre-graph-api-go"
"github.com/opencloud-eu/opencloud/services/thumbnails/pkg/thumbnail"
"github.com/opencloud-eu/opencloud/services/graph/pkg/errorcode"
)
@@ -44,30 +42,8 @@ func (g Graph) GetSharedByMe(w http.ResponseWriter, r *http.Request) {
expandThumbnails := strings.Contains(expand, "thumbnails")
if expandThumbnails {
for k, item := range driveItems {
mt := item.GetFile().MimeType
if mt == nil {
continue
}
_, match := thumbnail.SupportedMimeTypes[*mt]
if match {
baseUrl := fmt.Sprintf("%s/dav/spaces/%s?scalingup=0&preview=1&processor=thumbnail",
g.config.Commons.OpenCloudURL,
item.GetId())
smallUrl := baseUrl + "&x=36&y=36"
mediumUrl := baseUrl + "&x=48&y=48"
largeUrl := baseUrl + "&x=96&y=96"
item.SetThumbnails([]libregraph.ThumbnailSet{
{
Small: &libregraph.Thumbnail{Url: &smallUrl},
Medium: &libregraph.Thumbnail{Url: &mediumUrl},
Large: &libregraph.Thumbnail{Url: &largeUrl},
},
})
driveItems[k] = item // assign modified item back to the map
}
setShareThumbnails(&item, item.GetId(), g.config.Commons.OpenCloudURL)
driveItems[k] = item
}
}
+2 -26
View File
@@ -2,7 +2,6 @@ package svc
import (
"context"
"fmt"
"net/http"
"strings"
@@ -13,7 +12,6 @@ import (
"github.com/opencloud-eu/reva/v2/pkg/share"
"github.com/opencloud-eu/opencloud/services/graph/pkg/errorcode"
"github.com/opencloud-eu/opencloud/services/thumbnails/pkg/thumbnail"
)
// ListSharedWithMe lists the files shared with the current user.
@@ -73,30 +71,8 @@ func (g Graph) listSharedWithMe(ctx context.Context, expandThumbnails bool) ([]l
if expandThumbnails {
for k, item := range driveItems {
mt := item.GetFile().MimeType
if mt == nil {
continue
}
_, match := thumbnail.SupportedMimeTypes[*mt]
if match {
baseUrl := fmt.Sprintf("%s/dav/spaces/%s?scalingup=0&preview=1&processor=thumbnail",
g.config.Commons.OpenCloudURL,
item.RemoteItem.GetId())
smallUrl := baseUrl + "&x=36&y=36"
mediumUrl := baseUrl + "&x=48&y=48"
largeUrl := baseUrl + "&x=96&y=96"
item.SetThumbnails([]libregraph.ThumbnailSet{
{
Small: &libregraph.Thumbnail{Url: &smallUrl},
Medium: &libregraph.Thumbnail{Url: &mediumUrl},
Large: &libregraph.Thumbnail{Url: &largeUrl},
},
})
driveItems[k] = item // assign modified item back to the map
}
setShareThumbnails(&item, item.RemoteItem.GetId(), g.config.Commons.OpenCloudURL)
driveItems[k] = item
}
}
@@ -0,0 +1,68 @@
package svc
import (
"fmt"
"net/http"
provider "github.com/cs3org/go-cs3apis/cs3/storage/provider/v1beta1"
libregraph "github.com/opencloud-eu/libre-graph-api-go"
"github.com/opencloud-eu/reva/v2/pkg/storagespace"
"github.com/opencloud-eu/opencloud/services/thumbnails/pkg/thumbnail"
)
const (
thumbnailBoxSmall = 36
thumbnailBoxMedium = 48
thumbnailBoxLarge = 96
)
func (g Graph) setDriveItemsThumbnails(r *http.Request, items []libregraph.DriveItem, infos []*provider.ResourceInfo) {
if !driveItemRelationExpanded(r, _expandThumbnails) {
return
}
for i := range items {
if i < len(infos) {
setDriveItemThumbnails(&items[i], infos[i], g.config.Commons.OpenCloudURL)
}
}
}
func setDriveItemThumbnails(item *libregraph.DriveItem, res *provider.ResourceInfo, baseURL string) {
if set := previewThumbnailSet(res, baseURL); set != nil {
item.SetThumbnails([]libregraph.ThumbnailSet{*set})
}
}
// previewThumbnailSet returns nil when the thumbnailer cannot render the resource.
func previewThumbnailSet(res *provider.ResourceInfo, baseURL string) *libregraph.ThumbnailSet {
if !thumbnail.IsMimeTypeSupported(res.GetMimeType()) {
return nil
}
return thumbnailSetFor(baseURL, storagespace.FormatResourceID(res.GetId()))
}
// thumbnailSetFor builds the urls of the WebDAV preview endpoint.
func thumbnailSetFor(baseURL, itemID string) *libregraph.ThumbnailSet {
base := fmt.Sprintf("%s/dav/spaces/%s?scalingup=0&preview=1&processor=thumbnail", baseURL, itemID)
return &libregraph.ThumbnailSet{
Small: previewThumbnail(base, thumbnailBoxSmall),
Medium: previewThumbnail(base, thumbnailBoxMedium),
Large: previewThumbnail(base, thumbnailBoxLarge),
}
}
func previewThumbnail(base string, box int32) *libregraph.Thumbnail {
url := fmt.Sprintf("%s&x=%d&y=%d", base, box, box)
return &libregraph.Thumbnail{Url: &url}
}
// setShareThumbnails works off the driveItem, the share listings have no resource
// info. The id comes separately, a received share carries it on its remote item.
func setShareThumbnails(item *libregraph.DriveItem, itemID, baseURL string) {
mimeType := item.GetFile().MimeType
if itemID == "" || mimeType == nil || !thumbnail.IsMimeTypeSupported(*mimeType) {
return
}
item.SetThumbnails([]libregraph.ThumbnailSet{*thumbnailSetFor(baseURL, itemID)})
}
@@ -0,0 +1,212 @@
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
# Translators:
# Roy-Magne Mo, 2026
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: EMAIL\n"
"POT-Creation-Date: 2026-09-07 23:16+0000\n"
"PO-Revision-Date: 2025-01-27 10:17+0000\n"
"Last-Translator: Roy-Magne Mo, 2026\n"
"Language-Team: Norwegian (https://app.transifex.com/opencloud-eu/teams/204053/no/)\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Language: no\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
#. UnsharedSpace email template, resolves via {{ .CallToAction }}
#: pkg/email/templates.go:78
msgid "Click here to check it: {ShareLink}"
msgstr "Klikk her for å sjekke det: {ShareLink}"
#. ShareCreated email template, resolves via {{ .CallToAction }}
#. SharedSpace email template, resolves via {{ .CallToAction }}
#: pkg/email/templates.go:23 pkg/email/templates.go:63
msgid "Click here to view it: {ShareLink}"
msgstr "Klikk her for å se den: {ShareLink}"
#: pkg/email/templates.go:139
msgid "Hello {RecipientName},"
msgstr "Hei {RecipientName},"
#. ShareCreated email template, resolves via {{ .Greeting }}
#: pkg/email/templates.go:19
msgid "Hello {ShareGrantee}"
msgstr "Hei {ShareGrantee}"
#. ShareRemoved email template, resolves via {{ .Greeting }}
#. ShareExpired email template, resolves via {{ .Greeting }}
#: pkg/email/templates.go:32 pkg/email/templates.go:45
msgid "Hello {ShareGrantee},"
msgstr "Hei {ShareGrantee},"
#. SharedSpace email template, resolves via {{ .Greeting }}
#. UnsharedSpace email template, resolves via {{ .Greeting }}
#. MembershipExpired email template, resolves via {{ .Greeting }}
#: pkg/email/templates.go:59 pkg/email/templates.go:72
#: pkg/email/templates.go:87
msgid "Hello {SpaceGrantee},"
msgstr "Hei {SpaceGrantee},"
#. Grouped email template, resolves via {{ .Greeting }}
#: pkg/email/templates.go:131
msgid "Hi {DisplayName},"
msgstr "Hei {DisplayName},"
#. ScienceMeshInviteTokenGenerated email template, resolves via {{ .Greeting
#. }}
#. ScienceMeshInviteTokenGeneratedWithoutShareLink email template, resolves
#. via {{ .Greeting }}
#: pkg/email/templates.go:100 pkg/email/templates.go:117
msgid "Hi,"
msgstr "Hei,"
#. MembershipExpired email template, Subject field (resolves directly)
#: pkg/email/templates.go:85
msgid "Membership of '{SpaceName}' expired at {ExpiredAt}"
msgstr "Medlemskap i '{SpaceName}' utløpt {ExpiredAt}"
#. Grouped email template, Subject field (resolves directly)
#: pkg/email/templates.go:129
msgid "Report"
msgstr "Rapport"
#. ScienceMeshInviteTokenGenerated email template, Subject field (resolves
#. directly)
#. ScienceMeshInviteTokenGeneratedWithoutShareLink email template, Subject
#. field (resolves directly)
#: pkg/email/templates.go:98 pkg/email/templates.go:115
msgid "ScienceMesh: {InitiatorName} wants to collaborate with you"
msgstr "ScienceMesh: {InitiatorName} ønsker å samarbeide med deg"
#. ShareExpired email template, Subject field (resolves directly)
#: pkg/email/templates.go:43
msgid "Share to '{ShareFolder}' expired at {ExpiredAt}"
msgstr "Deling til '{ShareFolder}' utløpt {ExpiredAt}"
#: pkg/email/templates.go:141
msgid "You can view the mention here: {ResourceLink}"
msgstr "Du kan se omtalen her: {ResourceLink}"
#: pkg/email/templates.go:138
msgid "You were mentioned in '{ResourceName}'"
msgstr "Du ble nevnt i «{ResourceName}»"
#. MembershipExpired email template, resolves via {{ .MessageBody }}
#: pkg/email/templates.go:89
msgid ""
"Your membership of space {SpaceName} has expired at {ExpiredAt}\n"
"\n"
"Even though this membership has expired you still might have access through other shares and/or space memberships"
msgstr ""
"Medlemskapet ditt i rom {SpaceName} utløp {ExpiredAt}\n"
"\n"
"Selv om dette medlemskapet har utløpt, kan du fortsatt ha tilgang gjennom andre delinger og/eller rommedlemskap"
#. ShareExpired email template, resolves via {{ .MessageBody }}
#: pkg/email/templates.go:47
msgid ""
"Your share to {ShareFolder} has expired at {ExpiredAt}\n"
"\n"
"Even though this share has been revoked you still might have access through other shares and/or space memberships."
msgstr ""
"Deltakelsen din i {ShareFolder} utløp {ExpiredAt}\n"
"\n"
"Selv om denne deltakelsen er trukket tilbake, kan du fortsatt ha tilgang gjennom andre deltakelser og/eller plassmedlemskap."
#: pkg/email/templates.go:140
msgid "{AuthorName} mentioned you in \"{ResourceName}\"."
msgstr "{AuthorName} nevnte deg i «{ResourceName}»."
#. ScienceMeshInviteTokenGeneratedWithoutShareLink email template, resolves
#. via {{ .MessageBody }}
#: pkg/email/templates.go:119
msgid ""
"{ShareSharer} ({ShareSharerMail}) wants to start sharing collaboration resources with you.\n"
"Please visit your federation settings and use the following details:\n"
" Token: {Token}\n"
" ProviderDomain: {ProviderDomain}"
msgstr ""
"{ShareSharer} ({ShareSharerMail}) ønsker å begynne å dele samarbeidsressurser med deg.\n"
"Vennligst besøk føderasjonsinnstillingene dine og bruk følgende detaljer:\n"
" Token: {Token}\n"
" ProviderDomain: {ProviderDomain}"
#. ScienceMeshInviteTokenGenerated email template, resolves via {{
#. .MessageBody }}
#: pkg/email/templates.go:102
msgid ""
"{ShareSharer} ({ShareSharerMail}) wants to start sharing collaboration resources with you.\n"
"To accept the invite, please visit the following URL:\n"
"{ShareLink}\n"
"\n"
"Alternatively, you can visit your federation settings and use the following details:\n"
" Token: {Token}\n"
" ProviderDomain: {ProviderDomain}"
msgstr ""
"{ShareSharer} ({ShareSharerMail}) ønsker å begynne å dele samarbeidsressurser med deg.\n"
"For å akseptere invitasjonen, besøk følgende URL:\n"
"{ShareLink}\n"
"\n"
"Alternativt kan du besøke dine føderasjonsinnstillinger og bruke følgende detaljer:\n"
" Token: {Token}\n"
" ProviderDomain: {ProviderDomain}"
#. ShareCreated email template, resolves via {{ .MessageBody }}
#: pkg/email/templates.go:21
msgid "{ShareSharer} has shared \"{ShareFolder}\" with you."
msgstr "{ShareSharer} har delt «{ShareFolder}» med deg."
#. ShareRemoved email template, resolves via {{ .MessageBody }}
#: pkg/email/templates.go:34
msgid ""
"{ShareSharer} has unshared '{ShareFolder}' with you.\n"
"\n"
"Even though this share has been revoked you still might have access through other shares and/or space memberships."
msgstr ""
"{ShareSharer} har fjernet deling '{ShareFolder}' til deg.\n"
"\n"
"Selv om denne delingen er trukket tilbake, kan du fortsatt ha tilgang gjennom andre delinger og/eller medlemskap i rommet."
#. ShareCreated email template, Subject field (resolves directly)
#: pkg/email/templates.go:17
msgid "{ShareSharer} shared '{ShareFolder}' with you"
msgstr "{ShareSharer} delte '{ShareFolder}' med deg"
#. ShareRemoved email template, Subject field (resolves directly)
#: pkg/email/templates.go:30
msgid "{ShareSharer} unshared '{ShareFolder}' with you"
msgstr "{ShareSharer} deler ikke '{ShareFolder}' med deg"
#. SharedSpace email template, resolves via {{ .MessageBody }}
#: pkg/email/templates.go:61
msgid "{SpaceSharer} has invited you to join \"{SpaceName}\"."
msgstr "{SpaceSharer} har invitert deg til å bli med i «{SpaceName}»."
#. UnsharedSpace email template, resolves via {{ .MessageBody }}
#: pkg/email/templates.go:74
msgid ""
"{SpaceSharer} has removed you from \"{SpaceName}\".\n"
"\n"
"You might still have access through your other groups or direct membership."
msgstr ""
"{SpaceSharer} har fjernet deg fra «{SpaceName}».\n"
"\n"
"Du kan fortsatt ha tilgang gjennom andre grupper eller direkte medlemskap."
#. SharedSpace email template, Subject field (resolves directly)
#: pkg/email/templates.go:57
msgid "{SpaceSharer} invited you to join {SpaceName}"
msgstr "{SpaceSharer} inviterte deg til å bli med i {SpaceName}"
#. UnsharedSpace email template, Subject field (resolves directly)
#: pkg/email/templates.go:70
msgid "{SpaceSharer} removed you from {SpaceName}"
msgstr "{SpaceSharer} fjernet deg fra {SpaceName}"
+2 -2
View File
@@ -29,7 +29,7 @@ type Service struct {
// GRPC defines the available grpc configuration.
type GRPC struct {
Disabled bool `yaml:"disabled" env:"POLICIES_GRPC_DISABLED" desc:"Disables listening for GRPC API calls. Set this to true if the service should only handle requests through events." introductionVersion:"%NEXT%"`
Disabled bool `yaml:"disabled" env:"POLICIES_GRPC_DISABLED" desc:"Disables listening for GRPC API calls. Set this to true if the service should only handle requests through events." introductionVersion:"%%NEXT%%"`
Addr string `yaml:"addr" env:"POLICIES_GRPC_ADDR" desc:"The bind address of the GRPC service." introductionVersion:"1.0.0"`
Namespace string `yaml:"-"`
TLS *shared.GRPCServiceTLS `yaml:"tls"`
@@ -50,7 +50,7 @@ type Postprocessing struct {
// Events combines the configuration options for the event bus.
type Events struct {
Disabled bool `yaml:"disabled" env:"POLICIES_EVENTS_DISABLED" desc:"Disables listening for events. Set this to true if the service should only handle GRPC requests." introductionVersion:"%NEXT%"`
Disabled bool `yaml:"disabled" env:"POLICIES_EVENTS_DISABLED" desc:"Disables listening for events. Set this to true if the service should only handle GRPC requests." introductionVersion:"%%NEXT%%"`
Endpoint string `yaml:"endpoint" env:"OC_EVENTS_ENDPOINT;POLICIES_EVENTS_ENDPOINT" desc:"The address of the event system. The event system is the message queuing service. It is used as message broker for the microservice architecture." introductionVersion:"1.0.0"`
Cluster string `yaml:"cluster" env:"OC_EVENTS_CLUSTER;POLICIES_EVENTS_CLUSTER" desc:"The clusterID of the event system. The event system is the message queuing service. It is used as message broker for the microservice architecture. Mandatory when using NATS as event system." introductionVersion:"1.0.0"`
TLSInsecure bool `yaml:"tls_insecure" env:"OC_INSECURE;OC_EVENTS_TLS_INSECURE;POLICIES_EVENTS_TLS_INSECURE" desc:"Whether the server should skip the client certificate verification during the TLS handshake." introductionVersion:"1.0.0"`
+3 -6
View File
@@ -4,6 +4,7 @@ import (
"errors"
occfg "github.com/opencloud-eu/opencloud/pkg/config"
"github.com/opencloud-eu/opencloud/pkg/shared"
"github.com/opencloud-eu/opencloud/services/policies/pkg/config"
"github.com/opencloud-eu/opencloud/services/policies/pkg/config/defaults"
@@ -33,12 +34,8 @@ func ParseConfig(cfg *config.Config) error {
}
func Validate(cfg *config.Config) error {
if cfg.GRPC.Disabled && cfg.Events.Disabled {
// might be debatable, but this situation should be treated as an error,
// as the process wouldn't be able to serve either API and would thus be
// completely useless -- in that case, just don't start this service
// in the first place (especially since it's optional)
return errors.New("both gRPC and events APIs are disabled by configuration; at least one must be enabled")
if cfg.Events.Disabled && cfg.GRPC.Disabled {
return shared.AllComponentsDisabledError(cfg.Service.Name)
}
return nil
}
+19 -1
View File
@@ -8,6 +8,7 @@ import (
"strings"
"time"
"github.com/golang-jwt/jwt/v5"
"github.com/pkg/errors"
"github.com/vmihailenco/msgpack/v5"
"go-micro.dev/v4/store"
@@ -186,6 +187,13 @@ func verifyExpiresAt(claims map[string]any, cmp time.Time) bool {
return cmp.Before(expiry)
}
// isExpectedTokenTimeError reports whether the error is caused by a token that
// failed its expiry (or not-before) validation. These are expected as part of
// the normal token renewal cycle and should not be logged at error level.
func isExpectedTokenTimeError(err error) bool {
return errors.Is(err, jwt.ErrTokenExpired) || errors.Is(err, jwt.ErrTokenNotValidYet)
}
func (m OIDCAuthenticator) shouldServe(req *http.Request) bool {
if m.OIDCIss == "" {
return false
@@ -212,7 +220,17 @@ func (m *OIDCAuthenticator) Authenticate(r *http.Request) (*http.Request, bool)
claims, newSession, err := m.getClaims(token, r)
if err != nil {
host, port, _ := net.SplitHostPort(r.RemoteAddr)
m.Logger.Error().
// Expired (or not yet valid) tokens are a normal part of the token
// renewal cycle: the client hits the proxy with a token that is just
// past its expiry, receives a 401 challenge and refreshes. This is not
// a failure of the proxy, so we log it at debug level to avoid noise.
logEvent := m.Logger.Debug()
if !isExpectedTokenTimeError(err) {
logEvent = m.Logger.Error()
}
logEvent.
Err(err).
Str("authenticator", "oidc").
Str("path", r.URL.Path).
@@ -1,6 +1,7 @@
package middleware
import (
"bytes"
"net/http"
"net/http/httptest"
"time"
@@ -11,6 +12,7 @@ import (
"github.com/opencloud-eu/opencloud/pkg/log"
"github.com/opencloud-eu/opencloud/pkg/oidc"
oidcmocks "github.com/opencloud-eu/opencloud/pkg/oidc/mocks"
"github.com/rs/zerolog"
"github.com/stretchr/testify/mock"
"go-micro.dev/v4/store"
)
@@ -106,4 +108,69 @@ var _ = Describe("Authenticating requests", Label("OIDCAuthenticator"), func() {
Expect(req2).ToNot(BeNil())
})
})
When("the request contains an expired token", func() {
var buf bytes.Buffer
expiredAuthenticator := &OIDCAuthenticator{
OIDCIss: "http://idp.example.com",
Logger: log.Logger{zerolog.New(&buf).Level(zerolog.DebugLevel)},
oidcClient: &oidcmocks.OIDCClient{},
userInfoCache: store.NewMemoryStore(),
skipUserInfo: true,
}
BeforeEach(func() {
expiredAuthenticator.oidcClient.(*oidcmocks.OIDCClient).On("VerifyAccessToken", mock.Anything, mock.Anything).Return(
oidc.RegClaimsWithSID{}, jwt.MapClaims{}, jwt.ErrTokenExpired,
)
})
It("should reject the request", func() {
req := httptest.NewRequest(http.MethodGet, "http://example.com/example/path", http.NoBody)
req.Header.Set(_headerAuthorization, "Bearer jwt.token.sig")
req2, valid := expiredAuthenticator.Authenticate(req)
Expect(valid).To(Equal(false))
Expect(req2).To(BeNil())
})
It("should log the authentication failure at debug level", func() {
req := httptest.NewRequest(http.MethodGet, "http://example.com/example/path", http.NoBody)
req.Header.Set(_headerAuthorization, "Bearer jwt.token.sig")
expiredAuthenticator.Authenticate(req)
Expect(buf.String()).To(ContainSubstring("failed to authenticate the request"))
Expect(buf.String()).To(ContainSubstring(`"level":"debug"`))
Expect(buf.String()).ToNot(ContainSubstring(`"level":"error"`))
})
})
When("the request contains a token that cannot be verified", func() {
var buf bytes.Buffer
failingAuthenticator := &OIDCAuthenticator{
OIDCIss: "http://idp.example.com",
Logger: log.Logger{zerolog.New(&buf).Level(zerolog.DebugLevel)},
oidcClient: &oidcmocks.OIDCClient{},
userInfoCache: store.NewMemoryStore(),
skipUserInfo: true,
}
BeforeEach(func() {
failingAuthenticator.oidcClient.(*oidcmocks.OIDCClient).On("VerifyAccessToken", mock.Anything, mock.Anything).Return(
oidc.RegClaimsWithSID{}, jwt.MapClaims{}, jwt.ErrTokenMalformed,
)
})
It("should log the authentication failure at error level", func() {
req := httptest.NewRequest(http.MethodGet, "http://example.com/example/path", http.NoBody)
req.Header.Set(_headerAuthorization, "Bearer jwt.token.sig")
failingAuthenticator.Authenticate(req)
Expect(buf.String()).To(ContainSubstring("failed to authenticate the request"))
Expect(buf.String()).To(ContainSubstring(`"level":"error"`))
})
})
})
@@ -236,6 +236,16 @@ func (m SignedURLAuthenticator) Authenticate(r *http.Request) (*http.Request, bo
}
func (m SignedURLAuthenticator) authenticate(r *http.Request) (*http.Request, bool) {
if err := m.requestMethodIsAllowed(r.Method); err != nil {
m.Logger.Error().
Err(err).
Str("authenticator", "signed_url_jwt").
Str("path", r.URL.Path).
Str("method", r.Method).
Msg("Request method not allowed for signed urls")
return nil, false
}
u := r.URL.String()
if !r.URL.IsAbs() {
u = "https://" + r.Host + u
@@ -2,11 +2,13 @@ package middleware
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
userpb "github.com/cs3org/go-cs3apis/cs3/identity/user/v1beta1"
"github.com/opencloud-eu/opencloud/pkg/log"
"github.com/opencloud-eu/opencloud/services/proxy/pkg/config"
revactx "github.com/opencloud-eu/reva/v2/pkg/ctx"
"github.com/opencloud-eu/reva/v2/pkg/signedurl"
@@ -71,6 +73,27 @@ func TestSignedURLAuth_shouldServe(t *testing.T) {
}
}
func TestSignedURLAuth_authenticateRejectsDisallowedMethods(t *testing.T) {
signer, err := signedurl.NewJWTSignedURL(signedurl.WithSecret("secret"))
if err != nil {
t.Fatalf("failed to create signer: %v", err)
}
signed, err := signer.Sign("https://example.com/dav/spaces/file.txt", "userid", time.Minute)
if err != nil {
t.Fatalf("failed to sign url: %v", err)
}
pua := SignedURLAuthenticator{Logger: log.NewLogger(), URLVerifier: signer}
pua.PreSignedURLConfig.AllowedHTTPMethods = []string{"GET"}
for _, method := range []string{http.MethodPost, http.MethodPut, http.MethodDelete, "PROPFIND", "MOVE"} {
r := httptest.NewRequest(method, signed, nil)
if _, ok := pua.authenticate(r); ok {
t.Errorf("expected %s with a signed url to be rejected", method)
}
}
}
func TestSignedURLAuth_allRequiredParametersPresent(t *testing.T) {
pua := SignedURLAuthenticator{}
baseURL := "https://example.com/example.jpg?"
@@ -34,6 +34,10 @@ func ParseConfig(cfg *config.Config) error {
}
func Validate(cfg *config.Config) error {
if cfg.Events.Disabled && cfg.GRPC.Disabled {
return shared.AllComponentsDisabledError(cfg.Service.Name)
}
if cfg.TokenManager.JWTSecret == "" {
return shared.MissingJWTTokenError(cfg.Service.Name)
}
+15 -31
View File
@@ -82,6 +82,9 @@ func (t Tika) Extract(ctx context.Context, ri *provider.ResourceInfo) (Document,
if err != nil {
return doc, err
}
if len(metas) == 0 {
return doc, nil
}
for _, meta := range metas {
title, err := getFirstValue(meta, "dc:title")
@@ -98,40 +101,21 @@ func (t Tika) Extract(ctx context.Context, ri *provider.ResourceInfo) (Document,
} else if content, err := getFirstValue(meta, "X-TIKA:content"); err == nil {
doc.Content = strings.TrimSpace(fmt.Sprintf("%s %s", doc.Content, content))
}
// keep facets from earlier entries, an embedded resource's meta
// (e.g. cover art) must not reset them
if v := t.getLocation(meta); v != nil {
doc.Location = v
}
if v := t.getImage(meta); v != nil {
doc.Image = v
}
if v := t.getPhoto(meta); v != nil {
doc.Photo = v
}
if v := t.getAudio(meta); v != nil {
doc.Audio = v
}
if v := t.getLivePhoto(meta); v != nil {
doc.LivePhoto = v
}
}
if len(metas) > 0 {
// the video facet says the file is a video, so it comes from the file
// itself: the clip tika extracts from a motion photo must not make its
// image look like one
doc.Video = t.getVideo(metas[0])
}
// facets describe the file itself, not its embedded parts (cover art, clips)
m0 := metas[0]
doc.Location = t.getLocation(m0)
doc.Image = t.getImage(m0)
doc.Photo = t.getPhoto(m0)
doc.Audio = t.getAudio(m0)
doc.LivePhoto = t.getLivePhoto(m0)
doc.Video = t.getVideo(m0)
// a motion photo is the xmp on the file itself plus the video tika extracted
// from it. The xmp alone proves nothing: a share can keep it and strip the
// appended video.
if len(metas) > 0 {
if i := slices.IndexFunc(metas[1:], isVideo); i >= 0 {
doc.MotionPhoto = t.getMotionPhoto(metas[0], metas[i+1])
}
// a motion photo is the file's own xmp plus the video tika extracted from
// it; the xmp alone proves nothing, a share can strip the appended clip
if i := slices.IndexFunc(metas[1:], isVideo); i >= 0 {
doc.MotionPhoto = t.getMotionPhoto(m0, metas[i+1])
}
if langCode := t.detectLanguage(ctx, doc.Content); langCode != "" && t.CleanStopWords {
+4 -2
View File
@@ -170,7 +170,9 @@ var _ = Describe("Tika", func() {
Expect(doc.Content).To(Equal("body test stop words!!!"))
})
It("keeps the audio facet when an embedded resource follows", func() {
It("takes facets from the main document, not from an embedded resource", func() {
// metas[0] is the file (audio), metas[1] its embedded cover art. The
// cover must not give the track an image facet.
fullResponse = `[{"Content-Type": "audio/mpeg", "dc:title": "Sucker", "tk:content": "lyrics"}, {"Content-Type": "image/jpeg", "tiff:ImageWidth": "500"}]`
doc, err := tika.Extract(context.TODO(), &provider.ResourceInfo{
@@ -180,7 +182,7 @@ var _ = Describe("Tika", func() {
Expect(err).ToNot(HaveOccurred())
Expect(doc.Audio).ToNot(BeNil())
Expect(doc.Audio.Title).To(Equal(libregraph.PtrString("Sucker")))
Expect(doc.Image).ToNot(BeNil())
Expect(doc.Image).To(BeNil())
})
It("adds no audio facet to non-audio documents", func() {
@@ -11,7 +11,7 @@ msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: EMAIL\n"
"POT-Creation-Date: 2026-08-17 23:18+0000\n"
"POT-Creation-Date: 2026-09-07 23:16+0000\n"
"PO-Revision-Date: 2025-01-27 10:17+0000\n"
"Last-Translator: Jiri Grönroos <jiri.gronroos@iki.fi>, 2026\n"
"Language-Team: Finnish (https://app.transifex.com/opencloud-eu/teams/204053/fi/)\n"
+6 -1
View File
@@ -35,7 +35,12 @@ Share behavior can be configured via environment variables:
- Auto-acceptance of shares
- Share permissions and restrictions
See the `frontend` service README for more details on share-related configuration options.
### Auto-Accept Shares
When setting the `SHARING_AUTO_ACCEPT_SHARES` to `true` (sharing service), all
incoming shares will be accepted automatically. Users can overwrite this
setting individually in their profile. The deprecated
`FRONTEND_AUTO_ACCEPT_SHARES` is still supported for backwards compatibility.
## Scalability
@@ -0,0 +1,105 @@
//go:build !enable_vips
package preprocessor
import (
"bytes"
"encoding/base64"
"encoding/json"
"image"
"image/jpeg"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
thumbnailerErrors "github.com/opencloud-eu/opencloud/services/thumbnails/pkg/errors"
)
// craftDimensionBomb encodes a tiny valid grayscale JPEG, then overwrites the
// SOF0 width/height so the header declares huge dimensions while the payload
// stays tiny.
func craftDimensionBomb(width, height uint16) []byte {
var buf bytes.Buffer
Expect(jpeg.Encode(&buf, image.NewGray(image.Rect(0, 0, 8, 8)), &jpeg.Options{Quality: 10})).To(Succeed())
b := buf.Bytes()
for i := 0; i+9 < len(b); i++ {
if b[i] == 0xff && b[i+1] == 0xc0 {
b[i+5], b[i+6] = byte(height>>8), byte(height)
b[i+7], b[i+8] = byte(width>>8), byte(width)
return b
}
}
Fail("no SOF0 marker in encoded jpeg")
return nil
}
var _ = Describe("ImageDecoder dimension guard", func() {
It("rejects a source whose declared dimensions exceed the limit before decoding", func() {
dec := ImageDecoder{limit: decodeLimit{maxWidth: 7680, maxHeight: 7680}}
_, err := dec.Convert(bytes.NewReader(craftDimensionBomb(20000, 20000)))
Expect(err).To(MatchError(thumbnailerErrors.ErrImageTooLarge))
})
It("decodes an image within the limit", func() {
var buf bytes.Buffer
Expect(jpeg.Encode(&buf, image.NewGray(image.Rect(0, 0, 800, 600)), nil)).To(Succeed())
dec := ImageDecoder{limit: decodeLimit{maxWidth: 7680, maxHeight: 7680}}
img, err := dec.Convert(bytes.NewReader(buf.Bytes()))
Expect(err).ToNot(HaveOccurred())
Expect(img).ToNot(BeNil())
})
It("applies no limit when the bounds are zero", func() {
dec := ImageDecoder{}
var buf bytes.Buffer
Expect(jpeg.Encode(&buf, image.NewGray(image.Rect(0, 0, 16, 16)), nil)).To(Succeed())
_, err := dec.Convert(bytes.NewReader(buf.Bytes()))
Expect(err).ToNot(HaveOccurred())
})
It("rejects an oversized gif before decoding", func() {
dec := GifDecoder{limit: decodeLimit{maxWidth: 7680, maxHeight: 7680}}
// LSD declares a 20000x20000 logical screen
g := []byte("GIF89a")
g = append(g, 0x20, 0x4e, 0x20, 0x4e, 0xf0, 0x00, 0x00) // 20000x20000, gct flag
g = append(g, bytes.Repeat([]byte{0}, 6)...) // minimal gct + terminator-ish
_, err := dec.Convert(bytes.NewReader(g))
Expect(err).To(MatchError(thumbnailerErrors.ErrImageTooLarge))
})
})
var _ = Describe("dimension limit propagation", func() {
limit := decodeLimit{maxWidth: 7680, maxHeight: 7680}
opts := map[string]any{"maxInputWidth": 7680, "maxInputHeight": 7680}
It("threads the limit into decoders that recurse into ForType", func() {
Expect(ForType("audio/mpeg", opts)).To(Equal(AudioDecoder{limit: limit}))
Expect(ForType("application/vnd.geogebra.pinboard", opts)).To(Equal(GgpDecoder{limit: limit}))
g, ok := ForType("application/vnd.geogebra.slides", opts).(GgsDecoder)
Expect(ok).To(BeTrue())
Expect(g.limit).To(Equal(limit))
})
It("rejects an oversized cover image embedded in a ggp file", func() {
bomb := craftDimensionBomb(20000, 20000)
payload := "data:image/jpeg;base64," + base64.StdEncoding.EncodeToString(bomb)
ggp := GGPStruct{}
ggp.Sections = append(ggp.Sections, struct {
Cards []struct {
Element struct {
Image struct{ Base64Image string }
}
}
}{Cards: []struct {
Element struct {
Image struct{ Base64Image string }
}
}{{Element: struct {
Image struct{ Base64Image string }
}{Image: struct{ Base64Image string }{Base64Image: payload}}}}})
raw, err := json.Marshal(ggp)
Expect(err).ToNot(HaveOccurred())
_, err = GgpDecoder{limit: limit}.Convert(bytes.NewReader(raw))
Expect(err).To(MatchError(thumbnailerErrors.ErrImageTooLarge))
})
})
@@ -30,10 +30,14 @@ type FileConverter interface {
}
// GifDecoder is a converter for the gif file
type GifDecoder struct{}
type GifDecoder struct{ limit decodeLimit }
// Convert reads the gif file and returns the thumbnail image
func (i GifDecoder) Convert(r io.Reader) (any, error) {
r, err := i.limit.guardDimensions(r, gif.DecodeConfig)
if err != nil {
return nil, err
}
img, err := gif.DecodeAll(r)
if err != nil {
return nil, errors.Wrap(err, `could not decode the image`)
@@ -42,7 +46,10 @@ func (i GifDecoder) Convert(r io.Reader) (any, error) {
}
// GgsDecoder is a converter for the geogebra slides file
type GgsDecoder struct{ thumbnailpath string }
type GgsDecoder struct {
thumbnailpath string
limit decodeLimit
}
// Convert reads the ggs file and returns the thumbnail image
func (g GgsDecoder) Convert(r io.Reader) (any, error) {
@@ -61,7 +68,7 @@ func (g GgsDecoder) Convert(r io.Reader) (any, error) {
if err != nil {
return nil, err
}
converter := ForType("image/png", nil)
converter := ForType("image/png", g.limit.opts())
if converter == nil {
return nil, thumbnailerErrors.ErrNoConverterForExtractedImageFromGgsFile
}
@@ -76,7 +83,7 @@ func (g GgsDecoder) Convert(r io.Reader) (any, error) {
}
// AudioDecoder is a converter for the audio file
type AudioDecoder struct{}
type AudioDecoder struct{ limit decodeLimit }
// Convert reads the audio file and extracts the thumbnail image from the id3 tag
func (i AudioDecoder) Convert(r io.Reader) (any, error) {
@@ -94,7 +101,7 @@ func (i AudioDecoder) Convert(r io.Reader) (any, error) {
return nil, thumbnailerErrors.ErrNoImageFromAudioFile
}
converter := ForType(picture.MIMEType, nil)
converter := ForType(picture.MIMEType, i.limit.opts())
if converter == nil {
return nil, thumbnailerErrors.ErrNoConverterForExtractedImageFromAudioFile
}
@@ -200,7 +207,7 @@ type GGPStruct struct {
}
// GgpDecoder is a converter for the geogebra pinboard file
type GgpDecoder struct{}
type GgpDecoder struct{ limit decodeLimit }
// Convert reads the ggp file and returns the first thumbnail image
func (j GgpDecoder) Convert(r io.Reader) (any, error) {
@@ -220,7 +227,14 @@ func (j GgpDecoder) Convert(r io.Reader) (any, error) {
return nil, err
}
img, _, err := image.Decode(bytes.NewReader(b))
r2, err := j.limit.guardDimensions(bytes.NewReader(b), func(rr io.Reader) (image.Config, error) {
cfg, _, err := image.DecodeConfig(rr)
return cfg, err
})
if err != nil {
return nil, err
}
img, _, err := image.Decode(r2)
return img, err
}
@@ -297,11 +311,61 @@ func drawWord(canvas *font.Drawer, word string, minX, maxX, incY, maxY fixed.Int
}
}
// decodeLimit bounds the source image dimensions a decoder accepts. A zero
// value on an axis disables the limit for that axis.
type decodeLimit struct {
maxWidth int
maxHeight int
}
func decodeLimitFromOpts(opts map[string]any) decodeLimit {
l := decodeLimit{}
if v, ok := opts["maxInputWidth"].(int); ok {
l.maxWidth = v
}
if v, ok := opts["maxInputHeight"].(int); ok {
l.maxHeight = v
}
return l
}
func (l decodeLimit) exceeded(width, height int) bool {
return (l.maxWidth > 0 && width > l.maxWidth) || (l.maxHeight > 0 && height > l.maxHeight)
}
// opts renders the limit back into an options map so decoders that recurse
// into ForType can forward it to the nested image decoder.
func (l decodeLimit) opts() map[string]any {
return map[string]any{"maxInputWidth": l.maxWidth, "maxInputHeight": l.maxHeight}
}
// guardDimensions reads only the image header (no pixel allocation) and
// rejects a source whose declared dimensions exceed the limit, before the
// full bitmap is decoded. It returns a reader that replays the consumed
// header so the caller can still decode from the start. A header that cannot
// be parsed is passed through unchecked, letting the real decoder report it.
func (l decodeLimit) guardDimensions(r io.Reader, config func(io.Reader) (image.Config, error)) (io.Reader, error) {
if l.maxWidth <= 0 && l.maxHeight <= 0 {
return r, nil
}
var head bytes.Buffer
cfg, err := config(io.TeeReader(r, &head))
replay := io.MultiReader(&head, r)
if err != nil {
return replay, nil
}
if l.exceeded(cfg.Width, cfg.Height) {
return nil, thumbnailerErrors.ErrImageTooLarge
}
return replay, nil
}
// ForType returns the converter for the specified mimeType
func ForType(mimeType string, opts map[string]any) FileConverter {
// We can ignore the error here because we parse it in IsMimeTypeSupported before and if it fails
// return the service call. So we should only get here when the mimeType parses fine.
mimeType, _, _ = mime.ParseMediaType(mimeType)
limit := decodeLimitFromOpts(opts)
switch mimeType {
case "text/plain":
fontFileMap := ""
@@ -333,18 +397,18 @@ func ForType(mimeType string, opts map[string]any) FileConverter {
fontLoader: fontLoader,
}
case "application/vnd.geogebra.slides":
return GgsDecoder{"_slide0/geogebra_thumbnail.png"}
return GgsDecoder{thumbnailpath: "_slide0/geogebra_thumbnail.png", limit: limit}
case "application/vnd.geogebra.pinboard":
return GgpDecoder{}
return GgpDecoder{limit: limit}
case "image/gif":
return GifDecoder{}
return GifDecoder{limit: limit}
case "audio/flac":
fallthrough
case "audio/mpeg":
fallthrough
case "audio/ogg":
return AudioDecoder{}
return AudioDecoder{limit: limit}
default:
return ImageDecoder{}
return ImageDecoder{limit: limit}
}
}
@@ -3,6 +3,7 @@
package preprocessor
import (
"image"
"io"
"github.com/kovidgoyal/imaging"
@@ -10,10 +11,20 @@ import (
)
// ImageDecoder is a converter for the image file
type ImageDecoder struct{}
type ImageDecoder struct{ limit decodeLimit }
// Convert reads the image file and returns the thumbnail image
func (i ImageDecoder) Convert(r io.Reader) (any, error) {
// bound the declared dimensions before imaging.Decode allocates the full
// pixel buffer: a crafted header (e.g. 65535x65535) would otherwise OOM
// the worker, the downstream dimension guard only runs after the decode
r, err := i.limit.guardDimensions(r, func(rr io.Reader) (image.Config, error) {
cfg, _, err := image.DecodeConfig(rr)
return cfg, err
})
if err != nil {
return nil, err
}
img, err := imaging.Decode(r, imaging.AutoOrientation(true))
if err != nil {
return nil, errors.Wrap(err, `could not decode the image`)
@@ -81,14 +81,14 @@ var _ = Describe("ImageDecoder", func() {
})
It("should decode a ggs", func() {
decoder := GgsDecoder{"_slide0/geogebra_thumbnail.png"}
decoder := GgsDecoder{thumbnailpath: "_slide0/geogebra_thumbnail.png"}
img, err := decoder.Convert(fileReader)
Expect(err).ToNot(HaveOccurred())
Expect(img).ToNot(BeNil())
})
It("should return an error if the ggs is invalid", func() {
decoder := GgsDecoder{"_slide0/geogebra_thumbnail.png"}
decoder := GgsDecoder{thumbnailpath: "_slide0/geogebra_thumbnail.png"}
img, err := decoder.Convert(bytes.NewReader([]byte("not a ggs")))
Expect(err).To(HaveOccurred())
Expect(img).To(BeNil())
@@ -6,15 +6,26 @@ import (
"io"
"github.com/davidbyttow/govips/v2/vips"
thumbnailerErrors "github.com/opencloud-eu/opencloud/services/thumbnails/pkg/errors"
)
func init() {
vips.LoggingSettings(nil, vips.LogLevelError)
}
type ImageDecoder struct{}
type ImageDecoder struct{ limit decodeLimit }
func (v ImageDecoder) Convert(r io.Reader) (interface{}, error) {
// NewImageFromReader is header-lazy, Width/Height read the header without
// materializing pixels: reject oversized sources before ThumbnailWithSize
img, err := vips.NewImageFromReader(r)
return img, err
if err != nil {
return nil, err
}
if v.limit.exceeded(img.Width(), img.Height()) {
img.Close()
return nil, thumbnailerErrors.ErrImageTooLarge
}
return img, nil
}
@@ -54,6 +54,8 @@ func NewService(opts ...Option) decorators.DecoratedService {
selector: options.GatewaySelector,
preprocessorOpts: PreprocessorOpts{
TxtFontFileMap: options.Config.Thumbnail.FontMapFile,
MaxInputWidth: options.Config.Thumbnail.MaxInputWidth,
MaxInputHeight: options.Config.Thumbnail.MaxInputHeight,
},
dataEndpoint: options.Config.Thumbnail.DataEndpoint,
transferSecret: options.Config.Thumbnail.TransferSecret,
@@ -78,6 +80,8 @@ type Thumbnail struct {
// PreprocessorOpts holds the options for the preprocessor
type PreprocessorOpts struct {
TxtFontFileMap string
MaxInputWidth int
MaxInputHeight int
}
// GetThumbnail retrieves a thumbnail for an image
@@ -166,10 +170,15 @@ func (g Thumbnail) handleCS3Source(ctx context.Context, req *thumbnailssvc.GetTh
defer r.Close()
ppOpts := map[string]any{
"fontFileMap": g.preprocessorOpts.TxtFontFileMap,
"fontFileMap": g.preprocessorOpts.TxtFontFileMap,
"maxInputWidth": g.preprocessorOpts.MaxInputWidth,
"maxInputHeight": g.preprocessorOpts.MaxInputHeight,
}
pp := preprocessor.ForType(sRes.GetInfo().GetMimeType(), ppOpts)
img, err := pp.Convert(r)
if errors.Is(err, terrors.ErrImageTooLarge) {
return "", merrors.Forbidden(g.serviceID, "%s", err.Error())
}
if err != nil {
g.logger.Error().Err(err).Msg("failed to convert image")
}
@@ -262,10 +271,15 @@ func (g Thumbnail) handleWebdavSource(ctx context.Context, req *thumbnailssvc.Ge
}
defer r.Close()
ppOpts := map[string]any{
"fontFileMap": g.preprocessorOpts.TxtFontFileMap,
"fontFileMap": g.preprocessorOpts.TxtFontFileMap,
"maxInputWidth": g.preprocessorOpts.MaxInputWidth,
"maxInputHeight": g.preprocessorOpts.MaxInputHeight,
}
pp := preprocessor.ForType(sRes.GetInfo().GetMimeType(), ppOpts)
img, err := pp.Convert(r)
if errors.Is(err, terrors.ErrImageTooLarge) {
return "", merrors.Forbidden(g.serviceID, "%s", err.Error())
}
if img == nil || err != nil {
return "", merrors.NotFound(g.serviceID, "could not get image")
}
@@ -0,0 +1,129 @@
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
# Translators:
# Roy-Magne Mo, 2026
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: EMAIL\n"
"POT-Creation-Date: 2026-09-07 23:16+0000\n"
"PO-Revision-Date: 2025-01-27 10:17+0000\n"
"Last-Translator: Roy-Magne Mo, 2026\n"
"Language-Team: Norwegian (https://app.transifex.com/opencloud-eu/teams/204053/no/)\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Language: no\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
#: pkg/service/templates.go:39
msgid "Access to Space {space} lost"
msgstr "Tilgang til rom {space} mistet"
#: pkg/service/templates.go:59
msgid "Access to {resource} expired"
msgstr "Tilgang til {resource} utløpt"
#: pkg/service/templates.go:64
msgid ""
"Attention! The instance will be shut down and deprovisioned on {date}. "
"Download all your data before that date as no access past that date is "
"possible."
msgstr ""
"Merk! Instansen vil bli slått av og avinstallert {date}. Last ned all data "
"før denne datoen, da ingen tilgang er mulig etter denne datoen."
#: pkg/service/templates.go:63
msgid "Instance will be shut down and deprovisioned"
msgstr "Instansen vil bli slått av og fjernet"
#: pkg/service/templates.go:38
msgid "Membership expired"
msgstr "Medlemskapet utløpt"
#: pkg/service/templates.go:13
msgid "Policies enforced"
msgstr "Retningslinjer håndhevet"
#: pkg/service/templates.go:23
msgid "Removed from Space"
msgstr "Fjernet fra rom"
#: pkg/service/templates.go:43
msgid "Resource shared"
msgstr "Ressurs delt"
#: pkg/service/templates.go:48
msgid "Resource unshared"
msgstr "Ressursen deles ikke"
#: pkg/service/templates.go:58
msgid "Share expired"
msgstr "Deling utløpt"
#: pkg/service/templates.go:33
msgid "Space deleted"
msgstr "Plass slettet"
#: pkg/service/templates.go:28
msgid "Space disabled"
msgstr "Rom deaktivert"
#: pkg/service/templates.go:18
msgid "Space shared"
msgstr "Delte rom"
#: pkg/service/templates.go:14
msgid ""
"The file {resource} was deleted because it violates the restrictions of this"
" cloud. This could be due to an unsupported file type, potentially harmful "
"or prohibited content, or file size limits."
msgstr ""
"Filen {resource} ble slettet fordi den bryter med denne skyens "
"restriksjoner. Dette kan skyldes en ulovlig filtype, potensielt skadelig "
"eller forbudt innhold, eller filstørrelsesgrenser."
#: pkg/service/templates.go:8
msgid "Virus found"
msgstr "Virus funnet"
#: pkg/service/templates.go:9
msgid "Virus found in {resource}. Upload not possible. Virus: {virus}"
msgstr "Virus funnet i {resource}. Opplasting ikke mulig. Virus: {virus}"
#: pkg/service/templates.go:53
msgid "You have been mentioned"
msgstr "Du har blitt nevnt"
#: pkg/service/templates.go:19
msgid "{user} added you to Space {space}"
msgstr "{user} la deg til i rom {space}"
#: pkg/service/templates.go:34
msgid "{user} deleted Space {space}"
msgstr "{user} slettet rom {space}"
#: pkg/service/templates.go:29
msgid "{user} disabled Space {space}"
msgstr "{user} deaktivert rom {space}"
#: pkg/service/templates.go:54
msgid "{user} mentioned you in {resource}"
msgstr "{user} nevnte deg i {resource}"
#: pkg/service/templates.go:24
msgid "{user} removed you from Space {space}"
msgstr "{user} fjernet deg fra rom {space}"
#: pkg/service/templates.go:44
msgid "{user} shared {resource} with you"
msgstr "{user} delt {resource} med deg"
#: pkg/service/templates.go:49
msgid "{user} unshared {resource} with you"
msgstr "{user} delte ikke {resource} med deg"
@@ -25,6 +25,7 @@ use GuzzleHttp\Exception\GuzzleException;
use PHPUnit\Framework\Assert;
use Psr\Http\Message\ResponseInterface;
use TestHelpers\WebDavHelper;
use TestHelpers\WaitHelper;
use TestHelpers\HttpRequestHelper;
use TestHelpers\BehatHelper;
use Behat\Step\Then;
@@ -37,6 +38,7 @@ require_once 'bootstrap.php';
*/
class SearchContext implements Context {
private FeatureContext $featureContext;
private array $lastSearchQuery = [];
/**
* @param string $user
@@ -149,6 +151,13 @@ class SearchContext implements Context {
// NOTE: because indexing of newly uploaded files or directories with OpenCloud is decoupled and occurs asynchronously
// short wait is necessary before searching
sleep(10);
// remember the query so "should eventually contain" steps can re-search
$this->lastSearchQuery = [
"user" => $user,
"pattern" => $pattern,
"limit" => $limit,
"properties" => $properties,
];
$response = $this->searchFiles($user, $pattern, $limit, null, null, null, $properties);
$this->featureContext->setResponse($response);
}
@@ -279,4 +288,83 @@ class SearchContext implements Context {
$response = $this-> searchFiles($user, $pattern, null, $scopeType, $scope, $spaceName);
$this->featureContext->setResponse($response);
}
/**
* re-run the last WebDAV search until the assertion passes or the WaitHelper
* timeout elapses, leaving the last response set for a final assertion by the
* caller. Indexing of newly uploaded resources is asynchronous, so a wanted
* file can be missing from an early search; OpenSearch never returns a partial
* document, so once the expected entries are present the result is complete.
*
* @param callable $assert
*
* @return void
*/
private function retrySearchUntilSatisfied(callable $assert): void {
Assert::assertNotEmpty(
$this->lastSearchQuery,
'No search to retry. Use a "searches for ... using the WebDAV API" step first.'
);
$query = $this->lastSearchQuery;
$response = WaitHelper::waitUntil(
fn () => $this->searchFiles(
$query["user"],
$query["pattern"],
$query["limit"],
null,
null,
null,
$query["properties"]
),
function ($response) use ($assert) {
$this->featureContext->setResponse($response);
try {
$assert();
return true;
} catch (\Throwable $e) {
return false;
}
}
);
$this->featureContext->setResponse($response);
}
/**
* @param string $user
* @param TableNode $expectedFiles
*
* @return void
*/
#[Then('/^the search result of user "([^"]*)" should eventually contain only these (?:files|entries):$/')]
public function theSearchResultShouldEventuallyContainOnlyEntries(string $user, TableNode $expectedFiles): void {
$assert = fn () => $this->featureContext->thePropfindResultShouldContainOnlyEntries($user, $expectedFiles);
$this->retrySearchUntilSatisfied($assert);
$assert();
}
/**
* @param string $user
* @param TableNode $expectedFiles
*
* @return void
*/
#[Then('/^the search result of user "([^"]*)" should eventually contain these (?:files|entries):$/')]
public function theSearchResultShouldEventuallyContainEntries(string $user, TableNode $expectedFiles): void {
$assert = fn () => $this->featureContext->thePropfindResultShouldContainEntries($user, '', $expectedFiles);
$this->retrySearchUntilSatisfied($assert);
$assert();
}
/**
* @param TableNode $expectedFiles
* @param string $expectedContent
*
* @return void
*/
#[Then('/^the search result should eventually contain these (?:files|entries) with highlight on keyword "([^"]*)"$/')]
public function theSearchResultShouldEventuallyContainEntriesWithHighlight(TableNode $expectedFiles, string $expectedContent): void {
$assert = fn () => $this->theSearchResultShouldContainEntriesWithHighlight($expectedFiles, $expectedContent);
$this->retrySearchUntilSatisfied($assert);
$assert();
}
}
+63 -12
View File
@@ -55,6 +55,9 @@ class SpacesContext implements Context {
* key is space name and value is the username that created the space
*/
private array $createdSpaces = [];
// the request of the last driveItem GET, so a later "should eventually
// match" step can re-run it (the counterpart of the stored response)
private array $lastDriveItemRequest = [];
private string $ocsApiUrl = '/ocs/v2.php/apps/files_sharing/api/v1/shares';
/**
@@ -4533,17 +4536,16 @@ class SpacesContext implements Context {
}
/**
* resolve the Graph URL of a file in a space
*
* @param string $user
* @param string $file
* @param string $space
*
* @return void
* @return string
*/
#[When('user :user gets the file :file from space :space using the Graph API')]
public function userGetsTheDriveItemInSpace(string $user, string $file, string $space): void {
private function getDriveItemUrl(string $user, string $file, string $space): string {
$spaceId = ($this->getSpaceByName($user, $space))["id"];
$itemId = '';
if ($space === "Shares") {
$itemId = GraphHelper::getShareMountId(
$this->featureContext->getBaseUrl(),
@@ -4555,10 +4557,51 @@ class SpacesContext implements Context {
} else {
$itemId = $this->getFileId($user, $space, $file);
}
$url = $this->featureContext->getBaseUrl() . "/graph/v1.0/drives/$spaceId/items/$itemId";
return $this->featureContext->getBaseUrl() . "/graph/v1.0/drives/$spaceId/items/$itemId";
}
// NOTE: extracting properties occurs asynchronously after upload, so we need to wait until the properties are available
$extractionFacets = ["image", "photo", "location", "audio", "video"];
/**
*
* @param string $user
* @param string $file
* @param string $space
*
* @return void
*/
#[When('user :user gets the file :file from space :space using the Graph API')]
public function userGetsTheDriveItemInSpace(string $user, string $file, string $space): void {
$this->lastDriveItemRequest = [
"url" => $this->getDriveItemUrl($user, $file, $space),
"user" => $user,
];
$response = HttpRequestHelper::get(
$this->lastDriveItemRequest["url"],
$this->featureContext->getStepLineRef(),
$user,
$this->featureContext->getPasswordForUser($user),
);
$this->featureContext->setResponse($response);
}
/**
*
* @param PyStringNode $schemaString
*
* @return void
*/
#[Then('the JSON data of the response should eventually match')]
public function theJsonDataOfTheResponseShouldEventuallyMatch(PyStringNode $schemaString): void {
Assert::assertNotEmpty(
$this->lastDriveItemRequest,
'no driveItem request to repeat, get the file using the Graph API first'
);
$url = $this->lastDriveItemRequest["url"];
$user = $this->lastDriveItemRequest["user"];
$schema = $this->featureContext->getJSONSchema($schemaString);
// Extraction is asynchronous, so re-fetch until the response satisfies the
// expected schema (a partial payload never matches) or the WaitHelper
// timeout elapses.
$response = WaitHelper::waitUntil(
fn () => HttpRequestHelper::get(
$url,
@@ -4566,16 +4609,24 @@ class SpacesContext implements Context {
$user,
$this->featureContext->getPasswordForUser($user),
),
function ($response) use ($extractionFacets) {
function ($response) use ($schema) {
if ($response->getStatusCode() !== 200) {
return true;
return false;
}
try {
$body = $this->featureContext->getJsonDecodedResponseBodyContent($response);
$this->featureContext->assertJsonDocumentMatchesSchema($body, $schema);
return true;
} catch (\Throwable $e) {
return false;
}
$body = $this->featureContext->getJsonDecodedResponseBodyContent($response);
return \is_object($body)
&& !empty(\array_intersect($extractionFacets, \array_keys((array) $body)));
}
);
$this->featureContext->setResponse($response);
$this->featureContext->assertJsonDocumentMatchesSchema(
$this->featureContext->getJsonDecodedResponseBodyContent($response),
$schema
);
}
}
@@ -16,7 +16,7 @@ Feature: content search
And user "Alice" has uploaded file with content "namaste from nepal" to "hello.txt"
When user "Alice" searches for "Content:hello" using the WebDAV API
Then the HTTP status code should be "207"
And the search result of user "Alice" should contain only these files:
And the search result of user "Alice" should eventually contain only these files:
| keywordAtStart.txt |
| keywordAtMiddle.txt |
| keywordAtLast.txt |
@@ -34,15 +34,15 @@ Feature: content search
And user "Alice" has uploaded file with content "alan@example.org want to say hello" to "findByEmail.docs"
When user "Alice" searches for "Content:k6" using the WebDAV API
Then the HTTP status code should be "207"
And the search result of user "Alice" should contain only these files:
And the search result of user "Alice" should eventually contain only these files:
| wordWithNumber.md |
When user "Alice" searches for "Content:https://opencloud.eu/" using the WebDAV API
Then the HTTP status code should be "207"
And the search result of user "Alice" should contain only these files:
And the search result of user "Alice" should eventually contain only these files:
| findByWebSite.txt |
When user "Alice" searches for "Content:alan@" using the WebDAV API
Then the HTTP status code should be "207"
And the search result of user "Alice" should contain only these files:
And the search result of user "Alice" should eventually contain only these files:
| findByEmail.docs |
Examples:
| dav-path-version |
@@ -71,11 +71,11 @@ Feature: content search
And user "Alice" has uploaded file with content "He has expirience, we must to have, I have to find ...." to "fileWithStopWords.txt"
When user "Alice" searches for 'Content:"he has"' using the WebDAV API
Then the HTTP status code should be "207"
And the search result of user "Alice" should contain only these files:
And the search result of user "Alice" should eventually contain only these files:
| fileWithStopWords.txt |
When user "Alice" searches for 'Content:"I have"' using the WebDAV API
Then the HTTP status code should be "207"
And the search result of user "Alice" should contain only these files:
And the search result of user "Alice" should eventually contain only these files:
| fileWithStopWords.txt |
Examples:
| dav-path-version |
@@ -101,7 +101,7 @@ Feature: content search
And user "Brian" has a share "uploadFolder" synced
When user "Brian" searches for "Content:hello" using the WebDAV API
Then the HTTP status code should be "207"
And the search result of user "Brian" should contain only these files:
And the search result of user "Brian" should eventually contain only these files:
| keywordAtStart.txt |
| keywordAtMiddle.txt |
| keywordAtLast.txt |
@@ -121,7 +121,7 @@ Feature: content search
And user "Alice" has deleted file "keywordAtLast.txt"
When user "Alice" searches for "Content:hello" using the WebDAV API
Then the HTTP status code should be "207"
And the search result of user "Alice" should contain only these files:
And the search result of user "Alice" should eventually contain only these files:
| keywordAtStart.txt |
| keywordAtMiddle.txt |
Examples:
@@ -139,7 +139,7 @@ Feature: content search
And user "Alice" has restored the file with original path "keywordAtStart.txt"
When user "Alice" searches for "Content:hello" using the WebDAV API
Then the HTTP status code should be "207"
And the search result of user "Alice" should contain only these files:
And the search result of user "Alice" should eventually contain only these files:
| keywordAtStart.txt |
Examples:
| dav-path-version |
@@ -154,7 +154,7 @@ Feature: content search
And user "Alice" has restored version index "1" of file "test.txt"
When user "Alice" searches for "Content:hello" using the WebDAV API
Then the HTTP status code should be "207"
And the search result of user "Alice" should contain only these files:
And the search result of user "Alice" should eventually contain only these files:
| test.txt |
Examples:
| dav-path-version |
@@ -175,7 +175,7 @@ Feature: content search
And using <dav-path-version> DAV path
When user "Alice" searches for "Content:hello" using the WebDAV API
Then the HTTP status code should be "207"
And the search result of user "Alice" should contain only these files:
And the search result of user "Alice" should eventually contain only these files:
| keywordAtStart.txt |
| keywordAtMiddle.txt |
| keywordAtLast.txt |
@@ -204,7 +204,7 @@ Feature: content search
And using <dav-path-version> DAV path
When user "Brian" searches for "Content:hello" using the WebDAV API
Then the HTTP status code should be "207"
And the search result of user "Alice" should contain only these files:
And the search result of user "Alice" should eventually contain only these files:
| keywordAtStart.txt |
| keywordAtMiddle.txt |
| keywordAtLast.txt |
@@ -224,10 +224,10 @@ Feature: content search
| technical task.txt | test |
When user "Alice" searches for '<pattern>' using the WebDAV API
Then the HTTP status code should be "207"
And the search result should contain "<result-count>" entries
And the search result of user "Alice" should contain these entries:
And the search result of user "Alice" should eventually contain these entries:
| <search-result-1> |
| <search-result-2> |
And the search result should contain "<result-count>" entries
Examples:
| pattern | result-count | search-result-1 | search-result-2 |
| Content:hello | 1 | technical task.txt | |
@@ -251,7 +251,7 @@ Feature: content search
And user "Alice" has uploaded a file inside space "project-space" with content "this is a simple odt file" to "test-odt-file.odt"
When user "Alice" searches for "Content:simple" using the WebDAV API
Then the HTTP status code should be "207"
And the search result should contain these entries with highlight on keyword "simple"
And the search result should eventually contain these entries with highlight on keyword "simple"
| test-text-file.txt |
| test-pdf-file.pdf |
| test-cpp-file.cpp |
@@ -131,7 +131,7 @@ Feature: propfind extracted props
Given user "Alice" has uploaded a file "filesForUpload/testaudio.mp3" to "testaudio.mp3" in space "Personal"
When user "Alice" gets the file "testaudio.mp3" from space "Personal" using the Graph API
Then the HTTP status code should be "200"
And the JSON data of the response should match
And the JSON data of the response should eventually match
"""
{
"type": "object",
@@ -179,7 +179,7 @@ Feature: propfind extracted props
Given user "Alice" has uploaded a file "filesForUpload/testavatar.jpg" to "testavatar.jpg" in space "Personal"
When user "Alice" gets the file "testavatar.jpg" from space "Personal" using the Graph API
Then the HTTP status code should be "200"
And the JSON data of the response should match
And the JSON data of the response should eventually match
"""
{
"type": "object",
@@ -251,7 +251,7 @@ Feature: propfind extracted props
And user "Alice" has uploaded a file "filesForUpload/testaudio.mp3" to "testaudio.mp3" in space "new-space"
When user "Alice" gets the file "testaudio.mp3" from space "new-space" using the Graph API
Then the HTTP status code should be "200"
And the JSON data of the response should match
And the JSON data of the response should eventually match
"""
{
"type": "object",
@@ -301,7 +301,7 @@ Feature: propfind extracted props
And user "Alice" has uploaded a file "filesForUpload/testavatar.jpg" to "testavatar.jpg" in space "new-space"
When user "Alice" gets the file "testavatar.jpg" from space "new-space" using the Graph API
Then the HTTP status code should be "200"
And the JSON data of the response should match
And the JSON data of the response should eventually match
"""
{
"type": "object",
@@ -379,7 +379,7 @@ Feature: propfind extracted props
And user "Brian" has a share "testaudio.mp3" synced
When user "Brian" gets the file "testaudio.mp3" from space "Shares" using the Graph API
Then the HTTP status code should be "200"
And the JSON data of the response should match
And the JSON data of the response should eventually match
"""
{
"type": "object",
@@ -435,7 +435,7 @@ Feature: propfind extracted props
And user "Brian" has a share "testavatar.jpg" synced
When user "Brian" gets the file "testavatar.jpg" from space "Shares" using the Graph API
Then the HTTP status code should be "200"
And the JSON data of the response should match
And the JSON data of the response should eventually match
"""
{
"type": "object",
@@ -34,7 +34,6 @@ import (
_ "github.com/opencloud-eu/reva/v2/internal/http/services/prometheus"
_ "github.com/opencloud-eu/reva/v2/internal/http/services/reverseproxy"
_ "github.com/opencloud-eu/reva/v2/internal/http/services/sciencemesh"
_ "github.com/opencloud-eu/reva/v2/internal/http/services/siteacc"
_ "github.com/opencloud-eu/reva/v2/internal/http/services/sysinfo"
_ "github.com/opencloud-eu/reva/v2/internal/http/services/wellknown"
// Add your own service here
@@ -96,6 +96,30 @@ var (
"width",
"height",
}
videoKeys = []string{
"audioBitsPerSample",
"audioChannels",
"audioFormat",
"audioSamplesPerSecond",
"bitrate",
"duration",
"fourCC",
"frameRate",
"height",
"width",
}
motionPhotoKeys = []string{
"presentationTimestampUs",
"version",
"videoSize",
}
livePhotoKeys = []string{
"auto",
"contentId",
"stillImageTimeUs",
"vitalityScore",
"vitalityScoringVersion",
}
photoKeys = []string{
"cameraMake",
"cameraModel",
@@ -887,6 +911,12 @@ func metadataKeys(pf XML) ([]string, []string) {
metadataKeys = append(metadataKeys, metadataKeysWithPrefix("libre.graph.image", imageKeys)...)
case "http://owncloud.org/ns/photo":
metadataKeys = append(metadataKeys, metadataKeysWithPrefix("libre.graph.photo", photoKeys)...)
case "http://owncloud.org/ns/video":
metadataKeys = append(metadataKeys, metadataKeysWithPrefix("libre.graph.video", videoKeys)...)
case "http://owncloud.org/ns/motion-photo":
metadataKeys = append(metadataKeys, metadataKeysWithPrefix("libre.graph.motionPhoto", motionPhotoKeys)...)
case "http://owncloud.org/ns/live-photo":
metadataKeys = append(metadataKeys, metadataKeysWithPrefix("libre.graph.livePhoto", livePhotoKeys)...)
default:
metadataKeys = append(metadataKeys, key)
}
@@ -944,7 +974,7 @@ func requiresExplicitFetching(n *xml.Name) bool {
}
case net.NsOwncloud:
switch n.Local {
case "favorite", "share-types", "checksums", "size", "tags", "audio", "location", "image", "photo":
case "favorite", "share-types", "checksums", "size", "tags", "audio", "location", "image", "photo", "video", "motion-photo", "live-photo":
return true
default:
return false
@@ -1292,6 +1322,9 @@ func mdToPropResponse(ctx context.Context, pf *XML, md *provider.ResourceInfo, p
appendMetadataProp(k, "oc", "location", "libre.graph.location", locationKeys)
appendMetadataProp(k, "oc", "image", "libre.graph.image", imageKeys)
appendMetadataProp(k, "oc", "photo", "libre.graph.photo", photoKeys)
appendMetadataProp(k, "oc", "video", "libre.graph.video", videoKeys)
appendMetadataProp(k, "oc", "motion-photo", "libre.graph.motionPhoto", motionPhotoKeys)
appendMetadataProp(k, "oc", "live-photo", "libre.graph.livePhoto", livePhotoKeys)
}
if md.Type == provider.ResourceType_RESOURCE_TYPE_CONTAINER {
@@ -1579,6 +1612,18 @@ func mdToPropResponse(ctx context.Context, pf *XML, md *provider.ResourceInfo, p
if k := md.GetArbitraryMetadata().GetMetadata(); k != nil {
appendMetadataProp(k, "oc", "photo", "libre.graph.photo", photoKeys)
}
case "video":
if k := md.GetArbitraryMetadata().GetMetadata(); k != nil {
appendMetadataProp(k, "oc", "video", "libre.graph.video", videoKeys)
}
case "motion-photo":
if k := md.GetArbitraryMetadata().GetMetadata(); k != nil {
appendMetadataProp(k, "oc", "motion-photo", "libre.graph.motionPhoto", motionPhotoKeys)
}
case "live-photo":
if k := md.GetArbitraryMetadata().GetMetadata(); k != nil {
appendMetadataProp(k, "oc", "live-photo", "libre.graph.livePhoto", livePhotoKeys)
}
case "name":
appendToOK(prop.Escaped("oc:name", md.Name))
case "shareid":
@@ -1,127 +0,0 @@
// Copyright 2018-2021 CERN
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// In applying this license, CERN does not waive the privileges and immunities
// granted to it by virtue of its status as an Intergovernmental Organization
// or submit itself to any jurisdiction.
package siteacc
import (
"net/http"
"github.com/go-viper/mapstructure/v2"
"github.com/opencloud-eu/reva/v2/pkg/siteacc"
"github.com/opencloud-eu/reva/v2/pkg/siteacc/config"
"github.com/pkg/errors"
"github.com/rs/zerolog"
"github.com/opencloud-eu/reva/v2/pkg/rhttp/global"
)
func init() {
global.Register(serviceName, New)
}
type svc struct {
conf *config.Configuration
log *zerolog.Logger
siteacc *siteacc.SiteAccounts
}
const (
serviceName = "siteacc"
)
// Close is called when this service is being stopped.
func (s *svc) Close() error {
return nil
}
// Prefix returns the main endpoint of this service.
func (s *svc) Prefix() string {
return s.conf.Prefix
}
// Unprotected returns all endpoints that can be queried without prior authorization.
func (s *svc) Unprotected() []string {
return s.siteacc.GetPublicEndpoints()
}
// Handler serves all HTTP requests.
func (s *svc) Handler() http.Handler {
return s.siteacc.RequestHandler()
}
func parseConfig(m map[string]interface{}) (*config.Configuration, error) {
conf := &config.Configuration{}
if err := mapstructure.Decode(m, &conf); err != nil {
return nil, errors.Wrap(err, "error decoding configuration")
}
applyDefaultConfig(conf)
conf.Cleanup()
if conf.Webserver.URL == "" {
return nil, errors.Errorf("no webserver URL specified")
}
return conf, nil
}
func applyDefaultConfig(conf *config.Configuration) {
if conf.Prefix == "" {
conf.Prefix = serviceName
}
if conf.Storage.Driver == "" {
conf.Storage.Driver = "file"
}
if conf.Mentix.DataEndpoint == "" {
conf.Mentix.DataEndpoint = "/sites"
}
if conf.Mentix.SiteRegistrationEndpoint == "" {
conf.Mentix.SiteRegistrationEndpoint = "/sitereg"
}
// Enforce a minimum session timeout of 1 minute (and default to 5 minutes)
if conf.Webserver.SessionTimeout < 60 {
conf.Webserver.SessionTimeout = 5 * 60
}
}
// New returns a new Site Accounts service.
func New(m map[string]interface{}, log *zerolog.Logger) (global.Service, error) {
// Prepare the configuration
conf, err := parseConfig(m)
if err != nil {
return nil, err
}
// Create the site accounts instance
siteacc, err := siteacc.New(conf, log)
if err != nil {
return nil, errors.Wrap(err, "error creating the site accounts service")
}
// Create the service
s := &svc{
conf: conf,
log: log,
siteacc: siteacc,
}
return s, nil
}
+1
View File
@@ -602,6 +602,7 @@ var mimeTypes = map[string]string{
"nc": "application/x-netcdf",
"ncx": "application/x-dtbncx+xml",
"nef": "image/x-nikon-nef",
"nrw": "image/x-nikon-nrw",
"nfo": "text/x-nfo",
"ngdat": "application/vnd.nokia.n-gage.data",
"nitf": "application/vnd.nitf",
+33 -11
View File
@@ -289,6 +289,34 @@ func (m *Manager) initialize(ctx context.Context) error {
return err
}
err = m.storage.MakeDirIfNotExist(ctx, "migrations")
if err != nil {
span.RecordError(err)
span.SetStatus(codes.Error, err.Error())
return err
}
// A missing "storages" directory means nobody has ever used this metadata
// tree, so there is nothing to migrate. Record all migrations as applied
// before creating "storages": any instance that observes "storages" is then
// guaranteed to also observe the state file, which keeps instances starting
// in parallel from disagreeing about whether migrations have to run.
_, err = m.storage.Stat(ctx, "storages")
switch err.(type) {
case nil:
// existing deployment, leave the migration state untouched
case errtypes.IsNotFound:
if err = migration.MarkAllApplied(ctx, m.storage); err != nil {
span.RecordError(err)
span.SetStatus(codes.Error, err.Error())
return err
}
default:
span.RecordError(err)
span.SetStatus(codes.Error, err.Error())
return err
}
err = m.storage.MakeDirIfNotExist(ctx, "storages")
if err != nil {
span.RecordError(err)
@@ -307,12 +335,6 @@ func (m *Manager) initialize(ctx context.Context) error {
span.SetStatus(codes.Error, err.Error())
return err
}
err = m.storage.MakeDirIfNotExist(ctx, "migrations")
if err != nil {
span.RecordError(err)
span.SetStatus(codes.Error, err.Error())
return err
}
span.SetStatus(codes.Ok, "initialized")
return nil
@@ -566,7 +588,7 @@ func (m *Manager) GetShare(ctx context.Context, ref *collaboration.ShareReferenc
if s.ResourceId.SpaceId == s.ResourceId.OpaqueId {
if err := events.Publish(ctx, m.eventStream, events.SpaceMembershipExpired{
SpaceOwner: s.GetOwner(),
SpaceID: &provider.StorageSpaceId{OpaqueId: s.ResourceId.StorageId + "$" + s.ResourceId.SpaceId},
SpaceID: &provider.StorageSpaceId{OpaqueId: storagespace.FormatStorageID(s.ResourceId.StorageId, s.ResourceId.SpaceId)},
ExpiredAt: time.Unix(int64(s.GetExpiration().GetSeconds()), int64(s.GetExpiration().GetNanos())),
GranteeUserID: s.GetGrantee().GetUserId(),
GranteeGroupID: s.GetGrantee().GetGroupId(),
@@ -780,7 +802,7 @@ func (m *Manager) listSharesByIDs(ctx context.Context, user *userv1beta1.User, f
if s.ResourceId.SpaceId == s.ResourceId.OpaqueId {
if err := events.Publish(ctx, m.eventStream, events.SpaceMembershipExpired{
SpaceOwner: s.GetOwner(),
SpaceID: &provider.StorageSpaceId{OpaqueId: s.ResourceId.StorageId + "$" + s.ResourceId.SpaceId},
SpaceID: &provider.StorageSpaceId{OpaqueId: storagespace.FormatStorageID(s.ResourceId.StorageId, s.ResourceId.SpaceId)},
ExpiredAt: time.Unix(int64(s.GetExpiration().GetSeconds()), int64(s.GetExpiration().GetNanos())),
GranteeUserID: s.GetGrantee().GetUserId(),
GranteeGroupID: s.GetGrantee().GetGroupId(),
@@ -911,7 +933,7 @@ func (m *Manager) listCreatedShares(ctx context.Context, user *userv1beta1.User,
if s.ResourceId.SpaceId == s.ResourceId.OpaqueId {
if err := events.Publish(ctx, m.eventStream, events.SpaceMembershipExpired{
SpaceOwner: s.GetOwner(),
SpaceID: &provider.StorageSpaceId{OpaqueId: s.ResourceId.StorageId + "$" + s.ResourceId.SpaceId},
SpaceID: &provider.StorageSpaceId{OpaqueId: storagespace.FormatStorageID(s.ResourceId.StorageId, s.ResourceId.SpaceId)},
ExpiredAt: time.Unix(int64(s.GetExpiration().GetSeconds()), int64(s.GetExpiration().GetNanos())),
GranteeUserID: s.GetGrantee().GetUserId(),
GranteeGroupID: s.GetGrantee().GetGroupId(),
@@ -1096,7 +1118,7 @@ func (m *Manager) ListReceivedShares(ctx context.Context, filters []*collaborati
if s.ResourceId.SpaceId == s.ResourceId.OpaqueId {
if err := events.Publish(ctx, m.eventStream, events.SpaceMembershipExpired{
SpaceOwner: s.GetOwner(),
SpaceID: &provider.StorageSpaceId{OpaqueId: s.ResourceId.StorageId + "$" + s.ResourceId.SpaceId},
SpaceID: &provider.StorageSpaceId{OpaqueId: storagespace.FormatStorageID(s.ResourceId.StorageId, s.ResourceId.SpaceId)},
ExpiredAt: time.Unix(int64(s.GetExpiration().GetSeconds()), int64(s.GetExpiration().GetNanos())),
GranteeUserID: s.GetGrantee().GetUserId(),
GranteeGroupID: s.GetGrantee().GetGroupId(),
@@ -1212,7 +1234,7 @@ func (m *Manager) getReceived(ctx context.Context, ref *collaboration.ShareRefer
if s.ResourceId.SpaceId == s.ResourceId.OpaqueId {
if err := events.Publish(ctx, m.eventStream, events.SpaceMembershipExpired{
SpaceOwner: s.GetOwner(),
SpaceID: &provider.StorageSpaceId{OpaqueId: s.ResourceId.StorageId + "$" + s.ResourceId.SpaceId},
SpaceID: &provider.StorageSpaceId{OpaqueId: storagespace.FormatStorageID(s.ResourceId.StorageId, s.ResourceId.SpaceId)},
ExpiredAt: time.Unix(int64(s.GetExpiration().GetSeconds()), int64(s.GetExpiration().GetNanos())),
GranteeUserID: s.GetGrantee().GetUserId(),
GranteeGroupID: s.GetGrantee().GetGroupId(),
@@ -139,6 +139,40 @@ func New(logger zerolog.Logger,
}
}
// LatestVersion returns the version of the most recent registered migration.
func LatestVersion() int {
version := 0
for _, mig := range migrations {
if mig.Version() > version {
version = mig.Version()
}
}
return version
}
// MarkAllApplied records every registered migration as applied without running
// any of them. It is meant for fresh deployments, where there is no data to
// migrate and running the migrations would needlessly block write operations
// while depending on services that may not be up yet.
//
// The state file is created atomically and an existing state file is never
// overwritten, so instances starting in parallel converge on the same state.
func MarkAllApplied(ctx context.Context, storage metadata.Storage) error {
data, err := json.Marshal(persistedState{Version: LatestVersion()})
if err != nil {
return err
}
_, err = storage.Upload(ctx, metadata.UploadRequest{
Path: stateFile,
Content: data,
IfNoneMatch: []string{"*"},
})
if err != nil && !isConflict(err) {
return err
}
return nil
}
// acquireLock tries to atomically create the lock file, blocking until the lock
// is obtained. It returns the etag of the lock file on success. It retries
// indefinitely until ctx is cancelled. A lock whose timestamp is older than
@@ -1,51 +0,0 @@
// Copyright 2018-2020 CERN
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// In applying this license, CERN does not waive the privileges and immunities
// granted to it by virtue of its status as an Intergovernmental Organization
// or submit itself to any jurisdiction.
package contact
import "github.com/opencloud-eu/reva/v2/pkg/siteacc/html"
// PanelTemplate is the content provider for the contact form.
type PanelTemplate struct {
html.ContentProvider
}
// GetTitle returns the title of the panel.
func (template *PanelTemplate) GetTitle() string {
return "ScienceMesh Contact Form"
}
// GetCaption returns the caption which is displayed on the panel.
func (template *PanelTemplate) GetCaption() string {
return "Contact the ScienceMesh administration!"
}
// GetContentJavaScript delivers additional JavaScript code.
func (template *PanelTemplate) GetContentJavaScript() string {
return tplJavaScript
}
// GetContentStyleSheet delivers additional stylesheet code.
func (template *PanelTemplate) GetContentStyleSheet() string {
return tplStyleSheet
}
// GetContentBody delivers the actual body content.
func (template *PanelTemplate) GetContentBody() string {
return tplBody
}
@@ -1,111 +0,0 @@
// Copyright 2018-2020 CERN
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// In applying this license, CERN does not waive the privileges and immunities
// granted to it by virtue of its status as an Intergovernmental Organization
// or submit itself to any jurisdiction.
package contact
const tplJavaScript = `
function verifyForm(formData) {
if (formData.getTrimmed("subject") == "") {
setState(STATE_ERROR, "Please enter a subject.", "form", "subject", true);
return false;
}
if (formData.getTrimmed("message") == "") {
setState(STATE_ERROR, "Please enter a message.", "form", "message", true);
return false;
}
return true;
}
function handleAction(action) {
const formData = new FormData(document.querySelector("form"));
if (!verifyForm(formData)) {
return;
}
setState(STATE_STATUS, "Sending message... this should only take a moment.", "form", null, false);
var xhr = new XMLHttpRequest();
xhr.open("POST", "{{getServerAddress}}/" + action);
xhr.setRequestHeader('Content-Type', 'application/json; charset=UTF-8');
xhr.onload = function() {
if (this.status == 200) {
setState(STATE_SUCCESS, "Your message was successfully sent! A copy of the message has been sent to your email address.");
} else {
var resp = JSON.parse(this.responseText);
setState(STATE_ERROR, "An error occurred while trying to send your message:<br><em>" + resp.error + "</em>", "form", null, true);
}
}
var postData = {
"subject": formData.getTrimmed("subject"),
"message": formData.getTrimmed("message")
};
xhr.send(JSON.stringify(postData));
}
`
const tplStyleSheet = `
html * {
font-family: arial !important;
}
.mandatory {
color: red;
font-weight: bold;
}
`
const tplBody = `
<div>
<p>Contact the ScienceMesh administration using the form below.</p>
<p style="margin-bottom: 0em;">Please include as much information as possible in your request, especially:</p>
<ul style="margin-top: 0em;">
<li>The site your request refers to (if not obvious from your account information)</li>
<li>Your role within the ScienceMesh site (e.g., administrator, operational team member, etc.)</li>
<li>Any specific reasons for your request</li>
<li>Anything else that might help to process your request</li>
</ul>
</div>
<div>&nbsp;</div>
<div>
<form id="form" method="POST" class="box container-inline" style="width: 100%;" onSubmit="handleAction('contact'); return false;">
<div style="grid-row: 1;"><label for="subject">Subject: <span class="mandatory">*</span></label></div>
<div style="grid-row: 2; grid-column: 1 / span 2;"><input type="text" id="subject" name="subject" {{if .Params.Subject}}value="{{.Params.Subject}}" readonly{{end}}/></div>
<div style="grid-row: 3;"><label for="message">Message: <span class="mandatory">*</span></label></div>
<div style="grid-row: 4; grid-column: 1 / span 2;">
<textarea rows="10" id="message" name="message" style="box-sizing: border-box; width: 100%;" {{if .Params.Info}}placeholder="{{.Params.Info}}"{{end}}>{{if .Params.Message}}{{.Params.Message}}{{end}}</textarea>
</div>
<div style="grid-row: 5; align-self: center;">
Fields marked with <span class="mandatory">*</span> are mandatory.
</div>
<div style="grid-row: 5; grid-column: 2; text-align: right;">
<button type="reset">Reset</button>
<button type="submit" style="font-weight: bold;">Send</button>
</div>
</form>
</div>
<div>
<p>Go <a href="{{getServerAddress}}/account/?path=manage">back</a> to the main account page.</p>
</div>
`
@@ -1,51 +0,0 @@
// Copyright 2018-2020 CERN
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// In applying this license, CERN does not waive the privileges and immunities
// granted to it by virtue of its status as an Intergovernmental Organization
// or submit itself to any jurisdiction.
package edit
import "github.com/opencloud-eu/reva/v2/pkg/siteacc/html"
// PanelTemplate is the content provider for the edit form.
type PanelTemplate struct {
html.ContentProvider
}
// GetTitle returns the title of the panel.
func (template *PanelTemplate) GetTitle() string {
return "ScienceMesh Site Administrator Account"
}
// GetCaption returns the caption which is displayed on the panel.
func (template *PanelTemplate) GetCaption() string {
return "Edit your ScienceMesh Site Administrator Account!"
}
// GetContentJavaScript delivers additional JavaScript code.
func (template *PanelTemplate) GetContentJavaScript() string {
return tplJavaScript
}
// GetContentStyleSheet delivers additional stylesheet code.
func (template *PanelTemplate) GetContentStyleSheet() string {
return tplStyleSheet
}
// GetContentBody delivers the actual body content.
func (template *PanelTemplate) GetContentBody() string {
return tplBody
}
@@ -1,158 +0,0 @@
// Copyright 2018-2020 CERN
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// In applying this license, CERN does not waive the privileges and immunities
// granted to it by virtue of its status as an Intergovernmental Organization
// or submit itself to any jurisdiction.
package edit
const tplJavaScript = `
function verifyForm(formData) {
if (formData.getTrimmed("fname") == "") {
setState(STATE_ERROR, "Please specify your first name.", "form", "fname", true);
return false;
}
if (formData.getTrimmed("lname") == "") {
setState(STATE_ERROR, "Please specify your last name.", "form", "lname", true);
return false;
}
if (formData.getTrimmed("role") == "") {
setState(STATE_ERROR, "Please specify your role within your site.", "form", "role", true);
return false;
}
if (formData.get("password") != "") {
if (formData.get("password2") == "") {
setState(STATE_ERROR, "Please confirm your new password.", "form", "password2", true);
return false;
}
if (formData.get("password") != formData.get("password2")) {
setState(STATE_ERROR, "The entered passwords do not match.", "form", "password2", true);
return false;
}
}
return true;
}
function handleAction(action) {
const formData = new FormData(document.querySelector("form"));
if (!verifyForm(formData)) {
return;
}
setState(STATE_STATUS, "Updating account... this should only take a moment.", "form", null, false);
var xhr = new XMLHttpRequest();
xhr.open("POST", "{{getServerAddress}}/" + action);
xhr.setRequestHeader('Content-Type', 'application/json; charset=UTF-8');
xhr.onload = function() {
if (this.status == 200) {
setState(STATE_SUCCESS, "Your account was successfully updated!", "form", null, true);
} else {
var resp = JSON.parse(this.responseText);
setState(STATE_ERROR, "An error occurred while trying to update your account:<br><em>" + resp.error + "</em>", "form", null, true);
}
}
var postData = {
"title": formData.getTrimmed("title"),
"firstName": formData.getTrimmed("fname"),
"lastName": formData.getTrimmed("lname"),
"role": formData.getTrimmed("role"),
"phoneNumber": formData.getTrimmed("phone"),
"password": {
"value": formData.get("password")
}
};
xhr.send(JSON.stringify(postData));
}
`
const tplStyleSheet = `
html * {
font-family: arial !important;
}
.mandatory {
color: red;
font-weight: bold;
}
`
const tplBody = `
<div>
<p>Edit your ScienceMesh Site Administrator Account information below.</p>
<p>Please note that you cannot modify your email address using this form.</p>
</div>
<div>&nbsp;</div>
<div>
<form id="form" method="POST" class="box container-inline" style="width: 100%;" onSubmit="handleAction('update?invoker=user'); return false;">
<div style="grid-row: 1;"><label for="title">Title: <span class="mandatory">*</span></label></div>
<div style="grid-row: 2;">
<select id="title" name="title">
{{$title := .Account.Title}}
{{range .Titles}}
<option value="{{.}}" {{if eq . $title}}selected{{end}}>{{.}}.</option>
{{end}}
</select>
</div>
<div style="grid-row: 3;"><label for="fname">First name: <span class="mandatory">*</span></label></div>
<div style="grid-row: 4;"><input type="text" id="fname" name="fname" value="{{.Account.FirstName}}"/></div>
<div style="grid-row: 3;"><label for="lname">Last name: <span class="mandatory">*</span></label></div>
<div style="grid-row: 4;"><input type="text" id="lname" name="lname" value="{{.Account.LastName}}"/></div>
<div style="grid-row: 5;"><label for="role">Role: <span class="mandatory">*</span></label></div>
<div style="grid-row: 6;"><input type="text" id="role" name="role" placeholder="Site administrator" value="{{.Account.Role}}"/></div>
<div style="grid-row: 5;"><label for="phone">Phone number:</label></div>
<div style="grid-row: 6;"><input type="text" id="phone" name="phone" placeholder="+49 030 123456" value="{{.Account.PhoneNumber}}"/></div>
<div style="grid-row: 7;">&nbsp;</div>
<div style="grid-row: 8; grid-column: 1 / span 2;">If you want to change your password, fill out the fields below. Otherwise, leave them empty to keep your current one.</div>
<div style="grid-row: 9;"><label for="password">New password:</label></div>
<div style="grid-row: 10;"><input type="password" id="password" name="password" autocomplete="new-password"/></div>
<div style="grid-row: 9"><label for="password2">Confirm new password:</label></div>
<div style="grid-row: 10;"><input type="password" id="password2" name="password2" autocomplete="new-password"/></div>
<div style="grid-row: 11; font-style: italic; font-size: 0.8em;">
The password must fulfil the following criteria:
<ul style="margin-top: 0em;">
<li>Must be at least 8 characters long</li>
<li>Must contain at least 1 lowercase letter</li>
<li>Must contain at least 1 uppercase letter</li>
<li>Must contain at least 1 digit</li>
</ul>
</div>
<div style="grid-row: 12; align-self: center;">
Fields marked with <span class="mandatory">*</span> are mandatory.
</div>
<div style="grid-row: 12; grid-column: 2; text-align: right;">
<button type="reset">Reset</button>
<button type="submit" style="font-weight: bold;">Save</button>
</div>
</form>
</div>
<div>
<p>Go <a href="{{getServerAddress}}/account/?path=manage">back</a> to the main account page.</p>
</div>
`
@@ -1,51 +0,0 @@
// Copyright 2018-2020 CERN
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// In applying this license, CERN does not waive the privileges and immunities
// granted to it by virtue of its status as an Intergovernmental Organization
// or submit itself to any jurisdiction.
package login
import "github.com/opencloud-eu/reva/v2/pkg/siteacc/html"
// PanelTemplate is the content provider for the login form.
type PanelTemplate struct {
html.ContentProvider
}
// GetTitle returns the title of the panel.
func (template *PanelTemplate) GetTitle() string {
return "ScienceMesh Site Administrator Account Login"
}
// GetCaption returns the caption which is displayed on the panel.
func (template *PanelTemplate) GetCaption() string {
return "Login to your ScienceMesh Site Administrator Account!"
}
// GetContentJavaScript delivers additional JavaScript code.
func (template *PanelTemplate) GetContentJavaScript() string {
return tplJavaScript
}
// GetContentStyleSheet delivers additional stylesheet code.
func (template *PanelTemplate) GetContentStyleSheet() string {
return tplStyleSheet
}
// GetContentBody delivers the actual body content.
func (template *PanelTemplate) GetContentBody() string {
return tplBody
}
@@ -1,137 +0,0 @@
// Copyright 2018-2020 CERN
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// In applying this license, CERN does not waive the privileges and immunities
// granted to it by virtue of its status as an Intergovernmental Organization
// or submit itself to any jurisdiction.
package login
const tplJavaScript = `
function verifyForm(formData, requirePassword = true) {
if (formData.getTrimmed("email") == "") {
setState(STATE_ERROR, "Please enter your email address.", "form", "email", true);
return false;
}
if (requirePassword) {
if (formData.get("password") == "") {
setState(STATE_ERROR, "Please enter your password.", "form", "password", true);
return false;
}
}
return true;
}
function handleAction(action) {
const formData = new FormData(document.querySelector("form"));
if (!verifyForm(formData)) {
return;
}
setState(STATE_STATUS, "Logging in... this should only take a moment.", "form", null, false);
var xhr = new XMLHttpRequest();
xhr.open("POST", "{{getServerAddress}}/" + action);
xhr.setRequestHeader('Content-Type', 'application/json; charset=UTF-8');
xhr.onload = function() {
if (this.status == 200) {
setState(STATE_SUCCESS, "Your login was successful! Redirecting...");
window.location.replace("{{getServerAddress}}/account/?path=manage");
} else {
var resp = JSON.parse(this.responseText);
setState(STATE_ERROR, "An error occurred while trying to login your account:<br><em>" + resp.error + "</em>", "form", null, true);
}
}
var postData = {
"email": formData.getTrimmed("email"),
"password": {
"value": formData.get("password")
}
};
xhr.send(JSON.stringify(postData));
}
function handleResetPassword() {
const formData = new FormData(document.querySelector("form"));
if (!verifyForm(formData, false)) {
return;
}
setState(STATE_STATUS, "Resetting password... this should only take a moment.", "form", null, false);
var xhr = new XMLHttpRequest();
xhr.open("POST", "{{getServerAddress}}/reset-password");
xhr.setRequestHeader('Content-Type', 'application/json; charset=UTF-8');
xhr.onload = function() {
if (this.status == 200) {
setState(STATE_SUCCESS, "Your password was successfully reset! Please check your inbox for your new password.", "form", null, true);
} else {
var resp = JSON.parse(this.responseText);
setState(STATE_ERROR, "An error occurred while trying to reset your password:<br><em>" + resp.error + "</em>", "form", null, true);
}
}
var postData = {
"email": formData.get("email")
};
xhr.send(JSON.stringify(postData));
}
`
const tplStyleSheet = `
html * {
font-family: arial !important;
}
.mandatory {
color: red;
font-weight: bold;
}
`
const tplBody = `
<div>
<p>Login to your ScienceMesh Site Administrator Account using the form below.</p>
</div>
<div>&nbsp;</div>
<div>
<form id="form" method="POST" class="box container-inline" style="width: 100%;" onSubmit="handleAction('login'); return false;">
<div style="grid-row: 1;"><label for="email">Email address: <span class="mandatory">*</span></label></div>
<div style="grid-row: 2;"><input type="text" id="email" name="email" placeholder="me@example.com"/></div>
<div style="grid-row: 1;"><label for="password">Password: <span class="mandatory">*</span></label></div>
<div style="grid-row: 2;"><input type="password" id="password" name="password"/></div>
<div style="grid-row: 3; grid-column: 2; font-style: italic; font-size: 0.8em;">
Forgot your password? Click <a href="#" onClick="handleResetPassword();">here</a> to reset it.
</div>
<div style="grid-row: 4; align-self: center;">
Fields marked with <span class="mandatory">*</span> are mandatory.
</div>
<div style="grid-row: 4; grid-column: 2; text-align: right;">
<button type="reset">Reset</button>
<button type="submit" style="font-weight: bold;">Login</button>
</div>
</form>
</div>
<div>
<p>Don't' have an account yet? Register <a href="{{getServerAddress}}/account/?path=register">here</a>.</p>
</div>
`
@@ -1,51 +0,0 @@
// Copyright 2018-2020 CERN
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// In applying this license, CERN does not waive the privileges and immunities
// granted to it by virtue of its status as an Intergovernmental Organization
// or submit itself to any jurisdiction.
package manage
import "github.com/opencloud-eu/reva/v2/pkg/siteacc/html"
// PanelTemplate is the content provider for the mangement form.
type PanelTemplate struct {
html.ContentProvider
}
// GetTitle returns the title of the panel.
func (template *PanelTemplate) GetTitle() string {
return "ScienceMesh Site Administrator Account"
}
// GetCaption returns the caption which is displayed on the panel.
func (template *PanelTemplate) GetCaption() string {
return "Welcome to your ScienceMesh Site Administrator Account!"
}
// GetContentJavaScript delivers additional JavaScript code.
func (template *PanelTemplate) GetContentJavaScript() string {
return tplJavaScript
}
// GetContentStyleSheet delivers additional stylesheet code.
func (template *PanelTemplate) GetContentStyleSheet() string {
return tplStyleSheet
}
// GetContentBody delivers the actual body content.
func (template *PanelTemplate) GetContentBody() string {
return tplBody
}
@@ -1,132 +0,0 @@
// Copyright 2018-2020 CERN
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// In applying this license, CERN does not waive the privileges and immunities
// granted to it by virtue of its status as an Intergovernmental Organization
// or submit itself to any jurisdiction.
package manage
const tplJavaScript = `
function handleAccountSettings() {
setState(STATE_STATUS, "Redirecting to the account settings...");
window.location.replace("{{getServerAddress}}/account/?path=settings");
}
function handleEditAccount() {
setState(STATE_STATUS, "Redirecting to the account editor...");
window.location.replace("{{getServerAddress}}/account/?path=edit");
}
function handleSiteSettings() {
setState(STATE_STATUS, "Redirecting to the site settings...");
window.location.replace("{{getServerAddress}}/account/?path=site");
}
function handleRequestAccess(scope) {
setState(STATE_STATUS, "Redirecting to the contact form...");
window.location.replace("{{getServerAddress}}/account/?path=contact&subject=" + encodeURIComponent("Request " + scope + " access"));
}
function handleLogout() {
var xhr = new XMLHttpRequest();
xhr.open("GET", "{{getServerAddress}}/logout");
xhr.setRequestHeader('Content-Type', 'application/json; charset=UTF-8');
setState(STATE_STATUS, "Logging out...");
xhr.onload = function() {
if (this.status == 200) {
setState(STATE_SUCCESS, "Done! Redirecting...");
window.location.replace("{{getServerAddress}}/account/?path=login");
} else {
setState(STATE_ERROR, "An error occurred while logging out: " + this.responseText);
}
}
xhr.send();
}
`
const tplStyleSheet = `
html * {
font-family: arial !important;
}
button {
min-width: 170px;
}
`
const tplBody = `
<div>
<p><strong>Hello {{.Account.FirstName}} {{.Account.LastName}},</strong></p>
<p>On this page, you can manage your ScienceMesh Site Administrator Account. This includes editing your personal information, requesting access to the GOCDB and more.</p>
</div>
<div>&nbsp;</div>
<div>
<strong>Personal information:</strong>
<ul style="margin-top: 0em;">
<li>Name: <em>{{.Account.Title}}. {{.Account.FirstName}} {{.Account.LastName}}</em></li>
<li>Email: <em><a href="mailto:{{.Account.Email}}">{{.Account.Email}}</a></em></li>
<li>ScienceMesh Site: <em>{{getSiteName .Account.Site false}} ({{getSiteName .Account.Site true}})</em></li>
<li>Role: <em>{{.Account.Role}}</em></li>
{{if .Account.PhoneNumber}}
<li>Phone: <em>{{.Account.PhoneNumber}}</em></li>
{{end}}
</ul>
</div>
<div>
<strong>Account data:</strong>
<ul style="margin-top: 0em;">
<li>Site access: <em>{{if .Account.Data.SiteAccess}}Granted{{else}}Not granted{{end}}</em></li>
<li>GOCDB access: <em>{{if .Account.Data.GOCDBAccess}}Granted{{else}}Not granted{{end}}</em></li>
</ul>
</div>
<div>
<form id="form" method="POST" class="box" style="width: 100%;">
<div>
<button type="button" onClick="handleAccountSettings();">Account settings</button>
<button type="button" onClick="handleEditAccount();">Edit account</button>
<span style="width: 25px;">&nbsp;</span>
{{if .Account.Data.SiteAccess}}
<button type="button" onClick="handleSiteSettings();">Site settings</button>
<span style="width: 25px;">&nbsp;</span>
{{end}}
<button type="button" onClick="handleLogout();" style="float: right;">Logout</button>
</div>
<div style="margin-top: 0.5em;">
<button type="button" onClick="handleRequestAccess('Site');" {{if .Account.Data.SiteAccess}}disabled{{end}}>Request Site access</button>
<button type="button" onClick="handleRequestAccess('GOCDB');" {{if .Account.Data.GOCDBAccess}}disabled{{end}}>Request GOCDB access</button>
</div>
</form>
</div>
<div style="font-size: 90%; margin-top: 1em;">
<div>
<div>Notes:</div>
<ul style="margin-top: 0em;">
<li>The <em>Site access</em> allows you to access and modify the global configuration of your site.</li>
<li>The <em>GOCDB access</em> allows you to log into the central database where all site metadata is stored.</li>
</ul>
</div>
<div>
<div>Quick links:</div>
<ul style="margin-top: 0em;">
<li><a href="https://gocdb.sciencemesh.uni-muenster.de" target="_blank">Central Database (GOCDB)</a></li>
<li><a href="https://developer.sciencemesh.io/docs/technical-documentation/central-database/" target="_blank">Central Database documentation</a></li>
</ul>
</div>
</div>
`
-225
View File
@@ -1,225 +0,0 @@
// Copyright 2018-2020 CERN
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// In applying this license, CERN does not waive the privileges and immunities
// granted to it by virtue of its status as an Intergovernmental Organization
// or submit itself to any jurisdiction.
package account
import (
"net/http"
"net/url"
"github.com/opencloud-eu/reva/v2/pkg/siteacc/account/contact"
"github.com/opencloud-eu/reva/v2/pkg/siteacc/account/edit"
"github.com/opencloud-eu/reva/v2/pkg/siteacc/account/login"
"github.com/opencloud-eu/reva/v2/pkg/siteacc/account/manage"
"github.com/opencloud-eu/reva/v2/pkg/siteacc/account/registration"
"github.com/opencloud-eu/reva/v2/pkg/siteacc/account/settings"
"github.com/opencloud-eu/reva/v2/pkg/siteacc/account/site"
"github.com/opencloud-eu/reva/v2/pkg/siteacc/config"
"github.com/opencloud-eu/reva/v2/pkg/siteacc/data"
"github.com/opencloud-eu/reva/v2/pkg/siteacc/html"
"github.com/pkg/errors"
"github.com/rs/zerolog"
"golang.org/x/text/cases"
"golang.org/x/text/language"
)
// Panel represents the account panel.
type Panel struct {
html.PanelProvider
conf *config.Configuration
htmlPanel *html.Panel
}
const (
templateLogin = "login"
templateManage = "manage"
templateSettings = "settings"
templateEdit = "edit"
templateSite = "site"
templateContact = "contact"
templateRegistration = "register"
)
func (panel *Panel) initialize(conf *config.Configuration, log *zerolog.Logger) error {
if conf == nil {
return errors.Errorf("no configuration provided")
}
panel.conf = conf
// Create the internal HTML panel
htmlPanel, err := html.NewPanel("account-panel", panel, conf, log)
if err != nil {
return errors.Wrap(err, "unable to create the account panel")
}
panel.htmlPanel = htmlPanel
// Add all templates
if err := panel.htmlPanel.AddTemplate(templateLogin, &login.PanelTemplate{}); err != nil {
return errors.Wrap(err, "unable to create the login template")
}
if err := panel.htmlPanel.AddTemplate(templateManage, &manage.PanelTemplate{}); err != nil {
return errors.Wrap(err, "unable to create the account management template")
}
if err := panel.htmlPanel.AddTemplate(templateSettings, &settings.PanelTemplate{}); err != nil {
return errors.Wrap(err, "unable to create the account settings template")
}
if err := panel.htmlPanel.AddTemplate(templateEdit, &edit.PanelTemplate{}); err != nil {
return errors.Wrap(err, "unable to create the account editing template")
}
if err := panel.htmlPanel.AddTemplate(templateSite, &site.PanelTemplate{}); err != nil {
return errors.Wrap(err, "unable to create the site template")
}
if err := panel.htmlPanel.AddTemplate(templateContact, &contact.PanelTemplate{}); err != nil {
return errors.Wrap(err, "unable to create the contact template")
}
if err := panel.htmlPanel.AddTemplate(templateRegistration, &registration.PanelTemplate{}); err != nil {
return errors.Wrap(err, "unable to create the registration template")
}
return nil
}
// GetActiveTemplate returns the name of the active template.
func (panel *Panel) GetActiveTemplate(session *html.Session, path string) string {
validPaths := []string{templateLogin, templateManage, templateSettings, templateEdit, templateSite, templateContact, templateRegistration}
template := templateLogin
// Only allow valid template paths; redirect to the login page otherwise
for _, valid := range validPaths {
if valid == path {
template = path
break
}
}
return template
}
// PreExecute is called before the actual template is being executed.
func (panel *Panel) PreExecute(session *html.Session, path string, w http.ResponseWriter, r *http.Request) (html.ExecutionResult, error) {
protectedPaths := []string{templateManage, templateSettings, templateEdit, templateSite, templateContact}
if user := session.LoggedInUser(); user != nil {
switch path {
case templateSite:
// If the logged in user doesn't have site access, redirect him back to the main account page
if !user.Account.Data.SiteAccess {
return panel.redirect(templateManage, w, r), nil
}
case templateLogin:
case templateRegistration:
// If a user is logged in and tries to login or register again, redirect to the main account page
return panel.redirect(templateManage, w, r), nil
}
} else {
// If no user is logged in, redirect protected paths to the login page
for _, protected := range protectedPaths {
if protected == path {
return panel.redirect(templateLogin, w, r), nil
}
}
}
return html.ContinueExecution, nil
}
// Execute generates the HTTP output of the form and writes it to the response writer.
func (panel *Panel) Execute(w http.ResponseWriter, r *http.Request, session *html.Session) error {
dataProvider := func(*html.Session) interface{} {
flatValues := make(map[string]string, len(r.URL.Query()))
c := cases.Title(language.Und)
for k, v := range r.URL.Query() {
flatValues[c.String(k)] = v[0]
}
availSites, err := data.QueryAvailableSites(panel.conf.Mentix.URL, panel.conf.Mentix.DataEndpoint)
if err != nil {
return errors.Wrap(err, "unable to query available sites")
}
type TemplateData struct {
Site *data.Site
Account *data.Account
Params map[string]string
Titles []string
Sites []data.SiteInformation
}
tplData := TemplateData{
Site: nil,
Account: nil,
Params: flatValues,
Titles: []string{"Mr", "Mrs", "Ms", "Prof", "Dr"},
Sites: availSites,
}
if user := session.LoggedInUser(); user != nil {
tplData.Site = panel.cloneUserSite(user.Site)
tplData.Account = user.Account
}
return tplData
}
return panel.htmlPanel.Execute(w, r, session, dataProvider)
}
func (panel *Panel) redirect(path string, w http.ResponseWriter, r *http.Request) html.ExecutionResult {
// Check if the original (full) URI path is stored in the request header; if not, use the request URI to get the path
fullPath := r.Header.Get("X-Replaced-Path")
if fullPath == "" {
uri, _ := url.Parse(r.RequestURI)
fullPath = uri.Path
}
// Modify the original request URL by replacing the path parameter
newURL, _ := url.Parse(fullPath)
params := newURL.Query()
params.Del("path")
params.Add("path", path)
newURL.RawQuery = params.Encode()
http.Redirect(w, r, newURL.String(), http.StatusFound)
return html.AbortExecution
}
func (panel *Panel) cloneUserSite(site *data.Site) *data.Site {
// Clone the user's site and decrypt the credentials for the panel
siteClone := site.Clone(true)
id, secret, err := site.Config.TestClientCredentials.Get(panel.conf.Security.CredentialsPassphrase)
if err == nil {
siteClone.Config.TestClientCredentials.ID = id
siteClone.Config.TestClientCredentials.Secret = secret
}
return siteClone
}
// NewPanel creates a new account panel.
func NewPanel(conf *config.Configuration, log *zerolog.Logger) (*Panel, error) {
form := &Panel{}
if err := form.initialize(conf, log); err != nil {
return nil, errors.Wrap(err, "unable to initialize the account panel")
}
return form, nil
}
@@ -1,51 +0,0 @@
// Copyright 2018-2020 CERN
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// In applying this license, CERN does not waive the privileges and immunities
// granted to it by virtue of its status as an Intergovernmental Organization
// or submit itself to any jurisdiction.
package registration
import "github.com/opencloud-eu/reva/v2/pkg/siteacc/html"
// PanelTemplate is the content provider for the registration form.
type PanelTemplate struct {
html.ContentProvider
}
// GetTitle returns the title of the panel.
func (template *PanelTemplate) GetTitle() string {
return "ScienceMesh Site Administrator Account Registration"
}
// GetCaption returns the caption which is displayed on the panel.
func (template *PanelTemplate) GetCaption() string {
return "Welcome to the ScienceMesh Site Administrator Account Registration!"
}
// GetContentJavaScript delivers additional JavaScript code.
func (template *PanelTemplate) GetContentJavaScript() string {
return tplJavaScript
}
// GetContentStyleSheet delivers additional stylesheet code.
func (template *PanelTemplate) GetContentStyleSheet() string {
return tplStyleSheet
}
// GetContentBody delivers the actual body content.
func (template *PanelTemplate) GetContentBody() string {
return tplBody
}
Loaded 100 of 167 files, more files were not shown because too many files have changed in this diff. Show more