Populates @microsoft.graph.downloadUrl on file driveItems when requested
via $select and implements GET .../items/{item-id}/content as a 302 to
the same URL: a by-id WebDAV URL signed with OC_URL_SIGNING_SECRET,
verified by the proxy, valid for 30 minutes. Folders answer 404 on
/content and never carry the annotation.
The annotation is available on the driveItem stat, the children and
root children listings and the share jail item endpoint.
drive.WebUrl, driveItem.WebUrl and the public share link WebUrl all
derive from the same config value (graph.spaces.webdav_base), but only
driveItem.WebUrl used the pre-parsed BaseGraphService.publicBaseURL.
The other two re-parsed the config on every call.
Add a webURLForResource method on BaseGraphService for the /f/<id> URLs
(used twice, with a *string return matching the libregraph DriveItem
field shape), and inline g.publicBaseURL for the single /s/<token>
share-link case. Convert cs3ResourceToDriveItem and formatDriveItems
from free functions into BaseGraphService methods so they pick up
logger and publicBaseURL from the receiver. This also aligns them with
the surrounding code: BaseGraphService already exposes ~15 similar
methods, so the two free functions were the odd ones out.
Net: all three WebUrls are now constructed from a single pre-parsed
URL, and the (g.logger, g.publicBaseURL) plumbing at 7 call sites
disappears.
$expand=thumbnails was only honored by sharedByMe and sharedWithMe. The
driveItem stat, the children listing and the root children listing now
honor it as well, so a client that lists a folder learns which items have
a preview instead of guessing from the mime type.
The thumbnails are set from the resource info the listing already has, so
a later preview check that needs more than the mime type has a single
place to sit. The two share listings carry driveItems only, they keep
matching on the mime type but share the url building.
* introduce a fix in the metrics tools introduced with 8821053ff8 in
order to deal with a struct that has fields that are not exported
(as is the case here)
* introduce a func in the toplevel metrics package that registers a
single metric, as that needs to be done explicitly for unexported
ones
* in the graph service, use RegisterAll and BuildInfo from the
toplevel metrics package, as introduced with 8821053ff8
Introducing gowrap as a build-time tool to generate interface delegate
structs from templates:
* added as a 'make go-generate' target in services/graph,
* added as a build-time dependency in .bingo/
Introduce an LDAP client abstraction interface to be able to wrap the
go-ldap client API with metrics transparently (and possibly hooks and
such in the future), in order to use delegation patterns to measure the
time LDAP (client) operations take to finish, as well as to track their
results (success, failure, not-found).
Has two implementations that are generated using gowrap:
* a go-ldap adapter implementation that directly delegates to a go-ldap
connection
* a time measuring and metrics collecting implementation that delegates
to another LdapClient
The metrics collecting one is disabled by default, can be enabled with
GRAPH_LDAP_METRICS_DISABLE=false
It collects durations of outbound LDAP client operations into a histogram, as
well as the number of concurrent outbound LDAP operations in a gauge (via an
atomic int and a gauge function, as that performs best).
Add an HTTP middleware that measures how long Graph HTTP API requests
take, storing taken time into a histogram along with labels for
* method,
* path pattern (from the chi routes),
* Graph API version prefix,
* Graph API resource name,
* and the resulting status code.
It also tracks the number of concurrent inbound Graph API HTTP requests
using a gauge (also using an atomic int and a gauge function).
Disabled by default, can be enabled with
GRAPH_HTTP_METRICS_DISABLE=false
Add Backend and EducationBackend delegate implementations that measure
execution time on the level of the higher API call operations there
(CreateUser, DeleteUser, ..., CreateSchool, ...), generated using
gowrap.
Disabled by default, can be enabled with
GRAPH_IDENTITY_BACKEND_METRICS_DISABLE=false
Also added a small k6 script to produce some read-only load on the Graph
API, for a casual test of the metrics, as well as k6 in mise.toml.
Make an internal changes to how singular LDAP entry searches work in the LDAP
identity backends:
* check whether searches for a singular entry returns more than one
result, in which case a new error TooManyResults is returned, instead
of leaving that undetected, blindly taking the first result, and
potentially risking data inconsistencies
Improve the loggers in identity backends by adding attributes for their
request targets (Reva gateway address or LDAP URI, respectively).
Also add a "backend" attribute for all Graph API logs (set to "ldap" or
"cs3"), to help debug potential issues, and remove them from all the logger
debug calls at the beginning of each LDAP-related function as those should
really be part of the logger and set beforehand.
The LDAP identity backend logger also has two new attributes to help
debugging with logs:
* write (bool): whether write operations are enabled
* refint (bool): whether refint is enabled or not
Also adds a dedicated counter metric for user password change operations.
Minor campfire improvements:
* add a constructor func for the CS3 backend
* add a constructor func for the LDAP backend
* in the LDAP identity backend, in searchLDAPEntryByFilter (used by all
search/get public functions), errors that occur when performing LDAP
SEARCH operations were blindly mapped to a ItemNotFound error,
instead of being analyzed as it could be caused by a technical error
* in the requireadmin middleware, add debug logging to explain why a
request is denied
* when an LDAP password change fails because the user entry was not
found in LDAP, we now have a log message that tracks that
In the scope of the broader issue #1312, this PR deals with performing
those changes for the `graph` service, namely to add the ability to
disable the HTTP API or to disable the events API handler by
configuration.
It also adds metrics for the events processing, and tests for the events
processing.
The previous implementation was combining the HTTP server service and
the events consumption, which is why this PR refactors the composition
of those services:
* the event consumption has been moved into its own service
* the identity.Backend is created beforehand, and then injected as a
collaborator in both the HTTP service as well as the event consumer
service
It also adds metrics, mainly for the event processing.
To encourage re-use in latter implementations and changes, it also
introduces two top-level package changes:
* internal/eventstest/events_test_helpers: contains a TestBus
implementation to unit-test event consumers without NATS
* internal/metricstest/metrics_test_helpers: contains assertion
functions to test Prometheus metrics
Build the bleve and OpenSearch index mappings from the Go struct via
reflection (json tags + per-field overrides) instead of hand-rolled
mappings and hit deserializers. New mapping package: BleveBuildMapping,
OpenSearchBuildMapping, Deserialize[T], PrepareForIndex; field decoding is
fail-soft. Mtime is typed as a date so mtime ranges are chronological on
both backends. Route CS3 facet parsing through mapping.DeserializeStringMap.
The any-valued (bleve hit) and string-valued (CS3 metadata) deserializers
share one generic fillStruct walker with a per-value setLeaf callback.
Addresses review feedback: a sub-router middleware can re-route after all,
as long as it rewrites chi.RouteContext().RoutePath instead of r.URL.Path.
Once chi has descended into a sub-router its routeHTTP matches against
rctx.RoutePath and ignores r.URL.Path, which is why the earlier top-level
registration was thought to be required.
Move ResolveGraphPath off the top-level mux.Use and attach it to the
/drives/{driveID} sub-routers (v1.0 + v1beta1). It now reads driveID from
chi.URLParam and matches against RoutePath (the part below the drive), so
the regexes drop the version + drive prefix entirely.
RoutePath carries the percent-encoded wire form (Graph.ServeHTTP sets
RawPath), so the captured driveID/itemID/path are PathUnescape'd exactly
once - reproducing the decoded r.URL.Path a normal handler would see,
without the previous RawPath/EscapedPath workaround. r.URL.Path is now
left untouched; only chi's internal RoutePath is rewritten.
Tests are reworked to drive requests through a chi router mirroring the
production nesting (including the Graph.ServeHTTP RawPath behavior), so
chi's sub-router middleware ordering, RoutePath encoding and param
round-trip are all covered indirectly: a chi upgrade that changes any of
them fails these tests instead of silently breaking colon-path lookups.
Adds explicit coverage for percent-decoding (%20, %252F) and the `$`/`!`
sub-delimiter id round-trip.
Adds a chi middleware that detects MS Graph colon-syntax URLs and rewrites
them to the canonical /items/{itemID}/... form before chi performs route
matching. Existing handlers, routes, and GetDriveAndItemIDParam stay
unchanged.
Two URL shapes are recognized at both /v1.0 and /v1beta1:
/drives/{driveID}/root:/<path>[:/<suffix>][:]
/drives/{driveID}/items/{itemID}:/<relativePath>[:/<suffix>][:]
Path resolution runs as the request user via CS3 Stat. Both NOT_FOUND and
PERMISSION_DENIED collapse to a 404 response so existence isn't disclosed
to unauthorized callers. URLs without colon syntax fast-path through with
a single substring check. The original URL is stashed in request context
under OriginalPathContextKey for downstream tracing/logging.
The middleware is registered as a top-level mux.Use so it runs before any
route matching: chi middleware on a sub-router runs after the prefix is
matched but cannot redirect to a different leaf route. Top-level
middleware lets URL rewriting actually re-route the request.
Tests cover regex matching across versions, all rewrite variants
(root/items anchored, with/without suffix, with/without trailing colon,
deep paths), NOT_FOUND -> 404, PERMISSION_DENIED -> 404 (security: no
existence disclosure), and original-URL preservation in request context.
Allow a "permission denied error" from reva to bubble up to the client.
Reva was fixed to return "permission denied" only when the space to be
delete can actually be listed by the user. Other wise it will return
"not found". See reva commit 1bf72cb76394671f373e87f15f23f978cf41ab08.
So when a user with the 'can manage' role tries to purge an already
disabled space it will now get "Forbidden" status instead of a "Not
found".
Also fixes the expected status codes in the tests.
GetRoleDefinition/s does now handle l10n correctly. Previsouly it just
returned the non-localized string. What made things worse was that
ListPermissions() mutated global list of available roles and replaced
some strings with translated values depending on the `accept-language`
header. Which resulted in GetRoleDefinition returning results in mixed
localization depending on who/what called ListPermissions before.
Fixes: #2800
The Libre Graph spec declares webUrl on driveItem but
cs3ResourceToDriveItem never set it. Build the /f/<resource-id>
permalink off the configured public base URL (same source drives.go
reads for drive.webUrl) and parse it once at service construction
so callers can use it as a plain field.
This gets us rid of quite a bit of special casing for space permission.
Also provides us with "real" permission IDs instead of those faked
"u:<userid>" ones.
Instead of doing a lexical sort we sort the drive/space in a more
"natural" order so that e.g. "Space 2" is sorted before "Space 10".
Related: https://github.com/opencloud-eu/web/issues/2430
Dominik Schmidt