Introducing gowrap as a build-time tool to generate interface delegate
structs from templates:
* added as a 'make go-generate' target in services/graph,
* added as a build-time dependency in .bingo/
Introduce an LDAP client abstraction interface to be able to wrap the
go-ldap client API with metrics transparently (and possibly hooks and
such in the future), in order to use delegation patterns to measure the
time LDAP (client) operations take to finish, as well as to track their
results (success, failure, not-found).
Has two implementations that are generated using gowrap:
* a go-ldap adapter implementation that directly delegates to a go-ldap
connection
* a time measuring and metrics collecting implementation that delegates
to another LdapClient
The metrics collecting one is disabled by default, can be enabled with
GRAPH_LDAP_METRICS_DISABLE=false
It collects durations of outbound LDAP client operations into a histogram, as
well as the number of concurrent outbound LDAP operations in a gauge (via an
atomic int and a gauge function, as that performs best).
Add an HTTP middleware that measures how long Graph HTTP API requests
take, storing taken time into a histogram along with labels for
* method,
* path pattern (from the chi routes),
* Graph API version prefix,
* Graph API resource name,
* and the resulting status code.
It also tracks the number of concurrent inbound Graph API HTTP requests
using a gauge (also using an atomic int and a gauge function).
Disabled by default, can be enabled with
GRAPH_HTTP_METRICS_DISABLE=false
Add Backend and EducationBackend delegate implementations that measure
execution time on the level of the higher API call operations there
(CreateUser, DeleteUser, ..., CreateSchool, ...), generated using
gowrap.
Disabled by default, can be enabled with
GRAPH_IDENTITY_BACKEND_METRICS_DISABLE=false
Also added a small k6 script to produce some read-only load on the Graph
API, for a casual test of the metrics, as well as k6 in mise.toml.
Make an internal changes to how singular LDAP entry searches work in the LDAP
identity backends:
* check whether searches for a singular entry returns more than one
result, in which case a new error TooManyResults is returned, instead
of leaving that undetected, blindly taking the first result, and
potentially risking data inconsistencies
Improve the loggers in identity backends by adding attributes for their
request targets (Reva gateway address or LDAP URI, respectively).
Also add a "backend" attribute for all Graph API logs (set to "ldap" or
"cs3"), to help debug potential issues, and remove them from all the logger
debug calls at the beginning of each LDAP-related function as those should
really be part of the logger and set beforehand.
The LDAP identity backend logger also has two new attributes to help
debugging with logs:
* write (bool): whether write operations are enabled
* refint (bool): whether refint is enabled or not
Also adds a dedicated counter metric for user password change operations.
Minor campfire improvements:
* add a constructor func for the CS3 backend
* add a constructor func for the LDAP backend
* in the LDAP identity backend, in searchLDAPEntryByFilter (used by all
search/get public functions), errors that occur when performing LDAP
SEARCH operations were blindly mapped to a ItemNotFound error,
instead of being analyzed as it could be caused by a technical error
* in the requireadmin middleware, add debug logging to explain why a
request is denied
* when an LDAP password change fails because the user entry was not
found in LDAP, we now have a log message that tracks that
In the scope of the broader issue #1312, this PR deals with performing
those changes for the `graph` service, namely to add the ability to
disable the HTTP API or to disable the events API handler by
configuration.
It also adds metrics for the events processing, and tests for the events
processing.
The previous implementation was combining the HTTP server service and
the events consumption, which is why this PR refactors the composition
of those services:
* the event consumption has been moved into its own service
* the identity.Backend is created beforehand, and then injected as a
collaborator in both the HTTP service as well as the event consumer
service
It also adds metrics, mainly for the event processing.
To encourage re-use in latter implementations and changes, it also
introduces two top-level package changes:
* internal/eventstest/events_test_helpers: contains a TestBus
implementation to unit-test event consumers without NATS
* internal/metricstest/metrics_test_helpers: contains assertion
functions to test Prometheus metrics
from now on, not all unified roles are enabled by default, instead the available roles are hand-picked in the default setup.
For advanced use-cases, the administrator is capable to enable the desired set of available roles.
Picking roles is not easy since the uid is NOT humanly readable, therefore a cli is contained which lists the available, disabled and enabled roles.
By setting GRAPH_LDAP_GROUP_CREATE_BASE_DN a distinct subtree can be
configured where new LDAP groups are created. That subtree needs to be
subordinate to GRAPH_LDAP_GROUP_BASE_DN. All groups outside for
GRAPH_LDAP_GROUP_CREATE_BASE_DN are considered read-only and only groups
below that DN can be updated and deleted.
This is introduced for a pretty specific usecase where most groups are managed
in an external source (e.g. a read-only replica of an LDAP tree). But we still
want to allow the local administrator to create groups in a writeable subtree
attached to that replica.
* bump libregraph-go lib
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* add appRoleAssignment stubs
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* add get application stub
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* fetch appRoles for application from settings service
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* initial list appRoleAssignments implementation
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* initial create appRoleAssignment implementation, extract assignmentToAppRoleAssignment, configurable app id and displayname
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* initial delete appRoleAssignment implementation, changed error handling and logging
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* initial expand appRoleAssignment on users
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* test user expand appRoleAssignment
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* test appRoleAssignment
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* fix education test by actually using the mocked roleManager
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* test getapplication
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* list assignments
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* use common not exists error handling
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* default to just 'ownCloud Infinite Scale' as application name
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* fix store_test
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* roll application uuid on init
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* fix tests
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* extract method
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
* Apply suggestions from code review
Co-authored-by: Michael Barz <mbarz@owncloud.com>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Michael Barz <mbarz@owncloud.com>
Pascal Bleser