Commit Graph

153 Commits

Author SHA1 Message Date
Jörn Friedrich Dreyer
c7bbdd2eae add special nodes to cache key
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-05-03 11:09:55 +02:00
Jörn Friedrich Dreyer
13369f8367 only stat if property is set
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-05-03 09:18:19 +02:00
Jörn Friedrich Dreyer
92533895c5 some fixes for the tests and noisy logs
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-05-03 00:21:57 +02:00
Jörn Friedrich Dreyer
40eba0909d fix mocks for stat based special resource handling
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-05-02 22:04:04 +02:00
Jörn Friedrich Dreyer
f2b86d8115 stat based on default paths
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-05-02 17:12:49 +02:00
Jörn Friedrich Dreyer
ca638ddc51 cache special drive items until space root changes
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-05-02 11:44:27 +02:00
Jörn Friedrich Dreyer
4bfda6183c Read quota from opaque (#6187)
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-04-28 15:55:29 +02:00
André Duffeck
0eca2a08f6 Speed up me/drives by gathering space information in parallel 2023-04-28 10:53:24 +02:00
Jörn Friedrich Dreyer
3489de1c42 listing drives sould use the user filter (#6103)
* listing drives shsould use the user filter

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* fix status code

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* check correct error

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Adapt expected failures

* Bump reva

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: André Duffeck <andre.duffeck@firondu.de>
2023-04-28 10:06:40 +02:00
Andre Duffeck
77bb3d8bcd [full-ci] Refactor stores (#6019)
* Streamline the store implementation with and into reva

* Adapt to the cache/store refactoring in reva

* Streamline config options and their env vars

* Apply suggestions from code review

Co-authored-by: Martin <github@diemattels.at>

* Use the same database for all stores

* Bump reva

* Configure stat and filemetadata cache separately

* Fix default config

---------

Co-authored-by: Martin <github@diemattels.at>
2023-04-24 15:13:35 +02:00
jkoberg
8f5c255225 return badrequest when exporting gdpr for other user
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-04-24 12:17:05 +02:00
Prajwol Amatya
e0b6ca0e29 [tests-only][full-ci] api test to get personal drive information of other users (#6018)
* api test to get personal drive information of other users

* fix the broken personal drive listing

* removed scenario from expected failure after issue fixed

---------

Co-authored-by: Michael Barz <mbarz@owncloud.com>
2023-04-21 16:59:16 +05:45
Daniel Swärd
0d6500a225 graph: Add comment to fix and remove expected test failures
Fixes: #5083
2023-04-20 10:32:40 +02:00
Daniel Swärd
8d789ce24d graph: Try to fix problems with unescaping values. 2023-04-20 09:33:23 +02:00
Daniël Franke
30edab9f69 Search users by username instead of by email
As some setups don't have email addresses setup or reuse email
addresses, the keycloak search has to be done by username as that
is guaranteed to always be unique and defined.

This PR changes that.
2023-04-18 11:43:13 +02:00
Daniël Franke
d81065c1b6 Adds JSON tags to PII report.
This will make the JSON export look a bit prettier.
2023-04-14 15:09:54 +02:00
jkoberg
c9d0e876f7 use mail instead id to get user
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-04-14 15:09:54 +02:00
jkoberg
cf6b1eac25 add events to personalDataExport
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-04-14 15:09:54 +02:00
Daniël Franke
8e32b4f5da Integrate keycloak and events data into graph.
This PR adds the data from keycloak and events into the GDPR export of
the graph service.
2023-04-14 15:09:54 +02:00
Ralf Haferkamp
120887abcc graph: new config option GRAPH_LDAP_GROUP_CREATE_BASE_DN
By setting GRAPH_LDAP_GROUP_CREATE_BASE_DN a distinct subtree can be
configured where new LDAP groups are created. That subtree needs to be
subordinate to GRAPH_LDAP_GROUP_BASE_DN. All groups outside for
GRAPH_LDAP_GROUP_CREATE_BASE_DN are considered read-only and only groups
below that DN can be updated and deleted.

This is introduced for a pretty specific usecase where most groups are managed
in an external source (e.g. a read-only replica of an LDAP tree). But we still
want to allow the local administrator to create groups in a writeable subtree
attached to that replica.
2023-04-04 15:56:57 +02:00
Prajwol Amatya
7baac4f050 [tests-only][full-ci] Api test for user trying to set own personal space quota (#5979)
* api test for user trying to set their own personal space quota

* removed duplicate scenarios for set quota

* updated expected scenario

* fix wrong status code

* updated expected failure scenario after wrong status code fix

---------

Co-authored-by: Michael Barz <mbarz@owncloud.com>
2023-04-04 12:41:34 +05:45
Daniel Swärd
620940ec3f graph: Add strict json decoding to return errors when receiving unknown parameters 2023-04-03 15:39:23 +02:00
jkoberg
e0b07dee70 use correct status code
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-03-30 14:57:41 +02:00
kobergj
db5c84816b fix typo in graph service
Co-authored-by: Daniël Franke <df@ponc.tech>
2023-03-30 14:17:56 +02:00
jkoberg
e906c9f1ec emit event after finishing data extraction
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-03-30 10:06:32 +02:00
jkoberg
25d1347a76 create folders if they are not existing
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-03-30 09:49:56 +02:00
jkoberg
9f5b33f661 use reva gateway client and mocks
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-03-29 16:06:10 +02:00
jkoberg
ebd730c778 use location from request
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-03-29 16:02:43 +02:00
jkoberg
442c53f796 add gdpr export endpoint
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-03-29 16:02:42 +02:00
Daniel Swärd
ab07525602 graph: Allow updating of group name via PATCH request 2023-03-28 13:06:53 +02:00
Jörn Friedrich Dreyer
6bec87f582 Proxy accesstoken cache store (#5829)
* refactor middleware options

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* use ocmemstore micro store implementaiton for token cache

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* refactor ocis store options, support redis sentinel

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* align cache configuration

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* database and tabe are used to build prefixes for inmemory stores

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add global persistent store options to userlog config

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* log cache errors but continue

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* drup unnecessary type conversion

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Better description for the default userinfo ttl

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* use global cache options for even more caches

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* don't log userinfo cache misses

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* default to stock memory store

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* use correct mem store typo string

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* split cache options, doc cleanup

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* mint and write userinfo to cache async

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* use hashed token as key

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* go mod tidy

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update docs

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update cache store naming

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* bring back depreceted ocis-pkg/store package for backwards compatability

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update changelog

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Apply suggestions from code review

Co-authored-by: kobergj <jkoberg@owncloud.com>

* revert ocis-pkg/cache to store rename

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add waiting for each step 50 milliseconds

* starlack check

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: kobergj <jkoberg@owncloud.com>
Co-authored-by: Viktor Scharf <scharf.vi@gmail.com>
2023-03-22 15:21:57 +01:00
Daniel Swärd
40ab22195d Merge pull request #5836 from owncloud/excds/bugfix/Fix_updating_accountEnabled_for_educationUser
graph: Fix so that accountEnabled updates work for educationUser.
2023-03-21 11:44:53 +01:00
Jörn Friedrich Dreyer
474c4b848d upgrade to go1.20 and auto set go mem limit (#5732)
* upgrade to go1.19 and set go mem limit

* create ocis-pkg memlimit package

* use std automemlimit import

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* import automemlimit in every ocis service, drop ocis-pkg/memlimit package

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* bump go to 1.20

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* drop unused config options and env vars

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update all version numbers, add doc

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* fix lint

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update bingo and mockery

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* bump golangci-lint

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* fix selector test

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Update changelog/unreleased/enhancement-memlimit.md

Co-authored-by: kobergj <juliankoberg@googlemail.com>

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Willy Kloucek <wkloucek@owncloud.com>
Co-authored-by: kobergj <juliankoberg@googlemail.com>
2023-03-20 17:22:02 +01:00
jkoberg
a503ace11f don't check quota permission in graph
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-03-17 12:51:28 +01:00
Daniel Swärd
525638588d graph: Fix so that accountEnabled updates work for educationUser. 2023-03-17 10:16:39 +01:00
Ralf Haferkamp
81c0f21f78 graph: Allow to disable the default role assignment on user creation
Introduces a switch ('GRAPH_ASSIGN_DEFAULT_USER_ROLE') to allow to disable the assignment of the default role "User" to newly created users.
This will be used for setups where the role-assignments are populated either manually or during first login (e.g. from OIDC claims)
2023-03-15 15:42:57 +01:00
Ralf Haferkamp
b4b0f7abe7 graph: Skip default role assignment for education/users
For the education/users endpoints the role assignment does not
need to happen when creating a new user.
2023-03-15 15:42:57 +01:00
Daniel Swärd
6327edf574 graph: Change so that educationUsers can be created without email. 2023-03-15 14:43:22 +01:00
Daniel Swärd
23ba180e8a graph: Use userType property to distinguish between Member and Guest accounts
Fixes 5603

- Calling POST /graph/v1.0/users with userType not set will create a user as "Member"
- Calling POST /graph/v1.0/users with userType set as "Member" or "Guest" will create a user as "Member" or "Guest"
- Calling POST /graph/v1.0/users with userType set as anything but "Member" or "Guest" returns error
- Calling POST /graph/v1.0/education/users with userType not set will create a user as "Member"
- Calling POST /graph/v1.0/education/users with userType set as "Member" will create a user as "Member" and primary role as parameter specifies
- Calling POST /graph/v1.0/education/users with userType set as "Guest" will create a user as "Guest" and primary role as parameter specifies
- Calling POST /graph/v1.0/education/users with userType not set as anything but "Member" or "Guest" returns error
- Calling PATCH on /users or /education/users will update attribute in the same way as for POST
2023-03-15 12:05:29 +01:00
David Christofas
4c3df7277d return the correct status code when updating a share fails 2023-03-09 14:07:05 +01:00
Daniel Swärd
ca746106d8 graph: Allow disabling user by adding to local group
* A new config option for disabling users with the options "none", "attribute" and "group".
* When set to "none", there will be no enabledAttribute returned in user info and trying to change enabledAttribute will return an error
* Disable/enable group name DN as config parameter
* Adding/removing users to specified group on user update
* Changing log level for service initialization failure to error
* Adding helper methods to check if user is enabled/disabled + tests

Fixes #5554
2023-03-06 09:31:23 +01:00
Ralf Haferkamp
db752dc5ac graph: Add support for "or" filter on /users
This add support for combining filters on the /users with a logical "or" operation.
E.g. the filter:

"filter=(memberOf/any(m:m/id eq 509a9dcd-bb37-4f4f-a01a-19dca27d9cfa) or memberOf/any(m:m/id eq 262982c1-2362-4afa-bfdf-8cbfef64a06e)"

will return all users that are a member of either of the referenced group.

Closes: #5667
2023-03-02 13:26:00 +01:00
David Christofas
0fadc79c32 make graph/education API errors more consistent 2023-02-28 15:25:45 +01:00
kobergj
e631b74032 Merge pull request #5600 from kobergj/EventHistory
Eventhistory Service
2023-02-21 14:08:01 +01:00
Ralf Haferkamp
1dab0f7a54 graph: Improve appRoleAssignemts filters
This should improve the processing of filters for appRoleAssignments
a bit when combining them with other filters. We try to avoid reading
the full user list if possible. And delay the processing of an
appRoleAssignments filter so we can apply it on a subset of user.

E.g. a filter:

`appRoleAssignments/any(m:m/appRoleId eq 71881883-1768-46bd-a24d-a356a2afdf7f) and memberOf/any(m:m/id eq 509a9dcd-bb37-4f4f-a01a-19dca27d9cfa)`

Will be reordered to first process the memberOf filter (which can be
executed without reading the full user list) and only apply the
appRoleAssignments filter on the resultset of the memberOf filter.
2023-02-21 12:30:59 +01:00
Ralf Haferkamp
1552f6df5a Add support for $filter on appRoleAssignment
This add support for filtering on the `appRoleAssignment` relation of
users.  E.g.

```
$filter=appRoleAssignments/any(m:m/appRoleId eq '262982c1-2362-4afa-bfdf-8cbfef64a06e')
```

combining it with a filter on groupMemberShip does also work:

```
$filter=memberOf/any(m:m/id eq '262982c1-2362-4afa-bfdf-8cbfef64a06e') and appRoleAssignments/any(m:m/appRoleId eq 'd7beeea8-8ff4-406b-8fb6-ab2dd81e6b11')
```

The filter is still very inefficient as it always needs to get the full
users list. We need to adapt it to only filter on a subset of users when
using this filter 'and' combined with other filters.

Closes: #5488
2023-02-21 12:30:59 +01:00
jkoberg
c9dfb778a9 sharpen eventhistory service
Signed-off-by: jkoberg <jkoberg@owncloud.com>
2023-02-21 10:51:32 +01:00
Jörn Friedrich Dreyer
92923f6801 Add http endpoint to list permissions (#5571)
* Add http endpoint to list permissions

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* extract handler registration

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* use generated protobuf

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* update permissions mock in graph service

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* add unit test

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* return correct userid

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* assert error message type in tests

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-02-15 14:24:19 +01:00
Ralf Haferkamp
b7ec7c92c4 graph: Initial support for $filter in /users (#5533)
This adds some initial support for using $filter (as defined in the
odata spec) on the /users endpoint. Currently the following filters are
supported:

A single filter on `id` property of the `memberOf` relation of users.
To list all users that are members of a specific group:

```
curl 'https://localhost:9200/graph/v1.0/users?$filter=memberOf/any(m:m/id eq '262982c1-2362-4afa-bfdf-8cbfef64a06e')
```

A logical AND filteri on the `id` property of the `memberOf` relation of users.

`$filter=memberOf/any(m:m/id eq 262982c1-2362-4afa-bfdf-8cbfef64a06e) and memberOf/any(m:m/id eq 6040aa17-9c64-4fef-9bd0-77234d71bad0)`

This will cause at least two queries on the identity backend. The `and`
operation is performed locally.

Closes: #5487
2023-02-14 10:32:32 +01:00
Daniel Swärd
e853b98918 Merge pull request #5534 from owncloud/excds/feature/5411_support_adding_and_removing_of_teachers_for_classes
graph: Add support for listing/adding/removing teachers to a class
2023-02-13 11:48:58 +01:00