Commit Graph
24545 Commits
Author SHA1 Message Date
Pascal Bleser ee03db134d groupware: add Groupware integration testing
* split off the Stalwart integration test boilerplate into a new
   jmaptest package, in order to be able to use it from the groupware
   package as well

 * remove the JMAP Session patching in tests and, instead, pick random
   free local ports for HTTP and IMAPS manually, and set those
   explicitly in the Stalwart testcontainer, so that we can set the
   PUBLIC_URL environment variable that Stalwart then uses to define the
   URLs that are served as part of the JMAP Sessions (API URL, etc...)
2026-10-30 23:20:51 +01:00
Pascal Bleser e3d635c25c groupware: introduce a new package 'jmaptest' in order to be able to reuse the Stalwart test fixture for Groupware tests 2026-10-30 23:20:51 +01:00
Pascal Bleser 73817a2b66 groupware: vscode launcher: enable /metrics endpoint on port 9203 2026-10-30 23:20:51 +01:00
Pascal Bleser a585c1ace8 groupware: add body tracing to WS, fix metrics registrations, add operation identifiers for metrics 2026-10-30 23:20:51 +01:00
Pascal Bleser ad66875349 groupware: add better HTTP logging
* introduce configuration settings for TLS insecurity, tracing requests
   and responses

 * use more secure implementation when replacing JMAP placeholders in
   URL templates, path escaping them to avoid injection

 * add more efficient tracing of HTTP requests and responses between the
   Groupware backend and the JMAP server, with the possibility of
   specifying a maximum body size to trace, to avoid blowing up the logs

 * use more efficient string building for HTTP Authorization headers

 * change internal API to use streams (io.Reader) instead of reading
   JSON responses fully into memory before parsing them
2026-10-30 23:20:51 +01:00
Pascal Bleser 473086531d groupware: slightly improve the implementation of e77250b7c4 2026-10-30 23:20:51 +01:00
Pascal Bleser 0560d7c9e9 groupware: opencloud_full: change Stalwart container healthcheck interval to 10m to reduce the noise in the logs 2026-10-30 23:20:51 +01:00
Pascal Bleser 56e561bd86 groupware: add workaround when using the master/impersonation account: in that case, don't use impersonation but the impersonation account directly 2026-10-30 23:20:51 +01:00
Pascal Bleser 8e678e797a groupware: enable GROUPWARE_ENABLE_MOCK_DATA for testing from within VSCode 2026-10-30 23:20:51 +01:00
Pascal Bleser 4af171aff8 groupware: fully implement suppliers for addressbooks and contacts
* implement supplier templating to allow addressbooks and contacts to
   come from various sources ("suppliers", thus) as opposed to solely
   from JMAP

 * add creating, deleting, updating, getting changes

 * add some search filter parameters, currently incomplete
2026-10-30 23:20:51 +01:00
Pascal Bleser 24ee690692 groupware: add suppliers support for object changes 2026-10-30 23:20:51 +01:00
Pascal Bleser 7c3aabb41c groupware: rename supplier implementations files 2026-10-30 23:20:51 +01:00
Pascal Bleser b5714c31a1 groupware: add support for multi-supplier getting a Contact by ID 2026-10-30 23:20:51 +01:00
Pascal Bleser 2ab47208ec groupware: rebump go from 1.25.0 to 1.25.8, as done by dependeabot in 5e27a70 2026-10-30 23:20:51 +01:00
Pascal Bleser 3ee15455d0 groupware: fix query objects in a container with multiple suppliers
* by adding a predicate that can analyze a filter and inform the
   template methods whether the filter is applicable for that supplier
   or not

 * this fixed /groupware/accounts/.../addressbooks/.../contacts
2026-10-30 23:20:51 +01:00
Pascal Bleser 64b03203f1 groupware: fix NPE when flattening durations in slist() 2026-10-30 23:20:51 +01:00
Pascal Bleser 9b78c967a3 groupware: upgrade traefik in opencloud_full from 3.3.1 to 3.7.5 2026-10-30 23:20:51 +01:00
Pascal Bleser d747b908a9 stalwart.yml: remove image tag for stalwart-import, which makes a 'docker compose pull' fail needlessly 2026-10-30 23:20:51 +01:00
Pascal Bleser f65a719381 stalwart-snapshot-passwords: fix for Node, and beautify 2026-10-30 23:20:51 +01:00
Pascal Bleser 17e5bfb71d groupware: introduce temporary configuration property GROUPWARE_ENABLE_MOCK_DATA that controls whether the mock addressbook and mock contacts should be injected or not; default is false/disabled since it is not fully functional yet 2026-10-30 23:20:51 +01:00
Pascal Bleser 3bcfea817f groupware: upgrade Stalwart to 0.16.9 in opencloud_full 2026-10-30 23:20:51 +01:00
Pascal Bleser 1a909bedb7 groupware: upgrade Stalwart to 0.16.9 for integration tests 2026-10-30 23:20:51 +01:00
Pascal Bleser 176598efc2 groupware: remove middleware.RealIP, as it has been deprecated and we currently do not have a use for it 2026-10-30 23:20:51 +01:00
Pascal Bleser a210032103 groupware: downgrade to Go 1.25 since our building images are only available for 1.25 at the moment
* retrofit from using self-referencing generics parameters that were
   introduced with Go 1.26, will re-enable when we upgrade to 1.26

 * re-introduce a 'ptr' func since we don't have the new 'new' func yet
   that comes with 1.26
2026-10-30 23:20:51 +01:00
Pascal Bleser a99b9b10e2 opencloud_full: add healthchecks for traefik, opencloud and stalwart to the docker compose project 2026-10-30 23:20:51 +01:00
Pascal Bleser b61ddf5758 groupware: fix GROUPWARE_JMAP_MASTER_USERNAME to Stalwart 0.16 requirements for using the primary email instead of the username 2026-10-30 23:20:51 +01:00
Pascal Bleser 4624236f23 groupware: fix building the stalwart:cli container
* fix the inline dockerfile for building the stalwart:cli container,
   cannot be based on scratch since we need a shell and wget, and it
   should thus retain the alpine container as its base

 * remove stdin_open and tty from the container attributes, not needed
2026-10-30 23:20:51 +01:00
Pascal Bleser 4cde585d39 groupware: fix the 'opencloud-certs' container to set necessary permissions on the files or the opencloud container can't access them 2026-10-30 23:20:51 +01:00
Pascal Bleser 38fc26e1ea groupware: update developer guide
* add instructions and container to create a key and certificate pair
   for the built-in IDM LDAP as we need it to listen on LDAPS for
   Stalwart, which has been disabled by default with
   https://github.com/opencloud-eu/opencloud/pull/2880

 * add environment variables to point to the certificates

 * update services/groupware/DEVELOPERS.md with up-to-date instructions,
   using Stalwart 0.16
2026-10-30 23:20:51 +01:00
Pascal Bleser 050323147d groupware: minor fix for the mock addressbook supplier 2026-10-30 23:20:51 +01:00
Pascal Bleser 439c3aa6e2 groupware: VSCode launcher: change GROUPWARE_JMAP_MASTER_USERNAME from 'master' to 'admin@example.org' to make it work with Stalwart >= 0.16 2026-10-30 23:20:51 +01:00
Pascal Bleser e6bf657ec9 groupware: move prefixed writer into export_integration_test.go + upgrade integration tests to Stalwart 0.16.8 + update Stalwart configuration to include necessary default roles 2026-10-30 23:20:51 +01:00
Pascal Bleser d057fc5e7b groupware: upgrade opencloud_full to Stalwart 0.16.8 and fix configuration by adding default roles as well as adding the 'impersonate' permission on the admin@example.org user 2026-10-30 23:20:51 +01:00
Pascal Bleser 527c6e668c groupware: jmap: add a test for Invocation seriaization 2026-10-30 23:20:51 +01:00
Pascal Bleser 5b936bae7b groupware: port opencloud_full to Stalwart 0.16 (only the built-in IDM LDAP option for now) 2026-10-30 23:20:51 +01:00
Pascal Bleser b8c0c06efa groupware: simplify JMAP integration tests with Stalwart 0.16
* don't run the Stalwart container in recovery mode first, just run it
   in "proper" mode, import the configuration and use it like that,
   which only requires skipping the "destroy" steps

 * add code documentation and logging

 * replace the text templating of the Stalwart snapshot by doing the
   same as the formatting script was doing, but directly in Go, removing
   an additional step to perform when modifying a Stalwart
   configuration to use in the integration tests
2026-10-30 23:20:51 +01:00
Pascal Bleser 13b74567ee groupware: port tests to Stalwart 0.16.7 2026-10-30 23:20:51 +01:00
Pascal Bleser 1c0124780a groupware: fix unit tests 2026-10-30 23:20:47 +01:00
Pascal Bleser 140bd899c2 groupware: add tracking of backend call durations
* add new configuration setting GROUPWARE_SEND_DURATIONS_RESPONSE
   (defaults to false)

 * keep track of lists of durations of backend calls

 * when enabled, report them as response headers Durations (human
   readable) and Durations-Nanos (as raw nanosecond values for machine
   consumption)
2026-10-30 23:20:47 +01:00
Pascal Bleser 492fb3251d groupware: add strongly typed aliases for AccountId, PrincipalId and SupplierId
Purpose is to make APIs and parameters easier to understand, since plain
strings are used all over the place for all sorts of identifiers.
2026-10-30 23:20:47 +01:00
Pascal Bleser 2845f2baae groupware: remove unneeded jmap.QueryCommand methods
* remove the following methods on the jmap.QueryCommand interface, as
   they ended up not being needed:
   - GetPosition()
   - GetAnchor()
   - GetAnchorOffset()
   - GetLimit()
2026-10-30 23:20:47 +01:00
Pascal Bleser 69aa24e35c groupware: refactoring for pagination and support for multiple query suppliers
* refactor APIs in JMAP and Groupware in order to implement pagination
   across multiple accountIds and multiple suppliers (currently
   implemented using a mock supplier for contacts)

 * requires go 1.26 due to use of self-reflecting generics type
   constraints

 * still missing: query criteria and sorting parameters

 * still missing: multi-accountId support for emails

 * errors are now all just 'error' in the APIs, instead of the
   specialized implementations, and are interpreted dynamically where
   necessary in order to transform them into HTTP responses

 * remove position, anchor, anchorOffset as individual query parameters
   as we now only support a 'next=...' token for subsequent pages
   (except in emails for now), and use jmap.QueryParams instead; those
   tokens have a header character for the format, followed by a JSON
   encoded QueryParams map, all wrapped into base62 to make it clearer
   that it is meant to be an opaque token, and not a parameter clients
   should tinker with or construct themselves

 * introduce QueryParamsSupplier as an interface to provide QueryParams
   for various scenarios (single supplier, multiple supplier, ...) per
   accountId

 * implement multi-supplier template methods slist and squery
2026-10-30 23:20:47 +01:00
Pascal Bleser 77484a59f8 groupware: API documentation updates 2026-10-30 23:20:47 +01:00
Pascal Bleser 8b145b0b27 groupware: refactor responses to a jmap.Response object
* in the JMAP API as well as in several places in the Groupware
   framework, use a single jmap.Response[T] object to return the
   payload, the language, the session state and the etag/state instead
   of individual multi-valued return values
2026-10-30 23:20:47 +01:00
Pascal Bleser bedf49b7a2 groupware: fix test logic error in the JMAP ws integration test 2026-10-30 23:20:47 +01:00
Pascal Bleser 458af63774 groupware: omit "position" when "anchor" is specified in queries
* the "position" attribute is always set to 0 when using an anchor for
   pagination, but it does not accurately reflect the position and thus
   we decided to remove the attribute from the resulting object for
   clarity

 * omit "canCalculateChanges" attribute when its value is "true", which
   is going to be the case in 100% of calls against Stalwart
2026-10-30 23:20:47 +01:00
Pascal Bleser 99828addb9 groupware: add support for anchor and anchor offset pagination
* add query parameters 'anchor' and 'offset':
   - anchor is an object identifier
   - offset is a numeric offset relative to the anchor
2026-10-30 23:20:47 +01:00
Pascal Bleser 5dc0064876 groupware: fix use of ?limit=0
* JMAP query limit of 0 is synonymous with "no limit", but we actually
   want to be able to perform queries without any results, for cases
   where we only want to count the total number of objects, and also
   because it makes more sense semantically

 * introduce query parameter validation checks, in order to only allow
   query parameters that are actually supported, which is going to be
   useful during development of clients
2026-10-30 23:20:47 +01:00
Pascal Bleser 1b2402143d groupware: add documentation for SearchResultsTemplate + update pnpm 2026-10-30 23:20:47 +01:00
Pascal Bleser dc93134a91 groupware: rename 'offset' to 'position' to be consistent with the JMAP specification 2026-10-30 23:20:47 +01:00