Commit Graph

22440 Commits

Author SHA1 Message Date
Pascal Bleser
df21fdf2e2 docs(groupware): add services/groupware/DEVELOPER.md 2025-12-09 09:15:37 +01:00
Pascal Bleser
6224ded8b5 refactor(groupware): add max requests check
* move jmap.request() to jmap.Client.request() and pass the Session
   and a Logger to introduce checking the number of methodCalls within a
   request not exceeding the limit of the Session, as well as error
   handling and logging there instead of in each caller

 * a few bugfixes:
   - add a few missing Send() calls in logs
   - correct the response tag matching for
     GetMailboxChangesForMultipleAccounts
   - fix typo in Identity.ReplyTo json serialization rune
   - fix response tag in pkg/jmap/testdata/mailboxes1.json after
     changing them to be prefixed by the accountId
2025-12-09 09:15:37 +01:00
Pascal Bleser
0da72cf346 groupware: minor typo fixes 2025-12-09 09:15:37 +01:00
Pascal Bleser
aab08dd3de chore(groupware): add launcher for OC + containers for services
* add a launcher for running OpenCloud from within VSCode, but using
   third-party services that are running within the docker compose
   'full' example setup
2025-12-09 09:15:37 +01:00
Pascal Bleser
f470462ead feat(groupware): add fetching all mailboxes for all accounts
* add URL to retrieve all the mailboxes for all the accounts of a user,
   as a first use-case for an all-accounts operation, as
   /accounts/all/mailboxes

 * add URL to retrieve mailbox changes for all the mailboxes of all the
   accounts of a user, as a first use-case for an all-accounts
   operation, as /accounts/all/mailboxes/changes

 * change the defaultAccountId from '*' to '_', as '*' rather indicates
   "all" than "default", and we might want to use that for "all
   accounts" operations in the future

 * refactor(groupware): remove the accountId parameter from the logger()
   function, as it is not used anyways, but also confusing for
   operations that support multiple account ids
2025-12-09 09:15:37 +01:00
Pascal Bleser
62cace14fe docs(groupware): OpenAPI improvements
* refactor some pkg/jmap and groupware methods to make more sense from
   an API point-of-view

 * add path parameter documentation, but automate it by injecting their
   definition into the OpenAPI YAML tree that is extracted from the
   source code using go-swagger as it is too cumbersome, repetitive and
   error-prine to document them in the source code; wrote a TypeScript
   file apidoc-process.ts to do so

 * add generating an offline HTML file for the OpenAPI documentation
   using redocly, and injecting a favicon into the resulting HTML; wrote
   a TypeScript file apidoc-postprocess-html.ts to do so
2025-12-09 09:15:37 +01:00
Pascal Bleser
a8c2beac3a test(groupware): add testcontainers based jmap test
* adds pkg/jmap/jmap_integration_test.go

 * uses ghcr.io/stalwartlabs/stalwart:v0.13.2-alpine

 * can be disabled by setting one of the following environment
   variables, in the same fashion as ca0493b28
   - CI=woodpecker
   - CI_SYSTEM_NAME=woodpecker
   - USE_TESTCONTAINERS=false

 * dependencies:
   - bump github.com/go-test/deep from 1.1.0 to 1.1.1
   - add github.com/cention-sany/utf7
   - add github.com/dustinkirkland/golang-petname
   - add github.com/emersion/go-imap/v2
   - add github.com/emersion/go-message
   - add github.com/emersion/go-sasl
   - add github.com/go-crypt/crypt
   - add github.com/go-crypt/x
   - add github.com/gogs/chardet
   - add github.com/inbucket/html2text
   - add github.com/jhilleryerd/enmime/v2
   - add github.com/ssor/bom
   - add gopkg.in/loremipsum.v1
2025-12-09 09:15:37 +01:00
Pascal Bleser
1b732b8bff refactor(groupware): session cache and DNS autodiscovery
* move the logging of the username and session state away from pkg/jmap
   and into services/groupware

 * introduce more decoupling for the session cache, as well as moving
   the implementation into groupware_session.go
2025-12-09 09:15:37 +01:00
Pascal Bleser
33cc3365ee groupware: add DNS auto-discovery (currently disabled, needs testing) 2025-12-09 09:15:37 +01:00
Pascal Bleser
3e48284295 add a .gitignore entry for debug binaries built by VSCode when running OpenCloud 2025-12-09 09:15:37 +01:00
Pascal Bleser
c9a4bb94cd groupware: session handling improvements
* remove the baseurl from the JMAP client configuration, and pass it to
   the session retrieval functions instead, as that is really the only
   place where it is relevant, and we gain flexibility to discover that
   session URL differently in the future without having to touch the
   JMAP client

 * move the default account identifier handling from the JMAP package to
   the Groupware one, as it really has nothing to do with JMAP itself,
   and is an opinionated feature of the Groupware REST API instead

 * add an event listener interface for JMAP events to be more flexible
   and universal, typically for metrics that are defined on the API
   level that uses the JMAP client

 * add errors for when default accounts cannot be determined

 * split groupware_framework.go into groupware_framework.go,
   groupware_request.go and groupware_response.go

 * move the accountId logging into the Groupware level instead of JMAP
   since it can also be relevant to other operations that might be
   worthy of logging before the JMAP client is even invoked
2025-12-09 09:15:37 +01:00
Pascal Bleser
3ac4bcfeeb groupware: fix debug server, was missing a lot of configuration options and was binding to :80 2025-12-09 09:15:37 +01:00
Pascal Bleser
58583a66bb docs(groupware): add Groupware related ADRs 2025-12-09 09:15:37 +01:00
Pascal Bleser
1d6433f1c9 refactor(groupware): logging and metrics improvements
* some minor code refactorings to improve logging and metrics

 * more code documentation
2025-12-09 09:15:37 +01:00
Pascal Bleser
fc938bc4bc jmap: minor logging improvements 2025-12-09 09:15:37 +01:00
Pascal Bleser
724c44567b groupware: improve metrics
* implement more metrics, in a more streamlined fashion

 * use concurrent-map to store SSE streams instead of a regular map with
   one big lock that will not scale when it grows, causing too much
   contention on that one lock

 * while testing error metrics, noticed a few bugs with error handling
   when Stalwart is down: fixed
2025-12-09 09:15:37 +01:00
Pascal Bleser
1fc75a9091 groupware: jmap: add metrics 2025-12-09 09:15:37 +01:00
Pascal Bleser
5b51804744 groupware: implement metrics
* implement a framework for metrics, with a few exemplary ones
2025-12-09 09:15:37 +01:00
Pascal Bleser
0f3dac0280 groupware: Etag handling
* implement correct Etag and If-None-Match handling, responding with
   304 Not Modified if they match

 * introduce SessionState and State string type aliases to ensure we are
   using the correct fields for those, respectively

 * extract the SessionState from the JMAP response bodies in the
   groupware framework instead of having to do that in every single
   groupware API

 * use uint instead of int in some places to clarify that the values are
   >= 0

 * trace-log how long a Session was held in cache before being evicted

 * add Trace-Id header handling: add to response when specified in
   request, and implement a custom request logger to include it as a
   field

 * implement a more compact trace-logging of all the methods and URIs
   that are served, to put them into a single log entry instead of
   creating one log entry for every URI
2025-12-09 09:15:36 +01:00
Pascal Bleser
d214cfa2b7 groupware: initial related emails implementation with SSE 2025-12-09 09:15:36 +01:00
Pascal Bleser
f97bc0e875 groupware: add /bootstrap
* add a GET /accounts/{a}/boostrap URI that delivers the same as GET /
   but also mailboxes for a given account, in case the UI remembers the
   last used account identifier, to avoid an additional roundtrip

 * streamline the use of simpleError()

 * add logging of errors at the calling site

 * add logging of evictions of Sessions from the cache

 * change default Session cache TTL to 5min instead of 30sec
2025-12-09 09:15:36 +01:00
Pascal Bleser
72ee47fdca groupware: swagger API documentation improvements
* add more documentation for properties

 * fixes after a bit of trial-and-error with go-swagger

 * fix email filter marshalling when there are no search criteria

 * introduce an apidoc.yml that contains Swagger data and is merged when
   generating the swagger.yml from sources
2025-12-09 09:15:36 +01:00
Pascal Bleser
8d9c3b0c4e Groupware improvements
* ensure that all the jmap responses contain the SessionState

 * implement missing errors that were marked as TODO

 * moved common functions from pkg/jmap and pkg/services/groupware to
   pkg/log and pkg/structs to commonalize them across both source trees

 * implement error handling for SetError occurences

 * Email: replace anonymous map[string]bool for mailbox rights with a
   MailboxRights struct, as the keys are well-defined, which allows for
   properly documenting them

 * introduce ObjectType as an "enum"

 * fix JSON marshalling and unmarshalling of EmailBodyStructure

 * move the swagger documentation structs from groupware_api.go to
   groupware_docs.go

 * fix: change verb for /groupware/accounts/*/vacation from POST to PUT
2025-12-09 09:15:36 +01:00
Pascal Bleser
084eb005e3 groupware: minor email searching response improvements + started implementing vacation response setting API 2025-12-09 09:15:36 +01:00
Pascal Bleser
2bdbc5a42e groupware: add identities of all accounts to the index resource 2025-12-09 09:15:36 +01:00
Pascal Bleser
0a5d13b916 groupware: fix email search, add variant that includes the full emails 2025-12-09 09:15:36 +01:00
Pascal Bleser
446a98dd62 groupware: fix email search, add variant that includes the full emails 2025-12-09 09:15:36 +01:00
Pascal Bleser
e6441e58d4 Groupware: refactor jmap package, implement Email/set, EmailSubmission
* refactor the jmap package to split it into several files as the
   jmap.api.go file was becoming too unwieldy

 * refactor the Groupware handler function response to be a Response
   object, to be more future-proof and avoid adding more and more
   return parameters while handling "no content" response as well

 * more godoc for the JMAP model

 * add Email creation, updating, deleting (Email/set,
   EmailSubmission/set)

 * add endpoints
   - POST /accounts/{accountid}/messages
   - PATCH|PUT /accounts/{accountid}/messages/{messageid}
   - DELETE /accounts/{accountid}/messages/{messageid}
2025-12-09 09:15:36 +01:00
Pascal Bleser
a64223fe7d groupware: implement message search with snippets 2025-12-09 09:15:36 +01:00
Pascal Bleser
6e4918b50b groupware: blob streaming (upload and download) 2025-12-09 09:15:36 +01:00
Pascal Bleser
ac8d2587c9 groupware: more JMAP operations implementation 2025-12-09 09:15:36 +01:00
Pascal Bleser
0d2a5e992c groupware: further implementation and improvements 2025-12-09 09:15:36 +01:00
Pascal Bleser
1b9249ecba upgrade Stalwart to 0.13.2 2025-12-09 09:15:36 +01:00
Pascal Bleser
94c932d6a7 refactored the Session object, refactored the services/groupware directory, and started Swagger documentation implementation 2025-12-09 09:15:36 +01:00
Pascal Bleser
8be4d11a5e groupware: refactoring the API mechanisms 2025-12-09 09:15:36 +01:00
Pascal Bleser
2191b1d011 groupware: implement JSON:API's error response format, with a revamped error handling in jmap and services/groupware 2025-12-09 09:15:36 +01:00
Pascal Bleser
07522ce79a Refactor groupware service after ADR decision on the Groupware API
* after having decided that the Groupware API should be a standalone
   independent custom REST API that is using JMAP data models as much as
   possible,
 * removed Groupware APIs from the Graph service
 * moved Groupware implementation to the Groupware service, and
   refactored a few things accordingly
2025-12-09 09:15:36 +01:00
Pascal Bleser
d544efdec7 Groupware and jmap: cleanup and API documentation 2025-12-09 09:15:36 +01:00
Pascal Bleser
8df4ef67a2 groupware: remove unneeded messages.go that was a remainder from an earlier implementation attempt, which also fixes compilation issues due to changes in main 2025-12-09 09:15:36 +01:00
Pascal Bleser
2412e64cc5 opencloud_full: upgrade Stalwart to 0.12.5, and use the ghcr.io container repository to avoid Hub limits 2025-12-09 09:15:36 +01:00
Pascal Bleser
3f8076aa46 Groupware improvements: refactoring, k6 tests
* refactored the models to be strongly typed with structs and mapstruct
   to decompose the dynamic parts of the JMAP payloads

 * externalized large JSON strings for tests into .json files under
   testdata/

 * added a couple of fantasy Graph groupware APIs to explore further
   options

 * added k6 scripts to test those graph/me/messages APIs, with a setup
   program to set up users in LDAP, fill their IMAP inbox, activate them
   in Stalwart, cleaning things up, etc...
2025-12-09 09:15:36 +01:00
Pascal Bleser
5920291ec7 fix Stalwart LDAP configuration 2025-12-09 09:15:36 +01:00
Pascal Bleser
772a902f6d Use password policy overlay in LDAP and configure Stalwart to use it 2025-12-09 09:15:36 +01:00
Pascal Bleser
e0ea733489 upgrade Stalwart to 0.12.4 2025-12-09 09:15:36 +01:00
Pascal Bleser
22e51bd4a1 groupware: removed debugging logs 2025-12-09 09:15:36 +01:00
Pascal Bleser
0f47e3aca8 jwkset: remove debugging printlns 2025-12-09 09:15:36 +01:00
Pascal Bleser
b3766abba5 auth-api: fix: was missing newly introduced metrics 2025-12-09 09:15:36 +01:00
Pascal Bleser
71cddaaf3c groupware and jmap improvements and refactoring 2025-12-09 09:15:36 +01:00
Pascal Bleser
a6cdb4e863 upgrade Stalwart to 0.12 2025-12-09 09:15:36 +01:00
Pascal Bleser
02f33bd1d8 minor corrections to the Stalwart configuration 2025-12-09 09:15:36 +01:00