Commit Graph

23409 Commits

Author SHA1 Message Date
Pascal Bleser
af3e40bd02 groupware: upgrade Stalwart in devtools from 0.13.2 to 0.13.4
* changes from 0.13.4:
   - JMAP: Protocol layer rewrite for zero-copy deserialization and
     architectural improvements.
   - IMAP: Unbounded memory allocation in request parser
     (CVE-2025-61600)
   - IMAP: Wrong permission checked for GETACL.
   - JMAP: References to previous method fail when there are no results
     (stalwartlabs#1507).
   - JMAP: Enforce quota checks on Blob/copy.
   - JMAP: Mailbox/get fails without accountId argument (stalwartlabs#1936).
   - JMAP: Do not return invalidProperties when email update doesn't
     contain changes (stalwartlabs#1139)
   - iTIP: Include date properties in REPLY (stalwartlabs#2102).
   - OIDC: Do not set username field if it is the same as the email field.
   - Telemetry: Fix calculateMetrics housekeeper task (stalwartlabs#2155).
   - Directory: Always use rsplit to extract the domain part from email
     addresses.

  * changes from 0.13.3:
   - CLI: Health checks
   - WebDAV: Assisted discovery v2
   - iTIP: Do not send a REPLY when deleting an event that was not
     accepted.
   - iTIP: Include event details in REPLY messages (stalwart#2102).
   - iTIP: Add organizer to iMIP replies if missing to deal with MS
     Exchange 2010 bug.
   - OIDC: Do not overwrite locally defined aliases (stalwart#2065).
   - HTTP: Scan ban should only be triggered by HTTP parse errors.
   - HTTP: Skip scanner fail2ban checks when the proxy client IP can't
     be parsed (stalwart#2121).
   - JMAP: Do not allow roles to be removed from system mailboxes
     (stalwart#1977).
   - JMAP WS: Fix panic when using invalid server url.
   - SMTP: Do no send EHLO twice when STARTTLS is unavailable
     (stalwart#2050).
   - IMAP: Allow ENABLE UTF8 in IMAPrev1.
   - IMAP: Include administer permission in ACL responses.
   - IMAP: Add owner rights to ACL get responses.
   - IMAP: Do not auto-train Bayes when moving messages from Junk to
     Trash.
   - IMAP/ManageSieve: Increase maximum quoted argument size
     (stalwart#2039).
   - CalDAV: Limit recurrence expansions in calendar reports
     (CVE-2025-59045).
   - WebDAV: Do not fix percent encoding on WebDAV FS (stalwart#2036).
2026-07-09 14:15:50 +02:00
Pascal Bleser
31fde546be groupware: add bootstrapping on / with quotas for all accounts 2026-07-09 14:15:50 +02:00
Pascal Bleser
1f37cb1952 groupware: add /quota for all accounts 2026-07-09 14:15:50 +02:00
Pascal Bleser
47c0aec39f groupware: improve instructions in DEVELOPER.md 2026-07-09 14:15:50 +02:00
Pascal Bleser
8181c131bd groupware: update @redocly/cli: 2.2.2 -> 2.3.0 2026-07-09 14:15:50 +02:00
Pascal Bleser
ab9c9f19dd groupware: add quota API + add support for Accept-Language and Content-Language 2026-07-09 14:15:50 +02:00
Pascal Bleser
df25f8875b groupware: minor improvements to the DEVELOPER.md 2026-07-09 14:15:50 +02:00
Pascal Bleser
2c2244cd85 groupware: add flag to currently ignore session capability checks for calendars, contacts and tasks, as those are not implemented in Stalwart yet; will need to remove it in the future 2026-07-09 14:15:50 +02:00
Pascal Bleser
d6f757b84f groupware: add JMAP capability checking (in part: for contacts, calendars, tasks) 2026-07-09 14:15:50 +02:00
Pascal Bleser
253770cfef /auth: add SkipXAccessToken:true 2026-07-09 14:15:50 +02:00
Pascal Bleser
79fd20e7b2 groupware: add mock endpoints for tasklists and tasks 2026-07-09 14:15:50 +02:00
Pascal Bleser
27890bc3e5 groupware: implement JMAP Task specification 2026-07-09 14:15:50 +02:00
Pascal Bleser
822edb1900 groupware: more mock data, added missing JMAP types 2026-07-09 14:15:50 +02:00
Pascal Bleser
9079be072c groupware: add mock endpoints for addressbooks and contacts 2026-07-09 14:15:50 +02:00
Pascal Bleser
3cf1a1702e opencloud_full: also keep the 'Trace-Id' HTTP header 2026-07-09 14:15:50 +02:00
Pascal Bleser
8c3fea6473 implement JSCalendar (RFC 8984) 2026-07-09 14:15:50 +02:00
Pascal Bleser
4c5de9d543 services/groupware/DEVELOPER.md: adapt to new path for the opencloud_full deployment 2026-07-09 14:15:50 +02:00
Pascal Bleser
ab08217213 JSContact: refactored after full test coverage, stronger typing for enumerations 2026-07-09 14:15:50 +02:00
Pascal Bleser
d15ac805ca Implement JSContact (RFC9553) Model
* add pkg/jscontact with the implementation of the RFC9553 data model

 * add JMAP Calendar session capabilities support in pkg/jmap
2026-07-09 14:15:50 +02:00
Pascal Bleser
9ef4ea7f80 Docker Compose Groupware improvements
* made a few changes in order to further simplify the setup for
   developers of the Groupware backend

 * add STALWART_DOMAIN to deployments/examples/opencloud_full/.env

 * adapt the Stalwart configuration file to not set server.hostname and,
   instead, pick it up from /etc/hostname, which is set by Docker
   Compose as we can use default values for STALWART_DOMAIN there, in an
   analogous fashion to the other containers in that project

 * add config/keycloak/clients/groupware.json to avoid requiring manual
   configuration of Keycloak via the admin web UI

 * Stalwart container:
   - listen for SMTPS on :1465
   - remove the stalwart-logs volume, not needed (logs are going to
     stdout)

 * updated services/groupware/DEVELOPER.md:
   - refer to a variable OCDIR to make instructions more copy-pasteable
   - remove manual Keycloak configuration section as it is now obsolete,
     replaced by provisioning a configuration file instead
2026-07-09 14:15:50 +02:00
Pascal Bleser
371f0c5537 start websocket implementation, add endpoint for email summaries
* feat(groupware): start implementing JMAP websocket support for push
   notifications (unfinished)

 * groupware: add GetLatestEmailsSummaryForAllAccounts

 * add new vendored dependency: github.com/gorilla/websocket

 * jmap: add QueryEmailSummaries

 * openapi: start adding examples

 * openapi: add new tooling for api-examples.yaml injection

 * apidoc-process.ts: make it more typescript-y

 * bump @redocly/cli from 2.0.8 to latest 2.2.0
2026-07-09 14:15:50 +02:00
Pascal Bleser
d85e37b1de feat(groupware): add WebsocketEndpoint to the JMAP Session 2026-07-09 14:15:50 +02:00
Pascal Bleser
dd3da84b4d refactor(groupware): just use a function for the attachment picker
Minor: be more Go idiomatic: just use a function to pick the attachment
from an Email's attachment list instead of using an interface with
multiple iplementation structs.
2026-07-09 14:15:50 +02:00
Pascal Bleser
7b95424bde groupware: improved attachment APIs
* feat(groupware): add /accounts/{}/emails/{}/attachments

 * feat(groupware): add
   /accounts/{}/emails/{}/attachments?partId=&name=&blobId=
2026-07-09 14:15:50 +02:00
Pascal Bleser
174bad61c1 jmap: modify GetBlob -> GetBlobMetadata
* fix(jmap): fix bug where CommandBlobUpload was used instead of
   CommandBlobGet in GetBlob (now GetBlobMetadata)

 * we currently don't need a variant of BlobGetCommand that also
   retrieves the content of the blob, instead we only use it for
   retrieving metadata about it
2026-07-09 14:15:50 +02:00
Pascal Bleser
e5978a26ca fix(groupware): update DEVELOPER.md imap-filler usage since it was updated to use flags instead of environment variables 2026-07-09 14:15:49 +02:00
Pascal Bleser
c035033626 docs(groupware): more developer instructions 2026-07-09 14:15:49 +02:00
Pascal Bleser
37494d94e3 more updates to the Groupware DEVELOPER.md 2026-07-09 14:15:49 +02:00
Pascal Bleser
d61efa347d docs(groupware): add configuration instructions to DEVELOPER.md 2026-07-09 14:15:49 +02:00
Pascal Bleser
ac4d53b39a fix(groupware): fix JMAP error handling
* the JMAP error handling was not working properly, fixed it and added
   error definitions accordingly

 * add operations to retrieve mailbox roles and mailboxes by role for
   all accounts
2026-07-09 14:15:49 +02:00
Pascal Bleser
4792430bff refactor(groupware): rename "Messages" to "Email" everywhere
There was really no reason to go with "Messages" as far as the
vocabulary of the Groupware API goes, since the objects those APIs serve
are "Emails", to stick with the wording of the JMAP specification.
2026-07-09 14:15:49 +02:00
Pascal Bleser
9475a39641 refactor(groupware): use a function for multi-account method call IDs
* introduce a function 'mcid' to assemble method call IDs per account
   instead of doing that inline in each function, in case the rules for
   doing so change in the future
2026-07-09 14:15:49 +02:00
Pascal Bleser
106c17a39a docs(groupware): add services/groupware/DEVELOPER.md 2026-07-09 14:15:49 +02:00
Pascal Bleser
64a1d8e2b8 refactor(groupware): add max requests check
* move jmap.request() to jmap.Client.request() and pass the Session
   and a Logger to introduce checking the number of methodCalls within a
   request not exceeding the limit of the Session, as well as error
   handling and logging there instead of in each caller

 * a few bugfixes:
   - add a few missing Send() calls in logs
   - correct the response tag matching for
     GetMailboxChangesForMultipleAccounts
   - fix typo in Identity.ReplyTo json serialization rune
   - fix response tag in pkg/jmap/testdata/mailboxes1.json after
     changing them to be prefixed by the accountId
2026-07-09 14:15:49 +02:00
Pascal Bleser
38e3a6a188 groupware: minor typo fixes 2026-07-09 14:15:49 +02:00
Pascal Bleser
f914025578 chore(groupware): add launcher for OC + containers for services
* add a launcher for running OpenCloud from within VSCode, but using
   third-party services that are running within the docker compose
   'full' example setup
2026-07-09 14:15:49 +02:00
Pascal Bleser
080b098358 feat(groupware): add fetching all mailboxes for all accounts
* add URL to retrieve all the mailboxes for all the accounts of a user,
   as a first use-case for an all-accounts operation, as
   /accounts/all/mailboxes

 * add URL to retrieve mailbox changes for all the mailboxes of all the
   accounts of a user, as a first use-case for an all-accounts
   operation, as /accounts/all/mailboxes/changes

 * change the defaultAccountId from '*' to '_', as '*' rather indicates
   "all" than "default", and we might want to use that for "all
   accounts" operations in the future

 * refactor(groupware): remove the accountId parameter from the logger()
   function, as it is not used anyways, but also confusing for
   operations that support multiple account ids
2026-07-09 14:15:49 +02:00
Pascal Bleser
082ad6c848 docs(groupware): OpenAPI improvements
* refactor some pkg/jmap and groupware methods to make more sense from
   an API point-of-view

 * add path parameter documentation, but automate it by injecting their
   definition into the OpenAPI YAML tree that is extracted from the
   source code using go-swagger as it is too cumbersome, repetitive and
   error-prine to document them in the source code; wrote a TypeScript
   file apidoc-process.ts to do so

 * add generating an offline HTML file for the OpenAPI documentation
   using redocly, and injecting a favicon into the resulting HTML; wrote
   a TypeScript file apidoc-postprocess-html.ts to do so
2026-07-09 14:15:49 +02:00
Pascal Bleser
83405b1f62 test(groupware): add testcontainers based jmap test
* adds pkg/jmap/jmap_integration_test.go

 * uses ghcr.io/stalwartlabs/stalwart:v0.13.2-alpine

 * can be disabled by setting one of the following environment
   variables, in the same fashion as ca0493b28
   - CI=woodpecker
   - CI_SYSTEM_NAME=woodpecker
   - USE_TESTCONTAINERS=false

 * dependencies:
   - bump github.com/go-test/deep from 1.1.0 to 1.1.1
   - add github.com/cention-sany/utf7
   - add github.com/dustinkirkland/golang-petname
   - add github.com/emersion/go-imap/v2
   - add github.com/emersion/go-message
   - add github.com/emersion/go-sasl
   - add github.com/go-crypt/crypt
   - add github.com/go-crypt/x
   - add github.com/gogs/chardet
   - add github.com/inbucket/html2text
   - add github.com/jhilleryerd/enmime/v2
   - add github.com/ssor/bom
   - add gopkg.in/loremipsum.v1
2026-07-09 14:15:49 +02:00
Pascal Bleser
b27628307e refactor(groupware): session cache and DNS autodiscovery
* move the logging of the username and session state away from pkg/jmap
   and into services/groupware

 * introduce more decoupling for the session cache, as well as moving
   the implementation into groupware_session.go
2026-07-09 14:15:49 +02:00
Pascal Bleser
6c68590de4 groupware: add DNS auto-discovery (currently disabled, needs testing) 2026-07-09 14:15:49 +02:00
Pascal Bleser
fde1d6b184 add a .gitignore entry for debug binaries built by VSCode when running OpenCloud 2026-07-09 14:15:49 +02:00
Pascal Bleser
956b9456d6 groupware: session handling improvements
* remove the baseurl from the JMAP client configuration, and pass it to
   the session retrieval functions instead, as that is really the only
   place where it is relevant, and we gain flexibility to discover that
   session URL differently in the future without having to touch the
   JMAP client

 * move the default account identifier handling from the JMAP package to
   the Groupware one, as it really has nothing to do with JMAP itself,
   and is an opinionated feature of the Groupware REST API instead

 * add an event listener interface for JMAP events to be more flexible
   and universal, typically for metrics that are defined on the API
   level that uses the JMAP client

 * add errors for when default accounts cannot be determined

 * split groupware_framework.go into groupware_framework.go,
   groupware_request.go and groupware_response.go

 * move the accountId logging into the Groupware level instead of JMAP
   since it can also be relevant to other operations that might be
   worthy of logging before the JMAP client is even invoked
2026-07-09 14:15:49 +02:00
Pascal Bleser
72de164c84 groupware: fix debug server, was missing a lot of configuration options and was binding to :80 2026-07-09 14:15:49 +02:00
Pascal Bleser
9bc8158733 docs(groupware): add Groupware related ADRs 2026-07-09 14:15:49 +02:00
Pascal Bleser
338df0d822 refactor(groupware): logging and metrics improvements
* some minor code refactorings to improve logging and metrics

 * more code documentation
2026-07-09 14:15:49 +02:00
Pascal Bleser
9cf4c6211b jmap: minor logging improvements 2026-07-09 14:15:49 +02:00
Pascal Bleser
ed56d490ec groupware: improve metrics
* implement more metrics, in a more streamlined fashion

 * use concurrent-map to store SSE streams instead of a regular map with
   one big lock that will not scale when it grows, causing too much
   contention on that one lock

 * while testing error metrics, noticed a few bugs with error handling
   when Stalwart is down: fixed
2026-07-09 14:15:49 +02:00
Pascal Bleser
5a030fd5f4 groupware: jmap: add metrics 2026-07-09 14:15:49 +02:00
Pascal Bleser
bc91ef9bf6 groupware: implement metrics
* implement a framework for metrics, with a few exemplary ones
2026-07-09 14:15:49 +02:00