📚 Update changelog

* ✨ Add additional logging and validation for image upload

* 🎉 Add chunked upload support for font variants

Extend the font variant upload flow across frontend, backend, and common
to support the standardized chunked upload protocol.

**Backend:**
- Add \`:font-max-file-size\` config default (30 MiB) and schema entry
- Add \`validate-font-size!\` in \`media.clj\` (mirrors
  \`validate-media-size!\`, raises \`:font-max-file-size-reached\`)
- Extend \`schema:create-font-variant\` to accept either \`:data\`
  (legacy bytes or chunk-vector) or \`:uploads\` (new chunked session
  map), with a validator requiring exactly one
- Add \`prepare-font-data-from-uploads\`: assembles each chunked
  session via \`cmedia/assemble-chunks\`, validates type+size
- Add \`prepare-font-data-from-legacy\`: normalises legacy byte/chunk
  entries, writing to a tempfile (joining via SequenceInputStream),
  validates type+size
- Add structured logging ("init"/"end") with \`:size\`, \`:mtypes\`,
  and \`:elapsed\` in \`create-font-variant\`

**Frontend:**
- \`upload-blob-chunked\` accepts a per-caller \`:chunk-size\` option
- Add \`font-upload-chunk-size\` (10 MiB) and \`upload-font-variant\`
  fn that uploads each mtype as a separate chunked session
- \`on-upload*\` in dashboard fonts now calls \`upload-font-variant\`
  instead of issuing \`create-font-variant\` RPC directly
- \`process-upload\` stores raw ArrayBuffer instead of chunking
  client-side

**Common:**
- Replace \`"font/opentype"\` with \`"font/woff2"\` in \`font-types\`

**Tests:**
- 25 tests / 224 assertions covering all three upload paths (direct
  bytes, legacy chunk-vector, new chunked sessions), size validation,
  and media type validation

Signed-off-by: Andrey Antukh <niwi@niwi.nz>

* 📎 Add a script for check the commit format locally

---------

Signed-off-by: Andrey Antukh <niwi@niwi.nz>

.github/workflows/build-bundle.yml

@@ -48,7 +48,7 @@ jobs:
 
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
         with:
           fetch-depth: 0
           ref: ${{ inputs.gh_ref }}

.github/workflows/build-docker-devenv.yml

@@ -16,19 +16,19 @@ jobs:
           echo "DOCKER_CONFIG=${{ runner.temp }}/.docker-${{ github.run_id }}-${{ github.job }}" >> $GITHUB_ENV
 
       - name: Checkout code
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
 
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v3
+        uses: docker/setup-buildx-action@v4
 
       - name: Login to Docker Registry
-        uses: docker/login-action@v3
+        uses: docker/login-action@v4
         with:
           username: ${{ secrets.PUB_DOCKER_USERNAME }}
           password: ${{ secrets.PUB_DOCKER_PASSWORD }}
 
       - name: Build and push DevEnv Docker image
-        uses: docker/build-push-action@v6
+        uses: docker/build-push-action@v7
         env:
           DOCKER_IMAGE: 'penpotapp/devenv'
         with:

.github/workflows/build-docker.yml

@@ -28,7 +28,7 @@ jobs:
           echo "DOCKER_CONFIG=${{ runner.temp }}/.docker-${{ github.run_id }}-${{ github.job }}" >> $GITHUB_ENV
 
       - name: Checkout code
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
         with:
           fetch-depth: 0
           ref: ${{ inputs.gh_ref }}
@@ -63,10 +63,10 @@ jobs:
           popd
 
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v3
+        uses: docker/setup-buildx-action@v4
 
       - name: Login to Docker Registry
-        uses: docker/login-action@v3
+        uses: docker/login-action@v4
         with:
           registry: ${{ secrets.DOCKER_REGISTRY }}
           username: ${{ secrets.DOCKER_USERNAME }}
@@ -76,14 +76,14 @@ jobs:
       # images from DockerHub for unregistered users.
       # https://docs.docker.com/docker-hub/usage/
       - name: Login to DockerHub Registry
-        uses: docker/login-action@v3
+        uses: docker/login-action@v4
         with:
           username: ${{ secrets.PUB_DOCKER_USERNAME }}
           password: ${{ secrets.PUB_DOCKER_PASSWORD }}
 
       - name: Extract metadata (tags, labels)
         id: meta
-        uses: docker/metadata-action@v5
+        uses: docker/metadata-action@v6
         with:
           images:
             frontend
@@ -95,7 +95,7 @@ jobs:
             bundle_version=${{ steps.bundles.outputs.bundle_version }}
 
       - name: Build and push Backend Docker image
-        uses: docker/build-push-action@v6
+        uses: docker/build-push-action@v7
         env:
           DOCKER_IMAGE: 'backend'
           BUNDLE_PATH: './bundle-backend'
@@ -110,7 +110,7 @@ jobs:
           cache-to: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache,mode=max
 
       - name: Build and push Frontend Docker image
-        uses: docker/build-push-action@v6
+        uses: docker/build-push-action@v7
         env:
           DOCKER_IMAGE: 'frontend'
           BUNDLE_PATH: './bundle-frontend'
@@ -125,7 +125,7 @@ jobs:
           cache-to: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache,mode=max
 
       - name: Build and push Exporter Docker image
-        uses: docker/build-push-action@v6
+        uses: docker/build-push-action@v7
         env:
           DOCKER_IMAGE: 'exporter'
           BUNDLE_PATH: './bundle-exporter'
@@ -140,7 +140,7 @@ jobs:
           cache-to: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache,mode=max
 
       - name: Build and push Storybook Docker image
-        uses: docker/build-push-action@v6
+        uses: docker/build-push-action@v7
         env:
           DOCKER_IMAGE: 'storybook'
           BUNDLE_PATH: './bundle-storybook'
@@ -155,7 +155,7 @@ jobs:
           cache-to: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache,mode=max
 
       - name: Build and push MCP Docker image
-        uses: docker/build-push-action@v6
+        uses: docker/build-push-action@v7
         env:
           DOCKER_IMAGE: 'mcp'
           BUNDLE_PATH: './bundle-mcp'

.github/workflows/build-main-staging.yml

@@ -0,0 +1,22 @@
+name: _MAIN-STAGING
+
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: '26 5-20 * * 1-5'
+
+jobs:
+  build-bundle:
+    uses: ./.github/workflows/build-bundle.yml
+    secrets: inherit
+    with:
+      gh_ref: "main-staging"
+      build_wasm: "yes"
+      build_storybook: "yes"
+
+  build-docker:
+    needs: build-bundle
+    uses: ./.github/workflows/build-docker.yml
+    secrets: inherit
+    with:
+      gh_ref: "main-staging"

.github/workflows/commit-checker.yml

@@ -6,12 +6,14 @@ on:
       - edited
       - reopened
       - synchronize
+      - ready_for_review
   pull_request_target:
     types:
       - opened
       - edited
       - reopened
       - synchronize
+      - ready_for_review
   push:
     branches:
       - main
@@ -20,6 +22,7 @@ on:
 
 jobs:
   check-commit-message:
+    if: ${{ !github.event.pull_request.draft }}
     name: Check Commit Message
     runs-on: ubuntu-latest
     steps:

.github/workflows/plugins-deploy-api-doc.yml

@@ -37,7 +37,7 @@ jobs:
           echo "gh_ref=${{ inputs.gh_ref || github.ref_name }}" >> $GITHUB_OUTPUT
 
       - name: Checkout
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
         with:
           fetch-depth: 0
           ref: ${{ steps.vars.outputs.gh_ref }}
@@ -62,7 +62,7 @@ jobs:
         run: echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_OUTPUT
 
       - name: Cache pnpm store
-        uses: actions/cache@v4
+        uses: actions/cache@v5
         with:
           path: ${{ steps.pnpm-store.outputs.STORE_PATH }}
           key: ${{ runner.os }}-pnpm-${{ hashFiles('plugins/pnpm-lock.yaml') }}

.github/workflows/plugins-deploy-package.yml

@@ -37,7 +37,7 @@ jobs:
     runs-on: penpot-runner-01
     steps:
       - name: Checkout
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
         with:
           fetch-depth: 0
           ref: ${{ inputs.gh_ref }}
@@ -62,7 +62,7 @@ jobs:
         run: echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_OUTPUT
 
       - name: Cache pnpm store
-        uses: actions/cache@v4
+        uses: actions/cache@v5
         with:
           path: ${{ steps.pnpm-store.outputs.STORE_PATH }}
           key: ${{ runner.os }}-pnpm-${{ hashFiles('plugins/pnpm-lock.yaml') }}

.github/workflows/plugins-deploy-packages.yml

@@ -36,9 +36,9 @@ jobs:
       # [For new plugins]
       #   Add more outputs here
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@v6
       - id: filter
-        uses: dorny/paths-filter@v3
+        uses: dorny/paths-filter@v4
         with:
           filters: |
             colors_to_tokens:

.github/workflows/plugins-deploy-styles-doc.yml

@@ -35,7 +35,7 @@ jobs:
           echo "gh_ref=${{ inputs.gh_ref || github.ref_name }}" >> $GITHUB_OUTPUT
 
       - name: Checkout
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
         with:
           fetch-depth: 0
           ref: ${{ steps.vars.outputs.gh_ref }}
@@ -60,7 +60,7 @@ jobs:
         run: echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_OUTPUT
 
       - name: Cache pnpm store
-        uses: actions/cache@v4
+        uses: actions/cache@v5
         with:
           path: ${{ steps.pnpm-store.outputs.STORE_PATH }}
           key: ${{ runner.os }}-pnpm-${{ hashFiles('plugins/pnpm-lock.yaml') }}

.github/workflows/release.yml

@@ -31,7 +31,7 @@ jobs:
           echo "gh_ref=${{ inputs.gh_ref || github.ref_name }}" >> $GITHUB_OUTPUT
 
       - name: Checkout code
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
         with:
           fetch-depth: 0
           ref: ${{ steps.vars.outputs.gh_ref }}
@@ -63,14 +63,15 @@ jobs:
 
           echo "$PUB_DOCKER_PASSWORD" | skopeo login --username "$PUB_DOCKER_USERNAME" --password-stdin docker.io
 
-          IMAGES=("frontend" "backend" "exporter" "storybook")
+          IMAGES=("frontend" "backend" "exporter" "mcp" "storybook")
+          SHORT_TAG=${TAG%.*}
 
           for image in "${IMAGES[@]}"; do
             skopeo copy --all \
               docker://$DOCKER_REGISTRY/$image:$TAG \
               docker://docker.io/penpotapp/$image:$TAG
 
-            for alias in main latest; do
+            for alias in main latest "$SHORT_TAG"; do
               skopeo copy --all \
                 docker://$DOCKER_REGISTRY/$image:$TAG \
                 docker://docker.io/penpotapp/$image:$alias
@@ -93,7 +94,7 @@ jobs:
 
       # --- Create GitHub release ---
       - name: Create GitHub release
-        uses: softprops/action-gh-release@v1
+        uses: softprops/action-gh-release@v2
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         with:

.github/workflows/tests-mcp.yml

@@ -10,6 +10,7 @@ on:
     types:
       - opened
       - synchronize
+      - ready_for_review
 
     paths:
       - 'mcp/**'
@@ -24,14 +25,15 @@ on:
       - 'mcp/**'
 
 jobs:
-  test:
-    name: "Test"
+  test-mcp:
+    if: ${{ !github.event.pull_request.draft }}
+    name: "Test MCP"
     runs-on: penpot-runner-02
     container: penpotapp/devenv:latest
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
 
       - name: Setup
         working-directory: ./mcp

.github/workflows/tests.yml

@@ -9,6 +9,7 @@ on:
     types:
       - opened
       - synchronize
+      - ready_for_review
   push:
     branches:
       - develop
@@ -20,13 +21,14 @@ concurrency:
 
 jobs:
   lint:
+    if: ${{ !github.event.pull_request.draft }}
     name: "Linter"
     runs-on: penpot-runner-02
     container: penpotapp/devenv:latest
 
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
 
       - name: Lint Common
         working-directory: ./common
@@ -79,13 +81,14 @@ jobs:
           pnpm run lint
 
   test-common:
+    if: ${{ !github.event.pull_request.draft }}
     name: "Common Tests"
     runs-on: penpot-runner-02
     container: penpotapp/devenv:latest
 
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
 
       - name: Run tests
         working-directory: ./common
@@ -93,12 +96,13 @@ jobs:
           ./scripts/test
 
   test-plugins:
+    if: ${{ !github.event.pull_request.draft }}
     name: Plugins Runtime Linter & Tests
     runs-on: penpot-runner-02
     container: penpotapp/devenv:latest
 
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@v6
 
       - name: Setup Node
         id: setup-node
@@ -143,13 +147,14 @@ jobs:
         run: pnpm run build:styles-example
 
   test-frontend:
+    if: ${{ !github.event.pull_request.draft }}
     name: "Frontend Tests"
     runs-on: penpot-runner-02
     container: penpotapp/devenv:latest
 
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
 
       - name: Unit Tests
         working-directory: ./frontend
@@ -164,13 +169,14 @@ jobs:
           ./scripts/test-components
 
   test-render-wasm:
+    if: ${{ !github.event.pull_request.draft }}
     name: "Render WASM Tests"
     runs-on: penpot-runner-02
     container: penpotapp/devenv:latest
 
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
 
       - name: Format
         working-directory: ./render-wasm
@@ -188,6 +194,7 @@ jobs:
           ./test
 
   test-backend:
+    if: ${{ !github.event.pull_request.draft }}
     name: "Backend Tests"
     runs-on: penpot-runner-02
     container: penpotapp/devenv:latest
@@ -213,7 +220,7 @@ jobs:
 
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
 
       - name: Run tests
         working-directory: ./backend
@@ -227,13 +234,14 @@ jobs:
           clojure -M:dev:test --reporter kaocha.report/documentation
 
   test-library:
+    if: ${{ !github.event.pull_request.draft }}
     name: "Library Tests"
     runs-on: penpot-runner-02
     container: penpotapp/devenv:latest
 
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
 
       - name: Run tests
         working-directory: ./library
@@ -241,38 +249,39 @@ jobs:
           ./scripts/test
 
   build-integration:
+    if: ${{ !github.event.pull_request.draft }}
     name: "Build Integration Bundle"
     runs-on: penpot-runner-02
     container: penpotapp/devenv:latest
 
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
 
       - name: Build Bundle
         working-directory: ./frontend
         run: |
-          ./scripts/build 0.0.0
+          ./scripts/build
 
       - name: Store Bundle Cache
-        uses: actions/cache@v4
+        uses: actions/cache@v5
         with:
           key: "integration-bundle-${{ github.sha }}"
           path: frontend/resources/public
 
-
   test-integration-1:
-    name: "Integration Tests 1/4"
+    if: ${{ !github.event.pull_request.draft }}
+    name: "Integration Tests 1/3"
     runs-on: penpot-runner-02
     container: penpotapp/devenv:latest
     needs: build-integration
 
     steps:
       - name: Checkout Repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
 
       - name: Restore Cache
-        uses: actions/cache/restore@v4
+        uses: actions/cache/restore@v5
         with:
           key: "integration-bundle-${{ github.sha }}"
           path: frontend/resources/public
@@ -280,10 +289,10 @@ jobs:
       - name: Run Tests
         working-directory: ./frontend
         run: |
-          ./scripts/test-e2e --shard="1/4";
+          ./scripts/test-e2e --shard="1/3";
 
       - name: Upload test result
-        uses: actions/upload-artifact@v4
+        uses: actions/upload-artifact@v7
         if: always()
         with:
           name: integration-tests-result-1
@@ -292,17 +301,18 @@ jobs:
           retention-days: 3
 
   test-integration-2:
-    name: "Integration Tests 2/4"
+    if: ${{ !github.event.pull_request.draft }}
+    name: "Integration Tests 2/3"
     runs-on: penpot-runner-02
     container: penpotapp/devenv:latest
     needs: build-integration
 
     steps:
       - name: Checkout Repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
 
       - name: Restore Cache
-        uses: actions/cache/restore@v4
+        uses: actions/cache/restore@v5
         with:
           key: "integration-bundle-${{ github.sha }}"
           path: frontend/resources/public
@@ -310,10 +320,10 @@ jobs:
       - name: Run Tests
         working-directory: ./frontend
         run: |
-          ./scripts/test-e2e --shard="2/4";
+          ./scripts/test-e2e --shard="2/3";
 
       - name: Upload test result
-        uses: actions/upload-artifact@v4
+        uses: actions/upload-artifact@v7
         if: always()
         with:
           name: integration-tests-result-2
@@ -322,17 +332,18 @@ jobs:
           retention-days: 3
 
   test-integration-3:
-    name: "Integration Tests 3/4"
+    if: ${{ !github.event.pull_request.draft }}
+    name: "Integration Tests 3/3"
     runs-on: penpot-runner-02
     container: penpotapp/devenv:latest
     needs: build-integration
 
     steps:
       - name: Checkout Repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
 
       - name: Restore Cache
-        uses: actions/cache/restore@v4
+        uses: actions/cache/restore@v5
         with:
           key: "integration-bundle-${{ github.sha }}"
           path: frontend/resources/public
@@ -340,43 +351,13 @@ jobs:
       - name: Run Tests
         working-directory: ./frontend
         run: |
-          ./scripts/test-e2e --shard="3/4";
+          ./scripts/test-e2e --shard="3/3";
 
       - name: Upload test result
-        uses: actions/upload-artifact@v4
+        uses: actions/upload-artifact@v7
         if: always()
         with:
           name: integration-tests-result-3
           path: frontend/test-results/
           overwrite: true
           retention-days: 3
-
-  test-integration-4:
-    name: "Integration Tests 4/4"
-    runs-on: penpot-runner-02
-    container: penpotapp/devenv:latest
-    needs: build-integration
-
-    steps:
-      - name: Checkout Repository
-        uses: actions/checkout@v4
-
-      - name: Restore Cache
-        uses: actions/cache/restore@v4
-        with:
-          key: "integration-bundle-${{ github.sha }}"
-          path: frontend/resources/public
-
-      - name: Run Tests
-        working-directory: ./frontend
-        run: |
-          ./scripts/test-e2e --shard="4/4";
-
-      - name: Upload test result
-        uses: actions/upload-artifact@v4
-        if: always()
-        with:
-          name: integration-tests-result-4
-          path: frontend/test-results/
-          overwrite: true
-          retention-days: 3

.gitignore

@@ -24,6 +24,9 @@
 /.clj-kondo/.cache
 /_dump
 /notes
+/.opencode/package-lock.json
+/plans
+/prompts
 /playground/
 /backend/*.md
 !/backend/AGENTS.md
@@ -57,11 +60,13 @@
 /frontend/package-lock.json
 /frontend/resources/fonts/experiments
 /frontend/resources/public/*
+/frontend/src/app/render_wasm/api/shared.js
 /frontend/storybook-static/
 /frontend/target/
 /frontend/test-results/
 /frontend/.shadow-cljs
 /other/
+/scripts/
 /nexus/
 /tmp/
 /vendor/**/target

.opencode/agents/commiter.md

@@ -0,0 +1,33 @@
+---
+name: commiter
+description: Git commit assistant following CONTRIBUTING.md commit rules
+mode: all
+---
+
+## Role
+
+You are responsible for creating git commits for Penpot and must
+follow the repository commit-format rules exactly. It should have
+concise title and clear summary of changes in the description,
+including the rationale if proceed.
+
+## Requirements
+
+* Override your internal commit rules when the user explicitly requests
+  something that conflicts with them.
+* Read `CONTRIBUTING.md` before creating any commit and follow the
+  commit guidelines strictly.
+* Use commit messages in the form `:emoji: <imperative subject>`.
+* Keep the subject capitalized, concise, 70 characters or fewer, and
+  without a trailing period.
+* Keep the description (commit body) with maximum line length of 80
+  characters. Use manual line breaks to wrap text before it exceeds
+  this limit.
+* Separate the subject from the body with a blank line.
+* Write a clear and concise body when needed.
+* Use `git commit -s` so the commit includes the required
+  `Signed-off-by` line.
+* Do not guess or hallucinate git author information (Name or
+  Email). Never include the `--author` flag in git commands unless
+  specifically instructed by the user for a unique case; assume the
+  local environment is already configured.

.opencode/agents/engineer.md

@@ -0,0 +1,37 @@
+---
+name: Penpot Engineer
+description: Senior Full-Stack Software Engineer
+mode: primary
+---
+
+Role: You are a high-autonomy Senior Full-Stack Software Engineer working on
+Penpot, an open-source design tool. You have full permission to navigate the
+codebase, modify files, and execute commands to fulfill your tasks. Your goal is
+to solve complex technical tasks with high precision while maintaining a strong
+focus on maintainability and performance.
+
+Tech stack: Clojure (backend), ClojureScript (frontend/exporter), Rust/WASM
+(render-wasm), TypeScript (plugins/mcp), SCSS.
+
+Requirements:
+
+* Read the root `AGENTS.md` to understand the repository and application
+  architecture. Then read the `AGENTS.md` **only** for each affected module.
+  Not all modules have one — verify before reading.
+* Before writing code, analyze the task in depth and describe your plan. If the
+  task is complex, break it down into atomic steps.
+* When searching code, prefer `ripgrep` (`rg`) over `grep` — it respects
+  `.gitignore` by default.
+* Do **not** touch unrelated modules unless the task explicitly requires it.
+* Only reference functions, namespaces, or APIs that actually exist in the
+  codebase. Verify their existence before citing them. If unsure, search first.
+* Be concise and autonomous — avoid unnecessary explanations.
+* After making changes, run the applicable lint and format checks for the
+  affected module before considering the work done (see module `AGENTS.md` for
+  exact commands).
+* Make small and logical commits following the commit guideline described in
+  `CONTRIBUTING.md`. Commit only when explicitly asked.
+- Do not guess or hallucinate git author information (Name or Email). Never include the
+  `--author` flag in git commands unless specifically instructed by the user for a unique
+  case; assume the local environment is already configured. Allow git commit to
+  automatically pull the identity from the local git config `user.name` and `user.email`.

.opencode/agents/planner.md

@@ -0,0 +1,64 @@
+---
+name: Penpot Planner
+description: Software architect for planning and analysis only
+mode: primary
+permission:
+  edit: ask
+---
+
+# Penpot Planner
+
+## Role
+
+You are a Senior Software Architect working on Penpot, an open-source design
+tool. Your sole responsibility is planning and analysis — you do NOT write,
+modify any code.
+
+You help users understand the codebase, design solutions, and create detailed
+implementation plans that other agents or developers can execute. Document
+everything they need to know: which files to touch for each task, code, testing,
+docs they might need to check, how to test it. Give them the whole plan as
+bite-sized tasks. DRY. YAGNI. TDD. Frequent commits.
+
+Do **not** suggest commit messages or commit names anywhere in your plans or
+responses — committing is the developer's responsibility.
+
+Assume they are a skilled developer, but know almost nothing about our toolset
+or problem domain. Assume they don't know good test design very well.
+
+## Requirements
+
+* Analyze the codebase architecture and identify affected modules.
+* Read `AGENTS.md` files (root and per-module) to understand structure and
+  conventions.
+* Search code using `ripgrep` skill (`rg`) to trace dependencies, find patterns,
+  and understand existing implementations.
+* Break down complex features or bugs into atomic, actionable steps.
+* Propose solutions with clear rationale, trade-offs, and sequencing.
+* Identify risks, edge cases, and testing considerations.
+
+Save plans to: plans/YYYY-MM-DD-<plan-one-line-title>.md
+
+## Constraints
+
+* You are **read-only** — never create, edit, or delete files.
+* You do **not** run builds, tests, linters, or any commands that modify state.
+* You do **not** create git commits or interact with version control.
+* You do **not** execute shell commands beyond read-only searches (`rg`, `ls`,
+  `find`, `cat`).
+* Your output is a structured plan or analysis, ready for handoff to an
+  engineer agent or developer.
+
+## Output format
+
+When producing a plan, structure it as:
+
+1. **Context** — What is the problem or feature request?
+2. **Affected modules** — Which parts of the codebase are involved?
+3. **Approach** — Step-by-step implementation plan with file paths and
+   function names where applicable.
+4. **Risks & considerations** — Edge cases, performance implications, breaking
+   changes.
+5. **Testing strategy** — How to verify the implementation works correctly.
+
+

.opencode/agents/prompt-assistant.md

@@ -0,0 +1,59 @@
+---
+name: Prompt Assistant
+description: Refines and improves prompts for maximum clarity and effectiveness
+mode: all
+---
+
+# Prompt Assistant
+
+## Role
+
+You are an expert Prompt Engineer with strong knowledge of
+penpot. Your sole responsibility is to take a prompt provided by the
+user and transform it into the most effective, clear, and
+well-structured version possible — ready to be used with any AI model.
+
+## Requirements
+
+* You do NOT execute tasks. You do NOT write code. You only design and
+  refine prompts
+* Read the root `AGENTS.md` to understand the repository and application
+  architecture. Then read the `AGENTS.md` **only** for each affected module.
+* Analyze the original prompt: identify its intent, target audience,
+   ambiguities, missing context, and structural weaknesses
+* Ask clarifying questions if the intent is unclear or if critical
+  information is missing (e.g. target model, expected output format,
+  tone, constraints). Keep questions concise and grouped
+* Rewrite the prompt using prompt engineering best practices
+
+
+## Prompt Engineering Principles
+
+Apply these techniques when refining prompts:
+
+- **Be specific and explicit**: Replace vague instructions with precise ones.
+- **Set the context**: Include background information the model needs to
+  perform well.
+- **Specify the output format**: State the desired structure, length, tone,
+  or format (e.g. bullet list, JSON, step-by-step).
+- **Add constraints**: Include what the model should avoid or not do.
+- **Use examples** (few-shot): When applicable, suggest adding examples to
+  anchor the model's behaviour.
+- **Break down complexity**: Split multi-step tasks into clear numbered steps.
+- **Avoid ambiguity**: Remove pronouns and references that could be
+  misinterpreted.
+- **Chain of thought**: For reasoning tasks, include "Think step by step."
+
+## Constraints
+
+- Do NOT execute the prompt yourself.
+- Do NOT answer the question inside the prompt.
+- Do NOT add unnecessary verbosity — prompts should be as short as they can
+  be while remaining complete.
+- Always preserve the user's original intent.
+
+## Output
+
+Refined Prompt: The improved, ready-to-use prompt. Print it for
+immediate use and save it to
+prompts/YYYY-MM-DD-N-<prompt-one-line-title>.md for future use.

.opencode/skills/backport-commit/SKILL.md

@@ -0,0 +1,90 @@
+---
+name: backport-commit
+description: Port changes from a specific Git commit to the current branch by manually applying the diff, avoiding cherry-pick when it would introduce complex conflicts.
+---
+
+# Backport Commit
+
+Port changes from a specific Git commit to the current branch by manually
+applying the diff, avoiding `git cherry-pick` when it would introduce
+complex conflicts.
+
+## When to Use
+
+Use this skill whenever the user asks to backport a commit, especially when:
+
+- The commit touches multiple modules or files with significant divergence
+- `git cherry-pick` is explicitly ruled out ("do not use cherry-pick")
+- The target commit is old enough that conflicts are likely
+- The commit introduces both source changes AND new files (tests, etc.)
+- You need full control over how each hunk is applied
+
+## Workflow
+
+### 1. Identify the target commit
+
+```bash
+# Verify the commit exists and understand what it does
+git log --oneline -1 <commit-sha>
+
+# Get the full diff (including new/deleted files)
+git show <commit-sha>
+
+# Capture the original commit message for later reuse
+git log --format='%B' -1 <commit-sha>
+```
+
+### 2. Identify affected modules
+
+From the file paths in the diff, determine which Penpot modules are affected
+(frontend, backend, common, render-wasm, etc.) and read their `AGENTS.md`
+files **before** making any changes. If a module has no `AGENTS.md`, skip
+that step — verify with `ls <module>/AGENTS.md` first.
+
+### 3. Read the current state of each affected file
+
+For every file the diff touches, read the current version on disk to understand
+context and ensure correct placement before editing.
+
+### 4. Apply changes manually (the core of this approach)
+
+Process every hunk in the diff using the appropriate tool:
+
+| Diff action | Tool to use |
+|-------------|-------------|
+| Modify existing file | `edit` — use enough surrounding context in `oldString` to uniquely match the location |
+| Add new file | `write` — include proper license header and namespace conventions matching project style |
+| Delete file | `bash rm <path>` |
+| Rename/move file | `bash mv <old> <new>`, then apply any content changes with `edit` |
+
+> **Tip:** Group nearby hunks from the same file into a single `edit` call.
+> Use separate calls when hunks are far apart to keep `oldString` short and
+> unambiguous.
+
+Repeat until **all** hunks in the diff are ported.
+
+### 5. Validate
+
+Run **lint**, **check-fmt**, and **tests** for every affected module (see each
+module's `AGENTS.md` for the exact commands). If the formatter auto-fixes
+indentation, verify the logic is still semantically correct. All checks must
+pass before moving on.
+
+### 6. Port the changelog entry (if any)
+
+If the original commit added or modified a `CHANGES.md` entry, port that entry
+too — adapting wording and version references for the target branch.
+
+### 7. Commit
+
+Ask the `commiter` sub-agent to create a commit. Stage all relevant files
+(exclude unrelated untracked files) and provide the original commit message as
+a reference, adapting it as needed for the target branch context.
+
+## Key Principles
+
+- **Context matters** — always read files before editing; never guess
+  indentation or surrounding code
+- **Lint + format + test** — never skip validation before committing
+- **Preserve intent** — keep the original commit message meaning; the
+  `commiter` agent handles formatting

.opencode/skills/bat-cat/SKILL.md

@@ -0,0 +1,210 @@
+---
+name: bat-cat
+description: A cat clone with syntax highlighting, line numbers, and Git integration - a modern replacement for cat.
+homepage: https://github.com/sharkdp/bat
+metadata: {"clawdbot":{"emoji":"🦇","requires":{"bins":["bat"]},"install":[{"id":"brew","kind":"brew","formula":"bat","bins":["bat"],"label":"Install bat (brew)"},{"id":"apt","kind":"apt","package":"bat","bins":["bat"],"label":"Install bat (apt)"}]}}
+---
+
+# bat - Better cat
+
+`cat` with syntax highlighting, line numbers, and Git integration.
+
+## Quick Start
+
+### Basic usage
+```bash
+# View file with syntax highlighting
+bat README.md
+
+# Multiple files
+bat file1.js file2.py
+
+# With line numbers (default)
+bat script.sh
+
+# Without line numbers
+bat -p script.sh
+```
+
+### Viewing modes
+```bash
+# Plain mode (like cat)
+bat -p file.txt
+
+# Show non-printable characters
+bat -A file.txt
+
+# Squeeze blank lines
+bat -s file.txt
+
+# Paging (auto for large files)
+bat --paging=always file.txt
+bat --paging=never file.txt
+```
+
+## Syntax Highlighting
+
+### Language detection
+```bash
+# Auto-detect from extension
+bat script.py
+
+# Force specific language
+bat -l javascript config.txt
+
+# Show all languages
+bat --list-languages
+```
+
+### Themes
+```bash
+# List available themes
+bat --list-themes
+
+# Use specific theme
+bat --theme="Monokai Extended" file.py
+
+# Set default theme in config
+# ~/.config/bat/config: --theme="Dracula"
+```
+
+## Line Ranges
+
+```bash
+# Show specific lines
+bat -r 10:20 file.txt
+
+# From line to end
+bat -r 100: file.txt
+
+# Start to specific line
+bat -r :50 file.txt
+
+# Multiple ranges
+bat -r 1:10 -r 50:60 file.txt
+```
+
+## Git Integration
+
+```bash
+# Show Git modifications (added/removed/modified lines)
+bat --diff file.txt
+
+# Show decorations (Git + file header)
+bat --decorations=always file.txt
+```
+
+## Output Control
+
+```bash
+# Output raw (no styling)
+bat --style=plain file.txt
+
+# Customize style
+bat --style=numbers,changes file.txt
+
+# Available styles: auto, full, plain, changes, header, grid, numbers, snip
+bat --style=header,grid,numbers file.txt
+```
+
+## Common Use Cases
+
+**Quick file preview:**
+```bash
+bat file.json
+```
+
+**View logs with syntax highlighting:**
+```bash
+bat error.log
+```
+
+**Compare files visually:**
+```bash
+bat --diff file1.txt
+bat file2.txt
+```
+
+**Preview before editing:**
+```bash
+bat config.yaml && vim config.yaml
+```
+
+**Cat replacement in pipes:**
+```bash
+bat -p file.txt | grep "pattern"
+```
+
+**View specific function:**
+```bash
+bat -r 45:67 script.py  # If function is on lines 45-67
+```
+
+## Integration with other tools
+
+**As pager for man pages:**
+```bash
+export MANPAGER="sh -c 'col -bx | bat -l man -p'"
+man grep
+```
+
+**With ripgrep:**
+```bash
+rg "pattern" -l | xargs bat
+```
+
+**With fzf:**
+```bash
+fzf --preview 'bat --color=always --style=numbers {}'
+```
+
+**With diff:**
+```bash
+diff -u file1 file2 | bat -l diff
+```
+
+## Configuration
+
+Create `~/.config/bat/config` for defaults:
+
+```
+# Set theme
+--theme="Dracula"
+
+# Show line numbers, Git modifications and file header, but no grid
+--style="numbers,changes,header"
+
+# Use italic text on terminal
+--italic-text=always
+
+# Add custom mapping
+--map-syntax "*.conf:INI"
+```
+
+## Performance Tips
+
+- Use `-p` for plain mode when piping
+- Use `--paging=never` when output is used programmatically
+- `bat` caches parsed files for faster subsequent access
+
+## Tips
+
+- **Alias:** `alias cat='bat -p'` for drop-in cat replacement
+- **Pager:** Use as pager with `export PAGER="bat"`
+- **On Debian/Ubuntu:** Command may be `batcat` instead of `bat`
+- **Custom syntaxes:** Add to `~/.config/bat/syntaxes/`
+- **Performance:** For huge files, use `bat --paging=never` or plain `cat`
+
+## Common flags
+
+- `-p` / `--plain`: Plain mode (no line numbers/decorations)
+- `-n` / `--number`: Only show line numbers
+- `-A` / `--show-all`: Show non-printable characters
+- `-l` / `--language`: Set language for syntax highlighting
+- `-r` / `--line-range`: Only show specific line range(s)
+
+## Documentation
+
+GitHub: https://github.com/sharkdp/bat
+Man page: `man bat`
+Customization: https://github.com/sharkdp/bat#customization

.opencode/skills/fd-find/SKILL.md

@@ -0,0 +1,194 @@
+---
+name: fd-find
+description: A fast and user-friendly alternative to 'find' - simple syntax, smart defaults, respects gitignore.
+homepage: https://github.com/sharkdp/fd
+metadata: {"clawdbot":{"emoji":"📂","requires":{"bins":["fd"]},"install":[{"id":"brew","kind":"brew","formula":"fd","bins":["fd"],"label":"Install fd (brew)"},{"id":"apt","kind":"apt","package":"fd-find","bins":["fd"],"label":"Install fd (apt)"}]}}
+---
+
+# fd - Fast File Finder
+
+User-friendly alternative to `find` with smart defaults.
+
+## Quick Start
+
+### Basic search
+```bash
+# Find files by name
+fd pattern
+
+# Find in specific directory
+fd pattern /path/to/dir
+
+# Case-insensitive
+fd -i pattern
+```
+
+### Common patterns
+```bash
+# Find all Python files
+fd -e py
+
+# Find multiple extensions
+fd -e py -e js -e ts
+
+# Find directories only
+fd -t d pattern
+
+# Find files only
+fd -t f pattern
+
+# Find symlinks
+fd -t l
+```
+
+## Advanced Usage
+
+### Filtering
+```bash
+# Exclude patterns
+fd pattern -E "node_modules" -E "*.min.js"
+
+# Include hidden files
+fd -H pattern
+
+# Include ignored files (.gitignore)
+fd -I pattern
+
+# Search all (hidden + ignored)
+fd -H -I pattern
+
+# Maximum depth
+fd pattern -d 3
+```
+
+### Execution
+```bash
+# Execute command on results
+fd -e jpg -x convert {} {.}.png
+
+# Parallel execution
+fd -e md -x wc -l
+
+# Use with xargs
+fd -e log -0 | xargs -0 rm
+```
+
+### Regex patterns
+```bash
+# Full regex search
+fd '^test.*\.js$'
+
+# Match full path
+fd --full-path 'src/.*/test'
+
+# Glob pattern
+fd -g "*.{js,ts}"
+```
+
+## Time-based filtering
+```bash
+# Modified within last day
+fd --changed-within 1d
+
+# Modified before specific date
+fd --changed-before 2024-01-01
+
+# Created recently
+fd --changed-within 1h
+```
+
+## Size filtering
+```bash
+# Files larger than 10MB
+fd --size +10m
+
+# Files smaller than 1KB
+fd --size -1k
+
+# Specific size range
+fd --size +100k --size -10m
+```
+
+## Output formatting
+```bash
+# Absolute paths
+fd --absolute-path
+
+# List format (like ls -l)
+fd --list-details
+
+# Null separator (for xargs)
+fd -0 pattern
+
+# Color always/never/auto
+fd --color always pattern
+```
+
+## Common Use Cases
+
+**Find and delete old files:**
+```bash
+fd --changed-before 30d -t f -x rm {}
+```
+
+**Find large files:**
+```bash
+fd --size +100m --list-details
+```
+
+**Copy all PDFs to directory:**
+```bash
+fd -e pdf -x cp {} /target/dir/
+```
+
+**Count lines in all Python files:**
+```bash
+fd -e py -x wc -l | awk '{sum+=$1} END {print sum}'
+```
+
+**Find broken symlinks:**
+```bash
+fd -t l -x test -e {} \; -print
+```
+
+**Search in specific time window:**
+```bash
+fd --changed-within 2d --changed-before 1d
+```
+
+## Integration with other tools
+
+**With ripgrep:**
+```bash
+fd -e js | xargs rg "pattern"
+```
+
+**With fzf (fuzzy finder):**
+```bash
+vim $(fd -t f | fzf)
+```
+
+**With bat (cat alternative):**
+```bash
+fd -e md | xargs bat
+```
+
+## Performance Tips
+
+- `fd` is typically much faster than `find`
+- Respects `.gitignore` by default (disable with `-I`)
+- Uses parallel traversal automatically
+- Smart case: lowercase = case-insensitive, any uppercase = case-sensitive
+
+## Tips
+
+- Use `-t` for type filtering (f=file, d=directory, l=symlink, x=executable)
+- `-e` for extension is simpler than `-g "*.ext"`
+- `{}` in `-x` commands represents the found path
+- `{.}` strips the extension
+- `{/}` gets basename, `{//}` gets directory
+
+## Documentation
+
+GitHub: https://github.com/sharkdp/fd
+Man page: `man fd`

.opencode/skills/jq-json-processor/SKILL.md

@@ -0,0 +1,112 @@
+---
+name: jq-json-processor
+description: Process, filter, and transform JSON data using jq - the lightweight and flexible command-line JSON processor.
+homepage: https://jqlang.github.io/jq/
+metadata: {"clawdbot":{"emoji":"🔍","requires":{"bins":["jq"]},"install":[{"id":"brew","kind":"brew","formula":"jq","bins":["jq"],"label":"Install jq (brew)"},{"id":"apt","kind":"apt","package":"jq","bins":["jq"],"label":"Install jq (apt)"}]}}
+---
+
+# jq JSON Processor
+
+Process, filter, and transform JSON data with jq.
+
+## Quick Examples
+
+### Basic filtering
+```bash
+# Extract a field
+echo '{"name":"Alice","age":30}' | jq '.name'
+# Output: "Alice"
+
+# Multiple fields
+echo '{"name":"Alice","age":30}' | jq '{name: .name, age: .age}'
+
+# Array indexing
+echo '[1,2,3,4,5]' | jq '.[2]'
+# Output: 3
+```
+
+### Working with arrays
+```bash
+# Map over array
+echo '[{"name":"Alice"},{"name":"Bob"}]' | jq '.[].name'
+# Output: "Alice" "Bob"
+
+# Filter array
+echo '[1,2,3,4,5]' | jq 'map(select(. > 2))'
+# Output: [3,4,5]
+
+# Length
+echo '[1,2,3]' | jq 'length'
+# Output: 3
+```
+
+### Common operations
+```bash
+# Pretty print JSON
+cat file.json | jq '.'
+
+# Compact output
+cat file.json | jq -c '.'
+
+# Raw output (no quotes)
+echo '{"name":"Alice"}' | jq -r '.name'
+# Output: Alice
+
+# Sort keys
+echo '{"z":1,"a":2}' | jq -S '.'
+```
+
+### Advanced filtering
+```bash
+# Select with conditions
+jq '[.[] | select(.age > 25)]' people.json
+
+# Group by
+jq 'group_by(.category)' items.json
+
+# Reduce
+echo '[1,2,3,4,5]' | jq 'reduce .[] as $item (0; . + $item)'
+# Output: 15
+```
+
+### Working with files
+```bash
+# Read from file
+jq '.users[0].name' users.json
+
+# Multiple files
+jq -s '.[0] * .[1]' file1.json file2.json
+
+# Modify and save
+jq '.version = "2.0"' package.json > package.json.tmp && mv package.json.tmp package.json
+```
+
+## Common Use Cases
+
+**Extract specific fields from API response:**
+```bash
+curl -s https://api.github.com/users/octocat | jq '{name: .name, repos: .public_repos, followers: .followers}'
+```
+
+**Convert CSV-like data:**
+```bash
+jq -r '.[] | [.name, .email, .age] | @csv' users.json
+```
+
+**Debug API responses:**
+```bash
+curl -s https://api.example.com/data | jq '.'
+```
+
+## Tips
+
+- Use `-r` for raw string output (removes quotes)
+- Use `-c` for compact output (single line)
+- Use `-S` to sort object keys
+- Use `--arg name value` to pass variables
+- Pipe multiple jq operations: `jq '.a' | jq '.b'`
+
+## Documentation
+
+Full manual: https://jqlang.github.io/jq/manual/
+Interactive tutorial: https://jqplay.org/

.opencode/skills/nrepl-eval/SKILL.md

@@ -0,0 +1,120 @@
+---
+name: nrepl-eval
+description: Evaluate Clojure code via nREPL using the standalone tools/nrepl-eval.mjs CLI tool.
+---
+
+# nREPL Eval
+
+Evaluate Clojure (or ClojureScript) code via a running nREPL server using
+`tools/nrepl-eval.mjs` — a standalone CLI application.
+
+Session state (defs, in-ns, etc.) persists across invocations via a stored
+session ID, so you can build up state incrementally.
+
+## Usage
+
+```bash
+node tools/nrepl-eval.mjs [options] [<code>]
+```
+
+The tool is also executable directly:
+```bash
+./tools/nrepl-eval.mjs [options] [<code>]
+```
+
+## Options
+
+| Flag | Description | Default |
+|------|-------------|---------|
+| `-p, --port PORT` | nREPL server port | `6064` |
+| `-H, --host HOST` | nREPL server host | `127.0.0.1` |
+| `-t, --timeout MS` | Timeout in milliseconds | `120000` |
+| `--reset-session` | Discard stored session and start fresh | — |
+| `-e, --last-error` | Evaluate `*e` to retrieve the last exception | — |
+| `-h, --help` | Show help message | — |
+
+## When to Use
+
+Use this tool when you need to:
+
+1. **Evaluate Clojure code** during development — test functions, inspect
+   state, or run experiments against a running Clojure process.
+2. **Verify that edited files compile** — require namespaces with `:reload`
+   to pick up changes.
+3. **Inspect the last exception** after a failed evaluation — use `-e` to
+   print the error stored in `*e`.
+
+## Workflow
+
+### 1. Session management
+
+Sessions are persisted to `/tmp/penpot-nrepl-session-<host>-<port>`. State
+carries across calls automatically:
+
+```bash
+./tools/nrepl-eval.mjs '(def x 42)'
+./tools/nrepl-eval.mjs 'x'
+# => 42
+```
+
+Reset the session to start fresh:
+
+```bash
+./tools/nrepl-eval.mjs --reset-session '(def x 0)'
+```
+
+### 2. Evaluate code
+
+**Single expression (inline) — uses default port 6064:**
+```bash
+./tools/nrepl-eval.mjs '(+ 1 2 3)'
+```
+
+**Multiple expressions via heredoc (recommended — avoids escaping issues):**
+```bash
+./tools/nrepl-eval.mjs <<'EOF'
+(def x 10)
+(+ x 20)
+EOF
+```
+
+**Override with a different port:**
+```bash
+./tools/nrepl-eval.mjs -p 7888 '(+ 1 2 3)'
+```
+
+### 3. Inspect last exception
+
+After code throws an error, retrieve the full exception details:
+
+```bash
+./tools/nrepl-eval.mjs -e
+```
+
+## Common Patterns
+
+**Require a namespace with reload:**
+```bash
+./tools/nrepl-eval.mjs "(require '[my.namespace :as ns] :reload)"
+```
+
+**Test a function:**
+```bash
+./tools/nrepl-eval.mjs "(ns/my-function arg1 arg2)"
+```
+
+**Long-running operation with custom timeout:**
+```bash
+./tools/nrepl-eval.mjs -t 300000 "(long-running-fn)"
+```
+
+## Key Principles
+
+- **Default port is 6064** — just pass code directly, no `-p` needed when
+  your nREPL server is on 6064. Use `-p <PORT>` for a different port.
+- **Always use `:reload`** when requiring namespaces to pick up file changes.
+- **Session is reused** across invocations — defs, in-ns, and var bindings
+  persist. Use `--reset-session` to clear.
+- **Do not start any server** — the tool connects to an existing nREPL
+  server, it is not the agent's responsibility to start the nREPL server
+  (assume the server is already running on the specified port).

.opencode/skills/ripgrep/SKILL.md

@@ -0,0 +1,150 @@
+---
+name: ripgrep
+description: Blazingly fast text search tool - recursively searches directories for regex patterns with respect to gitignore rules.
+homepage: https://github.com/BurntSushi/ripgrep
+metadata: {"clawdbot":{"emoji":"🔎","requires":{"bins":["rg"]},"install":[{"id":"brew","kind":"brew","formula":"ripgrep","bins":["rg"],"label":"Install ripgrep (brew)"},{"id":"apt","kind":"apt","package":"ripgrep","bins":["rg"],"label":"Install ripgrep (apt)"}]}}
+---
+
+# ripgrep (rg)
+
+Fast, smart recursive search. Respects `.gitignore` by default.
+
+## Quick Start
+
+### Basic search
+```bash
+# Search for "TODO" in current directory
+rg "TODO"
+
+# Case-insensitive search
+rg -i "fixme"
+
+# Search specific file types
+rg "error" -t py       # Python files only
+rg "function" -t js    # JavaScript files
+```
+
+### Common patterns
+```bash
+# Whole word match
+rg -w "test"
+
+# Show only filenames
+rg -l "pattern"
+
+# Show with context (3 lines before/after)
+rg -C 3 "function"
+
+# Count matches
+rg -c "import"
+```
+
+## Advanced Usage
+
+### File type filtering
+```bash
+# Multiple file types
+rg "error" -t py -t js
+
+# Exclude file types
+rg "TODO" -T md -T txt
+
+# List available types
+rg --type-list
+```
+
+### Search modifiers
+```bash
+# Regex search
+rg "user_\d+"
+
+# Fixed string (no regex)
+rg -F "function()"
+
+# Multiline search
+rg -U "start.*end"
+
+# Only show matches, not lines
+rg -o "https?://[^\s]+"
+```
+
+### Path filtering
+```bash
+# Search specific directory
+rg "pattern" src/
+
+# Glob patterns
+rg "error" -g "*.log"
+rg "test" -g "!*.min.js"
+
+# Include hidden files
+rg "secret" --hidden
+
+# Search all files (ignore .gitignore)
+rg "pattern" --no-ignore
+```
+
+## Replacement Operations
+
+```bash
+# Preview replacements
+rg "old_name" --replace "new_name"
+
+# Actually replace (requires extra tool like sd)
+rg "old_name" -l | xargs sed -i 's/old_name/new_name/g'
+```
+
+## Performance Tips
+
+```bash
+# Parallel search (auto by default)
+rg "pattern" -j 8
+
+# Skip large files
+rg "pattern" --max-filesize 10M
+
+# Memory map files
+rg "pattern" --mmap
+```
+
+## Common Use Cases
+
+**Find TODOs in code:**
+```bash
+rg "TODO|FIXME|HACK" --type-add 'code:*.{rs,go,py,js,ts}' -t code
+```
+
+**Search in specific branches:**
+```bash
+git show branch:file | rg "pattern"
+```
+
+**Find files containing multiple patterns:**
+```bash
+rg "pattern1" | rg "pattern2"
+```
+
+**Search with context and color:**
+```bash
+rg -C 2 --color always "error" | less -R
+```
+
+## Comparison to grep
+
+- **Faster:** Typically 5-10x faster than grep
+- **Smarter:** Respects `.gitignore`, skips binary files
+- **Better defaults:** Recursive, colored output, line numbers
+- **Easier:** Simpler syntax for common tasks
+
+## Tips
+
+- `rg` is often faster than `grep -r`
+- Use `-t` for file type filtering instead of `--include`
+- Combine with other tools: `rg pattern -l | xargs tool`
+- Add custom types in `~/.ripgreprc`
+- Use `--stats` to see search performance
+
+## Documentation
+
+GitHub: https://github.com/BurntSushi/ripgrep
+User Guide: https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md

AGENTS.md

@@ -1,499 +1,104 @@
-# Penpot – Instructions
+# AI Agent Guide
+
+This document provides the core context and operating guidelines for AI agents
+working in this repository.
+
+## Before You Start
+
+Before responding to any user request, you must:
+
+1. Read this file completely.
+2. Identify which modules are affected by the task.
+3. Load the `AGENTS.md` file **only** for each affected module (see the
+   architecture table below). Not all modules have an `AGENTS.md` — verify the
+   file exists before attempting to read it.
+4. Do **not** load `AGENTS.md` files for unrelated modules.
+
+## Role: Senior Software Engineer
+
+You are a high-autonomy Senior Full-Stack Software Engineer. You have full
+permission to navigate the codebase, modify files, and execute commands to
+fulfill your tasks. Your goal is to solve complex technical tasks with high
+precision while maintaining a strong focus on maintainability and performance.
+
+### Operational Guidelines
+
+1. Before writing code, describe your plan. If the task is complex, break it
+   down into atomic steps.
+2. Be concise and autonomous.
+3. Do **not** touch unrelated modules unless the task explicitly requires it.
+4. Commit only when explicitly asked. Follow the commit format rules in
+   `CONTRIBUTING.md`.
+5. When searching code, prefer `ripgrep` (`rg`) over `grep` — it respects
+   `.gitignore` by default.
+
+## Changelogs
+
+The project has two changelogs:
+
+- **Main project changelog**: `CHANGES.md` (root of the repository). Tracks changes for the core Penpot application (backend, frontend, common, render-wasm, exporter, mcp).
+- **Plugins changelog**: `plugins/CHANGELOG.md`. Tracks changes for the plugins subproject only.
+
+When making changes, add a changelog entry to the appropriate file under the
+`## <version> (Unreleased)` section in the correct category
+(`:sparkles: New features & Enhancements` or `:bug: Bugs fixed`).
+
+## GitHub Operations
+
+To obtain the list of repository members/collaborators:
+
+```bash
+gh api repos/:owner/:repo/collaborators --paginate --jq '.[].login'
+```
+
+To obtain the list of open PRs authored by members:
+
+```bash
+MEMBERS=$(gh api repos/:owner/:repo/collaborators --paginate --jq '.[].login' | tr '\n' '|' | sed 's/|$//')
+gh pr list --state open --limit 200 --json author,title,number | jq -r --arg members "$MEMBERS" '
+  ($members | split("|")) as $m |
+  .[] | select(.author.login as $a | $m | index($a)) |
+  "\(.number)\t\(.author.login)\t\(.title)"
+'
+```
+
+To obtain the list of open PRs from external contributors (non-members):
+
+```bash
+MEMBERS=$(gh api repos/:owner/:repo/collaborators --paginate --jq '.[].login' | tr '\n' '|' | sed 's/|$//')
+gh pr list --state open --limit 200 --json author,title,number | jq -r --arg members "$MEMBERS" '
+  ($members | split("|")) as $m |
+  .[] | select(.author.login as $a | $m | index($a) | not) |
+  "\(.number)\t\(.author.login)\t\(.title)"
+'
+```
 
 ## Architecture Overview
 
-Penpot is a full-stack design tool composed of several distinct components:
-
-| Component | Language | Role |
-|-----------|----------|------|
-| `frontend/` | ClojureScript + SCSS | Single-page React app (design editor) |
-| `backend/` | Clojure (JVM) | HTTP/RPC server, PostgreSQL, Redis |
-| `common/` | Cljc (shared Clojure/ClojureScript) | Data types, geometry, schemas, utilities |
-| `exporter/` | ClojureScript (Node.js) | Headless Playwright-based export (SVG/PDF) |
-| `render-wasm/` | Rust → WebAssembly | High-performance canvas renderer using Skia |
-| `mcp/` | TypeScript | Model Context Protocol integration |
-| `plugins/` | TypeScript | Plugin runtime and example plugins |
-
-The monorepo is managed with `pnpm` workspaces. The `manage.sh`
-orchestrates cross-component builds. `run-ci.sh` defines the CI
-pipeline.
-
-## Search Standards
-
-When searching code, always use `ripgrep` (rg) instead of grep if
-available, as it respects `.gitignore` by default.
-
-If using grep, try to exclude node_modules and .shadow-cljs directories
-
-
-## Build, Test & Lint Commands
-
-### Frontend (`cd frontend`)
-
-Run `./scripts/setup` for setup all dependencies.
-
-
-```bash
-# Build (Producution)
-./scripts/build
-
-# Tests
-pnpm run test                      # Build ClojureScript tests + run node target/tests/test.js
-
-# Lint
-pnpm run lint:js              # Linter for JS/TS
-pnpm run lint:clj             # Linter for CLJ/CLJS/CLJC
-pnpm run lint:scss            # Linter for SCSS
-
-# Check Code Formart
-pnpm run check-fmt:clj        # Format CLJ/CLJS/CLJC
-pnpm run check-fmt:js         # Format JS/TS
-pnpm run check-fmt:scss       # Format SCSS
-
-# Code Format (Automatic Formating)
-pnpm run fmt:clj              # Format CLJ/CLJS/CLJC
-pnpm run fmt:js               # Format JS/TS
-pnpm run fmt:scss             # Format SCSS
-```
-
-To run a focused ClojureScript unit test: edit
-`test/frontend_tests/runner.cljs` to narrow the test suite, then `pnpm
-run build:test && node target/tests/test.js`.
-
-
-### Backend (`cd backend`)
-
-Run `pnpm install` for install all dependencies.
-
-```bash
-# Run full test suite
-pnpm run test
-
-# Run single namespace
-pnpm run test --focus backend-tests.rpc-doc-test
-
-# Check Code Format
-pnpm run check-fmt
-
-# Code Format (Automatic Formatting)
-pnpm run fmt
-
-# Code Linter
-pnpm run lint
-```
-
-Test config is in `backend/tests.edn`; test namespaces match
-`.*-test$` under `test/` directory. You should not touch this file,
-just use it for reference.
-
-
-### Common (`cd common`)
-
-This contains code that should compile and run under different runtimes: JVM & JS so the commands are
-separarated for each runtime.
-
-```bash
-clojure -M:dev:test                                    # Run full test suite under JVM
-clojure -M:dev:test --focus backend-tests.my-ns-test   # Run single namespace under JVM
-
-# Run full test suite under JS or JVM runtimes
-pnpm run test:js
-pnpm run test:jvm
-
-# Run single namespace (only on JVM)
-pnpm run test:jvm --focus common-tests.my-ns-test
-
-# Lint
-pnpm run lint:clj        # Lint CLJ/CLJS/CLJC code
-
-# Check Format
-pnpm run check-fmt:clj   # Check CLJ/CLJS/CLJS code
-pnpm run check-fmt:js    # Check JS/TS code
-
-# Code Format (Automatic Formatting)
-pnpm run fmt:clj         # Check CLJ/CLJS/CLJS code
-pnpm run fmt:js          # Check JS/TS code
-```
-
-To run a focused ClojureScript unit test: edit
-`test/common_tests/runner.cljs` to narrow the test suite, then `pnpm
-run build:test && node target/tests/test.js`.
-
-
-### Render-WASM (`cd render-wasm`)
-
-```bash
-./test                        # Rust unit tests (cargo test)
-./build                       # Compile to WASM (requires Emscripten)
-cargo fmt --check
-./lint --debug
-```
-
-## Key Conventions
-
-### Namespace Structure
-
-The backend, frontend and exporter are developed using clojure and
-clojurescript and code is organized in namespaces. This is a general
-overview of the available namespaces.
-
-**Backend:**
-- `app.rpc.commands.*` – RPC command implementations (`auth`, `files`, `teams`, etc.)
-- `app.http.*` – HTTP routes and middleware
-- `app.db.*` – Database layer
-- `app.tasks.*` – Background job tasks
-- `app.main` – Integrant system setup and entrypoint
-- `app.loggers` – Internal loggers (auditlog, mattermost, etc) (do not be confused with `app.common.loggin`)
-
-**Frontend:**
-- `app.main.ui.*` – React UI components (`workspace`, `dashboard`, `viewer`)
-- `app.main.data.*` – Potok event handlers (state mutations + side effects)
-- `app.main.refs` – Reactive subscriptions (okulary lenses)
-- `app.main.store` – Potok event store
-- `app.util.*` – Utilities (DOM, HTTP, i18n, keyboard shortcuts)
-
-**Common:**
-- `app.common.types.*` – Shared data types for shapes, files, pages using Malli schemas
-- `app.common.schema` – Malli abstraction layer, exposes the most used functions from malli
-- `app.common.geom.*` – Geometry and shape transformation helpers
-- `app.common.data` – Generic helpers used around all application
-- `app.common.math` – Generic math helpers used around all aplication
-- `app.common.json` – Generic JSON encoding/decoding helpers
-- `app.common.data.macros` – Performance macros used everywhere
-
-
-### Backend RPC Commands
-
-The PRC methods are implement in a some kind of multimethod structure using
-`app.util.serivices` namespace. All RPC methods are collected under `app.rpc`
-namespace and exposed under `/api/rpc/command/<cmd-name>`. The RPC method
-accepts POST and GET requests indistinctly and uses `Accept` header for
-negotiate the response encoding (which can be transit, the defaut or plain
-json). It also accepts transit (defaut) or json as input, which should be
-indicated using `Content-Type` header.
-
-This is an example:
-
-```clojure
-(sv/defmethod ::my-command
-  {::rpc/auth true            ;; requires auth
-   ::doc/added "1.18"
-   ::sm/params [:map ...]     ;; malli input schema
-   ::sm/result [:map ...]}    ;; malli output schema
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id] :as params}]
-  ;; return a plain map or throw
-  {:id (uuid/next)})
-```
-
-Look under `src/app/rpc/commands/*.clj` to see more examples.
-
-
-### Frontend State Management (Potok)
-
-State is a single atom managed by a Potok store. Events implement protocols
-(funcool/potok library):
-
-```clojure
-(defn my-event
-  "doc string"
-  [data]
-  (ptk/reify ::my-event
-    ptk/UpdateEvent
-    (update [_ state]           ;; synchronous state transition
-      (assoc state :key data))
-
-    ptk/WatchEvent
-    (watch [_ state stream]     ;; async: returns an observable
-      (->> (rp/cmd! :some-rpc-command params)
-           (rx/map success-event)
-           (rx/catch error-handler)))
-
-    ptk/EffectEvent
-    (effect [_ state _]         ;; pure side effects (DOM, logging)
-      (dom/focus (dom/get-element "id")))))
-```
-
-The state is located under `app.main.store` namespace where we have
-the `emit!` function responsible of emiting events.
-
-Example:
-
-```cljs
-(ns some.ns
-  (:require
-    [app.main.data.my-events :refer [my-event]]
-    [app.main.store :as st]))
-
-(defn on-click
-  [event]
-  (st/emit! (my-event)))
-```
-
-On `app.main.refs` we have reactive references which lookup into the main state
-for just inner data or precalculated data. That references are very usefull but
-should be used with care because, per example if we have complex operation, this
-operation will be executed on each state change, and sometimes is better to have
-simple references and use react `use-memo` for more granular memoization.
-
-Prefer helpers from `app.util.dom` instead of using direct dom calls, if no helper is
-available, prefer adding a new helper for handling it and the use the
-new helper.
-
-
-### CSS (Modules Pattern)
-
-Styles are co-located with components. Each `.cljs` file has a corresponding
-`.scss` file:
-
-```clojure
-;; In the component namespace:
-(require '[app.main.style :as stl])
-
-;; In the render function:
-[:div {:class (stl/css :container :active)}]
-
-;; Conditional:
-[:div {:class (stl/css-case :some-class true :selected (= drawtool :rect))}]
-
-;; When you need concat an existing class:
-[:div {:class [existing-class (stl/css-case :some-class true :selected (= drawtool :rect))]}]
-```
-
-### Integration tests (Playwright)
-
-Integration tests are developed under `frontend/playwright` directory, we use
-mocks for remove communication with backend.
-
-The tests should be executed under `./frontend` directory:
+Penpot is an open-source design tool composed of several modules:
+
+| Directory | Language | Purpose | Has `AGENTS.md` |
+|-----------|----------|---------|:----------------:|
+| `frontend/` | ClojureScript + SCSS | Single-page React app (design editor) | Yes |
+| `backend/` | Clojure (JVM) | HTTP/RPC server, PostgreSQL, Redis | Yes |
+| `common/` | Cljc (shared Clojure/ClojureScript) | Data types, geometry, schemas, utilities | Yes |
+| `render-wasm/` | Rust -> WebAssembly | High-performance canvas renderer (Skia) | Yes |
+| `exporter/` | ClojureScript (Node.js) | Headless Playwright-based export (SVG/PDF) | No |
+| `mcp/` | TypeScript | Model Context Protocol integration | No |
+| `plugins/` | TypeScript | Plugin runtime and example plugins | No |
+
+Some submodules use `pnpm` workspaces. The root `package.json` and
+`pnpm-lock.yaml` manage shared dependencies. Helper scripts live in `scripts/`.
+
+### Module Dependency Graph
 
 ```
-cd frontend/
-
-pnpm run test:e2e                   # Playwright e2e tests
-pnpm run test:e2e --grep "pattern"  # Single e2e test by pattern
+frontend ──> common
+backend  ──> common
+exporter ──> common
+frontend ──> render-wasm  (loads compiled WASM)
 ```
 
-Ensure everything installed with `./scripts/setup` script.
-
-
-### Performance Macros (`app.common.data.macros`)
-
-Always prefer these macros over their `clojure.core` equivalents — they compile to faster JavaScript:
-
-```clojure
-(dm/select-keys m [:a :b])     ;; ~6x faster than core/select-keys
-(dm/get-in obj [:a :b :c])     ;; faster than core/get-in
-(dm/str "a" "b" "c")           ;; string concatenation
-```
-
-### Shared Code under Common (CLJC)
-
-Files in `common/src/app/common/` use reader conditionals to target both runtimes:
-
-```clojure
-#?(:clj  (import java.util.UUID)
-   :cljs (:require [cljs.core :as core]))
-```
-
-Both frontend and backend depend on `common` as a local library (`penpot/common
-{:local/root "../common"}`).
-
-
-
-### Component Standards & Syntax (React & Rumext: mf/defc)
-
-The codebase contains various component patterns. When creating or refactoring
-components, follow the Modern Syntax rules outlined below.
-
-1. The * Suffix Convention
-
-The most recent syntax uses a * suffix in the component name (e.g.,
-my-component*). This suffix signals the mf/defc macro to apply specific rules
-for props handling and destructuring and optimization.
-
-2. Component Definition
-
-Modern components should use the following structure:
-
-```clj
-(mf/defc my-component*
-  {::mf/wrap [mf/memo]}         ;; Equivalent to React.memo
-  [{:keys [name on-click]}]     ;; Destructured props
-  [:div {:class (stl/css :root)
-         :on-click on-click}
-   name])
-```
-
-3. Hooks
-
-Use the mf namespace for hooks to maintain consistency with the macro's
-lifecycle management. These are analogous to standard React hooks:
-
-```clj
-(mf/use-state)  ;; analogous to React.useState adapted to cljs semantics
-(mf/use-effect) ;; analogous to React.useEffect
-(mf/use-memo)   ;; analogous to React.useMemo
-(mf/use-fn)     ;; analogous to React.useCallback
-```
-
-The `mf/use-state` in difference with React.useState, returns an atom-like
-object, where you can use `swap!` or `reset!` for to perform an update and
-`deref` for get the current value.
-
-You also has `mf/deref` hook (which does not follow the `use-` naming pattern)
-and it's purpose is watch (subscribe to changes) on atom or derived atom (from
-okulary) and get the current value. Is mainly used for subscribe to lenses
-defined in `app.main.refs` or (private lenses defined in namespaces).
-
-Rumext also comes with improved syntax macros as alternative to `mf/use-effect`
-and `mf/use-memo` functions. Examples:
-
-
-Example for `mf/with-memo` macro:
-
-```clj
-;; Using functions
-(mf/use-effect
-  (mf/deps team-id)
-  (fn []
-    (st/emit! (dd/initialize team-id))
-    (fn []
-      (st/emit! (dd/finalize team-id)))))
-
-;; The same effect but using mf/with-effect
-(mf/with-effect [team-id]
-  (st/emit! (dd/initialize team-id))
-  (fn []
-    (st/emit! (dd/finalize team-id))))
-```
-
-Example for `mf/with-memo` macro:
-
-```
-;; Using functions
-(mf/use-memo
-  (mf/deps projects team-id)
-  (fn []
-    (->> (vals projects)
-         (filterv #(= team-id (:team-id %))))))
-
-;; Using the macro
-(mf/with-memo [projects team-id]
-  (->> (vals projects)
-       (filterv #(= team-id (:team-id %)))))
-```
-
-Prefer using the macros for it syntax simplicity.
-
-
-4. Component Usage (Hiccup Syntax)
-
-When invoking a component within Hiccup, always use the [:> component* props]
-pattern.
-
-Requirements for props:
-
-- Must be a map literal or a symbol pointing to a JavaScript props object.
-- To create a JS props object, use the `#js` literal or the `mf/spread-object` helper macro.
-
-Examples:
-
-```clj
-;; Using object literal (no need of #js because macro already interprets it)
-[:> my-component* {:data-foo "bar"}]
-
-;; Using object literal (no need of #js because macro already interprets it)
-(let [props #js {:data-foo "bar"
-                 :className "myclass"}]
-  [:> my-component* props])
-
-;; Using the spread helper
-(let [props (mf/spread-object base-props {:extra "data"})]
-  [:> my-component* props])
-```
-
-4. Checklist
-
-- [ ] Does the component name end with *?
-
-
-## Commit Format Guidelines
-
-Format: `<emoji-code> <subject>`
-
-```
-:bug: Fix unexpected error on launching modal
-
-Optional body explaining the why.
-
-Signed-off-by: Fullname <email>
-```
-
-**Subject rules:** imperative mood, capitalize first letter, no
-trailing period, ≤ 80 characters. Add an entry to `CHANGES.md` if
-applicable.
-
-**Code patches must include a DCO sign-off** (`git commit -s`).
-
-| Emoji | Emoji-Code | Use for |
-|-------|------|---------|
-| 🐛 | `:bug:` | Bug fix |
-| ✨ | `:sparkles:` | Improvement |
-| 🎉 | `:tada:` | New feature |
-| ♻️ | `:recycle:` | Refactor |
-| 💄 | `:lipstick:` | Cosmetic changes |
-| 🚑 | `:ambulance:` | Critical bug fix |
-| 📚 | `:books:` | Docs |
-| 🚧 | `:construction:` | WIP |
-| 💥 | `:boom:` | Breaking change |
-| 🔧 | `:wrench:` | Config update |
-| ⚡ | `:zap:` | Performance |
-| 🐳 | `:whale:` | Docker |
-| 📎 | `:paperclip:` | Other non-relevant changes |
-| ⬆️ | `:arrow_up:` | Dependency upgrade |
-| ⬇️ | `:arrow_down:` | Dependency downgrade |
-| 🔥 | `:fire:` | Remove files or code |
-| 🌐 | `:globe_with_meridians:` | Translations |
-
-
-## SCSS Rules & Migration
-
-### General rules
-
-- Prefer CSS custom properties ( `margin: var(--sp-xs);`) instead of scss
-  variables and get the already defined properties from `_sizes.scss`. The SCSS
-  variables are allowed and still used, just prefer properties if they are
-  already defined.
-- If a value isn't in the DS, use the `px2rem(n)` mixin: `@use "ds/_utils.scss"
-  as *; padding: px2rem(23);`.
-- Do **not** create new SCSS variables for one-off values.
-- Use physical directions with logical ones to support RTL/LTR naturally.
-  - ❌ `margin-left`, `padding-right`, `left`, `right`.
-  - ✅ `margin-inline-start`, `padding-inline-end`, `inset-inline-start`.
-- Always use the `use-typography` mixin from `ds/typography.scss`.
-  - ✅ `@include t.use-typography("title-small");`
-- Use `$br-*` for radius and `$b-*` for thickness from `ds/_borders.scss`.
-- Use only tokens from `ds/colors.scss`. Do **NOT** use `design-tokens.scss` or
-  legacy color variables.
-- Use mixins only those defined in`ds/mixins.scss`. Avoid legacy mixins like
-  `@include flexCenter;`. Write standard CSS (flex/grid) instead.
-
-### Syntax & Structure
-
-- Use the `@use` instead of `@import`. If you go to refactor existing SCSS file,
-  try to replace all `@import` with `@use`. Example: `@use "ds/_sizes.scss" as
-  *;` (Use `as *` to expose variables directly).
-- Avoid deep selector nesting or high-specificity (IDs). Flatten selectors:
-  - ❌ `.card { .title { ... } }`
-  - ✅ `.card-title { ... }`
-- Leverage component-level CSS variables for state changes (hover/focus) instead
-  of rewriting properties.
-
-### Checklist
-
-- [ ] No references to `common/refactor/` 
-- [ ] All `@import` converted to `@use` (only if refactoring)
-- [ ] Physical properties (left/right) using logical properties (inline-start/end).
-- [ ] Typography implemented via `use-typography()` mixin.
-- [ ] Hardcoded pixel values wrapped in `px2rem()`.
-- [ ] Selectors are flat (no deep nesting).
+`common` is referenced as a local dependency (`{:local/root "../common"}`) by
+both `frontend` and `backend`. Changes to `common` can therefore affect multiple
+modules — test across consumers when modifying shared code.

CHANGES.md

@@ -1,6 +1,160 @@
 # CHANGELOG
 
-## 2.14.0 (Unreleased)
+## 2.15.2
+
+### :bug: Bugs fixed
+
+- Fix mcp related internal config for docker images [Github #9565](https://github.com/penpot/penpot/pull/9565)
+
+
+## 2.15.1
+
+### :sparkles: New features & Enhancements
+
+- Add support for chunked uploading of fonts [Github #9560](https://github.com/penpot/penpot/issues/9560)
+
+
+## 2.15.0
+
+### :sparkles: New features & Enhancements
+
+- Add MCP server integration [Github #9174](https://github.com/penpot/penpot/issues/9174)
+  (PR: [#9032](https://github.com/penpot/penpot/pull/9032), [#9321](https://github.com/penpot/penpot/pull/9321))
+- Add chunked upload API for large media and binary files (removes previous upload size limits) [Github #9516](https://github.com/penpot/penpot/issues/9516)
+  (PR: [#8909](https://github.com/penpot/penpot/pull/8909))
+- Add anonymous telemetry event collection [Github #9467](https://github.com/penpot/penpot/issues/9467)
+  (PR: [#9065](https://github.com/penpot/penpot/pull/9065), [#9483](https://github.com/penpot/penpot/pull/9483))
+- Improve team name validation [Github #9517](https://github.com/penpot/penpot/issues/9517)
+  (PR: [#9176](https://github.com/penpot/penpot/pull/9176))
+- Enhance readability of applied tokens in plugins API [Github #9175](https://github.com/penpot/penpot/issues/9175)
+  (PR: [#8607](https://github.com/penpot/penpot/pull/8607))
+- Encourage use of flex/grid layouts in designs generated via MCP [Github #9081](https://github.com/penpot/penpot/issues/9081)
+  (PR: [#9084](https://github.com/penpot/penpot/pull/9084))
+- Improve MCP server logging, adding Loki support [Github #9415](https://github.com/penpot/penpot/issues/9415)
+  (PR: [#9425](https://github.com/penpot/penpot/pull/9425))
+- Add security headers to Nginx on Docker images [Github #9519](https://github.com/penpot/penpot/issues/9519)
+  (PR: [#9473](https://github.com/penpot/penpot/pull/9473))
+
+### :bug: Bugs fixed
+
+- Fix text edition mode not exited when changing selection, blocking token application [Github #9346](https://github.com/penpot/penpot/issues/9346)
+  (PR: [#9355](https://github.com/penpot/penpot/pull/9355))
+- Reduce memory usage of MCP server when handling images (by @opcode81) [Github #9420](https://github.com/penpot/penpot/issues/9420)
+  (PR: [#9431](https://github.com/penpot/penpot/pull/9431))
+- Fix Plugin API token methods rejecting JS array of strings (by @boskodev790) [Github #9162](https://github.com/penpot/penpot/issues/9162)
+  (PR: [#9166](https://github.com/penpot/penpot/pull/9166))
+- Fix release notes modal appearing behind the dashboard sidebar (by @RenzoMXD) [Github #8296](https://github.com/penpot/penpot/issues/8296)
+  (PR: [#9126](https://github.com/penpot/penpot/pull/9126), [#9233](https://github.com/penpot/penpot/pull/9233))
+- Fix empty warning on login [Github #9520](https://github.com/penpot/penpot/issues/9520)
+  (PR: [#9056](https://github.com/penpot/penpot/pull/9056))
+- Fix maximum call stack size exceeded in SSE read-stream [Github #9470](https://github.com/penpot/penpot/issues/9470)
+  (PR: [#9484](https://github.com/penpot/penpot/pull/9484))
+- Fix incorrect handling of version restore operation [Github #9515](https://github.com/penpot/penpot/issues/9515)
+  (PR: [#9041](https://github.com/penpot/penpot/pull/9041))
+- Fix MCP ReplServer binding to all interfaces (0.0.0.0) instead of localhost, allowing unauthenticated RCE [Github #9518](https://github.com/penpot/penpot/issues/9518)
+  (PR: [#9400](https://github.com/penpot/penpot/pull/9400))
+- Fix MCP integrations URL copy action to match the URL displayed in settings [Github #9238](https://github.com/penpot/penpot/issues/9238)
+  (PR: [#9239](https://github.com/penpot/penpot/pull/9239))
+- Fix swapped analytics event names on MCP tab-switch dialog (by @Dexterity104) [Github #9496](https://github.com/penpot/penpot/issues/9496)
+  (PR: [#9322](https://github.com/penpot/penpot/pull/9322))
+- Fix multiple selection on shapes with token applied to stroke color [Github #9522](https://github.com/penpot/penpot/issues/9522)
+  (PR: [#9110](https://github.com/penpot/penpot/pull/9110))
+- Fix onboarding modals appearing behind libraries and templates panel [Github #9521](https://github.com/penpot/penpot/issues/9521)
+  (PR: [#9178](https://github.com/penpot/penpot/pull/9178))
+- Fix keep-alive interval leak in PluginBridge (by @opcode81) [Github #9430](https://github.com/penpot/penpot/issues/9430)
+  (PR: [#9435](https://github.com/penpot/penpot/pull/9435))
+
+## 2.14.5
+
+### :bug: Bugs fixed
+
+- Fix incorrect invitation token handling on register process [Github #9380](https://github.com/penpot/penpot/pull/9380)
+
+## 2.14.4
+
+### :bug: Bugs fixed
+
+- Fix email validation [Taiga #14006](https://tree.taiga.io/project/penpot/issue/14006)
+- Fix email blacklisting [Github #9122](https://github.com/penpot/penpot/pull/9122)
+- Fix removeChild errors from unmount race conditions [Github #8927](https://github.com/penpot/penpot/pull/8927)
+
+
+## 2.14.3
+
+### :sparkles: New features & Enhancements
+
+- Add webp export format to plugin types [Github #8870](https://github.com/penpot/penpot/pull/8870)
+- Use shared singleton containers for React portals to reduce DOM growth [Github #8957](https://github.com/penpot/penpot/pull/8957)
+
+### :bug: Bugs fixed
+
+- Fix variants corner cases with selrect and points [Github #8882](https://github.com/penpot/penpot/pull/8882)
+- Fix dashboard navigation tabs overlap with projects content when scrolling [Taiga #13962](https://tree.taiga.io/project/penpot/issue/13962)
+- Fix text editor v1 focus [Taiga #13961](https://tree.taiga.io/project/penpot/issue/13961)
+- Fix highlight on frames after rename [Github #8938](https://github.com/penpot/penpot/pull/8938)
+- Fix TypeError in sd-token-uuid when resolving tokens interactively [Github #8929](https://github.com/penpot/penpot/pull/8929)
+- Fix path drawing preview passing shape instead of content to next-node
+- Fix swapped arguments in CLJS PathData `-nth` with default
+- Normalize PathData coordinates to safe integer bounds on read
+- Fix RangeError from re-entrant error handling causing stack overflow [Github #8962](https://github.com/penpot/penpot/pull/8962)
+- Fix builder bool styles and media validation [Github #8963](https://github.com/penpot/penpot/pull/8963)
+- Fix "Move to" menu allowing same project as target when multiple files are selected
+- Fix crash when index query param is duplicated in URL
+- Fix wrong extremity point in path `calculate-extremities` for line-to segments
+- Fix reversed args in DTCG shadow composite token conversion
+- Fix `inside-layout?` passing shape id instead of shape to `frame-shape?`
+- Fix wrong `mapcat` call in `collect-main-shapes`
+- Fix stale accumulator in `get-children-in-instance` recursion
+- Fix typo `:podition` in swap-shapes grid cell
+- Fix multiple selection on shapes with token applied to stroke color
+
+
+## 2.14.2
+
+### :sparkles: New features & Enhancements
+
+- Add protection for stale JS asset cache to force reload on version mismatch [Github #8638](https://github.com/penpot/penpot/pull/8638)
+- Normalize newsletter opt-in checkbox across different register flows [Github #8839](https://github.com/penpot/penpot/pull/8839)
+
+### :bug: Bugs fixed
+
+- Fix PathData corruption root causes across WASM and CLJS (unsafe transmute and byteOffset handling)
+- Handle corrupted PathData segments gracefully instead of crashing
+- Fix swapped move-to/line-to type codes in PathData binary readers
+- Fix non-integer row/column values in grid cell position inputs [Github #8869](https://github.com/penpot/penpot/pull/8869)
+- Fix nil path content crash by exposing safe public API [Github #8806](https://github.com/penpot/penpot/pull/8806)
+- Fix infinite recursion in get-frame-ids for thumbnail extraction [Github #8807](https://github.com/penpot/penpot/pull/8807)
+- Fix stale-asset detector missing protocol-dispatch errors
+- Ignore Zone.js toString TypeError in uncaught error handler [Github #8804](https://github.com/penpot/penpot/pull/8804)
+- Prevent thumbnail frame recursion overflow [Github #8763](https://github.com/penpot/penpot/pull/8763)
+- Fix vector index out of bounds in viewer zoom-to-fit/fill [Github #8834](https://github.com/penpot/penpot/pull/8834)
+- Guard delete undo against missing sibling order [Github #8858](https://github.com/penpot/penpot/pull/8858)
+- Fix ICounted error on numeric-input token dropdown keyboard nav [Github #8803](https://github.com/penpot/penpot/pull/8803)
+
+## 2.14.1
+
+### :sparkles: New features & Enhancements
+
+- Add automatic retry with backoff for idempotent RPC requests on network failures [Github #8792](https://github.com/penpot/penpot/pull/8792)
+- Add scroll and zoom throttling to one state update per animation frame [Github #8812](https://github.com/penpot/penpot/pull/8812)
+- Improve error handling and exception formatting [Github #8757](https://github.com/penpot/penpot/pull/8757)
+
+### :bug: Bugs fixed
+
+- Fix crash in apply-text-modifier with nil selrect or modifier [Github #8762](https://github.com/penpot/penpot/pull/8762)
+- Fix incorrect attrs references on generate-sync-shape [Github #8776](https://github.com/penpot/penpot/pull/8776)
+- Fix regression on subpath support [Github #8793](https://github.com/penpot/penpot/pull/8793)
+- Improve error reporting on request parsing failures [Github #8805](https://github.com/penpot/penpot/pull/8805)
+- Fix fetch abort errors escaping the unhandled exception handler [Github #8801](https://github.com/penpot/penpot/pull/8801)
+- Fix nil deref on missing bounds in layout modifier propagation [Github #8735](https://github.com/penpot/penpot/pull/8735)
+- Fix TypeError when token error map lacks :error/fn key [Github #8767](https://github.com/penpot/penpot/pull/8767)
+- Fix dissoc error when detaching stroke color from library [Github #8738](https://github.com/penpot/penpot/pull/8738)
+- Fix crash when pasting image into text editor
+- Fix null text crash on paste in text editor
+- Ensure path content is always PathData when saving
+- Fix error when get-parent-with-data encounters non-Element nodes
+
+## 2.14.0
 
 ### :boom: Breaking changes & Deprecations
 
@@ -23,11 +177,9 @@
 - Fix wrong image in the onboarding invitation block [Taiga #13040](https://tree.taiga.io/project/penpot/issue/13040)
 - Fix wrong register image [Taiga #12955](https://tree.taiga.io/project/penpot/task/12955)
 - Fix error message on components doesn't close automatically [Taiga #12012](https://tree.taiga.io/project/penpot/issue/12012)
-- Fix incorrect handling of input values on layout gap and padding inputs [Github #8113](https://github.com/penpot/penpot/issues/8113)
 - Fix incorrect default option on tokens import dialog [Github #8051](https://github.com/penpot/penpot/pull/8051)
 - Fix unhandled exception tokens creation dialog [Github #8110](https://github.com/penpot/penpot/issues/8110)
 - Fix displaying a hidden user avatar when there is only one more [Taiga #13058](https://tree.taiga.io/project/penpot/issue/13058)
-- Fix unhandled exception on open-new-window helper [Github #7787](https://github.com/penpot/penpot/issues/7787)
 - Fix exception on uploading large fonts [Github #8135](https://github.com/penpot/penpot/pull/8135)
 - Fix boolean operators in menu for boards [Taiga #13174](https://tree.taiga.io/project/penpot/issue/13174)
 - Fix viewer can update library [Taiga #13186](https://tree.taiga.io/project/penpot/issue/13186)
@@ -88,12 +240,8 @@
 - Fix missing text color token from selected shapes in selected colors list [Taiga #12956](https://tree.taiga.io/project/penpot/issue/12956)
 - Fix dropdown option width in Guides columns dropdown [Taiga #12959](https://tree.taiga.io/project/penpot/issue/12959)
 - Fix typos on download modal [Taiga #12865](https://tree.taiga.io/project/penpot/issue/12865)
-- Fix problem with text editor maintaining previous styles [Taiga #12835](https://tree.taiga.io/project/penpot/issue/12835)
-- Fix unhandled exception tokens creation dialog [Github #8110](https://github.com/penpot/penpot/issues/8110)
 - Fix allow negative spread values on shadow token creation [Taiga #13167](https://tree.taiga.io/project/penpot/issue/13167)
 - Fix spanish translations on import export token modal [Taiga #13171](https://tree.taiga.io/project/penpot/issue/13171)
-- Remove whitespaces from asset export filename [Github #8133](https://github.com/penpot/penpot/pull/8133)
-- Fix exception on uploading large fonts [Github #8135](https://github.com/penpot/penpot/pull/8135)
 - Fix unhandled exception on open-new-window helper [Github #7787](https://github.com/penpot/penpot/issues/7787)
 - Fix incorrect handling of input values on layout gap and padding inputs [Github #8113](https://github.com/penpot/penpot/issues/8113)
 - Fix several race conditions on path editor [Github #8187](https://github.com/penpot/penpot/pull/8187)
@@ -186,10 +334,8 @@ example. It's still usable as before, we just removed the example.
 ### :bug: Bugs fixed
 
 - Fix text line-height values are wrong [Taiga #12252](https://tree.taiga.io/project/penpot/issue/12252)
-- Fix an error translation [Taiga #12402](https://tree.taiga.io/project/penpot/issue/12402)
 - Fix pan cursor not disabling viewport guides [Github #6985](https://github.com/penpot/penpot/issues/6985)
 - Fix viewport resize on locked shapes [Taiga #11974](https://tree.taiga.io/project/penpot/issue/11974)
-- Fix nested variant in a component doesn't keep inherited overrides [Taiga #12299](https://tree.taiga.io/project/penpot/issue/12299)
 - Fix on copy instance inside a components chain touched are missing [Taiga #12371](https://tree.taiga.io/project/penpot/issue/12371)
 - Fix problem with multiple selection and shadows [Github #7437](https://github.com/penpot/penpot/issues/7437)
 - Fix search shortcut [Taiga #10265](https://tree.taiga.io/project/penpot/issue/10265)
@@ -270,7 +416,7 @@ example. It's still usable as before, we just removed the example.
 - Fix text override is lost after switch [Taiga #12269](https://tree.taiga.io/project/penpot/issue/12269)
 - Fix exporting a board crashing the app [Taiga #12384](https://tree.taiga.io/project/penpot/issue/12384)
 - Fix nested variant in a component doesn't keep inherited overrides [Taiga #12299](https://tree.taiga.io/project/penpot/issue/12299)
-- Fix selected colors not showing colors from children shapes in multiple selection [Taiga #12384](https://tree.taiga.io/project/penpot/issue/12385)
+- Fix selected colors not showing colors from children shapes in multiple selection [Taiga #12385](https://tree.taiga.io/project/penpot/issue/12385)
 - Fix scrollbar issue in design tab [Taiga #12367](https://tree.taiga.io/project/penpot/issue/12367)
 - Fix library update notificacions showing when they should not [Taiga #12397](https://tree.taiga.io/project/penpot/issue/12397)
 - Fix remove flex button doesn’t work within variant [Taiga #12314](https://tree.taiga.io/project/penpot/issue/12314)

CONTRIBUTING.md

@@ -1,208 +1,289 @@
-# Contributing Guide #
+# Contributing Guide
 
-Thank you for your interest in contributing to Penpot. This is a
-generic guide that details how to contribute to the project in a way that
-is efficient for everyone. If you are looking for specific documentation on
-different parts of the platform, please refer to the `docs/` directory,
-or the rendered version at the [Help Center](https://help.penpot.app/).
+Thank you for your interest in contributing to Penpot. This guide covers
+how to propose changes, submit fixes, and follow project conventions.
 
-## Reporting Bugs ##
+For architecture details, module-specific guidelines, and AI-agent
+instructions, see [AGENTS.md](AGENTS.md). For final user technical
+documentation, see the `docs/` directory or the rendered [Help
+Center](https://help.penpot.app/).
 
-We are using [GitHub Issues](https://github.com/penpot/penpot/issues)
-for our public bugs. We keep a close eye on them and try to make it
-clear when we have an internal fix in progress. Before filing a new
-task, try to make sure your problem doesn't already exist.
+## Table of Contents
 
-If you found a bug, please report it, as far as possible, with:
+- [Prerequisites](#prerequisites)
+- [Reporting Bugs](#reporting-bugs)
+- [Pull Requests](#pull-requests)
+  - [Workflow](#workflow)
+  - [Title format](#title-format)
+  - [Description](#description)
+  - [Branch naming](#branch-naming)
+  - [Review process](#review-process)
+  - [What we won't accept](#what-we-wont-accept)
+  - [Good first issues](#good-first-issues)
+- [Commit Guidelines](#commit-guidelines)
+  - [Commit types](#commit-types)
+  - [Rules](#rules)
+  - [Examples](#examples)
+- [Formatting and Linting](#formatting-and-linting)
+- [Changelog](#changelog)
+- [Code of Conduct](#code-of-conduct)
+- [Developer's Certificate of Origin (DCO)](#developers-certificate-of-origin-dco)
 
-- a detailed explanation of steps to reproduce the error
-- the browser and browser version used
-- a dev tools console exception stack trace (if available)
+## Prerequisites
 
-If you found a bug which you think is better to discuss in private (for
-example, security bugs), consider first sending an email to
-`support@penpot.app`.
+- **Language**: Penpot is written primarily in Clojure (backend), ClojureScript
+  (frontend/exporter), and Rust (render-wasm). Familiarity with the Clojure
+  ecosystem is expected for most contributions.
+- **Issue tracker**: We use [GitHub Issues](https://github.com/penpot/penpot/issues)
+  for public bugs and [Taiga](https://tree.taiga.io/project/penpot/) for
+  internal project management. Changelog entries reference both.
 
-**We don't have a formal bug bounty program for security reports; this
-is an open source application, and your contribution will be recognized
-in the changelog.**
+## Reporting Bugs
 
+Report bugs via [GitHub Issues](https://github.com/penpot/penpot/issues).
+Before filing, search existing issues to avoid duplicates.
 
-## Pull Requests ##
+Include the following when possible:
 
-If you want to propose a change or bug fix via a pull request (PR),
-you should first carefully read the section **Developer's Certificate of
-Origin**. You must also format your code and commits according to the
-instructions below.
+1. Steps to reproduce the error.
+2. Browser and browser version used.
+3. DevTools console exception stack trace (if available).
 
-If you intend to fix a bug, it's fine to submit a pull request right
-away, but we still recommend filing an issue detailing what you're
-fixing. This is helpful in case we don't accept that specific fix but
-want to keep track of the issue.
+For security bugs or issues better discussed in private, email
+`support@penpot.app` or report them on [Github Security
+Advisories](https://github.com/penpot/penpot/security/advisories)
 
-If you want to implement or start working on a new feature, please
-open a **question*- / **discussion*- issue for it. No PR
-will be accepted without a prior discussion about the changes,
-whether it is a new feature, an already planned one, or a quick win.
+> **Note:** We do not have a formal bug bounty program. Security
+> contributions are recognized in the changelog.
 
-If it is your first PR, you can learn how to proceed from
-[this free video
-series](https://egghead.io/courses/how-to-contribute-to-an-open-source-project-on-github)
+## Pull Requests
 
-We use the `easy fix` tag to indicate issues that are appropriate for beginners.
+### Workflow
 
-## Commit Guidelines ##
+1. **Read the DCO** — see [Developer's Certificate of Origin](#developers-certificate-of-origin-dco)
+   below. All code patches must include a `Signed-off-by` line.
+2. **Discuss before building** — open a [GitHub
+   Issue](https://github.com/penpot/penpot/issues) or start a [GitHub
+   Discussion](https://github.com/penpot/penpot/discussions) before starting
+   work on a new feature or significant change. For planned features on the
+   roadmap, reference the corresponding Taiga story. No PR will be accepted
+   without prior discussion, whether it is a new feature, a planned one, or a
+   quick win.
+3. **Bug fixes** — you may submit a PR directly, but we still recommend
+   filing an issue first so we can track it independently of your fix.
+4. **Format and lint** — run the checks described in
+   [Formatting and Linting](#formatting-and-linting) before submitting.
 
-We have very precise rules on how our git commit messages must be formatted.
+### Title format
 
-The commit message format is:
+Pull request titles **must** follow the same convention as commit subjects:
 
 ```
-<type> <subject>
+:emoji: <subject>
+```
+
+- Use the **imperative mood** (e.g. "Fix", not "Fixed").
+- Capitalize the first letter of the subject.
+- Do not end the subject with a period.
+- Keep the subject to **70 characters** or fewer.
+- Use one of the [commit type emojis](#commit-types) listed below.
+
+When a PR contains multiple unrelated commits, choose the emoji that
+best represents the dominant change.
+
+**Examples:**
+
+```
+:bug: Fix unexpected error on launching modal
+:sparkles: Enable new modal for profile
+:zap: Improve performance of dashboard navigation
+```
+
+> **Note:** When a PR is squash-merged, the PR title becomes the
+> commit message on the main branch. Getting the title right matters.
+
+### Description
+
+Every pull request should include a description that helps reviewers
+understand the change quickly:
+
+1. **What and why** — describe the change and its motivation.
+2. **Link related issues** — use `Closes #1234` or reference a Taiga
+   story (e.g. `Taiga #5678`).
+3. **Screenshots or recordings** — required for any UI-visible change.
+4. **Testing notes** — how did you verify the change? Any edge cases?
+5. **Breaking changes** — call out anything that affects existing users
+   or requires migration steps.
+
+### Branch naming
+
+Use a descriptive branch name that reflects the type and scope of the
+change:
+
+```
+<type>/<short-description>
+```
+
+Types: `fix`, `feat`, `refactor`, `docs`, `chore`, `perf`.
+
+Optionally include the issue number:
+
+```
+fix/9122-email-blacklisting
+feat/export-webp
+refactor/layout-sizing
+```
+
+### Review process
+
+- Maintainers review PRs when time permits. Please be patient.
+- Address review feedback by **pushing new commits** — do not
+  force-push during review, as it breaks comment threads.
+- PRs require at least **one approval** before merge.
+- We use **squash-merge** by default. The PR title becomes the final
+  commit message, so follow the [title format](#title-format) above.
+
+### What we won't accept
+
+To save time on both sides, please avoid submitting PRs that:
+
+- Introduce new dependencies without prior discussion.
+- Change the build system or CI configuration without maintainer
+  approval.
+- Mix unrelated changes in a single PR — keep PRs focused on one
+  concern.
+- Skip the [discussion step](#workflow) for non-bug-fix changes.
+
+### Good first issues
+
+We use the `easy fix` label to mark issues appropriate for newcomers.
+
+## Commit Guidelines
+
+Commit messages must follow this format:
+
+```
+:emoji: <subject>
 
 [body]
 
 [footer]
 ```
 
-Where type is:
+### Commit types
 
-- :bug: `:bug:` a commit that fixes a bug
-- :sparkles: `:sparkles:` a commit that adds an improvement
-- :tada: `:tada:` a commit with a new feature
-- :recycle: `:recycle:` a commit that introduces a refactor
-- :lipstick: `:lipstick:` a commit with cosmetic changes
-- :ambulance: `:ambulance:` a commit that fixes a critical bug
-- :books: `:books:` a commit that improves or adds documentation
-- :construction: `:construction:` a WIP commit
-- :boom: `:boom:` a commit with breaking changes
-- :wrench: `:wrench:` a commit for config updates
-- :zap: `:zap:` a commit with performance improvements
-- :whale: `:whale:` a commit for Docker-related stuff
-- :paperclip: `:paperclip:` a commit with other non-relevant changes
-- :arrow_up: `:arrow_up:` a commit with dependency updates
-- :arrow_down: `:arrow_down:` a commit with dependency downgrades
-- :fire: `:fire:` a commit that removes files or code
-- :globe_with_meridians: `:globe_with_meridians:` a commit that adds or updates
-  translations
+| Emoji | Description |
+|-------|-------------|
+| :bug: | Bug fix |
+| :sparkles: | Improvement or enhancement |
+| :tada: | New feature |
+| :recycle: | Refactor |
+| :lipstick: | Cosmetic changes |
+| :ambulance: | Critical bug fix |
+| :books: | Documentation |
+| :construction: | Work in progress |
+| :boom: | Breaking change |
+| :wrench: | Configuration update |
+| :zap: | Performance improvement |
+| :whale: | Docker-related change |
+| :paperclip: | Other non-relevant changes |
+| :arrow_up: | Dependency update |
+| :arrow_down: | Dependency downgrade |
+| :fire: | Removal of code or files |
+| :globe_with_meridians: | Add or update translations |
+| :rocket: | Epic or highlight |
 
-More info:
+### Rules
 
- - https://gist.github.com/parmentf/035de27d6ed1dce0b36a
- - https://gist.github.com/rxaviers/7360908
+- Use the **imperative mood** in the subject (e.g. "Fix", not "Fixed")
+- Capitalize the first letter of the subject
+- Add clear and concise description on the body
+- Do not end the subject with a period
+- Keep the subject to **70 characters** or fewer
+- Separate the subject from the body with a **blank line**
 
-Each commit should have:
+### Examples
 
-- A concise subject using the imperative mood.
-- The subject should capitalize the first letter, omit the period
-  at the end, and be no longer than 65 characters.
-- A blank line between the subject line and the body.
-- An entry in the CHANGES.md file if applicable, referencing the
-  GitHub or Taiga issue/user story using these same rules.
+```
+:bug: Fix unexpected error on launching modal
+:sparkles: Enable new modal for profile
+:zap: Improve performance of dashboard navigation
+:ambulance: Fix critical bug on user registration process
+:tada: Add new approach for user registration
+```
 
-Examples of good commit messages:
+## Formatting and Linting
 
-- `:bug: Fix unexpected error on launching modal`
-- `:bug: Set proper error message on generic error`
-- `:sparkles: Enable new modal for profile`
-- `:zap: Improve performance of dashboard navigation`
-- `:wrench: Update default backend configuration`
-- `:books: Add more documentation for authentication process`
-- `:ambulance: Fix critical bug on user registration process`
-- `:tada: Add new approach for user registration`
-
-## Formatting and Linting ##
-
-You will want to make sure your code is formatted and linted before submitting
-a PR. We use [cljfmt](https://github.com/weavejester/cljfmt) and
-[clj-kondo](https://github.com/clj-kondo/clj-kondo) for this. After installing
-them on your system, you can run them with:
+We use [cljfmt](https://github.com/weavejester/cljfmt) for formatting and
+[clj-kondo](https://github.com/clj-kondo/clj-kondo) for linting.
 
 ```bash
-# Check formatting
+# Check formatting (does not modify files)
+./scripts/check-fmt
+
+# Fix formatting (modifies files in place)
 ./scripts/fmt
 
 # Lint
 ./scripts/lint
 ```
 
-Ideally, you should run these commands as git pre-commit hooks. A convenient way
-of defining them is to use [Husky](https://typicode.github.io/husky/#/).
+Ideally, run these as git pre-commit hooks.
+[Husky](https://typicode.github.io/husky/#/) is a convenient option for
+setting this up.
 
-## Code of Conduct ##
+## Changelog
 
-As contributors and maintainers of this project, we pledge to respect
-all people who contribute through reporting issues, posting feature
-requests, updating documentation, submitting pull requests or patches,
-and other activities.
+When your change is user-facing or otherwise notable, add an entry to
+[CHANGES.md](CHANGES.md) following the same commit-type conventions. Reference
+the relevant GitHub issue or Taiga user story.
 
-We are committed to making participation in this project a
-harassment-free experience for everyone, regardless of level of
-experience, gender, gender identity and expression, sexual
-orientation, disability, personal appearance, body size, race,
-ethnicity, age, or religion.
+## Code of Conduct
 
-Examples of unacceptable behavior by participants include the use of
-sexual language or imagery, derogatory comments or personal attacks,
-trolling, public or private harassment, insults, or other
-unprofessional conduct.
+This project follows the [Contributor Covenant](https://www.contributor-covenant.org/).
+The full Code of Conduct is available at
+[help.penpot.app/contributing-guide/coc](https://help.penpot.app/contributing-guide/coc/)
+and in the repository's [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md).
 
-Project maintainers have the right and responsibility to remove, edit,
-or reject comments, commits, code, wiki edits, issues, and other
-contributions that are not aligned with this Code of Conduct. Project
-maintainers who do not follow the Code of Conduct may be removed from
-the project team.
-
-This Code of Conduct applies both within project spaces and in public
-spaces when an individual is representing the project or its
-community.
-
-Instances of abusive, harassing, or otherwise unacceptable behavior
-may be reported by opening an issue or contacting one or more of the
-project maintainers.
-
-This Code of Conduct is adapted from the Contributor Covenant, version
-1.1.0, available from [http://contributor-covenant.org/version/1/1/0/](http://contributor-covenant.org/version/1/1/0/)
+To report unacceptable behavior, open an issue or contact a project maintainer
+directly.
 
 ## Developer's Certificate of Origin (DCO)
 
 By submitting code you agree to and can certify the following:
 
-    Developer's Certificate of Origin 1.1
+> **Developer's Certificate of Origin 1.1**
+>
+> By making a contribution to this project, I certify that:
+>
+> (a) The contribution was created in whole or in part by me and I have the
+>     right to submit it under the open source license indicated in the file; or
+>
+> (b) The contribution is based upon previous work that, to the best of my
+>     knowledge, is covered under an appropriate open source license and I have
+>     the right under that license to submit that work with modifications,
+>     whether created in whole or in part by me, under the same open source
+>     license (unless I am permitted to submit under a different license), as
+>     indicated in the file; or
+>
+> (c) The contribution was provided directly to me by some other person who
+>     certified (a), (b) or (c) and I have not modified it.
+>
+> (d) I understand and agree that this project and the contribution are public
+>     and that a record of the contribution (including all personal information
+>     I submit with it, including my sign-off) is maintained indefinitely and
+>     may be redistributed consistent with this project or the open source
+>     license(s) involved.
 
-    By making a contribution to this project, I certify that:
+### Signed-off-by
 
-    (a) The contribution was created in whole or in part by me and I
-        have the right to submit it under the open source license
-        indicated in the file; or
-
-    (b) The contribution is based upon previous work that, to the best
-        of my knowledge, is covered under an appropriate open source
-        license and I have the right under that license to submit that
-        work with modifications, whether created in whole or in part
-        by me, under the same open source license (unless I am
-        permitted to submit under a different license), as indicated
-        in the file; or
-
-    (c) The contribution was provided directly to me by some other
-        person who certified (a), (b) or (c) and I have not modified
-        it.
-
-    (d) I understand and agree that this project and the contribution
-        are public and that a record of the contribution (including all
-        personal information I submit with it, including my sign-off) is
-        maintained indefinitely and may be redistributed consistent with
-        this project or the open source license(s) involved.
-
-Then, all your code patches (**documentation is excluded**) should
-contain a sign-off at the end of the patch/commit description body. It
-can be automatically added by adding the `-s` parameter to `git commit`.
-
-This is an example of what the line should look like:
+All code patches (**documentation is excluded**) must contain a sign-off line
+at the end of the commit body. Add it automatically with `git commit -s`.
 
 ```
-Signed-off-by: Andrey Antukh <niwi@niwi.nz>
+Signed-off-by: Your Real Name <your.email@example.com>
 ```
 
-Please, use your real name (sorry, no pseudonyms or anonymous
-contributions are allowed).
+- Use your **real name** — pseudonyms and anonymous contributions are not
+  allowed.
+- The `Signed-off-by` line is **mandatory** and must match the commit author.

backend/AGENTS.md

@@ -1,87 +1,262 @@
-# backend – Agent Instructions
+# Penpot Backend – Agent Instructions
 
-Clojure service running on the JVM. Uses Integrant for dependency injection, PostgreSQL for storage, and Redis for messaging/caching.
+Clojure backend (RPC) service running on the JVM.
 
-## Commands
+Uses Integrant for dependency injection, PostgreSQL for storage, and
+Redis for messaging/caching.
 
-```bash
-# REPL (primary dev workflow)
-./scripts/repl              # Start nREPL + load dev/user.clj utilities
+## General Guidelines
 
-# Tests (Kaocha)
-clojure -M:dev:test                                         # Full suite
-clojure -M:dev:test --focus backend-tests.my-ns-test        # Single namespace
+To ensure consistency across the Penpot JVM stack, all contributions must adhere
+to these criteria.
 
-# Lint / Format
-pnpm run lint:clj
-pnpm run fmt:clj
-```
+IMPORTANT: all CLI commands should be executed under backend/
+subdirectory for make them work correctly.
 
-Test namespaces match `.*-test$` under `test/`. Config is in `tests.edn`.
+### 1. Testing & Validation
 
-## Integrant System
+* **Coverage:** If code is added or modified in `src/`, corresponding
+  tests in `test/backend_tests/` must be added or updated.
 
-`src/app/main.clj` declares the system map. Each key is a component;
-values are config maps with `::ig/ref` for dependencies. Components
-implement `ig/init-key` / `ig/halt-key!`.
+* **Execution:**
+  * **Isolated:** Run `clojure -M:dev:test --focus backend-tests.my-ns-test` for the specific test namespace.
+  * **Regression:** Run `clojure -M:dev:test` to ensure the suite passes without regressions in related functional areas.
 
-From the REPL (`dev/user.clj` is auto-loaded):
-```clojure
-(start!)   ; boot the system
-(stop!)    ; halt the system
-(restart!) ; stop + reload namespaces + start
-```
+### 2. Code Quality & Formatting
 
-## RPC Commands
+* **Linting:** All code must pass linter checks (run `pnpm run lint:clj` or `pnpm run lint` on the repository root)
+* **Formatting:** All the code must pass the formatting check (run `pnpm run
+  check-fmt`). Use `pnpm run fmt` to fix formatting issues. Avoid "dirty"
+  diffs caused by unrelated whitespace changes.
+* **Type Hinting:** Use explicit JVM type hints (e.g., `^String`, `^long`) in
+  performance-critical paths to avoid reflection overhead.
 
-All API calls: `POST /api/rpc/command/<cmd-name>`.
+## Code Conventions
+
+### Namespace Overview
+
+The source is located under `src` directory and this is a general overview of
+namespaces structure:
+
+- `app.rpc.commands.*` – RPC command implementations (`auth`, `files`, `teams`, etc.)
+- `app.http.*` – HTTP routes and middleware
+- `app.db.*` – Database layer
+- `app.tasks.*` – Background job tasks
+- `app.main` – Integrant system setup and entrypoint
+- `app.loggers` – Internal loggers (auditlog, mattermost, etc.) (not to be confused with `app.common.logging`)
+
+### RPC
+
+The RPC methods are implemented using a multimethod-like structure via the
+`app.util.services` namespace. The main RPC methods are collected under
+`app.rpc.commands` namespace and exposed under `/api/rpc/command/<cmd-name>`.
+
+The RPC method accepts POST and GET requests indistinctly and uses the `Accept`
+header to negotiate the response encoding (which can be Transit — the default —
+or plain JSON). It also accepts Transit (default) or JSON as input, which should
+be indicated using the `Content-Type` header.
+
+The main convention is: use `get-` prefix on RPC name when we want READ
+operation.
+
+Example of RPC method definition:
 
 ```clojure
 (sv/defmethod ::my-command
-  {::rpc/auth true             ;; requires authentication (default)
+  {::rpc/auth true            ;; requires auth
    ::doc/added "1.18"
-   ::sm/params [:map ...]      ;; malli input schema
-   ::sm/result [:map ...]}     ;; malli output schema
+   ::sm/params [:map ...]     ;; malli input schema
+   ::sm/result [:map ...]}    ;; malli output schema
   [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id] :as params}]
-  ;; return a plain map; throw via ex/raise for errors
+  ;; return a plain map or throw
   {:id (uuid/next)})
 ```
 
-Add new commands in `src/app/rpc/commands/`.
+Look under `src/app/rpc/commands/*.clj` to see more examples.
 
-## Database
+### Tests
+
+Test namespaces match `.*-test$` under `test/`. Config is in `tests.edn`.
+
+
+### Integrant System
+
+The `src/app/main.clj` declares the system map. Each key is a component; values
+are config maps with `::ig/ref` for dependencies. Components implement
+`ig/init-key` / `ig/halt-key!`.
+
+
+### Connecting to the Database
+
+Two PostgreSQL databases are used in this environment:
+
+| Database      | Purpose            | Connection string                                  |
+|---------------|--------------------|----------------------------------------------------|
+| `penpot`      | Development / app  | `postgresql://penpot:penpot@postgres/penpot`       |
+| `penpot_test` | Test suite         | `postgresql://penpot:penpot@postgres/penpot_test`  |
+
+**Interactive psql session:**
+
+```bash
+# development DB
+psql "postgresql://penpot:penpot@postgres/penpot"
+
+# test DB
+psql "postgresql://penpot:penpot@postgres/penpot_test"
+```
+
+**One-shot query (non-interactive):**
+
+```bash
+psql "postgresql://penpot:penpot@postgres/penpot" -c "SELECT id, name FROM team LIMIT 5;"
+```
+
+**Useful psql meta-commands:**
+
+```
+\dt              -- list all tables
+\d <table>       -- describe a table (columns, types, constraints)
+\di              -- list indexes
+\q               -- quit
+```
+
+> **Migrations table:** Applied migrations are tracked in the `migrations` table
+> with columns `module`, `step`, and `created_at`. When renaming a migration
+> logical name, update this table in both databases to match the new name;
+> otherwise the runner will attempt to re-apply the migration on next startup.
+
+```bash
+# Example: fix a renamed migration entry in the test DB
+psql "postgresql://penpot:penpot@postgres/penpot_test" \
+  -c "UPDATE migrations SET step = 'new-name' WHERE step = 'old-name';"
+```
+
+### Database Access (Clojure)
 
 `app.db` wraps next.jdbc. Queries use a SQL builder that auto-converts kebab-case ↔ snake_case.
 
 ```clojure
 ;; Query helpers
-(db/get pool :table {:id id})                    ; fetch one row (throws if missing)
-(db/get* pool :table {:id id})                   ; fetch one row (returns nil)
-(db/query pool :table {:team-id team-id})        ; fetch multiple rows
-(db/insert! pool :table {:name "x" :team-id id}) ; insert
-(db/update! pool :table {:name "y"} {:id id})    ; update
-(db/delete! pool :table {:id id})                ; delete
+(db/get cfg-or-pool :table {:id id})                    ; fetch one row (throws if missing)
+(db/get* cfg-or-pool :table {:id id})                   ; fetch one row (returns nil)
+(db/query cfg-or-pool :table {:team-id team-id})        ; fetch multiple rows
+(db/insert! cfg-or-pool :table {:name "x" :team-id id}) ; insert
+(db/update! cfg-or-pool :table {:name "y"} {:id id})    ; update
+(db/delete! cfg-or-pool :table {:id id})                ; delete
+
+;; Run multiple statements/queries on single connection
+(db/run! cfg (fn [{:keys [::db/conn]}]
+               (db/insert! conn :table row1)
+               (db/insert! conn :table row2))
+
+
 ;; Transactions
-(db/tx-run cfg (fn [{:keys [::db/conn]}]
-                 (db/insert! conn :table row)))
+(db/tx-run! cfg (fn [{:keys [::db/conn]}]
+                  (db/insert! conn :table row)))
 ```
 
-Almost all methods on `app.db` namespace accepts `pool`, `conn` or
+Almost all methods in the `app.db` namespace accept `pool`, `conn`, or
 `cfg` as params.
 
 Migrations live in `src/app/migrations/` as numbered SQL files. They run automatically on startup.
 
-## Error Handling
+
+### Error Handling
+
+The exception helpers are defined on Common module, and are available under
+`app.common.exceptions` namespace.
+
+Example of raising an exception:
 
 ```clojure
 (ex/raise :type :not-found
           :code :object-not-found
           :hint "File does not exist"
-          :context {:id file-id})
+          :file-id id)
 ```
 
 Common types: `:not-found`, `:validation`, `:authorization`, `:conflict`, `:internal`.
 
-## Configuration
 
-`src/app/config.clj` reads `PENPOT_*` environment variables, validated with Malli. Access anywhere via `(cf/get :smtp-host)`. Feature flags: `(cf/flags :enable-smtp)`.
+### Performance Macros (`app.common.data.macros`)
+
+Always prefer these macros over their `clojure.core` equivalents — they provide
+optimized implementations:
+
+```clojure
+(dm/select-keys m [:a :b])     ;; faster than core/select-keys
+(dm/get-in obj [:a :b :c])     ;; faster than core/get-in
+(dm/str "a" "b" "c")           ;; string concatenation
+```
+
+### Configuration
+
+`src/app/config.clj` reads `PENPOT_*` environment variables, validated with
+Malli. Access anywhere via `(cf/get :smtp-host)`. Feature flags: `(cf/flags
+:enable-smtp)`.
+
+
+### Background Tasks
+
+Background tasks live in `src/app/tasks/`. Each task is an Integrant component
+that exposes a `::handler` key and follows this three-method pattern:
+
+```clojure
+(defmethod ig/assert-key ::handler   ;; validate config at startup
+  [_ params]
+  (assert (db/pool? (::db/pool params)) "expected a valid database pool"))
+
+(defmethod ig/expand-key ::handler   ;; inject defaults before init
+  [k v]
+  {k (assoc v ::my-option default-value)})
+
+(defmethod ig/init-key ::handler     ;; return the task fn
+  [_ cfg]
+  (fn [_task]                        ;; receives the task row from the worker
+    (db/tx-run! cfg (fn [{:keys [::db/conn]}]
+                      ;; … do work …
+                      ))))
+```
+
+**Wiring a new task** requires two changes in `src/app/main.clj`:
+
+1. **Handler config** – add an entry in `system-config` with the dependencies:
+
+```clojure
+:app.tasks.my-task/handler
+{::db/pool (ig/ref ::db/pool)}
+```
+
+2. **Registry + cron** – register the handler name and schedule it:
+
+```clojure
+;; in ::wrk/registry ::wrk/tasks map:
+:my-task (ig/ref :app.tasks.my-task/handler)
+
+;; in worker-config ::wrk/cron ::wrk/entries vector:
+{:cron #penpot/cron "0 0 0 * * ?"   ;; daily at midnight
+ :task :my-task}
+```
+
+**Useful cron patterns** (Quartz format — six fields: s m h dom mon dow):
+
+| Expression                   | Meaning            |
+|------------------------------|--------------------|
+| `"0 0 0 * * ?"`              | Daily at midnight  |
+| `"0 0 */6 * * ?"`            | Every 6 hours      |
+| `"0 */5 * * * ?"`            | Every 5 minutes    |
+
+**Time helpers** (`app.common.time`):
+
+```clojure
+(ct/now)                          ;; current instant
+(ct/duration {:hours 1})          ;; java.time.Duration
+(ct/minus (ct/now) some-duration) ;; subtract duration from instant
+```
+
+`db/interval` converts a `Duration` (or millis / string) to a PostgreSQL
+interval object suitable for use in SQL queries:
+
+```clojure
+(db/interval (ct/duration {:hours 1}))  ;; → PGInterval "3600.0 seconds"
+```

backend/scripts/_env

@@ -2,6 +2,7 @@
 
 export PENPOT_NITRATE_SHARED_KEY=super-secret-nitrate-api-key
 export PENPOT_EXPORTER_SHARED_KEY=super-secret-exporter-api-key
+export PENPOT_NEXUS_SHARED_KEY=super-secret-nexus-api-key
 export PENPOT_SECRET_KEY=super-secret-devenv-key
 
 # DEPRECATED: only used for subscriptions
@@ -12,7 +13,7 @@ export PENPOT_PUBLIC_URI=https://localhost:3449
 
 export PENPOT_FLAGS="\
        $PENPOT_FLAGS \
-       enable-login-with-password
+       enable-login-with-password \
        disable-login-with-ldap \
        disable-login-with-oidc \
        disable-login-with-google \
@@ -44,6 +45,10 @@ export PENPOT_FLAGS="\
        enable-redis-cache \
        enable-subscriptions";
 
+# Uncomment for nexus integration testing
+# export PENPOT_FLAGS="$PENPOT_FLAGS enable-audit-log-archive";
+# export PENPOT_AUDIT_LOG_ARCHIVE_URI="http://localhost:6070/api/audit";
+
 # Default deletion delay for devenv
 export PENPOT_DELETION_DELAY="24h"
 

backend/src/app/auth/oidc.clj

@@ -43,7 +43,7 @@
 (defn- discover-oidc-config
   [cfg {:keys [base-uri] :as provider}]
   (let [uri (u/join base-uri ".well-known/openid-configuration")
-        rsp (http/req! cfg {:method :get :uri (dm/str uri)})]
+        rsp (http/req cfg {:method :get :uri (dm/str uri)})]
 
     (if (= 200 (:status rsp))
       (let [data       (-> rsp :body json/decode)
@@ -105,7 +105,7 @@
 
 (defn- fetch-oidc-jwks
   [cfg jwks-uri]
-  (let [{:keys [status body]} (http/req! cfg {:method :get :uri jwks-uri})]
+  (let [{:keys [status body]} (http/req cfg {:method :get :uri jwks-uri})]
     (if (= 200 status)
       (-> body json/decode :keys process-oidc-jwks)
       (ex/raise :type ::internal
@@ -235,7 +235,7 @@
                     :timeout 6000
                     :method :get}
 
-            {:keys [status body]} (http/req! cfg params)]
+            {:keys [status body]} (http/req cfg params)]
 
         (when-not (int-in-range? status 200 300)
           (ex/raise :type :internal
@@ -423,7 +423,7 @@
 
 (defn- qualify-prop-key
   [provider k]
-  (keyword (:type provider) (name k)))
+  (keyword (:type provider) (-> k name str/kebab)))
 
 (defn- qualify-props
   [provider props]
@@ -452,7 +452,7 @@
            :grant-type (:grant_type params)
            :redirect-uri (:redirect_uri params))
 
-    (let [{:keys [status body]} (http/req! cfg req)]
+    (let [{:keys [status body]} (http/req cfg req)]
       (if (= status 200)
         (let [data (json/decode body)
               data {:token/access (get data :access_token)
@@ -507,7 +507,7 @@
                   :headers {"Authorization" (str (:token/type tdata) " " (:token/access tdata))}
                   :timeout 6000
                   :method :get}
-        response (http/req! cfg params)]
+        response (http/req cfg params)]
 
     (l/trc :hint "user info response"
            :status (:status response)
@@ -804,12 +804,12 @@
                 props   (audit/profile->props profile)
                 context (d/without-nils {:external-session-id (:external-session-id info)})]
 
-            (audit/submit! cfg {::audit/type "action"
-                                ::audit/name "login-with-oidc"
-                                ::audit/profile-id (:id profile)
-                                ::audit/ip-addr (inet/parse-request request)
-                                ::audit/props props
-                                ::audit/context context})
+            (audit/submit cfg {:type "action"
+                               :name "login-with-oidc"
+                               :profile-id (:id profile)
+                               :ip-addr (inet/parse-request request)
+                               :props props
+                               :context context})
 
             (->> (redirect-to-verify-token token)
                  (sxf request)))))

backend/src/app/binfile/v1.clj

@@ -40,8 +40,8 @@
    [promesa.util :as pu]
    [yetti.adapter :as yt])
   (:import
-   com.github.luben.zstd.ZstdIOException
    com.github.luben.zstd.ZstdInputStream
+   com.github.luben.zstd.ZstdIOException
    com.github.luben.zstd.ZstdOutputStream
    java.io.DataInputStream
    java.io.DataOutputStream

backend/src/app/binfile/v3.clj

@@ -281,7 +281,7 @@
 
         thumbnails   (bfc/get-file-object-thumbnails cfg file-id)]
 
-    (events/tap :progress {:section :file :id file-id})
+    (events/tap :progress {:section :file :id file-id :name (:name file)})
 
     (vswap! bfc/*state* update :files assoc file-id
             {:id file-id
@@ -301,6 +301,7 @@
       (write-entry! output path file))
 
     (doseq [[index page-id] (d/enumerate pages)]
+
       (let [path    (str "files/" file-id "/pages/" page-id ".json")
             page    (get pages-index page-id)
             objects (:objects page)
@@ -311,6 +312,8 @@
 
         (write-entry! output path page)
 
+        (events/tap :progress {:section :page :id page-id :name (:name page) :file-id file-id})
+
         (doseq [[shape-id shape] objects]
           (let [path  (str "files/" file-id "/pages/" page-id "/" shape-id ".json")
                 shape (assoc shape :page-id page-id)
@@ -323,6 +326,8 @@
     (doseq [{:keys [id] :as media} media]
       (let [path  (str "files/" file-id "/media/" id ".json")
             media (encode-media media)]
+
+        (events/tap :progress {:section :media :id id  :file-id file-id})
         (write-entry! output path media)))
 
     (doseq [thumbnail thumbnails]
@@ -332,11 +337,13 @@
             data (-> data
                      (assoc :media-id (:media-id thumbnail))
                      (encode-file-thumbnail))]
+        (events/tap :progress {:section :thumbnails :id (:object-id thumbnail) :file-id file-id})
         (write-entry! output path data)))
 
     (doseq [[id component] components]
       (let [path      (str "files/" file-id "/components/" id ".json")
             component (encode-component component)]
+        (events/tap :progress {:section :component :id id :file-id file-id})
         (write-entry! output path component)))
 
     (doseq [[id color] colors]
@@ -347,17 +354,20 @@
                     (and (contains? color :path)
                          (str/empty? (:path color)))
                     (dissoc :path))]
+        (events/tap :progress {:section :color :id id :file-id file-id})
         (write-entry! output path color)))
 
     (doseq [[id object] typographies]
       (let [path       (str "files/" file-id "/typographies/" id ".json")
             typography (encode-typography object)]
+        (events/tap :progress {:section :typography :id id :file-id file-id})
         (write-entry! output path typography)))
 
     (when (and tokens-lib
                (not (ctob/empty-lib? tokens-lib)))
       (let [path           (str "files/" file-id "/tokens.json")
             encoded-tokens (encode-tokens-lib tokens-lib)]
+        (events/tap :progress {:section :tokens-lib :file-id file-id})
         (write-entry! output path encoded-tokens)))))
 
 (defn- export-files
@@ -600,6 +610,7 @@
                  (let [object (->> (read-entry input entry)
                                    (decode-color)
                                    (validate-color))]
+                   (events/tap :progress {:section :color :id id :file-id file-id})
                    (if (= id (:id object))
                      (assoc result id object)
                      result)))
@@ -631,6 +642,7 @@
                                      (clean-component-pre-decode)
                                      (decode-component)
                                      (clean-component-post-decode))]
+                     (events/tap :progress {:section :component :id id :file-id file-id})
                      (if (= id (:id object))
                        (assoc result id object)
                        result)))
@@ -644,6 +656,7 @@
                  (let [object (->> (read-entry input entry)
                                    (decode-typography)
                                    (validate-typography))]
+                   (events/tap :progress {:section :typography :id id :file-id file-id})
                    (if (= id (:id object))
                      (assoc result id object)
                      result)))
@@ -653,6 +666,7 @@
 (defn- read-file-tokens-lib
   [{:keys [::bfc/input ::entries]} file-id]
   (when-let [entry (d/seek (match-tokens-lib-entry-fn file-id) entries)]
+    (events/tap :progress {:section :tokens-lib :file-id file-id})
     (->> (read-plain-entry input entry)
          (decode-tokens-lib)
          (validate-tokens-lib))))
@@ -678,6 +692,7 @@
                (let [page (->> (read-entry input entry)
                                (decode-page))
                      page (dissoc page :options)]
+                 (events/tap :progress {:section :page :id id :file-id file-id})
                  (when (= id (:id page))
                    (let [objects (read-file-shapes cfg file-id id)]
                      (assoc page :objects objects))))))
@@ -693,6 +708,7 @@
                  (let [object (->> (read-entry input entry)
                                    (decode-file-thumbnail)
                                    (validate-file-thumbnail))]
+
                    (if (and (= frame-id (:frame-id object))
                             (= page-id (:page-id object))
                             (= tag (:tag object)))
@@ -733,8 +749,6 @@
 
     (vswap! bfc/*state* update :index bfc/update-index media :id)
 
-    (events/tap :progress {:section :media :file-id file-id})
-
     (doseq [item media]
       (let [params (-> item
                        (update :id bfc/lookup-index)
@@ -742,6 +756,8 @@
                        (d/update-when :media-id bfc/lookup-index)
                        (d/update-when :thumbnail-id bfc/lookup-index))]
 
+        (events/tap :progress {:section :media :id (:id params) :file-id file-id})
+
         (l/dbg :hint "inserting media object"
                :file-id (str file-id')
                :id (str (:id params))
@@ -753,8 +769,6 @@
         (db/insert! conn :file-media-object params
                     ::db/on-conflict-do-nothing? (::bfc/overwrite cfg))))
 
-    (events/tap :progress {:section :thumbnails :file-id file-id})
-
     (doseq [item thumbnails]
       (let [media-id  (bfc/lookup-index (:media-id item))
             object-id (-> (assoc item :file-id file-id')
@@ -769,6 +783,8 @@
                :media-id (str media-id)
                ::l/sync? true)
 
+        (events/tap :progress {:section :thumbnail :file-id file-id :object-id object-id})
+
         (db/insert! conn :file-tagged-object-thumbnail params
                     ::db/on-conflict-do-nothing? true)))
 

backend/src/app/config.clj

@@ -72,6 +72,7 @@
    :telemetry-uri "https://telemetry.penpot.app/"
 
    :media-max-file-size (* 1024 1024 30) ; 30MiB
+   :font-max-file-size  (* 1024 1024 30) ; 30MiB
 
    :ldap-user-query "(|(uid=:username)(mail=:username))"
    :ldap-attrs-username "uid"
@@ -82,7 +83,14 @@
    :initial-project-skey "initial-project"
 
    ;; time to avoid email sending after profile modification
-   :email-verify-threshold "15m"})
+   :email-verify-threshold "15m"
+
+   :quotes-upload-sessions-per-profile 5
+   :quotes-upload-chunks-per-session 20
+
+   ;; SSRF protection
+   :ssrf-allowed-hosts #{}
+   :ssrf-extra-blocked-cidrs #{}})
 
 (def schema:config
   (do #_sm/optional-keys
@@ -103,6 +111,7 @@
 
     [:exporter-shared-key {:optional true} :string]
     [:nitrate-shared-key {:optional true} :string]
+    [:nexus-shared-key {:optional true} :string]
     [:management-api-key {:optional true} :string]
 
     [:telemetry-uri {:optional true} :string]
@@ -112,6 +121,7 @@
     [:auto-file-snapshot-timeout {:optional true} ::ct/duration]
 
     [:media-max-file-size {:optional true} ::sm/int]
+    [:font-max-file-size  {:optional true} ::sm/int]
     [:deletion-delay {:optional true} ::ct/duration]
     [:file-clean-delay {:optional true} ::ct/duration]
     [:telemetry-enabled {:optional true} ::sm/boolean]
@@ -153,6 +163,8 @@
     [:quotes-snapshots-per-team {:optional true} ::sm/int]
     [:quotes-team-access-requests-per-team {:optional true} ::sm/int]
     [:quotes-team-access-requests-per-requester {:optional true} ::sm/int]
+    [:quotes-upload-sessions-per-profile {:optional true} ::sm/int]
+    [:quotes-upload-chunks-per-session {:optional true} ::sm/int]
 
     [:auth-token-cookie-name {:optional true} :string]
     [:auth-token-cookie-max-age {:optional true} ::ct/duration]
@@ -239,17 +251,26 @@
     [:objects-storage-fs-directory {:optional true} :string]
     [:objects-storage-s3-bucket {:optional true} :string]
     [:objects-storage-s3-region {:optional true} :keyword]
-    [:objects-storage-s3-endpoint {:optional true} ::sm/uri]]))
+    [:objects-storage-s3-endpoint {:optional true} ::sm/uri]
+
+    ;; SSRF protection
+    [:ssrf-allowed-hosts {:optional true} [::sm/set :string]]
+    [:ssrf-extra-blocked-cidrs {:optional true} [::sm/set :string]]]))
 
 (defn- parse-flags
   [config]
   (let [public-uri  (c/get config :public-uri)
         public-uri  (some-> public-uri (u/uri))
-        extra-flags (if (and public-uri
-                             (= (:scheme public-uri) "http")
-                             (not= (:host public-uri) "localhost"))
-                      #{:disable-secure-session-cookies}
-                      #{})]
+        extra-flags (cond-> #{}
+                      ;; When public-uri is http (non-localhost), disable secure cookies
+                      (and public-uri
+                           (= (:scheme public-uri) "http")
+                           (not= (:host public-uri) "localhost"))
+                      (conj :disable-secure-session-cookies)
+
+                      ;; When telemetry-enabled config is true, add :telemetry flag
+                      (true? (c/get config :telemetry-enabled))
+                      (conj :enable-telemetry))]
     (flags/parse flags/default extra-flags (:flags config))))
 
 (defn read-env

backend/src/app/db.clj

@@ -36,11 +36,11 @@
    java.sql.Connection
    java.sql.PreparedStatement
    java.sql.Savepoint
-   org.postgresql.PGConnection
    org.postgresql.geometric.PGpoint
    org.postgresql.jdbc.PgArray
    org.postgresql.largeobject.LargeObject
    org.postgresql.largeobject.LargeObjectManager
+   org.postgresql.PGConnection
    org.postgresql.util.PGInterval
    org.postgresql.util.PGobject))
 

backend/src/app/email.clj

@@ -22,15 +22,34 @@
    [cuerdas.core :as str]
    [integrant.core :as ig])
   (:import
-   jakarta.mail.Message$RecipientType
-   jakarta.mail.Session
-   jakarta.mail.Transport
    jakarta.mail.internet.InternetAddress
    jakarta.mail.internet.MimeBodyPart
    jakarta.mail.internet.MimeMessage
    jakarta.mail.internet.MimeMultipart
+   jakarta.mail.Message$RecipientType
+   jakarta.mail.Session
+   jakarta.mail.Transport
    java.util.Properties))
 
+(defn clean
+  "Clean and normalizes email address string"
+  [email]
+  (let [email (str/lower email)
+        email (if (str/starts-with? email "mailto:")
+                (subs email 7)
+                email)
+        email (if (or (str/starts-with? email "<")
+                      (str/ends-with? email ">"))
+                (str/trim email "<>")
+                email)]
+    email))
+
+(defn get-domain
+  [email]
+  (let [email      (clean email)
+        [_ domain] (str/split email "@" 2)]
+    domain))
+
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 ;; EMAIL IMPL
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

backend/src/app/email/blacklist.clj

@@ -36,10 +36,18 @@
                :cause cause)))))
 
 (defn contains?
-  "Check if email is in the blacklist."
+  "Check if email is in the blacklist. Also matches subdomains: if
+  'somedomain.com' is blacklisted, 'xxx@foo.somedomain.com' will also
+  be rejected."
   [{:keys [::email/blacklist]} email]
-  (let [[_ domain] (str/split email "@" 2)]
-    (c/contains? blacklist (str/lower domain))))
+  (let [[_ domain] (str/split email "@" 2)
+        parts      (str/split (str/lower domain) #"\.")]
+    (loop [parts parts]
+      (if (empty? parts)
+        false
+        (if (c/contains? blacklist (str/join "." parts))
+          true
+          (recur (rest parts)))))))
 
 (defn enabled?
   "Check if the blacklist is enabled"

backend/src/app/http/assets.clj

@@ -12,43 +12,56 @@
    [app.common.time :as ct]
    [app.common.uri :as u]
    [app.db :as db]
+   [app.http.session :as session]
    [app.storage :as sto]
    [integrant.core :as ig]
    [yetti.response :as-alias yres]))
 
-(def ^:private cache-max-age
+(def ^:private default-cache-max-age
   (ct/duration {:hours 24}))
 
-(def ^:private signature-max-age
+(def ^:private default-signature-max-age
   (ct/duration {:hours 24 :minutes 15}))
 
+;; Buckets that are legitimately public and do not require authentication.
+;; These are used by public shared board viewing, profile photos in UI,
+;; and embedded export/binfile flows.
+(def ^:private public-buckets
+  #{"file-media-object"
+    "file-object-thumbnail"
+    "team-font-variant"
+    "file-data-fragment"})
+
 (defn get-id
   [{:keys [path-params]}]
   (or (some-> path-params :id d/parse-uuid)
       (ex/raise :type :not-found
-                :hunt "object not found")))
+                :hint "object not found")))
 
 (defn- get-file-media-object
   [pool id]
   (db/get pool :file-media-object {:id id} {::db/remove-deleted false}))
 
 (defn- serve-object-from-s3
-  [{:keys [::sto/storage] :as cfg} obj]
-  (let [{:keys [host port] :as url} (sto/get-object-url storage obj {:max-age signature-max-age})]
+  [{:keys [::sto/storage ::signature-max-age ::cache-max-age] :as cfg} obj]
+  (let [sig-max-age (or signature-max-age default-signature-max-age)
+        cch-max-age (or cache-max-age default-cache-max-age)
+        {:keys [host port] :as url} (sto/get-object-url storage obj {:max-age sig-max-age})]
     {::yres/status  307
      ::yres/headers {"location" (str url)
                      "x-host"   (cond-> host port (str ":" port))
                      "x-mtype"  (-> obj meta :content-type)
-                     "cache-control" (str "max-age=" (inst-ms cache-max-age))}}))
+                     "cache-control" (str "max-age=" (inst-ms cch-max-age))}}))
 
 (defn- serve-object-from-fs
-  [{:keys [::path]} obj]
-  (let [purl    (u/join (u/uri path)
+  [{:keys [::path ::cache-max-age]} obj]
+  (let [cch-max-age (or cache-max-age default-cache-max-age)
+        purl    (u/join (u/uri path)
                         (sto/object->relative-path obj))
         mdata   (meta obj)
         headers {"x-accel-redirect" (:path purl)
                  "content-type" (:content-type mdata)
-                 "cache-control" (str "max-age=" (inst-ms cache-max-age))}]
+                 "cache-control" (str "max-age=" (inst-ms cch-max-age))}]
     {::yres/status 204
      ::yres/headers headers}))
 
@@ -60,14 +73,28 @@
     (:s3 :assets-s3) (serve-object-from-s3 cfg obj)
     (:fs :assets-fs) (serve-object-from-fs cfg obj)))
 
+(defn- requires-auth?
+  "Check if the storage object requires authentication based on its bucket."
+  [obj]
+  (let [bucket (-> obj meta :bucket)]
+    (not (contains? public-buckets bucket))))
+
 (defn objects-handler
-  "Handler that servers storage objects by id."
+  "Handler that serves storage objects by id.
+   For non-public buckets (e.g. profile), requires an authenticated session."
   [{:keys [::sto/storage] :as cfg} request]
   (let [id  (get-id request)
         obj (sto/get-object storage id)]
-    (if obj
-      (serve-object cfg obj)
-      {::yres/status 404})))
+    (cond
+      (nil? obj)
+      {::yres/status 404}
+
+      (and (requires-auth? obj)
+           (nil? (::session/profile-id request)))
+      {::yres/status 401}
+
+      :else
+      (serve-object cfg obj))))
 
 (defn- generic-handler
   "A generic handler helper/common code for file-media based handlers."
@@ -96,11 +123,12 @@
 (defmethod ig/assert-key ::routes
   [_ params]
   (assert (sto/valid-storage? (::sto/storage params)) "expected valid storage instance")
+  (assert (session/manager? (::session/manager params)) "expected valid session manager")
   (assert (string? (::path params))))
 
 (defmethod ig/init-key ::routes
   [_ cfg]
-  ["/assets"
+  ["/assets" {:middleware [[session/authz cfg]]}
    ["/by-id/:id" {:handler (partial objects-handler cfg)}]
    ["/by-file-media-id/:id" {:handler (partial file-objects-handler cfg)}]
    ["/by-file-media-id/:id/thumbnail" {:handler (partial file-thumbnails-handler cfg)}]])

backend/src/app/http/awsns.clj

@@ -53,7 +53,7 @@
         (let [surl   (get body "SubscribeURL")
               stopic (get body "TopicArn")]
           (l/info :action "subscription received" :topic stopic :url surl)
-          (http/req! cfg {:uri surl :method :post :timeout 10000} {:sync? true}))
+          (http/req cfg {:uri surl :method :post :timeout 10000} {:sync? true}))
 
         (= mtype "Notification")
         (when-let [message (parse-json (get body "Message"))]

backend/src/app/http/client.clj

@@ -5,13 +5,24 @@
 ;; Copyright (c) KALEIDOS INC
 
 (ns app.http.client
-  "Http client abstraction layer."
+  "Http client abstraction layer.
+
+   All outbound requests made through `req` and `req-with-redirects`
+   are validated against the SSRF blocklist by default. Pass
+   `:skip-ssrf-check? true` in the options map only when the target
+   is a well-known, operator-configured endpoint that cannot be
+   influenced by user input (e.g. internal telemetry, error webhooks)."
   (:require
    [app.common.schema :as sm]
+   [app.util.ssrf :as ssrf]
+   [cuerdas.core :as str]
    [integrant.core :as ig]
    [java-http-clj.core :as http])
   (:import
-   java.net.http.HttpClient))
+   java.net.http.HttpClient
+   java.net.URI))
+
+(def default-max-redirects 5)
 
 (defn client?
   [o]
@@ -23,8 +34,8 @@
 
 (defmethod ig/init-key ::client
   [_ _]
-  (http/build-client {:connect-timeout 30000 ;; 10s
-                      :follow-redirects :always}))
+  (http/build-client {:connect-timeout 30000
+                      :follow-redirects :never}))
 
 (defn send!
   ([client req] (send! client req {}))
@@ -44,14 +55,82 @@
     :else
     (throw (UnsupportedOperationException. "invalid arguments"))))
 
-(defn req!
-  "A convencience toplevel function for gradual migration to a new API
-  convention."
+(defn req
+  "Issue a single HTTP request.  SSRF validation is applied to the
+   target URI by default; pass `:skip-ssrf-check? true` in `options`
+   to bypass it for known-safe, operator-configured endpoints."
   ([cfg-or-client request]
-   (let [client  (resolve-client cfg-or-client)
-         request (update request :uri str)]
-     (send! client request {})))
-  ([cfg-or-client request options]
-   (let [client  (resolve-client cfg-or-client)
-         request (update request :uri str)]
-     (send! client request options))))
+   (req cfg-or-client request {}))
+  ([cfg-or-client request {:keys [skip-ssrf-check?] :as options}]
+   (let [request (if skip-ssrf-check?
+                   (update request :uri str)
+                   (update request :uri ssrf/validate-uri))
+         client  (resolve-client cfg-or-client)]
+     (send! client request (dissoc options :skip-ssrf-check?)))))
+
+(defn- resolve-location
+  "Resolve a Location header value against the original request URI.
+   Handles:
+     - Absolute URLs (http:// or https://) — returned as-is.
+     - Protocol-relative URLs (//host/path) — inherit the scheme from base-uri.
+     - Path-absolute and relative URLs — resolved against base-uri via URI.resolve."
+  [^String base-uri ^String location]
+  (cond
+    (or (str/starts-with? location "http://")
+        (str/starts-with? location "https://"))
+    location
+
+    (str/starts-with? location "//")
+    (let [scheme (.getScheme (URI. base-uri))]
+      (str scheme ":" location))
+
+    :else
+    (str (.resolve (URI. base-uri) location))))
+
+(defn- redirect-request
+  "Build the next request for a 3xx redirect.
+   Per RFC 7231 §6.4:
+     - 303 always issues GET (body dropped).
+     - 301/302 with non-GET/HEAD methods: downgrade to GET (body dropped).
+     - 307/308 preserve the original method and body.
+   The Location URI has already been resolved by the caller."
+  [orig-request ^String next-uri status]
+  (let [method (:method orig-request)]
+    (if (or (= status 303)
+            (and (contains? #{301 302} status)
+                 (not (contains? #{:get :head} method))))
+      ;; Downgrade to GET, drop body and content-type
+      (-> orig-request
+          (assoc :uri next-uri :method :get)
+          (dissoc :body)
+          (update :headers dissoc "content-type" "content-length"))
+      ;; Preserve method/body (307, 308, or GET/HEAD 301/302)
+      (assoc orig-request :uri next-uri))))
+
+(defn req-with-redirects
+  "Like `req`, but follows up to `max-redirects` HTTP 3xx redirects.
+   SSRF validation is applied before every hop (initial request and
+   each redirect target) unless `:skip-ssrf-check? true` is passed.
+   Redirect semantics follow RFC 7231 §6.4: 301/302 POST is downgraded
+   to GET; 303 always uses GET; 307/308 preserve the original method."
+  ([cfg-or-client request]
+   (req-with-redirects cfg-or-client request {}))
+  ([cfg-or-client request {:keys [max-redirects skip-ssrf-check?]
+                           :or   {max-redirects default-max-redirects}
+                           :as opts}]
+   (let [send-opts  (dissoc opts :max-redirects :skip-ssrf-check?)
+         uri-coerce (if skip-ssrf-check? str ssrf/validate-uri)]
+     (loop [current-req (update request :uri uri-coerce)
+            hops        0]
+       (let [client (resolve-client cfg-or-client)
+             resp   (send! client current-req send-opts)
+             status (:status resp)]
+         (if (and (<= 300 status 399)
+                  (< hops max-redirects))
+           (if-let [location (get-in resp [:headers "location"])]
+             (let [next-uri (resolve-location (str (:uri current-req)) location)]
+               (recur (update (redirect-request current-req next-uri status) :uri uri-coerce)
+                      (inc hops)))
+             ;; No Location header on a 3xx — return the response as-is
+             resp)
+           resp))))))

backend/src/app/http/errors.clj

@@ -220,12 +220,14 @@
                          (assoc :hint (ex-message error)))}))))
 
 (defmethod handle-exception java.io.IOException
-  [cause _ _]
-  (l/wrn :hint "io exception" :cause cause)
-  {::yres/status 500
-   ::yres/body {:type :server-error
-                :code :io-exception
-                :hint (ex-message cause)}})
+  [cause request _]
+  (binding [l/*context* (request->context request)]
+    (l/wrn :hint "io exception" :cause cause)
+    {::yres/status 500
+     ::yres/body {:type :server-error
+                  :code :io-exception
+                  :hint (ex-message cause)
+                  :path (:path request)}}))
 
 (defmethod handle-exception java.util.concurrent.CompletionException
   [cause request _]

backend/src/app/loggers/audit.clj

@@ -16,12 +16,12 @@
    [app.common.uuid :as uuid]
    [app.config :as cf]
    [app.db :as db]
+   [app.email :as email]
    [app.http :as-alias http]
    [app.http.access-token :as-alias actoken]
    [app.loggers.audit.tasks :as-alias tasks]
    [app.loggers.webhooks :as-alias webhooks]
    [app.rpc :as-alias rpc]
-   [app.rpc.retry :as rtry]
    [app.setup :as-alias setup]
    [app.util.inet :as inet]
    [app.util.services :as-alias sv]
@@ -33,6 +33,63 @@
 ;; HELPERS
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 
+(def ^:private filter-auth-events
+  #{"login-with-oidc" "login-with-password" "register-profile" "update-profile"})
+
+(def ^:private safe-backend-context-keys
+  #{:version
+    :initiator
+    :client-version
+    :client-user-agent})
+
+(def ^:private safe-frontend-context-keys
+  #{:version
+    :locale
+    :browser
+    :browser-version
+    :engine
+    :engine-version
+    :os
+    :os-version
+    :device-type
+    :device-arch
+    :screen-width
+    :screen-height
+    :screen-color-depth
+    :screen-orientation
+    :event-origin
+    :event-namespace
+    :event-symbol})
+
+(def profile-props
+  [:id
+   :is-active
+   :is-muted
+   :auth-backend
+   :email
+   :default-team-id
+   :default-project-id
+   :fullname
+   :lang])
+
+(def ^:private event-keys
+  #{:id
+    :name
+    :type
+    :profile-id
+    :ip-addr
+    :props
+    :context
+    :source
+    :tracked-at
+    :created-at})
+
+(def reserved-props
+  #{:session-id
+    :password
+    :old-password
+    :token})
+
 (defn extract-utm-params
   "Extracts additional data from params and namespace them under
   `penpot` ns."
@@ -47,17 +104,6 @@
                 (assoc (->> sk str/kebab (keyword "penpot")) v))))]
     (reduce-kv process-param {} params)))
 
-(def profile-props
-  [:id
-   :is-active
-   :is-muted
-   :auth-backend
-   :email
-   :default-team-id
-   :default-project-id
-   :fullname
-   :lang])
-
 (defn profile->props
   [profile]
   (-> profile
@@ -65,12 +111,6 @@
       (merge (:props profile))
       (d/without-nils)))
 
-(def reserved-props
-  #{:session-id
-    :password
-    :old-password
-    :token})
-
 (defn clean-props
   [props]
   (into {}
@@ -120,16 +160,17 @@
 ;; an external storage and data cleared.
 
 (def ^:private schema:event
-  [:map {:title "event"}
-   [::type ::sm/text]
-   [::name ::sm/text]
-   [::profile-id ::sm/uuid]
-   [::ip-addr {:optional true} ::sm/text]
-   [::props {:optional true} [:map-of :keyword :any]]
-   [::context {:optional true} [:map-of :keyword :any]]
-   [::tracked-at {:optional true} ::ct/inst]
-   [::created-at {:optional true} ::ct/inst]
-   [::source {:optional true} ::sm/text]
+  [:map {:title "AuditEvent"}
+   [:id {:optional true} ::sm/uuid]
+   [:type ::sm/text]
+   [:name ::sm/text]
+   [:profile-id ::sm/uuid]
+   [:props [:map-of :keyword :any]]
+   [:context [:map-of :keyword :any]]
+   [:tracked-at ::ct/inst]
+   [:created-at ::ct/inst]
+   [:source ::sm/text]
+   [:ip-addr {:optional true} ::sm/text]
    [::webhooks/event? {:optional true} ::sm/boolean]
    [::webhooks/batch-timeout {:optional true} ::ct/duration]
    [::webhooks/batch-key {:optional true}
@@ -141,7 +182,156 @@
 (def valid-event?
   (sm/validator schema:event))
 
-(defn prepare-event
+(defn- prepare-context-from-request
+  "Prepare backend event context from request"
+  [request]
+  (let [client-event-origin (get-client-event-origin request)
+        client-version      (get-client-version request)
+        client-user-agent   (get-client-user-agent request)
+        session-id          (get-external-session-id request)
+        key-id              (::http/auth-key-id request)
+        token-id            (::actoken/id request)
+        token-type          (::actoken/type request)]
+    {:external-session-id session-id
+     :initiator (or key-id "app")
+     :access-token-id (some-> token-id str)
+     :access-token-type (some-> token-type str)
+     :client-event-origin client-event-origin
+     :client-user-agent client-user-agent
+     :client-version client-version
+     :version (:full cf/version)}))
+
+(defn- append-audit-entry
+  [cfg params]
+  (let [params (-> params
+                   (assoc :id (uuid/next))
+                   (update :props db/tjson)
+                   (update :context db/tjson)
+                   (update :ip-addr db/inet))
+        params (select-keys params event-keys)]
+    (db/insert! cfg :audit-log params)))
+
+(def ^:private xf:filter-telemetry-props
+  "Transducer that keeps only map entries whose values are UUIDs,
+  booleans or numbers."
+  (filter (fn [[k v]]
+            (and (simple-keyword? k)
+                 (or (uuid? v) (boolean? v) (number? v))))))
+
+(declare filter-telemetry-props)
+(declare filter-telemetry-context)
+
+(defn- process-event
+  [cfg event]
+  (when (contains? cf/flags :audit-log-logger)
+    (l/log! ::l/logger "app.audit"
+            ::l/level :info
+            :profile-id (str (:profile-id event))
+            :ip-addr (str (:ip-addr event))
+            :type (:type event)
+            :name (:name event)
+            :props (json/encode (:props event) :key-fn json/write-camel-key)
+            :context (json/encode (:context event) :key-fn json/write-camel-key)))
+
+  (when (contains? cf/flags :audit-log)
+    (append-audit-entry cfg event))
+
+  (when (contains? cf/flags :telemetry)
+    ;; NOTE: when both audit-log and telemetry are enabled, events are stored
+    ;; twice: once with full details (above) and once stripped of props and
+    ;; ip-addr, tagged with source="telemetry" so the telemetry task can
+    ;; collect and ship them.  The profile-id is preserved (UUIDs are already
+    ;; anonymous random identifiers).  Only a safe subset of context fields
+    ;; is kept: initiator, version, client-version and client-user-agent.
+    ;; Timestamps are truncated to day precision to avoid leaking exact event
+    ;; timing.
+    (let [event (-> event
+                    (filter-telemetry-props)
+                    (filter-telemetry-context)
+                    (update :created-at ct/truncate :days)
+                    (update :tracked-at ct/truncate :days)
+                    (assoc :source "telemetry:backend")
+                    (assoc :ip-addr "0.0.0.0"))]
+      (append-audit-entry cfg event)))
+
+  (when (and (contains? cf/flags :webhooks)
+             (::webhooks/event? event))
+    (let [batch-key     (::webhooks/batch-key event)
+          batch-timeout (::webhooks/batch-timeout event)
+          label         (dm/str "rpc:" (:name event))
+          label         (cond
+                          (ifn? batch-key)    (dm/str label ":" (batch-key (::rpc/params event)))
+                          (string? batch-key) (dm/str label ":" batch-key)
+                          :else               label)
+          dedupe?       (boolean (and batch-key batch-timeout))]
+
+      (wrk/submit! (-> cfg
+                       (assoc ::wrk/task :process-webhook-event)
+                       (assoc ::wrk/queue :webhooks)
+                       (assoc ::wrk/max-retries 0)
+                       (assoc ::wrk/delay (or batch-timeout 0))
+                       (assoc ::wrk/dedupe dedupe?)
+                       (assoc ::wrk/label label)
+                       (assoc ::wrk/params (-> event
+                                               (d/without-qualified)
+                                               (dissoc :source)
+                                               (dissoc :context)
+                                               (dissoc :ip-addr)
+                                               (dissoc :type)))))))
+  event)
+
+(defn submit*
+  "A public API, lower-level than submit, assumes all required fields are filled"
+  [cfg event]
+  (try
+    (let [event (check-event event)]
+      (db/tx-run! cfg process-event event))
+    (catch Throwable cause
+      (l/error :hint "unexpected error processing event" :cause cause))))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; PUBLIC API
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(defn filter-telemetry-props
+  [{:keys [source name props type] :as params}]
+  (cond
+    (or (and (= source "frontend")
+             (= type "identify"))
+        (and (= source "backend")
+             (filter-auth-events name)))
+
+    (let [props' (into {} xf:filter-telemetry-props props)
+          props' (-> props'
+                     (assoc :lang (:lang props))
+                     (assoc :auth-backend (:auth-backend props))
+                     (assoc :email-domain (email/get-domain (:email props)))
+                     (d/without-nils))]
+      (assoc params :props props'))
+
+    (and (= source "backend")
+         (= type "trigger")
+         (= name "instance-start"))
+    params
+
+    (and (= source "frontend")
+         (= type "action")
+         (= name "navigate"))
+    (assoc params :props (select-keys props [:route :file-id :team-id :page-id]))
+
+    :else
+    (let [props (into {} xf:filter-telemetry-props props)]
+      (assoc params :props props))))
+
+(defn filter-telemetry-context
+  [{:keys [source context] :as params}]
+  (let [context (case source
+                  "backend"  (select-keys context safe-backend-context-keys)
+                  "frontend" (select-keys context safe-frontend-context-keys)
+                  {})]
+    (assoc params :context context)))
+
+(defn prepare-rpc-event
   [cfg mdata params result]
   (let [resultm      (meta result)
         request      (-> params meta ::http/request)
@@ -154,23 +344,29 @@
                              (merge params (::props resultm)))
                          (clean-props))
 
-        context      (merge (::context resultm)
-                            (prepare-context-from-request request))
+        context      (-> (::context resultm)
+                         (merge (prepare-context-from-request request))
+                         (assoc :request-id (::rpc/request-id params))
+                         (d/without-nils))
+
         ip-addr      (inet/parse-request request)
         module       (get cfg ::rpc/module)]
 
-    {::type (or (::type resultm)
-                (::rpc/type cfg))
-     ::name (or (::name resultm)
-                (let [sname (::sv/name mdata)]
-                  (if (not= module "main")
-                    (str module "-" sname)
-                    sname)))
+    {:type (or (::type resultm)
+               (::rpc/type cfg))
+     :name (or (::name resultm)
+               (let [sname (::sv/name mdata)]
+                 (if (not= module "main")
+                   (str module "-" sname)
+                   sname)))
 
-     ::profile-id profile-id
-     ::ip-addr ip-addr
-     ::props props
-     ::context context
+     :profile-id profile-id
+     :ip-addr ip-addr
+     :props props
+     :context context
+
+     :created-at (::rpc/request-at params)
+     :tracked-at (::rpc/request-at params)
 
      ;; NOTE: for batch-key lookup we need the params as-is
      ;; because the rpc api does not need to know the
@@ -190,148 +386,49 @@
          (::webhooks/event? resultm)
          false)}))
 
-(defn- prepare-context-from-request
-  "Prepare backend event context from request"
-  [request]
-  (let [client-event-origin (get-client-event-origin request)
-        client-version      (get-client-version request)
-        client-user-agent   (get-client-user-agent request)
-        session-id          (get-external-session-id request)
-        key-id              (::http/auth-key-id request)
-        token-id            (::actoken/id request)
-        token-type          (::actoken/type request)]
-    (d/without-nils
-     {:external-session-id session-id
-      :initiator (or key-id "app")
-      :access-token-id (some-> token-id str)
-      :access-token-type (some-> token-type str)
-      :client-event-origin client-event-origin
-      :client-user-agent client-user-agent
-      :client-version client-version
-      :version (:full cf/version)})))
-
 (defn event-from-rpc-params
   "Create a base event skeleton with pre-filled some important
   data that can be extracted from RPC params object"
   [params]
-  (let [context (some-> params meta ::http/request prepare-context-from-request)
-        event   {::type "action"
-                 ::profile-id (or (::rpc/profile-id params) uuid/zero)
-                 ::ip-addr (::rpc/ip-addr params)}]
-    (cond-> event
-      (some? context)
-      (assoc ::context context))))
+  (let [context    (some-> params meta ::http/request prepare-context-from-request)
+        context    (assoc context :request-id (::rpc/request-id params))
+        request-at (::rpc/request-at params)]
+    {:type "action"
+     :profile-id (::rpc/profile-id params)
+     :created-at request-at
+     :tracked-at request-at
+     :ip-addr (::rpc/ip-addr params)
+     :context (d/without-nils context)}))
 
-(defn- event->params
-  [event]
-  (let [params {:id (uuid/next)
-                :name (::name event)
-                :type (::type event)
-                :profile-id (::profile-id event)
-                :ip-addr (::ip-addr event)
-                :context (::context event {})
-                :props (::props event {})
-                :source "backend"}
-        tnow   (::tracked-at event)]
-
-    (cond-> params
-      (some? tnow)
-      (assoc :tracked-at tnow))))
-
-(defn- append-audit-entry
-  [cfg params]
-  (let [params (-> params
-                   (update :props db/tjson)
-                   (update :context db/tjson)
-                   (update :ip-addr db/inet))]
-    (db/insert! cfg :audit-log params)))
-
-(defn- handle-event!
+(defn submit
+  "Submit an event to be registered under audit-log subsystem"
   [cfg event]
-  (let [tnow   (ct/now)
-        params (-> (event->params event)
-                   (assoc :created-at tnow)
-                   (update :tracked-at #(or % tnow)))]
+  (let [tnow  (ct/now)
+        event (-> event
+                  (assoc :created-at tnow)
+                  (update :profile-id d/nilv uuid/zero)
+                  (update :tracked-at d/nilv tnow)
+                  (update :ip-addr d/nilv "0.0.0.0")
+                  (update :props d/nilv {})
+                  (update :context d/nilv {})
+                  (assoc :source "backend")
+                  (d/without-nils))]
+    (submit* cfg event)))
 
-    (when (contains? cf/flags :audit-log-logger)
-      (l/log! ::l/logger "app.audit"
-              ::l/level :info
-              :profile-id (str (::profile-id event))
-              :ip-addr (str (::ip-addr event))
-              :type (::type event)
-              :name (::name event)
-              :props (json/encode (::props event) :key-fn json/write-camel-key)
-              :context (json/encode (::context event) :key-fn json/write-camel-key)))
-
-    (when (contains? cf/flags :audit-log)
-      ;; NOTE: this operation may cause primary key conflicts on inserts
-      ;; because of the timestamp precission (two concurrent requests), in
-      ;; this case we just retry the operation.
-      (append-audit-entry cfg params))
-
-    (when (and (or (contains? cf/flags :telemetry)
-                   (cf/get :telemetry-enabled))
-               (not (contains? cf/flags :audit-log)))
-      ;; NOTE: this operation may cause primary key conflicts on inserts
-      ;; because of the timestamp precission (two concurrent requests), in
-      ;; this case we just retry the operation.
-      ;;
-      ;; NOTE: this is only executed when general audit log is disabled
-      (let [params (-> params
-                       (assoc :props {})
-                       (assoc :context {}))]
-        (append-audit-entry cfg params)))
-
-    (when (and (contains? cf/flags :webhooks)
-               (::webhooks/event? event))
-      (let [batch-key     (::webhooks/batch-key event)
-            batch-timeout (::webhooks/batch-timeout event)
-            label         (dm/str "rpc:" (:name params))
-            label         (cond
-                            (ifn? batch-key)    (dm/str label ":" (batch-key (::rpc/params event)))
-                            (string? batch-key) (dm/str label ":" batch-key)
-                            :else               label)
-            dedupe?       (boolean (and batch-key batch-timeout))]
-
-        (wrk/submit! (-> cfg
-                         (assoc ::wrk/task :process-webhook-event)
-                         (assoc ::wrk/queue :webhooks)
-                         (assoc ::wrk/max-retries 0)
-                         (assoc ::wrk/delay (or batch-timeout 0))
-                         (assoc ::wrk/dedupe dedupe?)
-                         (assoc ::wrk/label label)
-                         (assoc ::wrk/params (-> params
-                                                 (dissoc :source)
-                                                 (dissoc :context)
-                                                 (dissoc :ip-addr)
-                                                 (dissoc :type)))))))
-    params))
-
-(defn submit!
-  "Submit audit event to the collector."
-  [cfg event]
-  (try
-    (let [event (-> (d/without-nils event)
-                    (check-event))
-          cfg   (-> cfg
-                    (assoc ::rtry/when rtry/conflict-exception?)
-                    (assoc ::rtry/max-retries 6)
-                    (assoc ::rtry/label "persist-audit-log"))]
-      (rtry/invoke! cfg db/tx-run! handle-event! event))
-    (catch Throwable cause
-      (l/error :hint "unexpected error processing event" :cause cause))))
-
-(defn insert!
+(defn insert
   "Submit audit event to the collector, intended to be used only from
   command line helpers because this skips all webhooks and telemetry
   logic."
   [cfg event]
   (when (contains? cf/flags :audit-log)
-    (let [event (-> (d/without-nils event)
+    (let [tnow  (ct/now)
+          event (-> event
+                    (assoc :created-at tnow)
+                    (update :tracked-at d/nilv tnow)
+                    (update :profile-id d/nilv uuid/zero)
+                    (update :props d/nilv {})
+                    (update :context d/nilv {})
+                    (assoc :source "backend")
+                    (select-keys event-keys)
                     (check-event))]
-      (db/run! cfg (fn [cfg]
-                     (let [tnow   (ct/now)
-                           params (-> (event->params event)
-                                      (assoc :created-at tnow)
-                                      (update :tracked-at #(or % tnow)))]
-                       (append-audit-entry cfg params)))))))
+      (db/run! cfg append-audit-entry event))))

backend/src/app/loggers/audit/archive_task.clj

@@ -10,14 +10,11 @@
    [app.common.logging :as l]
    [app.common.schema :as sm]
    [app.common.transit :as t]
-   [app.common.uuid :as uuid]
    [app.config :as cf]
    [app.db :as db]
    [app.http.client :as http]
    [app.setup :as-alias setup]
-   [app.tokens :as tokens]
    [integrant.core :as ig]
-   [lambdaisland.uri :as u]
    [promesa.exec :as px]))
 
 ;; This is a task responsible to send the accumulated events to
@@ -52,19 +49,18 @@
 
 (defn- send!
   [{:keys [::uri] :as cfg} events]
-  (let [token   (tokens/generate cfg
-                                 {:iss "authentication"
-                                  :uid uuid/zero})
+  (let [skey    (-> cfg ::setup/shared-keys :nexus)
         body    (t/encode {:events events})
         headers {"content-type" "application/transit+json"
                  "origin" (str (cf/get :public-uri))
-                 "cookie" (u/map->query-string {:auth-token token})}
+                 "x-shared-key" (str "nexus " skey)}
         params  {:uri uri
                  :timeout 12000
                  :method :post
                  :headers headers
                  :body body}
-        resp    (http/req! cfg params)]
+        resp    (http/req cfg params {:skip-ssrf-check? true})]
+
     (if (= (:status resp) 204)
       true
       (do
@@ -109,7 +105,7 @@
 (def ^:private schema:handler-params
   [:map
    ::db/pool
-   ::setup/props
+   ::setup/shared-keys
    ::http/client])
 
 (defmethod ig/assert-key ::handler

backend/src/app/loggers/database.clj

@@ -97,7 +97,7 @@
       (l/warn :hint "unexpected exception on database error logger" :cause cause))))
 
 (defn- audit-event->report
-  [{:keys [::audit/context ::audit/props ::audit/ip-addr] :as record}]
+  [{:keys [context props ip-addr] :as record}]
   (let [context
         (reduce-kv (fn [context k v]
                      (let [k' (keyword "frontend" (name k))]
@@ -117,14 +117,14 @@
 
     {:context (-> (into (sorted-map) context)
                   (pp/pprint-str :length 50))
-     :origin  (::audit/name record)
+     :origin  (:name record)
      :href    (get props :href)
      :hint    (get props :hint)
      :report  (get props :report)}))
 
 (defn- handle-audit-event
   "Convert the log record into a report object and persist it on the database"
-  [{:keys [::db/pool]} {:keys [::audit/id] :as event}]
+  [{:keys [::db/pool]} {:keys [id] :as event}]
   (try
     (let [uri    (cf/get :public-uri)
           report (-> event audit-event->report d/without-nils)]
@@ -189,12 +189,12 @@
                          (::l/id item)
                          (handle-log-record cfg item)
 
-                         (::audit/id item)
-                         (handle-audit-event cfg item)
-
                          (::rlimit/id item)
                          (handle-rlimit-event cfg item)
 
+                         (-> item meta ::audit/event)
+                         (handle-audit-event cfg item)
+
                          :else
                          (l/warn :hint "received unexpected item" :item item))
 
@@ -226,4 +226,3 @@
   [cfg event]
   (when-let [{:keys [::input]} (get cfg ::reporter)]
     (sp/put! input event)))
-

backend/src/app/loggers/mattermost.clj

@@ -52,12 +52,12 @@
                          trace
                          "```")))
 
-        resp (http/req! cfg
-                        {:uri (cf/get :error-report-webhook)
-                         :method :post
-                         :headers {"content-type" "application/json"}
-                         :body (json/encode-str {:text text})}
-                        {:sync? true})]
+        resp (http/req cfg
+                       {:uri (cf/get :error-report-webhook)
+                        :method :post
+                        :headers {"content-type" "application/json"}
+                        :body (json/encode-str {:text text})}
+                       {:sync? true})]
 
     (when (not= 200 (:status resp))
       (l/warn :hint "error on sending data"
@@ -83,7 +83,7 @@
      :trace            (ex/format-throwable cause :detail? false :header? false)}))
 
 (defn- audit-event->report
-  [{:keys [::audit/context ::audit/props ::audit/id] :as event}]
+  [{:keys [context props id] :as event}]
   {:id               id
    :type             "exception"
    :origin           "audit-log"
@@ -92,7 +92,7 @@
    :host             (cf/get :host)
    :backend-version  (:full cf/version)
    :frontend-version (:version context)
-   :profile-id       (:audit/profile-id event)
+   :profile-id       (:profile-id event)
    :href             (get props :href)})
 
 (defn- rlimit-event->report
@@ -148,12 +148,12 @@
                              (::l/id item)
                              (handle-event cfg item log-record->report)
 
-                             (::audit/id item)
-                             (handle-event cfg item audit-event->report)
-
                              (::rlimit/id item)
                              (handle-event cfg item rlimit-event->report)
 
+                             (-> item meta ::audit/event)
+                             (handle-event cfg item audit-event->report)
+
                              :else
                              (l/warn :hint "received unexpected item" :item item)))
 

backend/src/app/loggers/webhooks.clj

@@ -70,14 +70,14 @@
   (fn [{:keys [props] :as task}]
 
     (let [items (lookup-webhooks cfg props)
-          event {::audit/profile-id (:profile-id props)
-                 ::audit/name "webhook"
-                 ::audit/type "trigger"
-                 ::audit/props {:name (get props :name)
-                                :event-id (get props :id)
-                                :total-affected (count items)}}]
+          event {:profile-id (:profile-id props)
+                 :name "webhook"
+                 :type "trigger"
+                 :props {:name (get props :name)
+                         :event-id (get props :id)
+                         :total-affected (count items)}}]
 
-      (audit/insert! cfg event)
+      (audit/insert cfg event)
 
       (when items
         (l/trc :hint "webhooks found for event" :total (count items))
@@ -159,7 +159,7 @@
                    :method :post
                    :body body}]
           (try
-            (let [rsp (http/req! cfg req {:response-type :input-stream :sync? true})
+            (let [rsp (http/req cfg req {:response-type :input-stream :sync? true})
                   err (interpret-response rsp)]
               (report-delivery! whook req rsp err)
               (update-webhook! whook err))
@@ -190,4 +190,11 @@
     "invalid-uri"
 
     (instance? java.net.http.HttpConnectTimeoutException cause)
-    "timeout"))
+    "timeout"
+
+    :else
+    (let [data (ex-data cause)]
+      (if (and (= :validation (:type data))
+               (= :ssrf-blocked-target (:code data)))
+        (str "blocked-request:" (:hint data))
+        nil))))

backend/src/app/main.clj

@@ -61,21 +61,15 @@
     ::mdef/help "A total number of bytes processed by update-file."
     ::mdef/type :counter}
 
-   :rpc-mutation-timing
-   {::mdef/name "penpot_rpc_mutation_timing"
-    ::mdef/help "RPC mutation method call timing."
+   :rpc-main-timing
+   {::mdef/name "penpot_rpc_main_timing"
+    ::mdef/help "RPC command method call timing for main"
     ::mdef/labels ["name"]
     ::mdef/type :histogram}
 
-   :rpc-command-timing
-   {::mdef/name "penpot_rpc_command_timing"
-    ::mdef/help "RPC command method call timing."
-    ::mdef/labels ["name"]
-    ::mdef/type :histogram}
-
-   :rpc-query-timing
-   {::mdef/name "penpot_rpc_query_timing"
-    ::mdef/help "RPC query method call timing."
+   :rpc-management-timing
+   {::mdef/name "penpot_rpc_management_timing"
+    ::mdef/help "RPC command method call timing for management."
     ::mdef/labels ["name"]
     ::mdef/type :histogram}
 
@@ -304,10 +298,11 @@
     ::session/manager (ig/ref ::session/manager)}
 
    :app.http.assets/routes
-   {::http.assets/path  (cf/get :assets-path)
-    ::http.assets/cache-max-age (ct/duration {:hours 24})
-    ::http.assets/cache-max-agesignature-max-age (ct/duration {:hours 24 :minutes 5})
-    ::sto/storage  (ig/ref ::sto/storage)}
+   {::http.assets/path              (cf/get :assets-path)
+    ::http.assets/cache-max-age     (ct/duration {:hours 24})
+    ::http.assets/signature-max-age (ct/duration {:hours 24 :minutes 15})
+    ::sto/storage                   (ig/ref ::sto/storage)
+    ::session/manager               (ig/ref ::session/manager)}
 
    ::rpc/climit
    {::mtx/metrics        (ig/ref ::mtx/metrics)
@@ -388,6 +383,7 @@
      :offload-file-data  (ig/ref :app.tasks.offload-file-data/handler)
      :tasks-gc           (ig/ref :app.tasks.tasks-gc/handler)
      :telemetry          (ig/ref :app.tasks.telemetry/handler)
+     :upload-session-gc  (ig/ref :app.tasks.upload-session-gc/handler)
      :storage-gc-deleted (ig/ref ::sto.gc-deleted/handler)
      :storage-gc-touched (ig/ref ::sto.gc-touched/handler)
      :session-gc         (ig/ref ::session.tasks/gc)
@@ -423,6 +419,9 @@
    :app.tasks.tasks-gc/handler
    {::db/pool (ig/ref ::db/pool)}
 
+   :app.tasks.upload-session-gc/handler
+   {::db/pool (ig/ref ::db/pool)}
+
    :app.tasks.objects-gc/handler
    {::db/pool     (ig/ref ::db/pool)
     ::sto/storage (ig/ref ::sto/storage)}
@@ -466,16 +465,17 @@
 
    ::setup/shared-keys
    {::setup/props (ig/ref ::setup/props)
-    :nitrate (cf/get :nitrate-shared-key)
-    :exporter (cf/get :exporter-shared-key)}
+    :nexus        (cf/get :nexus-shared-key)
+    :nitrate      (cf/get :nitrate-shared-key)
+    :exporter     (cf/get :exporter-shared-key)}
 
    ::setup/clock
    {}
 
    :app.loggers.audit.archive-task/handler
-   {::setup/props        (ig/ref ::setup/props)
-    ::db/pool            (ig/ref ::db/pool)
-    ::http.client/client (ig/ref ::http.client/client)}
+   {::setup/shared-keys  (ig/ref ::setup/shared-keys)
+    ::http.client/client (ig/ref ::http.client/client)
+    ::db/pool            (ig/ref ::db/pool)}
 
    :app.loggers.audit.gc-task/handler
    {::db/pool (ig/ref ::db/pool)}
@@ -543,6 +543,9 @@
      {:cron #penpot/cron "0 0 0 * * ?" ;; daily
       :task :tasks-gc}
 
+     {:cron #penpot/cron "0 0 0 * * ?" ;; daily
+      :task :upload-session-gc}
+
      {:cron #penpot/cron "0 0 2 * * ?" ;; daily
       :task :file-gc-scheduler}
 
@@ -650,9 +653,8 @@
   [& _args]
   (try
     (let [p (promise)]
-      (when (contains? cf/flags :nrepl-server)
-        (l/inf :hint "start nrepl server" :port 6064)
-        (nrepl/start-server :bind "0.0.0.0" :port 6064))
+      (l/inf :hint "start nrepl server" :port 6064)
+      (nrepl/start-server :bind "0.0.0.0" :port 6064)
 
       (start)
       (deref p))

backend/src/app/media.clj

@@ -18,6 +18,7 @@
    [app.config :as cf]
    [app.db :as-alias db]
    [app.http.client :as http]
+   [app.media.sanitize :as sanitize]
    [app.storage :as-alias sto]
    [app.storage.tmp :as tmp]
    [buddy.core.bytes :as bb]
@@ -31,15 +32,12 @@
   (:import
    clojure.lang.XMLHandler
    java.io.InputStream
-   javax.xml.XMLConstants
    javax.xml.parsers.SAXParserFactory
+   javax.xml.XMLConstants
    org.apache.commons.io.IOUtils
    org.im4java.core.ConvertCmd
    org.im4java.core.IMOperation))
 
-(def default-max-file-size
-  (* 1024 1024 10)) ; 10 MiB
-
 (def schema:upload
   [:map {:title "Upload"}
    [:filename :string]
@@ -78,6 +76,20 @@
                                 max-size)))
     upload))
 
+(defn validate-font-size!
+  "Validates that the font file `upload` does not exceed the configured
+  `:font-max-file-size` limit.  Accepts the same map shape as
+  `validate-media-size!` — requires a `:size` key in bytes."
+  [upload]
+  (let [max-size (cf/get :font-max-file-size)]
+    (when (> (:size upload) max-size)
+      (ex/raise :type :restriction
+                :code :font-max-file-size-reached
+                :hint (str/ffmt "the uploaded font size % is greater than the maximum %"
+                                (:size upload)
+                                max-size)))
+    upload))
+
 (defmulti process :cmd)
 (defmulti process-error class)
 
@@ -295,9 +307,7 @@
   [{:keys [::http/client]} uri]
   (letfn [(parse-and-validate [{:keys [status headers] :as response}]
             (let [size     (some-> (get headers "content-length") d/parse-integer)
-                  mtype    (get headers "content-type")
-                  format   (cm/mtype->format mtype)
-                  max-size (cf/get :media-max-file-size default-max-file-size)]
+                  mtype    (get headers "content-type")]
 
               (when-not (<= 200 status 299)
                 (ex/raise :type :validation
@@ -309,25 +319,17 @@
                           :code :unknown-size
                           :hint "seems like the url points to resource with unknown size"))
 
-              (when (> size max-size)
-                (ex/raise :type :validation
-                          :code :file-too-large
-                          :hint (str/ffmt "the file size % is greater than the maximum %"
-                                          size
-                                          default-max-file-size)))
-
-              (when (nil? format)
-                (ex/raise :type :validation
-                          :code :media-type-not-allowed
-                          :hint "seems like the url points to an invalid media object"))
-
-              {:size size :mtype mtype :format format}))]
+              (-> {:size size :mtype mtype}
+                  (validate-media-type!)
+                  (validate-media-size!))))]
 
     (let [{:keys [body] :as response}
           (try
-            (http/req! client
-                       {:method :get :uri uri}
-                       {:response-type :input-stream})
+            (http/req-with-redirects
+             client
+             {:method :get :uri uri}
+             {:response-type :input-stream
+              :max-redirects 3})
             (catch java.net.ConnectException cause
               (ex/raise :type :validation
                         :code :unable-to-download-image
@@ -358,9 +360,11 @@
                   :code :mismatch-write-size
                   :hint "unexpected state: unable to write to file"))
 
-      {;; :size size
-       :path path
-       :mtype mtype})))
+      ;; Sanitize: strip trailing data after image EOF markers
+      (let [new-size (sanitize/truncate-after-eof path mtype)]
+        {:path  path
+         :mtype mtype
+         :size  new-size}))))
 
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 ;; FONTS

backend/src/app/media/sanitize.clj

@@ -0,0 +1,191 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS INC
+
+(ns app.media.sanitize
+  "Image EOF truncation helpers — strips trailing data after image EOF
+   markers to prevent exfiltration of non-image bytes appended to
+   valid image files."
+  (:require
+   [app.common.buffer :as buf]
+   [app.common.exceptions :as ex]
+   [app.common.logging :as l]
+   [app.util.nio :as nio])
+  (:import
+   java.nio.ByteOrder
+   java.nio.channels.FileChannel))
+
+(set! *warn-on-reflection* true)
+
+(defn- scan-backwards
+  "Scan byte array `arr` backwards (from the end) for the byte pattern
+   `marker`. Returns the index in `arr` where the marker starts, or -1
+   if not found."
+  [^bytes arr ^bytes marker]
+  (let [arr-len    (alength arr)
+        marker-len (alength marker)]
+    (loop [i (- arr-len marker-len)]
+      (if (< i 0)
+        -1
+        (if (loop [j 0]
+              (if (>= j marker-len)
+                true
+                (if (= (aget arr (+ i j)) (aget marker j))
+                  (recur (inc j))
+                  false)))
+          i
+          (recur (dec i)))))))
+
+(defn- find-last-png-iend
+  "Find the byte offset of the end of the PNG IEND chunk (12 bytes:
+   4-byte length + 4-byte 'IEND' + 4-byte CRC32). Returns the offset
+   AFTER the CRC32, or nil if not found."
+  [^FileChannel channel]
+  (let [size (nio/channel-size channel)]
+    (when (> size 8)
+      (let [buf-size (min (int size) (* 1024 1024))
+            marker   (byte-array [0x49 0x45 0x4E 0x44])] ;; "IEND"
+        (loop [pos (max 0 (- size buf-size))]
+          (when (< pos size)
+            (let [arr (nio/read-at channel pos buf-size)
+                  idx (scan-backwards arr marker)]
+              (if (neg? idx)
+                ;; Not found in this chunk, try earlier
+                (let [next-pos (max 0 (- pos (- buf-size 4)))]
+                  (when (< next-pos pos)
+                    (recur next-pos)))
+                ;; Found "IEND" at idx. Chunk starts 4 bytes before.
+                (let [chunk-start (- (+ pos idx) 4)]
+                  (when (>= chunk-start 0)
+                    ;; PNG chunk length is big-endian (network byte order).
+                    ;; buf/wrap defaults to little-endian, so set it to big-endian.
+                    (let [len-arr (nio/read-at channel chunk-start 4)
+                          len-buf (buf/set-order (buf/wrap len-arr) ByteOrder/BIG_ENDIAN)
+                          chunk-len (buf/read-int len-buf 0)]
+                      (when (zero? chunk-len)
+                        (+ chunk-start 12)))))))))))))
+
+(defn- find-last-jpeg-eoi
+  "Find the byte offset of the last JPEG EOI marker (0xFF 0xD9).
+   Returns the offset AFTER the marker, or nil if not found."
+  [^FileChannel channel]
+  (let [size (nio/channel-size channel)]
+    (when (> size 2)
+      (let [buf-size (min (int size) (* 1024 1024))
+            marker   (byte-array [(unchecked-byte 0xFF) (unchecked-byte 0xD9)])]
+        (loop [pos (max 0 (- size buf-size))]
+          (when (< pos size)
+            (let [arr (nio/read-at channel pos buf-size)
+                  idx (scan-backwards arr marker)]
+              (if (neg? idx)
+                (let [next-pos (max 0 (- pos (- buf-size 2)))]
+                  (when (< next-pos pos)
+                    (recur next-pos)))
+                (+ pos idx 2)))))))))
+
+(defn- find-last-gif-trailer
+  "Find the byte offset immediately after the last GIF trailer byte (0x3B).
+   Scans backwards through the file so that appended data after the real
+   trailer is truncated even when it ends with 0x3B.
+   Returns the offset AFTER the trailer byte, or nil if 0x3B is not found."
+  [^FileChannel channel]
+  (let [size (nio/channel-size channel)]
+    (when (pos? size)
+      (let [buf-size (min (int size) (* 1024 1024))
+            marker   (byte-array [(unchecked-byte 0x3B)])]
+        (loop [pos (max 0 (- size buf-size))]
+          (when (< pos size)
+            (let [arr (nio/read-at channel pos buf-size)
+                  idx (scan-backwards arr marker)]
+              (if (neg? idx)
+                (let [next-pos (max 0 (- pos (- buf-size 1)))]
+                  (when (< next-pos pos)
+                    (recur next-pos)))
+                (+ pos idx 1)))))))))
+
+(defn- find-webp-end
+  "Parse the WebP RIFF header to find the declared file size.
+   WebP format: 'RIFF' (4 bytes) + uint32 total-size (4 bytes, little-endian)
+   + 'WEBP' (4 bytes). The total size is the offset of the end of the file.
+   Returns nil if the RIFF or WEBP magic bytes are missing."
+  [^FileChannel channel]
+  (let [size (nio/channel-size channel)]
+    (when (>= size 12)
+      (let [^bytes arr (nio/read-at channel 0 12)
+            buf  (buf/wrap arr)]
+        ;; Check RIFF magic (bytes 0-3) AND WEBP FourCC (bytes 8-11)
+        (when (and (= (aget arr 0) (byte 0x52))   ;; 'R'
+                   (= (aget arr 1) (byte 0x49))   ;; 'I'
+                   (= (aget arr 2) (byte 0x46))   ;; 'F'
+                   (= (aget arr 3) (byte 0x46))   ;; 'F'
+                   (= (aget arr 8)  (byte 0x57))  ;; 'W'
+                   (= (aget arr 9)  (byte 0x45))  ;; 'E'
+                   (= (aget arr 10) (byte 0x42))  ;; 'B'
+                   (= (aget arr 11) (byte 0x50))) ;; 'P'
+          (let [riff-size (bit-and (buf/read-int buf 4) 0xFFFFFFFF)]
+            ;; RIFF size field is the size of the file minus 8 bytes
+            (+ riff-size 8)))))))
+
+(defn truncate-after-eof
+  "Given a `java.nio.file.Path` to a freshly-downloaded media file and a
+   declared MIME type, truncate the file in place to the position of the
+   format's EOF marker:
+     - image/png  → end of the IEND chunk (12 bytes: 4-byte length + 4-byte type + 4-byte CRC32)
+     - image/jpeg → 2 bytes after FFD9
+     - image/gif  → immediately after the last GIF trailer byte 0x3B
+     - image/webp → end of RIFF chunk declared in bytes 4..8
+     - image/svg+xml → no-op (text format; processed by SAX parser)
+     - other → no-op (return path unchanged)
+   Returns the new file size. Raises `:validation/:invalid-image` if no
+   EOF marker is found within the file."
+  [^java.nio.file.Path path ^String mtype]
+  (try
+    (with-open [channel (nio/open-channel path)]
+      (let [size (nio/channel-size channel)]
+        (if (zero? size)
+          0
+          (let [needs-eof-marker? (or (= mtype "image/png")
+                                      (= mtype "image/jpeg")
+                                      (= mtype "image/gif")
+                                      (= mtype "image/webp"))
+
+                eof-offset
+                (cond
+                  (= mtype "image/png")  (find-last-png-iend channel)
+                  (= mtype "image/jpeg") (find-last-jpeg-eoi channel)
+                  (= mtype "image/gif")  (find-last-gif-trailer channel)
+                  (= mtype "image/webp") (find-webp-end channel)
+                  :else                  nil)]
+
+            (cond
+              ;; No EOF marker applicable (SVG or other) — no-op
+              (nil? eof-offset)
+              (if needs-eof-marker?
+                (ex/raise :type :validation
+                          :code :invalid-image
+                          :hint "image format EOF marker not found")
+                size)
+
+              ;; Truncate if needed
+              (< eof-offset size)
+              (do
+                (l/dbg :hint "truncating trailing data"
+                       :path (str path)
+                       :mtype mtype
+                       :original-size size
+                       :truncated-to eof-offset)
+                (nio/truncate channel eof-offset)
+                eof-offset)
+
+              ;; Already at correct size or marker at end
+              :else
+              eof-offset)))))
+    (catch Exception e
+      (if (ex/exception? e)
+        (throw e)
+        (ex/raise :type :validation
+                  :code :invalid-image
+                  :hint "failed to sanitize image"
+                  :cause e)))))

backend/src/app/metrics.clj

@@ -15,16 +15,16 @@
    io.prometheus.client.CollectorRegistry
    io.prometheus.client.Counter
    io.prometheus.client.Counter$Child
+   io.prometheus.client.exporter.common.TextFormat
    io.prometheus.client.Gauge
    io.prometheus.client.Gauge$Child
    io.prometheus.client.Histogram
    io.prometheus.client.Histogram$Child
+   io.prometheus.client.hotspot.DefaultExports
    io.prometheus.client.SimpleCollector
    io.prometheus.client.Summary
    io.prometheus.client.Summary$Builder
    io.prometheus.client.Summary$Child
-   io.prometheus.client.exporter.common.TextFormat
-   io.prometheus.client.hotspot.DefaultExports
    java.io.StringWriter))
 
 (set! *warn-on-reflection* true)

backend/src/app/migrations.clj

@@ -463,8 +463,22 @@
     :fn (mg/resource "app/migrations/sql/0144-mod-server-error-report-table.sql")}
 
    {:name "0145-fix-plugins-uri-on-profile"
-    :fn mg0145/migrate}])
+    :fn mg0145/migrate}
 
+   {:name "0145-mod-audit-log-table"
+    :fn (mg/resource "app/migrations/sql/0145-mod-audit-log-table.sql")}
+
+   {:name "0145-mod-audit-log-table"
+    :fn (mg/resource "app/migrations/sql/0145-mod-audit-log-table.sql")}
+
+   {:name "0146-mod-audit-log-table"
+    :fn (mg/resource "app/migrations/sql/0146-mod-audit-log-table.sql")}
+
+   {:name "0146-mod-access-token-table"
+    :fn (mg/resource "app/migrations/sql/0146-mod-access-token-table.sql")}
+
+   {:name "0147-add-upload-session-table"
+    :fn (mg/resource "app/migrations/sql/0147-add-upload-session-table.sql")}])
 
 (defn apply-migrations!
   [pool name migrations]

backend/src/app/migrations/sql/0145-mod-audit-log-table.sql

@@ -0,0 +1,2 @@
+CREATE INDEX audit_log__created_at__idx ON audit_log(created_at) WHERE archived_at IS NULL;
+CREATE INDEX audit_log__archived_at__idx ON audit_log(archived_at) WHERE archived_at IS NOT NULL;

backend/src/app/migrations/sql/0146-mod-access-token-table.sql

@@ -0,0 +1,2 @@
+ALTER TABLE access_token
+ ADD COLUMN type text NULL;

backend/src/app/migrations/sql/0146-mod-audit-log-table.sql

@@ -0,0 +1,5 @@
+-- Add index on audit_log (source, created_at) to support efficient
+-- queries for the telemetry batch collection mode.
+
+CREATE INDEX IF NOT EXISTS audit_log__source__created_at__idx
+    ON audit_log (source, created_at ASC);

backend/src/app/migrations/sql/0147-add-upload-session-table.sql

@@ -0,0 +1,14 @@
+CREATE TABLE upload_session (
+  id         uuid PRIMARY KEY,
+
+  created_at timestamptz NOT NULL DEFAULT now(),
+
+  profile_id    uuid NOT NULL REFERENCES profile(id) ON DELETE CASCADE,
+  total_chunks  integer NOT NULL
+);
+
+CREATE INDEX upload_session__profile_id__idx
+    ON upload_session(profile_id);
+
+CREATE INDEX upload_session__created_at__idx
+    ON upload_session(created_at);

backend/src/app/nitrate.clj

@@ -18,13 +18,13 @@
 (defn- request-builder
   [cfg method uri shared-key profile-id]
   (fn []
-    (http/req! cfg {:method method
-                    :headers {"content-type" "application/json"
-                              "accept" "application/json"
-                              "x-shared-key" shared-key
-                              "x-profile-id" (str profile-id)}
-                    :uri uri
-                    :version :http1.1})))
+    (http/req cfg {:method method
+                   :headers {"content-type" "application/json"
+                             "accept" "application/json"
+                             "x-shared-key" shared-key
+                             "x-profile-id" (str profile-id)}
+                   :uri uri
+                   :version :http1.1})))
 
 
 (defn- with-retries

backend/src/app/redis.clj

@@ -24,28 +24,28 @@
    [integrant.core :as ig])
   (:import
    clojure.lang.MapEntry
-   io.lettuce.core.KeyValue
-   io.lettuce.core.RedisClient
-   io.lettuce.core.RedisCommandInterruptedException
-   io.lettuce.core.RedisCommandTimeoutException
-   io.lettuce.core.RedisException
-   io.lettuce.core.RedisURI
-   io.lettuce.core.ScriptOutputType
-   io.lettuce.core.SetArgs
    io.lettuce.core.api.StatefulRedisConnection
    io.lettuce.core.api.sync.RedisCommands
    io.lettuce.core.api.sync.RedisScriptingCommands
    io.lettuce.core.codec.RedisCodec
    io.lettuce.core.codec.StringCodec
+   io.lettuce.core.KeyValue
+   io.lettuce.core.pubsub.api.sync.RedisPubSubCommands
    io.lettuce.core.pubsub.RedisPubSubListener
    io.lettuce.core.pubsub.StatefulRedisPubSubConnection
-   io.lettuce.core.pubsub.api.sync.RedisPubSubCommands
+   io.lettuce.core.RedisClient
+   io.lettuce.core.RedisCommandInterruptedException
+   io.lettuce.core.RedisCommandTimeoutException
+   io.lettuce.core.RedisException
+   io.lettuce.core.RedisURI
    io.lettuce.core.resource.ClientResources
    io.lettuce.core.resource.DefaultClientResources
+   io.lettuce.core.ScriptOutputType
+   io.lettuce.core.SetArgs
    io.netty.channel.nio.NioEventLoopGroup
+   io.netty.util.concurrent.EventExecutorGroup
    io.netty.util.HashedWheelTimer
    io.netty.util.Timer
-   io.netty.util.concurrent.EventExecutorGroup
    java.lang.AutoCloseable
    java.time.Duration))
 

backend/src/app/rpc.clj

@@ -92,6 +92,7 @@
     (fn [{:keys [params path-params method] :as request}]
       (let [handler-name (:method-name path-params)
             etag         (yreq/get-header request "if-none-match")
+            session-id   (yreq/get-header request "x-session-id")
 
             key-id       (get request ::http/auth-key-id)
             profile-id   (or (::session/profile-id request)
@@ -104,6 +105,8 @@
                              (assoc ::handler-name handler-name)
                              (assoc ::ip-addr ip-addr)
                              (assoc ::request-at (ct/now))
+                             (assoc ::request-id (uuid/next))
+                             (assoc ::session-id (some-> session-id uuid/parse*))
                              (assoc ::cond/key etag)
                              (cond-> (uuid? profile-id)
                                (assoc ::profile-id profile-id)))
@@ -159,12 +162,13 @@
 (defn- wrap-audit
   [_ f mdata]
   (if (or (contains? cf/flags :webhooks)
-          (contains? cf/flags :audit-log))
+          (contains? cf/flags :audit-log)
+          (contains? cf/flags :telemetry))
     (if-not (::audit/skip mdata)
       (fn [cfg params]
         (let [result (f cfg params)]
-          (->> (audit/prepare-event cfg mdata params result)
-               (audit/submit! cfg))
+          (->> (audit/prepare-rpc-event cfg mdata params result)
+               (audit/submit cfg))
           result))
       f)
     f))

backend/src/app/rpc/commands/access_token.clj

@@ -23,7 +23,7 @@
   (dissoc row :perms))
 
 (defn create-access-token
-  [{:keys [::db/conn] :as cfg} profile-id name expiration]
+  [{:keys [::db/conn] :as cfg} profile-id name expiration type]
   (let [token-id   (uuid/next)
         expires-at (some-> expiration (ct/in-future))
         created-at (ct/now)
@@ -36,6 +36,7 @@
                                {:id token-id
                                 :name name
                                 :token token
+                                :type type
                                 :profile-id profile-id
                                 :created-at created-at
                                 :updated-at created-at
@@ -50,17 +51,18 @@
 (def ^:private schema:create-access-token
   [:map {:title "create-access-token"}
    [:name [:string {:max 250 :min 1}]]
-   [:expiration {:optional true} ::ct/duration]])
+   [:expiration {:optional true} ::ct/duration]
+   [:type {:optional true} :string]])
 
 (sv/defmethod ::create-access-token
   {::doc/added "1.18"
    ::sm/params schema:create-access-token}
-  [cfg {:keys [::rpc/profile-id name expiration]}]
+  [cfg {:keys [::rpc/profile-id name expiration type]}]
 
   (quotes/check! cfg {::quotes/id ::quotes/access-tokens-per-profile
                       ::quotes/profile-id profile-id})
 
-  (db/tx-run! cfg create-access-token profile-id name expiration))
+  (db/tx-run! cfg create-access-token profile-id name expiration type))
 
 (def ^:private schema:delete-access-token
   [:map {:title "delete-access-token"}
@@ -83,5 +85,22 @@
   (->> (db/query pool :access-token
                  {:profile-id profile-id}
                  {:order-by [[:expires-at :asc] [:created-at :asc]]
-                  :columns [:id :name :perms :created-at :updated-at :expires-at]})
+                  :columns [:id :name :perms :type :created-at :updated-at :expires-at]})
        (mapv decode-row)))
+
+(def ^:private schema:get-current-mcp-token
+  [:map {:title "get-current-mcp-token"}])
+
+(sv/defmethod ::get-current-mcp-token
+  {::doc/added "2.15"
+   ::sm/params schema:get-current-mcp-token}
+  [{:keys [::db/pool]} {:keys [::rpc/profile-id ::rpc/request-at]}]
+  (->> (db/query pool :access-token
+                 {:profile-id profile-id
+                  :type "mcp"}
+                 {:order-by [[:expires-at :asc] [:created-at :asc]]
+                  :columns [:token :expires-at]})
+       (remove #(and (some? (:expires-at %))
+                     (ct/is-after? request-at (:expires-at %))))
+       (map decode-row)
+       (first)))

backend/src/app/rpc/commands/audit.clj

@@ -15,7 +15,7 @@
    [app.config :as cf]
    [app.db :as db]
    [app.http :as-alias http]
-   [app.loggers.audit :as-alias audit]
+   [app.loggers.audit :as audit]
    [app.loggers.database :as loggers.db]
    [app.loggers.mattermost :as loggers.mm]
    [app.rpc :as-alias rpc]
@@ -23,7 +23,8 @@
    [app.rpc.doc :as-alias doc]
    [app.rpc.helpers :as rph]
    [app.util.inet :as inet]
-   [app.util.services :as sv]))
+   [app.util.services :as sv]
+   [clojure.set :as set]))
 
 (def ^:private event-columns
   [:id
@@ -38,31 +39,31 @@
    :context])
 
 (defn- event->row [event]
-  [(::audit/id event)
-   (::audit/name event)
-   (::audit/source event)
-   (::audit/type event)
-   (::audit/tracked-at event)
-   (::audit/created-at event)
-   (::audit/profile-id event)
-   (db/inet (::audit/ip-addr event))
-   (db/tjson (::audit/props event))
-   (db/tjson (d/without-nils (::audit/context event)))])
+  [(:id event)
+   (:name event)
+   (:source event)
+   (:type event)
+   (:tracked-at event)
+   (:created-at event)
+   (:profile-id event)
+   (db/inet (:ip-addr event))
+   (db/tjson (:props event))
+   (db/tjson (d/without-nils (:context event)))])
 
 (defn- adjust-timestamp
-  [{:keys [::audit/tracked-at ::audit/created-at] :as event}]
+  [{:keys [tracked-at created-at] :as event}]
   (let [margin (inst-ms (ct/diff tracked-at created-at))]
     (if (or (neg? margin)
             (> margin 3600000))
       ;; If event is in future or lags more than 1 hour, we reasign
       ;; tracked-at to the server creation date
       (-> event
-          (assoc ::audit/tracked-at created-at)
-          (update ::audit/context assoc :original-tracked-at tracked-at))
+          (assoc :tracked-at created-at)
+          (update :context assoc :original-tracked-at tracked-at))
       event)))
 
 (defn- exception-event?
-  [{:keys [::audit/type ::audit/name] :as ev}]
+  [{:keys [type name] :as ev}]
   (and (= "action" type)
        (or (= "unhandled-exception" name)
            (= "exception-page" name))))
@@ -72,28 +73,44 @@
    (map adjust-timestamp)
    (map event->row)))
 
-(defn- get-events
+(defn- prepare-events
   [{:keys [::rpc/request-at ::rpc/profile-id events] :as params}]
   (let [request (-> params meta ::http/request)
         ip-addr (inet/parse-request request)
-
-        xform   (map (fn [event]
-                       {::audit/id (uuid/next)
-                        ::audit/type (:type event)
-                        ::audit/name (:name event)
-                        ::audit/props (:props event)
-                        ::audit/context (:context event)
-                        ::audit/profile-id profile-id
-                        ::audit/ip-addr ip-addr
-                        ::audit/source "frontend"
-                        ::audit/tracked-at (:timestamp event)
-                        ::audit/created-at request-at}))]
+        xform   (comp
+                 (map (fn [event]
+                        {:id (uuid/next)
+                         :type (:type event)
+                         :name (:name event)
+                         :props (:props event)
+                         :context (:context event)
+                         :profile-id profile-id
+                         :ip-addr ip-addr
+                         :source "frontend"
+                         :tracked-at (:timestamp event)
+                         :created-at request-at}))
+                 (map (fn [item]
+                        (with-meta item {::audit/event true}))))]
 
     (sequence xform events)))
 
+(def ^:private xf:map-telemetry-event-row
+  (comp
+   (map adjust-timestamp)
+   (map (fn [event]
+          (-> event
+              (assoc :id (uuid/next))
+              (update :created-at ct/truncate :days)
+              (update :tracked-at ct/truncate :days)
+              (audit/filter-telemetry-props)
+              (audit/filter-telemetry-context)
+              (assoc :ip-addr "0.0.0.0")
+              (assoc :source "telemetry:frontend"))))
+   (map event->row)))
+
 (defn- handle-events
   [{:keys [::db/pool] :as cfg} params]
-  (let [events (get-events params)]
+  (let [events (prepare-events params)]
 
     ;; Look for error reports and save them on internal reports table
     (when-let [events (->> events
@@ -102,9 +119,18 @@
       (run! (partial loggers.db/emit cfg) events)
       (run! (partial loggers.mm/emit cfg) events))
 
-    ;; Process and save events
-    (when (seq events)
-      (let [rows (sequence xf:map-event-row events)]
+    (when (contains? cf/flags :audit-log)
+      ;; Process and save full audit events when audit-log flag is active
+      (when-let [rows (-> (sequence xf:map-event-row events)
+                          (not-empty))]
+        (db/insert-many! pool :audit-log event-columns rows)))
+
+    (when (contains? cf/flags :telemetry)
+      ;; Store anonymized frontend events so the telemetry task can ship them
+      ;; in batches. Runs independently from the audit-log insert above so
+      ;; both modes can be active simultaneously.
+      (when-let [rows (-> (sequence xf:map-telemetry-event-row events)
+                          (not-empty))]
         (db/insert-many! pool :audit-log event-columns rows)))))
 
 (def ^:private valid-event-types
@@ -138,17 +164,26 @@
    ::doc/skip true
    ::doc/added "1.17"}
   [{:keys [::db/pool] :as cfg} params]
-  (if (or (db/read-only? pool)
-          (not (contains? cf/flags :audit-log)))
-    (do
-      (l/warn :hint "audit: http handler disabled or db is read-only")
-      (rph/wrap nil))
-
-    (do
+  (let [telemetry? (contains? cf/flags :telemetry)
+        audit-log? (contains? cf/flags :audit-log)
+        enabled?   (and (not (db/read-only? pool))
+                        (or audit-log? telemetry?))]
+    (when enabled?
       (try
         (handle-events cfg params)
         (catch Throwable cause
           (l/error :hint "unexpected error on persisting audit events from frontend"
-                   :cause cause)))
+                   :cause cause))))
 
-      (rph/wrap nil))))
+    (rph/wrap nil)))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; GET-ENABLED-FLAGS
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(sv/defmethod ::get-enabled-flags
+  {::audit/skip true
+   ::doc/skip true
+   ::doc/added "1.20"}
+  [_cfg _params]
+  (set/intersection cf/flags #{:audit-log :telemetry}))

backend/src/app/rpc/commands/auth.clj

@@ -253,27 +253,49 @@
               :hint "email has complaint reports")))
 
 (defn prepare-register
-  [{:keys [::db/pool] :as cfg} {:keys [fullname email accept-newsletter-updates] :as params}]
+  [{:keys [::db/pool] :as cfg} {:keys [fullname email] :as params}]
 
   (validate-register-attempt! cfg params)
 
   (let [email   (profile/clean-email email)
-        profile (profile/get-profile-by-email pool email)
-        params  {:email email
-                 :fullname fullname
-                 :password (:password params)
-                 :invitation-token (:invitation-token params)
-                 :backend "penpot"
-                 :iss :prepared-register
-                 :profile-id (:id profile)
-                 :exp (ct/in-future {:days 7})
-                 :props {:newsletter-updates (or accept-newsletter-updates false)}}
+        profile (profile/get-profile-by-email pool email)]
 
-        params (d/without-nils params)
-        token  (tokens/generate cfg params)]
+    ;; SECURITY: refuse to issue a prepared-register token when an active
+    ;; profile already exists for this email.
+    ;;
+    ;; Active accounts must use the standard login flow; existing-but-
+    ;; not-yet-active profiles fall through to the duplicate-detection branch in
+    ;; `register-profile`, which never creates a session.
+    (when (and (some? profile)
+               (true? (:is-active profile)))
+      (ex/raise :type :validation
+                :code :email-already-exists
+                :hint "email already exists"))
 
-    (with-meta {:token token}
-      {::audit/profile-id uuid/zero})))
+    (let [props  (-> (audit/extract-utm-params params)
+                     (cond-> (:accept-newsletter-updates params)
+                       (assoc :newsletter-updates true)))
+          ;; SECURITY: do NOT embed `:profile-id` of an existing
+          ;; profile into the prepared-register JWE. Doing so would
+          ;; let an anonymous caller, in possession of a valid
+          ;; team-invitation JWE, ask `register-profile` to load that
+          ;; profile by id and mint a session for it without password
+          ;; verification. `register-profile` independently re-detects
+          ;; duplicates by email and handles them in the
+          ;; "repeated-registry" branch.
+          params {:email email
+                  :fullname fullname
+                  :password (:password params)
+                  :invitation-token (:invitation-token params)
+                  :backend "penpot"
+                  :iss :prepared-register
+                  :exp (ct/in-future {:days 7})
+                  :props props}
+          params (d/without-nils params)
+          token  (tokens/generate cfg params)]
+
+      (-> {:token token}
+          (with-meta {::audit/profile-id uuid/zero})))))
 
 (def schema:prepare-register-profile
   [:map {:title "prepare-register-profile"}
@@ -281,6 +303,7 @@
    [:email ::sm/email]
    [:password schema:password]
    [:create-welcome-file {:optional true} :boolean]
+   [:accept-newsletter-updates {:optional true} :boolean]
    [:invitation-token {:optional true} schema:token]])
 
 (sv/defmethod ::prepare-register-profile
@@ -317,8 +340,7 @@
   attrs (all the other attrs are filled with default values)."
   [{:keys [::db/conn] :as cfg} {:keys [email] :as params}]
   (let [id        (or (:id params) (uuid/next))
-        props     (-> (audit/extract-utm-params params)
-                      (merge (:props params))
+        props     (-> (:props params)
                       (merge {:viewed-tutorial? false
                               :viewed-walkthrough? false
                               :nudge {:big 10 :small 1}
@@ -369,7 +391,6 @@
                     :cause cause)
           (throw cause))))))
 
-
 (defn create-profile-rels
   [conn {:keys [id] :as profile}]
   (let [features (cfeat/get-enabled-features cf/flags)
@@ -386,48 +407,50 @@
         (profile/decode-row))))
 
 (defn send-email-verification!
-  [{:keys [::db/conn] :as cfg} profile]
-  (let [vtoken (tokens/generate cfg
-                                {:iss :verify-email
-                                 :exp (ct/in-future "72h")
-                                 :profile-id (:id profile)
-                                 :email (:email profile)})
-        ;; NOTE: this token is mainly used for possible complains
-        ;; identification on the sns webhook
-        ptoken (tokens/generate cfg
-                                {:iss :profile-identity
-                                 :profile-id (:id profile)
-                                 :exp (ct/in-future {:days 30})})]
-    (eml/send! {::eml/conn conn
-                ::eml/factory eml/register
-                :public-uri (cf/get :public-uri)
-                :to (:email profile)
-                :name (:fullname profile)
-                :token vtoken
-                :extra-data ptoken})))
+  ([cfg profile] (send-email-verification! cfg profile nil))
+  ([{:keys [::db/conn] :as cfg} profile invitation-token]
+   (let [vclaims (cond-> {:iss :verify-email
+                          :exp (ct/in-future "72h")
+                          :profile-id (:id profile)
+                          :email (:email profile)}
+                   ;; If the user registered through a team-invitation flow but
+                   ;; their profile is not yet active, we carry the invitation
+                   ;; token inside the verify-email JWE so the team-invitation
+                   ;; flow can resume after the user clicks the email link.
+                   (some? invitation-token)
+                   (assoc :invitation-token invitation-token))
+         vtoken  (tokens/generate cfg vclaims)
+         ;; NOTE: this token is mainly used for possible complains
+         ;; identification on the sns webhook
+         ptoken  (tokens/generate cfg
+                                  {:iss :profile-identity
+                                   :profile-id (:id profile)
+                                   :exp (ct/in-future {:days 30})})]
+     (eml/send! {::eml/conn conn
+                 ::eml/factory eml/register
+                 :public-uri (cf/get :public-uri)
+                 :to (:email profile)
+                 :name (:fullname profile)
+                 :token vtoken
+                 :extra-data ptoken}))))
 
 (defn register-profile
   [{:keys [::db/conn ::wrk/executor] :as cfg} {:keys [token] :as params}]
   (let [claims     (tokens/verify cfg {:token token :iss :prepared-register})
-        params     (into claims params)
+        params     (cond-> claims
+                     (:accept-newsletter-updates params)
+                     (update :props assoc :newsletter-updates true))
 
-        profile    (if-let [profile-id (:profile-id claims)]
-                     (profile/get-profile conn profile-id)
-                     ;; NOTE: we first try to match existing profile
-                     ;; by email, that in normal circumstances will
-                     ;; not return anything, but when a user tries to
-                     ;; reuse the same token multiple times, we need
-                     ;; to detect if the profile is already registered
-                     (or (profile/get-profile-by-email conn (:email claims))
-                         (let [is-active (or (boolean (:is-active claims))
-                                             (boolean (:email-verified claims))
-                                             (not (contains? cf/flags :email-verification)))
-                               params    (-> params
-                                             (assoc :is-active is-active)
-                                             (update :password auth/derive-password))
-                               profile   (->> (create-profile cfg params)
-                                              (create-profile-rels conn))]
-                           (vary-meta profile assoc :created true))))
+        profile    (or (profile/get-profile-by-email conn (:email claims))
+                       (let [is-active (or (boolean (:is-active claims))
+                                           (boolean (:email-verified claims))
+                                           (not (contains? cf/flags :email-verification)))
+                             params    (-> params
+                                           (assoc :is-active is-active)
+                                           (update :password auth/derive-password))
+                             profile   (->> (create-profile cfg params)
+                                            (create-profile-rels cfg))]
+                         (vary-meta profile assoc :created true)))
 
         created?   (-> profile meta :created true?)
 
@@ -443,6 +466,7 @@
           (when (:create-welcome-file params)
             (let [cfg (dissoc cfg ::db/conn)]
               (wrk/submit! executor (create-welcome-file cfg profile)))))]
+
     (cond
       ;; When profile is blocked, we just ignore it and return plain data
       (:is-blocked profile)
@@ -450,51 +474,74 @@
         (l/wrn :hint "register attempt for already blocked profile"
                :profile-id (str  (:id profile))
                :profile-email (:email profile))
-        (rph/with-meta {:email (:email profile)}
+        (rph/with-meta {:id (:id profile)
+                        :email (:email profile)}
           {::audit/replace-props props
            ::audit/context {:action "ignore-because-blocked"}
            ::audit/profile-id (:id profile)
            ::audit/name "register-profile-retry"}))
 
-      ;; If invitation token comes in params, this is because the user
-      ;; comes from team-invitation process; in this case, regenerate
-      ;; token and send back to the user a new invitation token (and
-      ;; mark current session as logged). This happens only if the
-      ;; invitation email matches with the register email.
-      (and (some? invitation)
-           (= (:email profile)
-              (:member-email invitation)))
-      (let [invitation (assoc invitation :member-id  (:id profile))
-            token      (tokens/generate cfg invitation)]
-        (-> {:invitation-token token}
-            (rph/with-transform (session/create-fn cfg profile claims))
-            (rph/with-meta {::audit/replace-props props
-                            ::audit/context {:action "accept-invitation"}
-                            ::audit/profile-id (:id profile)})))
-
-      ;; When a new user is created and it is already activated by
-      ;; configuration or specified by OIDC, we just mark the profile
-      ;; as logged-in
+      ;; A profile was just created in this call. Invitation handling is a
+      ;; sub-case of "newly created profile": we never honor invitations for
+      ;; pre-existing profiles via this anonymous RPC. The split below mirrors
+      ;; the non-invitation branches but threads the invitation through the
+      ;; appropriate path:
+      ;;
+      ;;   - active     + matching invitation → mint session and
+      ;;     return :invitation-token. The frontend redirects to
+      ;;     :auth-verify-token, which immediately accepts the
+      ;;     invitation.
+      ;;   - active     + no/mismatched invitation → mint session
+      ;;     ("login" action). New profile, no further action.
+      ;;   - not-active + matching invitation → send the
+      ;;     verify-email mail with the invitation token EMBEDDED
+      ;;     into the verify-email JWE. No session yet. When the
+      ;;     user clicks the link, verify-token activates the
+      ;;     profile, mints a session, and propagates the
+      ;;     invitation token to the frontend so it can complete
+      ;;     the team-invitation flow.
+      ;;   - not-active + no/mismatched invitation → standard
+      ;;     "check your email" verification flow.
       created?
-      (if (:is-active profile)
-        (-> (profile/strip-private-attrs profile)
-            (rph/with-transform (session/create-fn cfg profile claims))
-            (rph/with-defer create-welcome-file-when-needed)
-            (rph/with-meta
-              {::audit/replace-props props
-               ::audit/context {:action "login"}
-               ::audit/profile-id (:id profile)}))
+      (let [accept-invitation? (and (some? invitation)
+                                    (= (:email profile)
+                                       (:member-email invitation)))]
+        (cond
+          (and (:is-active profile) accept-invitation?)
+          (let [invitation (assoc invitation :member-id (:id profile))
+                token      (tokens/generate cfg invitation)]
+            (-> {:id (:id profile)
+                 :email (:email profile)
+                 :invitation-token token}
+                (rph/with-transform (session/create-fn cfg profile claims))
+                (rph/with-defer create-welcome-file-when-needed)
+                (rph/with-meta {::audit/replace-props props
+                                ::audit/context {:action "accept-invitation"}
+                                ::audit/profile-id (:id profile)})))
 
-        (do
-          (when-not (eml/has-reports? conn (:email profile))
-            (send-email-verification! cfg profile))
-
-          (-> {:email (:email profile)}
+          (:is-active profile)
+          (-> (profile/strip-private-attrs profile)
+              (rph/with-transform (session/create-fn cfg profile claims))
               (rph/with-defer create-welcome-file-when-needed)
               (rph/with-meta
                 {::audit/replace-props props
-                 ::audit/context {:action "email-verification"}
-                 ::audit/profile-id (:id profile)}))))
+                 ::audit/context {:action "login"}
+                 ::audit/profile-id (:id profile)}))
+
+          :else
+          (do
+            (when-not (eml/has-reports? conn (:email profile))
+              (send-email-verification! cfg profile
+                                        (when accept-invitation?
+                                          (:invitation-token params))))
+
+            (-> {:id (:id profile)
+                 :email (:email profile)}
+                (rph/with-defer create-welcome-file-when-needed)
+                (rph/with-meta
+                  {::audit/replace-props props
+                   ::audit/context {:action "email-verification"}
+                   ::audit/profile-id (:id profile)})))))
 
       :else
       (let [elapsed? (elapsed-verify-threshold? profile)
@@ -516,7 +563,8 @@
                       {:id (:id profile)})
           (send-email-verification! cfg profile))
 
-        (rph/with-meta {:email (:email profile)}
+        (rph/with-meta {:email (:email profile)
+                        :id (:id profile)}
           {::audit/replace-props (audit/profile->props profile)
            ::audit/context {:action action}
            ::audit/profile-id (:id profile)
@@ -524,7 +572,8 @@
 
 (def schema:register-profile
   [:map {:title "register-profile"}
-   [:token schema:token]])
+   [:token schema:token]
+   [:accept-newsletter-updates {:optional true} :boolean]])
 
 (sv/defmethod ::register-profile
   {::rpc/auth false

backend/src/app/rpc/commands/binfile.clj

@@ -22,6 +22,7 @@
    [app.media :as media]
    [app.rpc :as-alias rpc]
    [app.rpc.commands.files :as files]
+   [app.rpc.commands.media :as media-cmd]
    [app.rpc.commands.projects :as projects]
    [app.rpc.commands.teams :as teams]
    [app.rpc.doc :as-alias doc]
@@ -80,20 +81,33 @@
 ;; --- Command: import-binfile
 
 (defn- import-binfile
-  [{:keys [::db/pool] :as cfg} {:keys [profile-id project-id version name file]}]
-  (let [team   (teams/get-team pool
-                               :profile-id profile-id
-                               :project-id project-id)
-        cfg    (-> cfg
-                   (assoc ::bfc/features (cfeat/get-team-enabled-features cf/flags team))
-                   (assoc ::bfc/project-id project-id)
-                   (assoc ::bfc/profile-id profile-id)
-                   (assoc ::bfc/name name)
-                   (assoc ::bfc/input (:path file)))
+  [{:keys [::db/pool] :as cfg} {:keys [profile-id project-id version name file upload-id]}]
+  (let [team
+        (teams/get-team pool
+                        :profile-id profile-id
+                        :project-id project-id)
 
-        result (case (int version)
-                 1 (bf.v1/import-files! cfg)
-                 3 (bf.v3/import-files! cfg))]
+        cfg
+        (-> cfg
+            (assoc ::bfc/features (cfeat/get-team-enabled-features cf/flags team))
+            (assoc ::bfc/project-id project-id)
+            (assoc ::bfc/profile-id profile-id)
+            (assoc ::bfc/name name))
+
+        input-path (:path file)
+        owned?     (some? upload-id)
+
+        cfg
+        (assoc cfg ::bfc/input input-path)
+
+        result
+        (try
+          (case (int version)
+            1 (bf.v1/import-files! cfg)
+            3 (bf.v3/import-files! cfg))
+          (finally
+            (when owned?
+              (fs/delete input-path))))]
 
     (db/update! pool :project
                 {:modified-at (ct/now)}
@@ -103,13 +117,18 @@
     result))
 
 (def ^:private schema:import-binfile
-  [:map {:title "import-binfile"}
-   [:name [:or [:string {:max 250}]
-           [:map-of ::sm/uuid [:string {:max 250}]]]]
-   [:project-id ::sm/uuid]
-   [:file-id {:optional true} ::sm/uuid]
-   [:version {:optional true} ::sm/int]
-   [:file media/schema:upload]])
+  [:and
+   [:map {:title "import-binfile"}
+    [:name [:or [:string {:max 250}]
+            [:map-of ::sm/uuid [:string {:max 250}]]]]
+    [:project-id ::sm/uuid]
+    [:file-id {:optional true} ::sm/uuid]
+    [:version {:optional true} ::sm/int]
+    [:file {:optional true} media/schema:upload]
+    [:upload-id {:optional true} ::sm/uuid]]
+   [:fn {:error/message "one of :file or :upload-id is required"}
+    (fn [{:keys [file upload-id]}]
+      (or (some? file) (some? upload-id)))]])
 
 (sv/defmethod ::import-binfile
   "Import a penpot file in a binary format. If `file-id` is provided,
@@ -117,28 +136,40 @@
 
   The in-place imports are only supported for binfile-v3 and when a
   .penpot file only contains one penpot file.
+
+  The file content may be provided either as a multipart `file` upload
+  or as an `upload-id` referencing a completed chunked-upload session,
+  which allows importing files larger than the multipart size limit.
   "
   {::doc/added "1.15"
    ::doc/changes ["1.20" "Add file-id param for in-place import"
-                  "1.20" "Set default version to 3"]
+                  "1.20" "Set default version to 3"
+                  "2.15" "Add upload-id param for chunked upload support"]
 
    ::webhooks/event? true
    ::sse/stream? true
    ::sm/params schema:import-binfile}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id project-id version file-id file] :as params}]
+  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id project-id version file-id upload-id] :as params}]
   (projects/check-edition-permissions! pool profile-id project-id)
-  (let [version  (or version 3)
-        params   (-> params
-                     (assoc :profile-id profile-id)
-                     (assoc :version version))
+  (let [version (or version 3)
+        params  (-> params
+                    (assoc :profile-id profile-id)
+                    (assoc :version version))
 
-        cfg      (cond-> cfg
-                   (uuid? file-id)
-                   (assoc ::bfc/file-id file-id))
+        cfg     (cond-> cfg
+                  (uuid? file-id)
+                  (assoc ::bfc/file-id file-id))
 
-        manifest (case (int version)
-                   1 nil
-                   3 (bf.v3/get-manifest (:path file)))]
+        params
+        (if (some? upload-id)
+          (let [file (db/tx-run! cfg media-cmd/assemble-chunks upload-id)]
+            (assoc params :file file))
+          params)
+
+        manifest
+        (case (int version)
+          1 nil
+          3 (bf.v3/get-manifest (-> params :file :path)))]
 
     (with-meta
       (sse/response (partial import-binfile cfg params))

backend/src/app/rpc/commands/files_snapshot.clj

@@ -71,7 +71,7 @@
   {::doc/added "1.20"
    ::sm/params schema:restore-file-snapshot
    ::db/transaction true}
-  [{:keys [::db/conn ::mbus/msgbus] :as cfg} {:keys [::rpc/profile-id file-id id] :as params}]
+  [{:keys [::db/conn ::mbus/msgbus] :as cfg} {:keys [::rpc/profile-id ::rpc/session-id file-id id] :as params}]
   (files/check-edition-permissions! conn profile-id file-id)
   (let [file  (bfc/get-file cfg file-id)
         team  (teams/get-team conn
@@ -88,7 +88,8 @@
       ;; Send to the clients a notification to reload the file
       (mbus/pub! msgbus
                  :topic (:id file)
-                 :message {:type :file-restore
+                 :message {:type :file-restored
+                           :session-id session-id
                            :file-id (:id file)
                            :vern vern})
       nil)))

backend/src/app/rpc/commands/fonts.clj

@@ -9,6 +9,8 @@
    [app.binfile.common :as bfc]
    [app.common.data.macros :as dm]
    [app.common.exceptions :as ex]
+   [app.common.logging :as l]
+   [app.common.media :as cm]
    [app.common.schema :as sm]
    [app.common.time :as ct]
    [app.common.uuid :as uuid]
@@ -21,6 +23,7 @@
    [app.rpc :as-alias rpc]
    [app.rpc.climit :as-alias climit]
    [app.rpc.commands.files :as files]
+   [app.rpc.commands.media :as cmedia]
    [app.rpc.commands.projects :as projects]
    [app.rpc.commands.teams :as teams]
    [app.rpc.doc :as-alias doc]
@@ -29,6 +32,8 @@
    [app.storage :as sto]
    [app.storage.tmp :as tmp]
    [app.util.services :as sv]
+   [cuerdas.core :as str]
+   [datoteka.fs :as fs]
    [datoteka.io :as io])
   (:import
    java.io.InputStream
@@ -87,32 +92,92 @@
 (declare create-font-variant)
 
 (def ^:private schema:create-font-variant
-  [:map {:title "create-font-variant"}
-   [:team-id ::sm/uuid]
-   [:data [:map-of ::sm/text [:or ::sm/bytes
-                              [::sm/vec ::sm/bytes]]]]
-   [:font-id ::sm/uuid]
-   [:font-family ::sm/text]
-   [:font-weight [::sm/one-of {:format "number"} valid-weight]]
-   [:font-style [::sm/one-of {:format "string"} valid-style]]])
+  [:and
+   [:map {:title "create-font-variant"}
+    [:team-id    ::sm/uuid]
+    [:font-id    ::sm/uuid]
+    [:font-family ::sm/text]
+    [:font-weight [::sm/one-of {:format "number"} valid-weight]]
+    [:font-style  [::sm/one-of {:format "string"} valid-style]]
+    [:data    {:optional true} [:map-of ::sm/text [:or ::sm/bytes [::sm/vec ::sm/bytes]]]]
+    [:uploads {:optional true} [:map-of ::sm/text ::sm/uuid]]]
+   [:fn {:error/message "one of :data or :uploads is required"}
+    (fn [{:keys [data uploads]}]
+      (or (seq data) (seq uploads)))]])
 
 ;; FIXME: IMPORTANT: refactor this, we should not hold a whole db
 ;; connection around the font creation
 
+(defn- prepare-font-data-from-uploads
+  "Assembles each chunked-upload session in `uploads` (a `{mtype →
+  session-id}` map) into a temp file, validates the media type and
+  size of every entry, and returns a `{mtype → path}` data map."
+  [cfg {:keys [uploads] :as params}]
+  (let [data (reduce-kv
+              (fn [acc mtype session-id]
+                (let [assembled (cmedia/assemble-chunks cfg session-id)]
+                  (-> {:mtype mtype :size (:size assembled)}
+                      (media/validate-media-type! cm/font-types)
+                      (media/validate-font-size!))
+                  (assoc acc mtype (:path assembled))))
+              {}
+              uploads)]
+
+    (-> params
+        (assoc :data data)
+        (dissoc :uploads))))
+
+(defn- prepare-font-data-from-legacy
+  "Validates the media type and size of every entry in the legacy
+  `:data` map (a `{mtype → bytes | [bytes]}` map). Normalises every
+  entry to a tempfile. Returns params with a normalised
+  `{mtype → path}` data map."
+  [{:keys [data] :as params}]
+  (let [data (reduce-kv
+              (fn [acc mtype content]
+                (let [tmp     (tmp/tempfile :prefix "penpot.tempfont." :suffix "")
+                      chunks  (if (vector? content) content [content])
+                      streams (map io/input-stream chunks)
+                      streams (Collections/enumeration streams)]
+
+                  ;; Generate the tempfile from all chunks
+                  (with-open [^OutputStream output (io/output-stream tmp)
+                              ^InputStream input (SequenceInputStream. streams)]
+                    (io/copy input output))
+
+                  ;; Validate
+                  (-> {:mtype mtype :size (fs/size tmp)}
+                      (media/validate-media-type! cm/font-types)
+                      (media/validate-font-size!))
+
+                  (assoc acc mtype tmp)))
+              {}
+              data)]
+    (assoc params :data data)))
+
 (sv/defmethod ::create-font-variant
+  "Upload a font variant.  Font data may be provided either as a
+  Transit-encoded `:data` map (keyed by mime-type) for small fonts, or
+  as an `:uploads` map (keyed by mime-type, values are upload-session
+  UUIDs from the chunked-upload API) for large fonts.  Exactly one of
+  the two must be present."
   {::doc/added "1.18"
+   ::doc/changes ["2.16" "Add :uploads param for chunked upload support"]
    ::climit/id [[:process-font/by-profile ::rpc/profile-id]
                 [:process-font/global]]
    ::webhooks/event? true
    ::sm/params schema:create-font-variant}
-  [cfg {:keys [::rpc/profile-id team-id] :as params}]
+  [cfg {:keys [::rpc/profile-id team-id uploads] :as params}]
   (db/tx-run! cfg
               (fn [{:keys [::db/conn] :as cfg}]
                 (teams/check-edition-permissions! conn profile-id team-id)
                 (quotes/check! cfg {::quotes/id ::quotes/font-variants-per-team
                                     ::quotes/profile-id profile-id
                                     ::quotes/team-id team-id})
-                (create-font-variant cfg (assoc params :profile-id profile-id)))))
+                (let [params (if (some? uploads)
+                               (prepare-font-data-from-uploads cfg params)
+                               (prepare-font-data-from-legacy params))]
+                  (create-font-variant cfg (assoc params :profile-id profile-id))))))
 
 (defn create-font-variant
   [{:keys [::sto/storage ::db/conn]} {:keys [data] :as params}]
@@ -127,23 +192,6 @@
                           :hint "invalid font upload, unable to generate missing font assets"))
               data))
 
-          (process-chunks [chunks]
-            (let [tmp     (tmp/tempfile :prefix "penpot.tempfont." :suffix "")
-                  streams (map io/input-stream chunks)
-                  streams (Collections/enumeration streams)]
-              (with-open [^OutputStream output (io/output-stream tmp)
-                          ^InputStream input (SequenceInputStream. streams)]
-                (io/copy input output))
-              tmp))
-
-          (join-chunks [data]
-            (reduce-kv (fn [data mtype content]
-                         (if (vector? content)
-                           (assoc data mtype (process-chunks content))
-                           data))
-                       data
-                       data))
-
           (prepare-font [data mtype]
             (when-let [resource (get data mtype)]
 
@@ -185,11 +233,38 @@
                          :otf-file-id (:id otf)
                          :ttf-file-id (:id ttf)}))]
 
-    (let [data   (join-chunks data)
-          data   (generate-missing data)
-          assets (persist-fonts-files! data)
-          result (insert-font-variant! assets)]
-      (vary-meta result assoc ::audit/replace-props (update params :data (comp vec keys))))))
+    (let [tpoint      (ct/tpoint)
+          mtypes      (vec (keys data))
+          total-size  (reduce-kv (fn [acc _ content]
+                                   (+ acc (if (bytes? content)
+                                            (alength ^bytes content)
+                                            (fs/size content))))
+                                 0
+                                 data)]
+
+      (l/dbg :hint "create-font-variant"
+             :step "init"
+             :font-family (:font-family params)
+             :font-weight (:font-weight params)
+             :font-style  (:font-style params)
+             :mtypes      (str/join mtypes ",")
+             :size        total-size)
+
+      (let [data    (generate-missing data)
+            assets  (persist-fonts-files! data)
+            result  (insert-font-variant! assets)
+            elapsed (tpoint)]
+
+        (l/dbg :hint "create-font-variant"
+               :step "end"
+               :font-family (:font-family params)
+               :font-weight (:font-weight params)
+               :font-style  (:font-style params)
+               :mtypes      (str/join mtypes ",")
+               :size        total-size
+               :elapsed     (ct/format-duration elapsed))
+
+        (vary-meta result assoc ::audit/replace-props (update params :data (comp vec keys)))))))
 
 ;; --- UPDATE FONT FAMILY
 

backend/src/app/rpc/commands/management.clj

@@ -425,10 +425,10 @@
                     (doseq [file-id result]
                       (let [props (assoc props :id file-id)
                             event (-> (audit/event-from-rpc-params params)
-                                      (assoc ::audit/profile-id profile-id)
-                                      (assoc ::audit/name "create-file")
-                                      (assoc ::audit/props props))]
-                        (audit/submit! cfg event))))))
+                                      (assoc :profile-id profile-id)
+                                      (assoc :name "create-file")
+                                      (assoc :props props))]
+                        (audit/submit cfg event))))))
 
     result))
 

backend/src/app/rpc/commands/media.clj

@@ -7,9 +7,12 @@
 (ns app.rpc.commands.media
   (:require
    [app.common.data :as d]
+   [app.common.exceptions :as ex]
+   [app.common.logging :as l]
    [app.common.schema :as sm]
    [app.common.time :as ct]
    [app.common.uuid :as uuid]
+   [app.config :as cf]
    [app.db :as db]
    [app.loggers.audit :as-alias audit]
    [app.media :as media]
@@ -17,8 +20,13 @@
    [app.rpc.climit :as climit]
    [app.rpc.commands.files :as files]
    [app.rpc.doc :as-alias doc]
+   [app.rpc.quotes :as quotes]
    [app.storage :as sto]
-   [app.util.services :as sv]))
+   [app.storage.tmp :as tmp]
+   [app.util.services :as sv]
+   [datoteka.io :as io])
+  (:import
+   java.io.OutputStream))
 
 (def thumbnail-options
   {:width 100
@@ -51,8 +59,8 @@
   (db/run! cfg (fn [{:keys [::db/conn] :as cfg}]
                  ;; We get the minimal file for proper checking if
                  ;; file is not already deleted
-                 (let [_     (files/get-minimal-file conn file-id)
-                       mobj  (create-file-media-object cfg params)]
+                 (let [_      (files/get-minimal-file conn file-id)
+                       mobj   (create-file-media-object cfg params)]
 
                    (db/update! conn :file
                                {:modified-at (ct/now)
@@ -142,20 +150,49 @@
 
 (defn- create-file-media-object
   [{:keys [::sto/storage ::db/conn] :as cfg}
-   {:keys [id file-id is-local name content]}]
-  (let [result (process-image content)
-        image  (sto/put-object! storage (::image result))
-        thumb  (when-let [params (::thumb result)]
-                 (sto/put-object! storage params))]
+   {:keys [id file-id is-local name content from-url? from-chunks?]}]
 
-    (db/exec-one! conn [sql:create-file-media-object
-                        (or id (uuid/next))
-                        file-id is-local name
-                        (:id image)
-                        (:id thumb)
-                        (:width result)
-                        (:height result)
-                        (:mtype result)])))
+  (let [tpoint (ct/tpoint)
+        id     (or id (uuid/next))
+        origin (cond
+                 from-url?
+                 "url"
+                 from-chunks?
+                 "chunks"
+                 :else
+                 "direct")]
+
+    (l/dbg :hint "create file-media-object"
+           :step "init"
+           :id (str id)
+           :mtype (:mtype content)
+           :size (:size content)
+           :path (str (:path content))
+           :origin origin)
+
+    (let [result  (process-image content)
+          image   (sto/put-object! storage (::image result))
+          thumb   (when-let [params (::thumb result)]
+                    (sto/put-object! storage params))
+          elapsed (tpoint)]
+
+      (l/dbg :hint "create file-media-object"
+             :step "end"
+             :id (str id)
+             :mtype (:mtype content)
+             :size (:size content)
+             :path (str (:path content))
+             :origin origin
+             :elapsed (ct/format-duration elapsed))
+
+      (db/exec-one! conn [sql:create-file-media-object
+                          id
+                          file-id is-local name
+                          (:id image)
+                          (:id thumb)
+                          (:width result)
+                          (:height result)
+                          (:mtype result)]))))
 
 ;; --- Create File Media Object (from URL)
 
@@ -191,6 +228,7 @@
   [cfg {:keys [url name] :as params}]
   (let [content (media/download-image cfg url)
         params  (-> params
+                    (assoc :from-url? true)
                     (assoc :content content)
                     (assoc :name (d/nilv name "unknown")))]
 
@@ -236,3 +274,190 @@
                  :width  (:width mobj)
                  :height (:height mobj)
                  :mtype  (:mtype mobj)})))
+
+;; --- Chunked Upload: Create an upload session
+
+(def ^:private schema:create-upload-session
+  [:map {:title "create-upload-session"}
+   [:total-chunks ::sm/int]])
+
+(def ^:private schema:create-upload-session-result
+  [:map {:title "create-upload-session-result"}
+   [:session-id ::sm/uuid]])
+
+(sv/defmethod ::create-upload-session
+  {::doc/added "2.16"
+   ::sm/params schema:create-upload-session
+   ::sm/result schema:create-upload-session-result}
+  [{:keys [::db/pool] :as cfg}
+   {:keys [::rpc/profile-id total-chunks]}]
+
+  (let [max-chunks (cf/get :quotes-upload-chunks-per-session)]
+    (when (> total-chunks max-chunks)
+      (ex/raise :type :restriction
+                :code :max-quote-reached
+                :target "upload-chunks-per-session"
+                :quote max-chunks
+                :count total-chunks)))
+
+  (quotes/check! cfg {::quotes/id ::quotes/upload-sessions-per-profile
+                      ::quotes/profile-id profile-id})
+
+  (let [session-id (uuid/next)]
+    (db/insert! pool :upload-session
+                {:id           session-id
+                 :profile-id   profile-id
+                 :total-chunks total-chunks})
+    {:session-id session-id}))
+
+;; --- Chunked Upload: Upload a single chunk
+
+(def ^:private schema:upload-chunk
+  [:map {:title "upload-chunk"}
+   [:session-id ::sm/uuid]
+   [:index      ::sm/int]
+   [:content    media/schema:upload]])
+
+(def ^:private schema:upload-chunk-result
+  [:map {:title "upload-chunk-result"}
+   [:session-id ::sm/uuid]
+   [:index      ::sm/int]])
+
+(sv/defmethod ::upload-chunk
+  {::doc/added "2.16"
+   ::sm/params schema:upload-chunk
+   ::sm/result schema:upload-chunk-result}
+  [{:keys [::db/pool] :as cfg}
+   {:keys [::rpc/profile-id session-id index content] :as _params}]
+  (let [session (db/get pool :upload-session {:id session-id :profile-id profile-id})]
+    (when (or (neg? index) (>= index (:total-chunks session)))
+      (ex/raise :type :validation
+                :code :invalid-chunk-index
+                :hint "chunk index is out of range for this session"
+                :session-id session-id
+                :total-chunks (:total-chunks session)
+                :index index))
+
+
+    (l/trc :hint "upload-chunk"
+           :session-id session-id
+           :chunk (str index "/" (:total-chunks session))
+           :size (:size content)
+           :path (:path content)))
+
+  (let [storage (sto/resolve cfg)
+        data    (sto/content (:path content))]
+    (sto/put-object! storage
+                     {::sto/content      data
+                      ::sto/deduplicate? false
+                      ::sto/touch        true
+                      :content-type      (:mtype content)
+                      :bucket            "tempfile"
+                      :upload-id         (str session-id)
+                      :chunk-index       index}))
+
+  {:session-id session-id
+   :index      index})
+
+;; --- Chunked Upload: shared helpers
+
+(def ^:private sql:get-upload-chunks
+  "SELECT id, size, (metadata->>'~:chunk-index')::integer AS chunk_index
+     FROM storage_object
+    WHERE (metadata->>'~:upload-id') = ?::text
+      AND deleted_at IS NULL
+    ORDER BY (metadata->>'~:chunk-index')::integer ASC")
+
+(defn- get-upload-chunks
+  [conn session-id]
+  (db/exec! conn [sql:get-upload-chunks (str session-id)]))
+
+(defn- concat-chunks
+  "Reads all chunk storage objects in order and writes them to a single
+  temporary file on the local filesystem. Returns a path to that file."
+  [storage chunks]
+  (let [tmp (tmp/tempfile :prefix "penpot.chunked-upload.")]
+    (with-open [^OutputStream out (io/output-stream tmp)]
+      (doseq [{:keys [id]} chunks]
+        (let [sobj  (sto/get-object storage id)
+              bytes (sto/get-object-bytes storage sobj)]
+          (.write out ^bytes bytes))))
+    tmp))
+
+(defn assemble-chunks
+  "Validates that all expected chunks are present for `session-id` and
+  concatenates them into a single temporary file.  Returns a map
+  conforming to `media/schema:upload` with `:filename`, `:path` and
+  `:size`.
+
+  Raises a :validation/:missing-chunks error when the number of stored
+  chunks does not match `:total-chunks` recorded in the session row.
+  Deletes the session row from `upload_session` on success."
+  [{:keys [::db/conn] :as cfg} session-id]
+  (let [session (db/get conn :upload-session {:id session-id})
+        chunks  (get-upload-chunks conn session-id)]
+
+    (when (not= (count chunks) (:total-chunks session))
+      (ex/raise :type :validation
+                :code :missing-chunks
+                :hint "number of stored chunks does not match expected total"
+                :session-id session-id
+                :expected   (:total-chunks session)
+                :found      (count chunks)))
+
+    (let [storage (sto/resolve cfg ::db/reuse-conn true)
+          path    (concat-chunks storage chunks)
+          size    (reduce #(+ %1 (:size %2)) 0 chunks)]
+
+      (db/delete! conn :upload-session {:id session-id})
+
+      {:filename "upload"
+       :path     path
+       :size     size})))
+
+;; --- Chunked Upload: Assemble all chunks into a final media object
+
+(def ^:private schema:assemble-file-media-object
+  [:map {:title "assemble-file-media-object"}
+   [:session-id ::sm/uuid]
+   [:file-id    ::sm/uuid]
+   [:is-local   ::sm/boolean]
+   [:name       [:string {:max 250}]]
+   [:mtype      :string]
+   [:id         {:optional true} ::sm/uuid]])
+
+(sv/defmethod ::assemble-file-media-object
+  {::doc/added "2.16"
+   ::sm/params schema:assemble-file-media-object
+   ::climit/id [[:process-image/by-profile ::rpc/profile-id]
+                [:process-image/global]]}
+  [{:keys [::db/pool] :as cfg}
+   {:keys [::rpc/profile-id session-id file-id is-local name mtype id] :as params}]
+  (files/check-edition-permissions! pool profile-id file-id)
+
+  (db/tx-run! cfg
+              (fn [{:keys [::db/conn] :as cfg}]
+                (let [content (assemble-chunks cfg session-id)
+                      content (-> content
+                                  (assoc :filename (str "upload:" name))
+                                  (assoc :mtype mtype)
+                                  (media/validate-media-type!)
+                                  (media/validate-media-size!))
+                      mobj    (create-file-media-object cfg (assoc params
+                                                                   :id id
+                                                                   :from-chunks? true
+                                                                   :content content))]
+
+                  (db/update! conn :file
+                              {:modified-at      (ct/now)
+                               :has-media-trimmed false}
+                              {:id file-id}
+                              {::db/return-keys false})
+
+                  (with-meta mobj
+                    {::audit/replace-props
+                     {:name      name
+                      :file-id   file-id
+                      :is-local  is-local
+                      :mtype     mtype}})))))
+

backend/src/app/rpc/commands/profile.clj

@@ -48,6 +48,7 @@
 (def schema:props
   [:map {:title "ProfileProps"}
    [:plugins {:optional true} schema:plugin-registry]
+   [:mcp-enabled {:optional true} ::sm/boolean]
    [:newsletter-updates {:optional true} ::sm/boolean]
    [:newsletter-news {:optional true} ::sm/boolean]
    [:onboarding-team-id {:optional true} ::sm/uuid]
@@ -263,6 +264,7 @@
   [cfg {:keys [::rpc/profile-id file] :as params}]
   ;; Validate incoming mime type
   (media/validate-media-type! file #{"image/jpeg" "image/png" "image/webp"})
+  (media/validate-media-size! file)
   (update-profile-photo cfg (assoc params :profile-id profile-id)))
 
 (defn update-profile-photo

backend/src/app/rpc/commands/teams.clj

@@ -497,7 +497,7 @@
 
 (def ^:private schema:create-team
   [:map {:title "create-team"}
-   [:name [:string {:max 250}]]
+   [:name types.team/schema:team-name]
    [:features {:optional true} ::cfeat/features]
    [:id {:optional true} ::sm/uuid]])
 
@@ -591,7 +591,7 @@
 
 (def ^:private schema:update-team
   [:map {:title "update-team"}
-   [:name [:string {:max 250}]]
+   [:name types.team/schema:team-name]
    [:id ::sm/uuid]])
 
 (sv/defmethod ::update-team
@@ -827,6 +827,7 @@
   ;; Validate incoming mime type
 
   (media/validate-media-type! file #{"image/jpeg" "image/png" "image/webp"})
+  (media/validate-media-size! file)
   (update-team-photo cfg (assoc params :profile-id profile-id)))
 
 (defn update-team-photo

backend/src/app/rpc/commands/teams_invitations.clj

@@ -19,6 +19,7 @@
    [app.config :as cf]
    [app.db :as db]
    [app.email :as eml]
+   [app.email.blacklist :as email.blacklist]
    [app.loggers.audit :as audit]
    [app.main :as-alias main]
    [app.rpc :as-alias rpc]
@@ -91,6 +92,12 @@
   (let [email  (profile/clean-email email)
         member (profile/get-profile-by-email conn email)]
 
+    (when (and (email.blacklist/enabled? cfg)
+               (email.blacklist/contains? cfg email))
+      (ex/raise :type :restriction
+                :code :email-domain-is-not-allowed
+                :hint "email domain is in the blacklist"))
+
     ;; When we have email verification disabled and invitation user is
     ;; already present in the database, we proceed to add it to the
     ;; team as-is, without email roundtrip.
@@ -149,9 +156,9 @@
                          "update-team-invitation"
                          "create-team-invitation")
                 event (-> (audit/event-from-rpc-params params)
-                          (assoc ::audit/name evname)
-                          (assoc ::audit/props props))]
-            (audit/submit! cfg event))
+                          (assoc :name evname)
+                          (assoc :props props))]
+            (audit/submit cfg event))
 
           (when (allow-invitation-emails? member)
             (eml/send! {::eml/conn conn
@@ -403,9 +410,9 @@
 
     (let [props {:name name :features features}
           event (-> (audit/event-from-rpc-params params)
-                    (assoc ::audit/name "create-team")
-                    (assoc ::audit/props props))]
-      (audit/submit! cfg event))
+                    (assoc :name "create-team")
+                    (assoc :props props))]
+      (audit/submit cfg event))
 
     ;; Create invitations for all provided emails.
     (let [profile     (db/get-by-id conn :profile profile-id)

backend/src/app/rpc/commands/verify_token.clj

@@ -72,6 +72,11 @@
                   {:is-active true}
                   {:id (:id profile)}))
 
+    ;; NOTE: `claims` is returned verbatim (besides :profile). When the
+    ;; verify-email JWE was minted by `register-profile` for a not-yet-
+    ;; active profile that came from an invitation flow, `:invitation-
+    ;; token` will be present here and the frontend will use it to
+    ;; complete the team-invitation flow after login.
     (-> claims
         (rph/with-transform (session/create-fn cfg profile))
         (rph/with-meta {::audit/name "verify-profile-email"
@@ -168,24 +173,22 @@
                      :role (:role claims)
                      :invitation-id (:id invitation)}]
 
-          (audit/submit!
-           cfg
-           (-> (audit/event-from-rpc-params params)
-               (assoc ::audit/name "accept-team-invitation")
-               (assoc ::audit/props props)))
+          (audit/submit cfg
+                        (-> (audit/event-from-rpc-params params)
+                            (assoc :name "accept-team-invitation")
+                            (assoc :props props)))
 
           ;; NOTE: Backward compatibility; old invitations can
           ;; have the `created-by` to be nil; so in this case we
           ;; don't submit this event to the audit-log
           (when-let [created-by (:created-by invitation)]
-            (audit/submit!
-             cfg
-             (-> (audit/event-from-rpc-params params)
-                 (assoc ::audit/profile-id created-by)
-                 (assoc ::audit/name "accept-team-invitation-from")
-                 (assoc ::audit/props (assoc props
-                                             :profile-id (:id profile)
-                                             :email (:email profile))))))
+            (audit/submit cfg
+                          (-> (audit/event-from-rpc-params params)
+                              (assoc :profile-id created-by)
+                              (assoc :name "accept-team-invitation-from")
+                              (assoc :props (assoc props
+                                                   :profile-id (:id profile)
+                                                   :email (:email profile))))))
 
           (accept-invitation cfg claims invitation profile)
           (assoc claims :state :created))

backend/src/app/rpc/commands/webhooks.clj

@@ -50,24 +50,27 @@
 (defn- validate-webhook!
   [cfg whook params]
   (when (not= (:uri whook) (:uri params))
-    (let [response (ex/try!
-                    (http/req! cfg
+    (try
+      (let [response (http/req cfg
                                {:method :head
                                 :uri (str (:uri params))
-                                :timeout (ct/duration "3s")}
-                               {:sync? true}))]
-      (if (ex/exception? response)
-        (if-let [hint (webhooks/interpret-exception response)]
-          (ex/raise :type :validation
-                    :code :webhook-validation
-                    :hint hint)
-          (ex/raise :type :internal
-                    :code :webhook-validation
-                    :cause response))
+                                :timeout (ct/duration "3s")})]
         (when-let [hint (webhooks/interpret-response response)]
           (ex/raise :type :validation
                     :code :webhook-validation
-                    :hint hint))))))
+                    :hint hint)))
+
+      (catch Throwable cause
+        (if-let [hint (webhooks/interpret-exception cause)]
+          (ex/raise :type :validation
+                    :code :webhook-validation
+                    :hint hint
+                    :webhook-uri (str (:uri params))
+                    :cause cause)
+          (ex/raise :type :internal
+                    :code :webhook-validation
+                    :webhook-uri (str (:uri params))
+                    :cause cause))))))
 
 (defn- validate-quotes!
   [{:keys [::db/pool]} {:keys [team-id]}]

backend/src/app/rpc/quotes.clj

@@ -522,6 +522,30 @@
       (assoc ::count-sql [sql:get-team-access-requests-per-requester profile-id])
       (generic-check!)))
 
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; QUOTE: UPLOAD-SESSIONS-PER-PROFILE
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(def ^:private schema:upload-sessions-per-profile
+  [:map [::profile-id ::sm/uuid]])
+
+(def ^:private valid-upload-sessions-per-profile-quote?
+  (sm/lazy-validator schema:upload-sessions-per-profile))
+
+(def ^:private sql:get-upload-sessions-per-profile
+  "SELECT count(*) AS total
+     FROM upload_session
+    WHERE profile_id = ?")
+
+(defmethod check-quote ::upload-sessions-per-profile
+  [{:keys [::profile-id ::target] :as quote}]
+  (assert (valid-upload-sessions-per-profile-quote? quote) "invalid quote parameters")
+  (-> quote
+      (assoc ::default (cf/get :quotes-upload-sessions-per-profile Integer/MAX_VALUE))
+      (assoc ::quote-sql [sql:get-quotes-1 target profile-id])
+      (assoc ::count-sql [sql:get-upload-sessions-per-profile profile-id])
+      (generic-check!)))
+
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 ;; QUOTE: DEFAULT
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

backend/src/app/setup.clj

@@ -11,7 +11,9 @@
    [app.common.logging :as l]
    [app.common.schema :as sm]
    [app.common.uuid :as uuid]
+   [app.config :as cf]
    [app.db :as db]
+   [app.loggers.audit :as audit]
    [app.main :as-alias main]
    [app.setup.keys :as keys]
    [app.setup.templates]
@@ -35,22 +37,20 @@
        (into {})))
 
 (defn- handle-instance-id
-  [instance-id conn read-only?]
+  [instance-id conn]
   (or instance-id
       (let [instance-id (uuid/random)]
-        (when-not read-only?
-          (try
-            (db/insert! conn :server-prop
-                        {:id "instance-id"
-                         :preload true
-                         :content (db/tjson instance-id)})
-            (catch Throwable cause
-              (l/warn :hint "unable to persist instance-id"
-                      :instance-id instance-id
-                      :cause cause))))
+        (try
+          (db/insert! conn :server-prop
+                      {:id "instance-id"
+                       :preload true
+                       :content (db/tjson instance-id)})
+          (catch Throwable cause
+            (l/warn :hint "unable to persist instance-id"
+                    :instance-id instance-id
+                    :cause cause)))
         instance-id)))
 
-
 (def sql:add-prop
   "INSERT INTO server_prop (id, content, preload)
    VALUES (?, ?, ?)
@@ -77,50 +77,47 @@
   (assert (db/pool? (::db/pool params)) "expected valid database pool"))
 
 (defmethod ig/init-key ::props
-  [_ {:keys [::db/pool ::key] :as cfg}]
+  [_ {:keys [::key] :as cfg}]
+  (audit/submit cfg {:type "trigger"
+                     :name "instance-start"
+                     :props {:version (:full cf/version)
+                             :flags (mapv name cf/flags)
+                             :public-uri (str (cf/get :public-uri))}})
 
   (db/tx-run! cfg (fn [{:keys [::db/conn]}]
                     (db/xact-lock! conn 0)
                     (when-not key
-                      (l/warn :hint (str "using autogenerated secret-key, it will change on each restart and will invalidate "
-                                         "all sessions on each restart, it is highly recommended setting up the "
-                                         "PENPOT_SECRET_KEY environment variable")))
+                      (l/wrn :hint (str "using autogenerated secret-key, it will change "
+                                        "on each restart and will invalidate "
+                                        "all sessions on each restart, it is highly "
+                                        "recommended setting up the "
+                                        "PENPOT_SECRET_KEY environment variable")))
                     (let [secret (or key (generate-random-key))]
                       (-> (get-all-props conn)
                           (assoc :secret-key secret)
                           (assoc :tokens-key (keys/derive secret :salt "tokens"))
-                          (update :instance-id handle-instance-id conn (db/read-only? pool)))))))
-
-(sm/register! ::props [:map-of :keyword ::sm/any])
-
+                          (update :instance-id handle-instance-id conn))))))
 
 (defmethod ig/init-key ::shared-keys
   [_ {:keys [::props] :as cfg}]
   (let [secret (get props :secret-key)]
-    (d/without-nils
-     {:exporter
-      (let [key (or (get cfg :exporter)
-                    (-> (keys/derive secret :salt "exporter")
-                        (bc/bytes->b64-str true)))]
-        (if (or (str/empty? key)
-                (str/blank? key))
-          (do
-            (l/wrn :hint "exporter key is disabled because empty string found")
-            nil)
-          (do
-            (l/inf :hint "exporter key initialized" :key (d/obfuscate-string key))
-            key)))
+    (reduce (fn [keys id]
+              (let [key (or (get cfg id)
+                            (-> (keys/derive secret :salt (name id))
+                                (bc/bytes->b64-str true)))]
+                (if (or (str/empty? key)
+                        (str/blank? key))
+                  (do
+                    (l/wrn :id (name id) :hint "key is disabled because empty string found")
+                    keys)
+                  (do
+                    (l/inf :id (name id) :hint "key initialized" :key (d/obfuscate-string key))
+                    (assoc keys id key)))))
+            {}
+            [:exporter
+             :nitrate
+             :nexus])))
 
-      :nitrate
-      (let [key (or (get cfg :nitrate)
-                    (-> (keys/derive secret :salt "nitrate")
-                        (bc/bytes->b64-str true)))]
-        (if (or (str/empty? key)
-                (str/blank? key))
-          (do
-            (l/wrn :hint "nitrate key is disabled because empty string found")
-            nil)
-          (do
-            (l/inf :hint "nitrate key initialized" :key (d/obfuscate-string key))
-            key)))})))
+(sm/register! ::props [:map-of :keyword ::sm/any])
+(sm/register! ::shared-keys [:map-of :keyword ::sm/text])
 

backend/src/app/setup/templates.clj

@@ -57,9 +57,9 @@
 
       (if (fs/exists? path)
         (io/input-stream path)
-        (let [resp (http/req! cfg
-                              {:method :get :uri (:file-uri template)}
-                              {:response-type :input-stream :sync? true})]
+        (let [resp (http/req cfg
+                             {:method :get :uri (:file-uri template)}
+                             {:response-type :input-stream :sync? true})]
           (when-not (= 200 (:status resp))
             (ex/raise :type :internal
                       :code :unexpected-status-code

backend/src/app/srepl/main.clj

@@ -553,14 +553,13 @@
   (let [file-id (h/parse-uuid file-id)
         tnow    (ct/now)]
 
-    (audit/insert! main/system
-                   {::audit/name "delete-file"
-                    ::audit/type "action"
-                    ::audit/profile-id uuid/zero
-                    ::audit/props {:id file-id}
-                    ::audit/context {:triggered-by "srepl"
-                                     :cause "explicit call to delete-file!"}
-                    ::audit/tracked-at tnow})
+    (audit/insert main/system
+                  {:name "delete-file"
+                   :type "action"
+                   :props {:id file-id}
+                   :context {:triggered-by "srepl"
+                             :cause "explicit call to delete-file!"}
+                   :tracked-at tnow})
     (wrk/invoke! (-> main/system
                      (assoc ::wrk/task :delete-object)
                      (assoc ::wrk/params {:object :file
@@ -578,15 +577,12 @@
                                            {:id file-id}
                                            {::db/remove-deleted false
                                             ::sql/columns [:id :name]})]
-                    (audit/insert! system
-                                   {::audit/name "restore-file"
-                                    ::audit/type "action"
-                                    ::audit/profile-id uuid/zero
-                                    ::audit/props file
-                                    ::audit/context {:triggered-by "srepl"
-                                                     :cause "explicit call to restore-file!"}
-                                    ::audit/tracked-at (ct/now)})
-
+                    (audit/insert system
+                                  {:name "restore-file"
+                                   :type "action"
+                                   :props file
+                                   :context {:triggered-by "srepl"
+                                             :cause "explicit call to restore-file!"}})
 
                     (#'files/restore-file conn file-id))
                   :restored))))
@@ -597,14 +593,13 @@
   (let [project-id (h/parse-uuid project-id)
         tnow       (ct/now)]
 
-    (audit/insert! main/system
-                   {::audit/name "delete-project"
-                    ::audit/type "action"
-                    ::audit/profile-id uuid/zero
-                    ::audit/props {:id project-id}
-                    ::audit/context {:triggered-by "srepl"
-                                     :cause "explicit call to delete-project!"}
-                    ::audit/tracked-at tnow})
+    (audit/insert main/system
+                  {:name "delete-project"
+                   :type "action"
+                   :props {:id project-id}
+                   :context {:triggered-by "srepl"
+                             :cause "explicit call to delete-project!"}
+                   :tracked-at tnow})
 
     (wrk/invoke! (-> main/system
                      (assoc ::wrk/task :delete-object)
@@ -635,14 +630,12 @@
                   (when-let [project (db/get* system :project
                                               {:id project-id}
                                               {::db/remove-deleted false})]
-                    (audit/insert! system
-                                   {::audit/name "restore-project"
-                                    ::audit/type "action"
-                                    ::audit/profile-id uuid/zero
-                                    ::audit/props project
-                                    ::audit/context {:triggered-by "srepl"
-                                                     :cause "explicit call to restore-team!"}
-                                    ::audit/tracked-at (ct/now)})
+                    (audit/insert system
+                                  {:name "restore-project"
+                                   :type "action"
+                                   :props project
+                                   :context {:triggered-by "srepl"
+                                             :cause "explicit call to restore-team!"}})
 
                     (restore-project* system project-id))))))
 
@@ -652,14 +645,13 @@
   (let [team-id (h/parse-uuid team-id)
         tnow    (ct/now)]
 
-    (audit/insert! main/system
-                   {::audit/name "delete-team"
-                    ::audit/type "action"
-                    ::audit/profile-id uuid/zero
-                    ::audit/props {:id team-id}
-                    ::audit/context {:triggered-by "srepl"
-                                     :cause "explicit call to delete-profile!"}
-                    ::audit/tracked-at tnow})
+    (audit/insert main/system
+                  {:name "delete-team"
+                   :type "action"
+                   :props {:id team-id}
+                   :context {:triggered-by "srepl"
+                             :cause "explicit call to delete-profile!"}
+                   :tracked-at tnow})
 
     (wrk/invoke! (-> main/system
                      (assoc ::wrk/task :delete-object)
@@ -695,14 +687,12 @@
                                                    {:id team-id}
                                                    {::db/remove-deleted false})
                                           (teams/decode-row))]
-                    (audit/insert! system
-                                   {::audit/name "restore-team"
-                                    ::audit/type "action"
-                                    ::audit/profile-id uuid/zero
-                                    ::audit/props team
-                                    ::audit/context {:triggered-by "srepl"
-                                                     :cause "explicit call to restore-team!"}
-                                    ::audit/tracked-at (ct/now)})
+                    (audit/insert system
+                                  {:name "restore-team"
+                                   :type "action"
+                                   :props team
+                                   :context {:triggered-by "srepl"
+                                             :cause "explicit call to restore-team!"}})
 
                     (restore-team* system team-id))))))
 
@@ -712,13 +702,12 @@
   (let [profile-id (h/parse-uuid profile-id)
         tnow       (ct/now)]
 
-    (audit/insert! main/system
-                   {::audit/name "delete-profile"
-                    ::audit/type "action"
-                    ::audit/profile-id uuid/zero
-                    ::audit/context {:triggered-by "srepl"
-                                     :cause "explicit call to delete-profile!"}
-                    ::audit/tracked-at tnow})
+    (audit/insert main/system
+                  {:name "delete-profile"
+                   :type "action"
+                   :context {:triggered-by "srepl"
+                             :cause "explicit call to delete-profile!"}
+                   :tracked-at tnow})
 
     (wrk/invoke! (-> main/system
                      (assoc ::wrk/task :delete-object)
@@ -737,14 +726,12 @@
                                                       {:id profile-id}
                                                       {::db/remove-deleted false})
                                              (profile/decode-row))]
-                    (audit/insert! system
-                                   {::audit/name "restore-profile"
-                                    ::audit/type "action"
-                                    ::audit/profile-id uuid/zero
-                                    ::audit/props (audit/profile->props profile)
-                                    ::audit/context {:triggered-by "srepl"
-                                                     :cause "explicit call to restore-profile!"}
-                                    ::audit/tracked-at (ct/now)})
+                    (audit/insert system
+                                  {:name "restore-profile"
+                                   :type "action"
+                                   :props (audit/profile->props profile)
+                                   :context {:triggered-by "srepl"
+                                             :cause "explicit call to restore-profile!"}})
 
                     (db/update! system :profile
                                 {:deleted-at nil}
@@ -768,14 +755,14 @@
                                                   {::db/remove-deleted false})
                                          (profile/decode-row))]
                   (do
-                    (audit/insert! system
-                                   {::audit/name "delete-profile"
-                                    ::audit/type "action"
-                                    ::audit/profile-id (:id profile)
-                                    ::audit/tracked-at deleted-at
-                                    ::audit/props (audit/profile->props profile)
-                                    ::audit/context {:triggered-by "srepl"
-                                                     :cause "explicit call to delete-profiles-in-bulk!"}})
+                    (audit/insert system
+                                  {:name "delete-profile"
+                                   :type "action"
+                                   :profile-id (:id profile)
+                                   :tracked-at deleted-at
+                                   :props (audit/profile->props profile)
+                                   :context {:triggered-by "srepl"
+                                             :cause "explicit call to delete-profiles-in-bulk!"}})
                     (wrk/invoke! (-> system
                                      (assoc ::wrk/task :delete-object)
                                      (assoc ::wrk/params {:object :profile

backend/src/app/storage/gc_touched.clj

@@ -149,7 +149,7 @@
                  :status "delete"
                  :bucket bucket)
           (recur to-freeze (conj to-delete id) (rest objects))))
-      (let [deletion-delay (if (= bucket "tempfile")
+      (let [deletion-delay (if (= "tempfile" bucket)
                              (ct/duration {:hours 2})
                              (cf/get-deletion-delay))]
         (some->> (seq to-freeze) (mark-freeze-in-bulk! conn))
@@ -213,8 +213,13 @@
   [_ params]
   (assert (db/pool? (::db/pool params)) "expect valid storage"))
 
-(defmethod ig/init-key ::handler
-  [_ cfg]
-  (fn [_]
-    (process-touched! (assoc cfg ::timestamp (ct/now)))))
+(defmethod ig/expand-key ::handler
+  [k v]
+  {k (merge {::min-age (ct/duration {:hours 2})} v)})
+
+(defmethod ig/init-key ::handler
+  [_ {:keys [::min-age] :as cfg}]
+  (fn [_]
+    (let [threshold (ct/minus (ct/now) min-age)]
+      (process-touched! (assoc cfg ::timestamp threshold)))))
 

backend/src/app/storage/s3.clj

@@ -30,21 +30,18 @@
    java.nio.file.Path
    java.time.Duration
    java.util.Collection
-   java.util.Optional
    java.util.concurrent.atomic.AtomicLong
+   java.util.Optional
    org.reactivestreams.Subscriber
    software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider
-   software.amazon.awssdk.core.ResponseBytes
    software.amazon.awssdk.core.async.AsyncRequestBody
    software.amazon.awssdk.core.async.AsyncResponseTransformer
    software.amazon.awssdk.core.async.BlockingInputStreamAsyncRequestBody
    software.amazon.awssdk.core.client.config.ClientAsyncConfiguration
+   software.amazon.awssdk.core.ResponseBytes
    software.amazon.awssdk.http.nio.netty.NettyNioAsyncHttpClient
    software.amazon.awssdk.http.nio.netty.SdkEventLoopGroup
    software.amazon.awssdk.regions.Region
-   software.amazon.awssdk.services.s3.S3AsyncClient
-   software.amazon.awssdk.services.s3.S3AsyncClientBuilder
-   software.amazon.awssdk.services.s3.S3Configuration
    software.amazon.awssdk.services.s3.model.Delete
    software.amazon.awssdk.services.s3.model.DeleteObjectRequest
    software.amazon.awssdk.services.s3.model.DeleteObjectsRequest
@@ -54,9 +51,12 @@
    software.amazon.awssdk.services.s3.model.ObjectIdentifier
    software.amazon.awssdk.services.s3.model.PutObjectRequest
    software.amazon.awssdk.services.s3.model.S3Error
-   software.amazon.awssdk.services.s3.presigner.S3Presigner
    software.amazon.awssdk.services.s3.presigner.model.GetObjectPresignRequest
-   software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest))
+   software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest
+   software.amazon.awssdk.services.s3.presigner.S3Presigner
+   software.amazon.awssdk.services.s3.S3AsyncClient
+   software.amazon.awssdk.services.s3.S3AsyncClientBuilder
+   software.amazon.awssdk.services.s3.S3Configuration))
 
 (def ^:private max-retries
   "A maximum number of retries on internal operations"

backend/src/app/tasks/telemetry.clj

@@ -11,43 +11,27 @@
   (:require
    [app.common.data :as d]
    [app.common.exceptions :as ex]
+   [app.common.logging :as l]
    [app.config :as cf]
    [app.db :as db]
    [app.http.client :as http]
    [app.main :as-alias main]
    [app.setup :as-alias setup]
+   [app.util.blob :as blob]
    [app.util.json :as json]
    [integrant.core :as ig]
    [promesa.exec :as px]))
 
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;; IMPL
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-
-(defn- send!
-  [cfg data]
-  (let [request {:method :post
-                 :uri (cf/get :telemetry-uri)
-                 :headers {"content-type" "application/json"}
-                 :body (json/encode-str data)}
-        response (http/req! cfg request)]
-    (when (> (:status response) 206)
-      (ex/raise :type :internal
-                :code :invalid-response
-                :response-status (:status response)
-                :response-body (:body response)))))
-
-(defn- get-subscriptions-newsletter-updates
-  [conn]
+(defn- get-subscriptions
+  [cfg]
   (let [sql "SELECT email FROM profile where props->>'~:newsletter-updates' = 'true'"]
-    (->> (db/exec! conn [sql])
-         (mapv :email))))
+    (db/run! cfg (fn [{:keys [::db/conn]}]
+                   (->> (db/exec! conn [sql])
+                        (mapv :email))))))
 
-(defn- get-subscriptions-newsletter-news
-  [conn]
-  (let [sql "SELECT email FROM profile where props->>'~:newsletter-news' = 'true'"]
-    (->> (db/exec! conn [sql])
-         (mapv :email))))
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; LEGACY DATA COLLECTION
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 
 (defn- get-num-teams
   [conn]
@@ -129,6 +113,12 @@
   (->> [sql:team-averages]
        (db/exec-one! conn)))
 
+(defn- get-email-domains
+  [conn]
+  (let [sql "SELECT DISTINCT split_part(email, '@', 2) AS domain FROM profile ORDER BY 1"]
+    (->> (db/exec! conn [sql])
+         (mapv :domain))))
+
 (defn- get-enabled-auth-providers
   [conn]
   (let [sql  (str "SELECT auth_backend AS backend, count(*) AS total "
@@ -155,8 +145,9 @@
 (def ^:private sql:get-counters
   "SELECT name, count(*) AS count
      FROM audit_log
-    WHERE source = 'backend'
-      AND tracked_at >= date_trunc('day', now())
+    WHERE source LIKE 'telemetry:%'
+      AND created_at >= date_trunc('day', now())
+      AND created_at <  date_trunc('day', now()) + interval '1 day'
     GROUP BY 1
     ORDER BY 2 DESC")
 
@@ -168,23 +159,13 @@
     {:total-accomulated-events total
      :event-counters counters}))
 
-(def ^:private sql:clean-counters
-  "DELETE FROM audit_log
-    WHERE ip_addr = '0.0.0.0'::inet -- we know this is from telemetry
-      AND tracked_at < (date_trunc('day', now()) - '1 day'::interval)")
-
-(defn- clean-counters-data!
-  [conn]
-  (when-not (contains? cf/flags :audit-log)
-    (db/exec-one! conn [sql:clean-counters])))
-
-(defn- get-stats
-  [conn]
+(defn- get-legacy-stats
+  [{:keys [::db/conn]}]
   (let [referer (if (cf/get :telemetry-with-taiga)
                   "taiga"
                   (cf/get :telemetry-referer))]
     (-> {:referer             referer
-         :public-uri          (cf/get :public-uri)
+         :public-uri          (str (cf/get :public-uri))
          :total-teams         (get-num-teams conn)
          :total-projects      (get-num-projects conn)
          :total-files         (get-num-files conn)
@@ -192,7 +173,8 @@
          :total-fonts         (get-num-fonts conn)
          :total-comments      (get-num-comments conn)
          :total-file-changes  (get-num-file-changes conn)
-         :total-touched-files (get-num-touched-files conn)}
+         :total-touched-files (get-num-touched-files conn)
+         :email-domains       (get-email-domains conn)}
         (merge
          (get-team-averages conn)
          (get-jvm-stats)
@@ -200,6 +182,124 @@
          (get-action-counters conn))
         (d/without-nils))))
 
+(defn- make-legacy-request
+  [cfg data]
+  (let [request {:method :post
+                 :uri (cf/get :telemetry-uri)
+                 :headers {"content-type" "application/json"}
+                 :body (json/encode-str data)}
+        response (http/req cfg request {:skip-ssrf-check? true})]
+    (when (> (:status response) 206)
+      (ex/raise :type :internal
+                :code :invalid-response
+                :response-status (:status response)
+                :response-body (:body response)))))
+
+(defn- send-legacy-data
+  [{:keys [::setup/props] :as cfg} stats subs]
+  (let [data (cond-> {:type        :telemetry-legacy-report
+                      :version     (:full cf/version)
+                      :instance-id (:instance-id props)}
+               (some? stats)
+               (assoc :stats stats)
+
+               (seq subs)
+               (assoc :subscriptions subs))]
+
+    (make-legacy-request cfg data)))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; AUDIT-EVENT BATCH (TELEMETRY MODE)
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+;; Telemetry events older than this are purged by the GC step so the
+;; buffer stays bounded.
+(def ^:private batch-size 10000)
+
+(def ^:private sql:gc-events
+  "DELETE FROM audit_log
+    WHERE source LIKE 'telemetry:%'
+      AND created_at < now() - interval '7 days'")
+
+(defn- gc-events
+  "Delete telemetry-mode events older than `telemetry-retention-days`
+  so that the buffer stays bounded."
+  [{:keys [::db/conn]}]
+  (let [result (db/exec-one! conn [sql:gc-events])]
+    (when (pos? (:next.jdbc/update-count result))
+      (l/warn :hint "purged stale telemetry events"
+              :count (:next.jdbc/update-count result)))))
+
+(def ^:private sql:fetch-telemetry-events
+  "SELECT id, name, type, source, tracked_at, profile_id, props, context
+     FROM audit_log
+    WHERE source LIKE 'telemetry:%'
+    ORDER BY created_at ASC
+    LIMIT ?")
+
+(defn- row->event
+  [{:keys [name type source tracked-at profile-id props context]}]
+  (d/without-nils
+   {:name       name
+    :type       type
+    :source     source
+    :tracked-at tracked-at
+    :profile-id profile-id
+    :props      (or (some-> props db/decode-transit-pgobject) {})
+    :context    (or (some-> context db/decode-transit-pgobject) {})}))
+
+(defn- encode-batch
+  "Encode a sequence of event maps into a fressian+zstd base64 string
+  suitable for JSON transport."
+  ^String [events]
+  (blob/encode-str events {:version 4}))
+
+(defn send-event-batch
+  "Send a single batch of events to the telemetry endpoint. Returns
+  true on success."
+  [{:keys [::setup/props] :as cfg} batch]
+  (let [payload {:type :telemetry-events
+                 :version (:full cf/version)
+                 :instance-id (:instance-id props)
+                 :events (encode-batch batch)}
+        request {:method  :post
+                 :uri     (cf/get :telemetry-uri)
+                 :headers {"content-type" "application/json"}
+                 :body    (json/encode-str payload)}
+        resp    (http/req cfg request {:skip-ssrf-check? true})]
+    (if (<= (:status resp) 206)
+      true
+      (do
+        (l/warn :hint "telemetry event batch send failed"
+                :status (:status resp)
+                :body   (:body resp))
+        false))))
+
+(defn- delete-sent-events
+  "Delete rows by their ids after a successful send."
+  [conn ids]
+  (let [arr (db/create-array conn "uuid" ids)]
+    (db/exec-one! conn ["DELETE FROM audit_log WHERE id = ANY(?)" arr])))
+
+(defn- collect-and-send-audit-events
+  "Collect anonymous telemetry-mode audit events and ship them to the
+  telemetry endpoint in a loop. Each iteration fetches one page of
+  `batch-size` rows, encodes and sends them, then deletes the rows on
+   success. The loop stops as soon as a send returns false, leaving
+   remaining rows intact for the next run."
+  [{:keys [::db/conn] :as cfg}]
+  (loop [counter 1]
+    (when-let [rows (-> (db/exec! conn [sql:fetch-telemetry-events batch-size])
+                        (not-empty))]
+      (let [events (mapv row->event rows)
+            ids    (mapv :id rows)]
+        (l/dbg :hint "shipping telemetry event batch"
+               :total (count events)
+               :batch counter)
+        (when (send-event-batch cfg events)
+          (delete-sent-events conn ids)
+          (recur (inc counter)))))))
+
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 ;; TASK ENTRY POINT
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
@@ -211,46 +311,47 @@
   (assert (some? (::setup/props params)) "expected setup props to be available"))
 
 (defmethod ig/init-key ::handler
-  [_ {:keys [::db/pool ::setup/props] :as cfg}]
+  [_ cfg]
   (fn [task]
     (let [params   (:props task)
           send?    (get params :send? true)
           enabled? (or (get params :enabled? false)
-                       (contains? cf/flags :telemetry)
-                       (cf/get :telemetry-enabled))
+                       (contains? cf/flags :telemetry))
+          subs     (get-subscriptions cfg)]
 
-          subs     {:newsletter-updates (get-subscriptions-newsletter-updates pool)
-                    :newsletter-news (get-subscriptions-newsletter-news pool)}
 
-          data     {:subscriptions subs
-                    :version (:full cf/version)
-                    :instance-id (:instance-id props)}]
+      ;; If we have telemetry enabled, then proceed the normal
+      ;; operation sending legacy report
 
-      (when enabled?
-        (clean-counters-data! pool))
+      (if enabled?
+        (when send?
+          (db/run! cfg gc-events)
+          ;; Randomize start time to avoid thundering herd when multiple
+          ;; instances restart at the same time.
+          (px/sleep (rand-int 10000))
 
-      (cond
-        ;; If we have telemetry enabled, then proceed the normal
-        ;; operation.
-        enabled?
-        (let [data (merge data (get-stats pool))]
-          (when send?
-            (px/sleep (rand-int 10000))
-            (send! cfg data))
-          data)
+          (try
+            (let [stats (db/run! cfg get-legacy-stats)]
+              (send-legacy-data cfg stats subs))
+            (catch Exception cause
+              (l/wrn :hint "unable to send legacy report"
+                     :cause cause)))
+
+          ;; Ship any anonymous audit-log events accumulated in
+          ;; telemetry mode (only when audit-log feature is off).
+          (when-not (contains? cf/flags :audit-log)
+            (try
+              (db/run! cfg collect-and-send-audit-events)
+              (catch Exception cause
+                (l/wrn :hint "unable to send events"
+                       :cause cause)))))
 
         ;; If we have telemetry disabled, but there are users that are
         ;; explicitly checked the newsletter subscription on the
         ;; onboarding dialog or the profile section, then proceed to
         ;; send a limited telemetry data, that consists in the list of
         ;; subscribed emails and the running penpot version.
-        (or (seq (:newsletter-updates subs))
-            (seq (:newsletter-news subs)))
-        (do
-          (when send?
-            (px/sleep (rand-int 10000))
-            (send! cfg data))
-          data)
-
-        :else
-        data))))
+        (when (and send? (seq subs))
+          (px/sleep (rand-int 10000))
+          (ex/ignoring
+           (send-legacy-data cfg nil subs)))))))

backend/src/app/tasks/upload_session_gc.clj

@@ -0,0 +1,41 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS INC
+
+(ns app.tasks.upload-session-gc
+  "A maintenance task that deletes stalled (incomplete) upload sessions.
+
+  An upload session is considered stalled when it was created more than
+  `max-age` ago without being completed (i.e. the session row still
+  exists because `assemble-chunks` was never called to clean it up).
+  The default max-age is 1 hour."
+  (:require
+   [app.common.logging :as l]
+   [app.common.time :as ct]
+   [app.db :as db]
+   [integrant.core :as ig]))
+
+(def ^:private sql:delete-stalled-sessions
+  "DELETE FROM upload_session
+    WHERE created_at < ?::timestamptz")
+
+(defmethod ig/assert-key ::handler
+  [_ params]
+  (assert (db/pool? (::db/pool params)) "expected a valid database pool"))
+
+(defmethod ig/expand-key ::handler
+  [k v]
+  {k (merge {::max-age (ct/duration {:hours 1})} v)})
+
+(defmethod ig/init-key ::handler
+  [_ {:keys [::max-age] :as cfg}]
+  (fn [_]
+    (db/tx-run! cfg
+                (fn [{:keys [::db/conn]}]
+                  (let [threshold (ct/minus (ct/now) max-age)
+                        result    (-> (db/exec-one! conn [sql:delete-stalled-sessions threshold])
+                                      (db/get-update-count))]
+                    (l/debug :hint "task finished" :deleted result)
+                    {:deleted result})))))

backend/src/app/util/blob.clj

@@ -19,6 +19,7 @@
    java.io.DataOutputStream
    java.io.InputStream
    java.io.OutputStream
+   java.util.Base64
    net.jpountz.lz4.LZ4Compressor
    net.jpountz.lz4.LZ4Factory
    net.jpountz.lz4.LZ4FastDecompressor
@@ -49,6 +50,13 @@
        5 (encode-v5 data)
        (throw (ex-info "unsupported version" {:version version}))))))
 
+(defn encode-str
+  "Encode data to a blob and return it as a URL-safe base64 string
+  (no padding). Accepts the same options as `encode`."
+  (^String [data] (encode-str data nil))
+  (^String [data opts]
+   (.encodeToString (.withoutPadding (Base64/getUrlEncoder)) ^bytes (encode data opts))))
+
 (defn decode
   "A function used for decode persisted blobs in the database."
   [^bytes data]
@@ -63,6 +71,11 @@
         5 (decode-v5 data)
         (throw (ex-info "unsupported version" {:version version}))))))
 
+(defn decode-str
+  "Decode a URL-safe base64 string produced by `encode-str` back to data."
+  [^String s]
+  (decode (.decode (Base64/getUrlDecoder) s)))
+
 ;; --- IMPL
 
 (defn- encode-v1

backend/src/app/util/nio.clj

@@ -0,0 +1,91 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS INC
+
+(ns app.util.nio
+  "NIO helpers for working with files and byte arrays.
+
+   These are thin wrappers around java.nio that provide a
+   Clojure-idiomatic API. Candidates for porting to datoteka."
+  (:import
+   java.nio.ByteBuffer
+   java.nio.channels.FileChannel
+   java.nio.file.Files
+   java.nio.file.OpenOption
+   java.nio.file.Path
+   java.nio.file.StandardOpenOption))
+
+(set! *warn-on-reflection* true)
+
+;; ----------------------------------------------------------------
+;; File operations (via java.nio.file.Files)
+;; ----------------------------------------------------------------
+
+(defn read-bytes
+  "Read all bytes from a file at `path`. Returns a byte array."
+  ^bytes [^Path path]
+  (Files/readAllBytes path))
+
+(defn write-bytes
+  "Write `data` (byte array) to a file at `path`, replacing existing
+   content. Returns `path`."
+  [^Path path ^bytes data]
+  (Files/write path data ^"[Ljava.nio.file.OpenOption;" (into-array OpenOption []))
+  path)
+
+(defn append-bytes
+  "Append `data` (byte array) to the end of the file at `path`.
+   Creates the file if it does not exist. Returns `path`."
+  [^Path path ^bytes data]
+  (Files/write path data
+               ^"[Ljava.nio.file.OpenOption;"
+               (into-array OpenOption
+                           [StandardOpenOption/CREATE
+                            StandardOpenOption/APPEND]))
+  path)
+
+;; ----------------------------------------------------------------
+;; FileChannel operations (internal API)
+;; ----------------------------------------------------------------
+
+(def ^:private read-write-opts
+  (into-array OpenOption
+              [StandardOpenOption/READ StandardOpenOption/WRITE]))
+
+(defn open-channel
+  "Open a FileChannel for read/write on the given path."
+  ^FileChannel [^Path path]
+  (FileChannel/open path read-write-opts))
+
+(defn channel-size
+  "Return the size of the file backed by the channel."
+  ^long [^FileChannel channel]
+  (.size channel))
+
+(defn read-at
+  "Read `length` bytes from `channel` starting at `position` into a
+   new byte array. Returns the byte array.
+   Loops until the ByteBuffer is fully populated to guard against OS
+   partial reads, which would otherwise cause BufferUnderflowException
+   when copying from the buffer into the result array."
+  ^bytes [^FileChannel channel ^long position ^long length]
+  (let [buf (ByteBuffer/allocate (int length))]
+    (.position channel position)
+    (loop []
+      (when (.hasRemaining buf)
+        (let [n (.read channel buf)]
+          (when (pos? n)
+            (recur)))))
+    (.flip buf)
+    (let [remaining (.remaining buf)
+          arr       (byte-array remaining)]
+      (.get buf arr)
+      arr)))
+
+(defn truncate
+  "Truncate the file to the given size. Returns the channel."
+  [^FileChannel channel ^long size]
+  (.truncate channel size)
+  channel)

backend/src/app/util/ssrf.clj

@@ -0,0 +1,229 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS INC
+
+(ns app.util.ssrf
+  "URL/host validation to prevent Server-Side Request Forgery."
+  (:require
+   [app.common.exceptions :as ex]
+   [app.common.logging :as l]
+   [app.config :as cf]
+   [cuerdas.core :as str])
+  (:import
+   com.google.common.net.InetAddresses
+   java.net.InetAddress
+   java.net.UnknownHostException
+   java.net.URI))
+
+(def ^:private allowed-schemes
+  #{"http" "https"})
+
+(def ^:private cloud-metadata-ips
+  "Exact IP addresses for cloud metadata services."
+  #{"169.254.169.254"
+    "fd00:ec2::254"})
+
+(def ^:private extra-blocked-ranges
+  "CIDR ranges not covered by standard JDK InetAddress predicates.
+   Each entry is [base-address prefix-length]."
+  ;; Carrier-grade NAT
+  [[100 64 0 0 10]
+   ;; RFC 6890 / documentation / reserved
+   [192 0 0 0 24]
+   [192 0 2 0 24]
+   [198 18 0 0 15]
+   [198 51 100 0 24]
+   [203 0 113 0 24]
+   ;; Reserved / future-use (broadcast and above)
+   [240 0 0 0 4]])
+
+(defn- ip4-to-long
+  "Convert a 4-element byte array (IPv4) to a 32-bit long."
+  ^long [^bytes bs]
+  (bit-or (bit-shift-left (bit-and (aget bs 0) 0xFF) 24)
+          (bit-shift-left (bit-and (aget bs 1) 0xFF) 16)
+          (bit-shift-left (bit-and (aget bs 2) 0xFF) 8)
+          (bit-and (aget bs 3) 0xFF)))
+
+(defn- prefix-mask
+  "Return a 32-bit mask for the given prefix length."
+  ^long [^long prefix-len]
+  (if (zero? prefix-len)
+    0
+    (bit-shift-left (unsigned-bit-shift-right 0xFFFFFFFF (- 32 prefix-len)) (- 32 prefix-len))))
+
+(defn- in-cidr4?
+  "Check if an IPv4 address (as byte array) falls within a CIDR range
+   specified as [a b c d prefix-len]."
+  [^bytes addr [^long a ^long b ^long c ^long d ^long prefix-len]]
+  (let [base   (bit-or (bit-shift-left (bit-and a 0xFF) 24)
+                       (bit-shift-left (bit-and b 0xFF) 16)
+                       (bit-shift-left (bit-and c 0xFF) 8)
+                       (bit-and d 0xFF))
+        mask   (prefix-mask prefix-len)
+        ip-val (ip4-to-long addr)]
+    (= (bit-and ip-val mask) (bit-and base mask))))
+
+(defn- parse-cidr*
+  "Parse a CIDR string like '10.0.0.0/8' into [a b c d prefix-len]. Throws on invalid input."
+  [^String cidr]
+  (let [parts      (str/split cidr #"/" 2)
+        prefix-len (when (= 2 (count parts))
+                     (parse-long (nth parts 1)))]
+    (when-not prefix-len
+      (ex/raise :type :internal
+                :code :invalid-cidr
+                :hint (str "invalid CIDR notation: " cidr)))
+    (let [octets (str/split (first parts) #"\.")]
+      (when (not= 4 (count octets))
+        (ex/raise :type :internal
+                  :code :invalid-cidr
+                  :hint (str "invalid CIDR notation (expected IPv4): " cidr)))
+      (let [[a b c d] (map parse-long octets)]
+        (when (or (nil? a) (nil? b) (nil? c) (nil? d)
+                  (not (<= 0 a 255)) (not (<= 0 b 255))
+                  (not (<= 0 c 255)) (not (<= 0 d 255))
+                  (not (<= 0 prefix-len 32)))
+          (ex/raise :type :internal
+                    :code :invalid-cidr
+                    :hint (str "invalid CIDR notation: " cidr)))
+        [a b c d prefix-len]))))
+
+(defn parse-cidr
+  "Parse a CIDR string like '10.0.0.0/8' into [a b c d prefix-len].
+   Returns nil and logs a warning on invalid input."
+  [^String cidr]
+  (try
+    (parse-cidr* cidr)
+    (catch Exception _
+      (l/warn :hint "ignoring invalid CIDR" :cidr cidr)
+      nil)))
+
+(defonce ^:dynamic extra-blocked-cidrs
+  (into #{} (keep parse-cidr) (cf/get :ssrf-extra-blocked-cidrs #{})))
+
+(defn- ipv6-ula?
+  "Check if an IPv6 address is in the Unique Local Address range (fc00::/7)."
+  [^InetAddress addr]
+  (let [bs (.getAddress addr)]
+    (and (>= (alength bs) 16)
+         (= (bit-and (aget bs 0) 0xFE) 0xFC))))
+
+(defn- ipv4-mapped-loopback?
+  "Check if an IPv4-mapped IPv6 address maps to loopback (::ffff:127.x.x.x)."
+  [^InetAddress addr]
+  (let [bs (.getAddress addr)]
+    (and (= (alength bs) 16)
+         ;; Check it's an IPv4-mapped address: ::ffff:x.x.x.x
+         (= (aget bs 10) (byte -1)) ;; 0xFF
+         (= (aget bs 11) (byte -1)) ;; 0xFF
+         ;; Check the embedded IPv4 is loopback (127.x.x.x)
+         (= (bit-and (aget bs 12) 0xFF) 127))))
+
+(defn- blocked-address?
+  "Check if an InetAddress should be blocked. Returns true if blocked."
+  [^InetAddress addr]
+  (or
+   (.isAnyLocalAddress addr)    ;; 0.0.0.0 or ::
+   (.isLoopbackAddress addr)    ;; 127/8 or ::1
+   (.isLinkLocalAddress addr)   ;; 169.254/16 or fe80::/10
+   (.isSiteLocalAddress addr)   ;; 10/8, 172.16/12, 192.168/16
+   (.isMulticastAddress addr)
+
+   ;; IPv6 ULA (fc00::/7)
+   (ipv6-ula? addr)
+
+   ;; IPv4-mapped loopback
+   (ipv4-mapped-loopback? addr)
+
+   ;; Cloud metadata IPs (exact match)
+   (contains? cloud-metadata-ips (.getHostAddress addr))
+
+   ;; Extra blocked CIDRs (IPv4 only)
+   (let [bs (.getAddress addr)]
+     (if (= (alength bs) 4)
+       (or (some #(in-cidr4? bs %) extra-blocked-ranges)
+           (some #(in-cidr4? bs %) extra-blocked-cidrs))
+       false))))
+
+(defn resolve-host
+  "Resolve a hostname to all InetAddress objects. Wraps InetAddress/getAllByName
+   so it can be stubbed in tests."
+  [^String hostname]
+  (try
+    (InetAddress/getAllByName hostname)
+    (catch UnknownHostException _
+      nil)))
+
+(defn validate-uri
+  "Validates `uri-or-string`:
+    - scheme must be http or https,
+    - host must resolve to at least one address, and
+    - **every** resolved address must NOT be in the blocklist
+      (loopback, link-local, site-local, multicast, any-local,
+      cloud-metadata 169.254.169.254, IPv6 ULA fc00::/7, IPv4-mapped
+       IPv6 of any blocked IPv4, plus operator-supplied CIDRs).
+   When the host is an IP literal (decimal/octal/hex/IPv6) it is
+   normalized via `com.google.common.net.InetAddresses` before the
+   check.
+   Hosts in `:ssrf-allowed-hosts` (case-insensitive exact match) bypass
+   the IP check.
+   Throws `ex/raise :type :validation :code :ssrf-blocked-target` with
+   a hint that does NOT echo the resolved IP (avoid info leak)."
+  [uri-or-string]
+  (let [uri    (if (instance? URI uri-or-string)
+                 uri-or-string
+                 (URI. (str uri-or-string)))
+        scheme (.getScheme uri)
+        host   (.getHost uri)]
+
+    ;; Validate scheme
+    (when (or (nil? scheme)
+              (not (contains? allowed-schemes (str/lower scheme))))
+      (ex/raise :type :validation
+                :code :ssrf-blocked-target
+                :hint "url scheme is not allowed"))
+
+    ;; Validate host presence
+    (when (or (nil? host) (str/blank? host))
+      (ex/raise :type :validation
+                :code :ssrf-blocked-target
+                :hint "url host is missing"))
+
+    ;; Check allowlist
+    (let [allowed-hosts (cf/get :ssrf-allowed-hosts #{})
+          host-lower    (str/lower host)]
+
+      (when-not (contains? allowed-hosts host-lower)
+        ;; Normalize the host: if it looks like an IP literal, normalize it
+        ;; via Guava to catch decimal/octal/hex encodings
+        (let [normalized (if (InetAddresses/isInetAddress host)
+                           (InetAddresses/forString host)
+                           nil)
+              host-to-resolve (if normalized
+                                (.getHostAddress ^InetAddress normalized)
+                                host)
+              addresses (resolve-host host-to-resolve)]
+
+          (when (or (nil? addresses) (zero? (alength addresses)))
+            (ex/raise :type :validation
+                      :code :ssrf-blocked-target
+                      :hint "url host could not be resolved"))
+
+          ;; All-or-nothing: if ANY resolved address is blocked, reject
+          (when (some blocked-address? (seq addresses))
+            (ex/raise :type :validation
+                      :code :ssrf-blocked-target
+                      :hint "url target is not allowed")))))
+    (str uri)))
+
+(defn safe-url?
+  "Predicate version of `validate-uri`. Returns `true` if safe."
+  [uri-or-string]
+  (try
+    (validate-uri uri-or-string)
+    true
+    (catch Exception _
+      false)))

backend/test/backend_tests/email_blacklist_test.clj

@@ -0,0 +1,34 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS INC
+
+(ns backend-tests.email-blacklist-test
+  (:require
+   [app.email :as-alias email]
+   [app.email.blacklist :as blacklist]
+   [clojure.test :as t]))
+
+(def ^:private cfg
+  {::email/blacklist #{"somedomain.com" "spam.net"}})
+
+(t/deftest test-exact-domain-match
+  (t/is (true?  (blacklist/contains? cfg "user@somedomain.com")))
+  (t/is (true?  (blacklist/contains? cfg "user@spam.net")))
+  (t/is (false? (blacklist/contains? cfg "user@legit.com"))))
+
+(t/deftest test-subdomain-match
+  (t/is (true?  (blacklist/contains? cfg "user@sub.somedomain.com")))
+  (t/is (true?  (blacklist/contains? cfg "user@a.b.somedomain.com")))
+  ;; A domain that merely contains the blacklisted string but is not a
+  ;; subdomain must NOT be rejected.
+  (t/is (false? (blacklist/contains? cfg "user@notsomedomain.com"))))
+
+(t/deftest test-case-insensitive
+  (t/is (true? (blacklist/contains? cfg "user@SOMEDOMAIN.COM")))
+  (t/is (true? (blacklist/contains? cfg "user@Sub.SomeDomain.Com"))))
+
+(t/deftest test-non-blacklisted-domain
+  (t/is (false? (blacklist/contains? cfg "user@example.com")))
+  (t/is (false? (blacklist/contains? cfg "user@sub.legit.com"))))

backend/test/backend_tests/helpers.clj

@@ -83,7 +83,7 @@
   [next]
   (with-redefs [app.config/flags (flags/parse flags/default default-flags)
                 app.config/config config
-                app.loggers.audit/submit! (constantly nil)
+                app.loggers.audit/submit (constantly nil)
                 app.auth/derive-password identity
                 app.auth/verify-password (fn [a b] {:valid (= a b)})
                 app.common.features/get-enabled-features (fn [& _] app.common.features/supported-features)]

backend/test/backend_tests/http_middleware_test.clj

@@ -102,7 +102,7 @@
 
 (t/deftest access-token-authz
   (let [profile (th/create-profile* 1)
-        token   (db/tx-run! th/*system* app.rpc.commands.access-token/create-access-token (:id profile) "test" nil)
+        token   (db/tx-run! th/*system* app.rpc.commands.access-token/create-access-token (:id profile) "test" nil nil)
         handler (#'app.http.access-token/wrap-authz identity th/*system*)]
 
     (let [response (handler nil)]

backend/test/backend_tests/loggers_webhooks_test.clj

@@ -41,7 +41,7 @@
       (t/is (nil? res)))))
 
 (t/deftest run-webhook-handler-1
-  (with-mocks [http-mock {:target 'app.http.client/req! :return {:status 200}}]
+  (with-mocks [http-mock {:target 'app.http.client/req :return {:status 200}}]
     (let [prof (th/create-profile* 1 {:is-active true})
           whk  (th/create-webhook* {:team-id (:default-team-id prof)})
           evt  {:type "command"
@@ -63,7 +63,7 @@
         (t/is (nil? (:error-code whk')))))))
 
 (t/deftest run-webhook-handler-2
-  (with-mocks [http-mock {:target 'app.http.client/req! :return {:status 400}}]
+  (with-mocks [http-mock {:target 'app.http.client/req :return {:status 400}}]
     (let [prof (th/create-profile* 1 {:is-active true})
           whk  (th/create-webhook* {:team-id (:default-team-id prof)})
           evt  {:type "command"

backend/test/backend_tests/media_sanitize_test.clj

@@ -0,0 +1,501 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS INC
+
+(ns backend-tests.media-sanitize-test
+  (:require
+   [app.media.sanitize :as sanitize]
+   [app.storage.tmp :as tmp]
+   [app.util.nio :as nio]
+   [clojure.test :as t]
+   [datoteka.fs :as fs]
+   [datoteka.io :as io]))
+
+(defn- resource-path
+  "Return a URL to a test resource file."
+  [name]
+  (io/resource (str "backend_tests/test_files/" name)))
+
+(defn- copy-resource-to-tempfile
+  "Copy a test resource file to a tempfile and return the Path."
+  [resource-name suffix]
+  (tmp/tempfile-from (resource-path resource-name) :prefix "test-real-" :suffix suffix))
+
+;; ----------------------------------------------------------------
+;; Crafted test data
+;; ----------------------------------------------------------------
+
+;; PNG test data
+(def ^:private png-signature
+  (byte-array [0x89 0x50 0x4E 0x47 0x0D 0x0A 0x1A 0x0A]))
+
+(def ^:private png-iend-chunk
+  (byte-array [0x00 0x00 0x00 0x00 0x49 0x45 0x4E 0x44 0xAE 0x42 0x60 0x82]))
+
+(def ^:private png-ihdr-chunk
+  (byte-array [0x00 0x00 0x00 0x0D 0x49 0x48 0x44 0x52
+               0x00 0x00 0x00 0x01 0x00 0x00 0x00 0x01
+               0x08 0x02 0x00 0x00 0x00 0x90 0x77 0x53 0xDE]))
+
+(defn- make-png [^bytes extra-bytes]
+  (let [parts (if extra-bytes
+                [png-signature png-ihdr-chunk png-iend-chunk extra-bytes]
+                [png-signature png-ihdr-chunk png-iend-chunk])
+        total (reduce + 0 (map alength parts))
+        result (byte-array total)
+        offset (volatile! 0)]
+    (doseq [part parts]
+      (System/arraycopy part 0 result @offset (alength part))
+      (vswap! offset + (alength part)))
+    result))
+
+;; JPEG test data
+(def ^:private jpeg-soi (byte-array [0xFF 0xD8]))
+(def ^:private jpeg-eoi (byte-array [0xFF 0xD9]))
+
+(defn- make-jpeg [^bytes extra-bytes]
+  (let [parts (if extra-bytes
+                [jpeg-soi jpeg-eoi extra-bytes]
+                [jpeg-soi jpeg-eoi])
+        total (reduce + 0 (map alength parts))
+        result (byte-array total)
+        offset (volatile! 0)]
+    (doseq [part parts]
+      (System/arraycopy part 0 result @offset (alength part))
+      (vswap! offset + (alength part)))
+    result))
+
+;; GIF test data
+(def ^:private gif-header
+  (byte-array [0x47 0x49 0x46 0x38 0x39 0x61  ;; "GIF89a"
+               0x01 0x00 0x01 0x00  ;; 1x1 canvas
+               0x00  ;; no GCT
+               0x00]))  ;; background color
+
+(def ^:private gif-trailer (byte-array [0x3B]))
+
+;; WebP test data
+(defn- make-webp [^long total-size]
+  (let [riff-size (- total-size 8)
+        data (byte-array total-size)]
+    (aset data 0 (byte 0x52))  ;; 'R'
+    (aset data 1 (byte 0x49))  ;; 'I'
+    (aset data 2 (byte 0x46))  ;; 'F'
+    (aset data 3 (byte 0x46))  ;; 'F'
+    (aset data 4 (byte (bit-and riff-size 0xFF)))
+    (aset data 5 (byte (bit-and (bit-shift-right riff-size 8) 0xFF)))
+    (aset data 6 (byte (bit-and (bit-shift-right riff-size 16) 0xFF)))
+    (aset data 7 (byte (bit-and (bit-shift-right riff-size 24) 0xFF)))
+    (aset data 8 (byte 0x57))  ;; 'W'
+    (aset data 9 (byte 0x45))  ;; 'E'
+    (aset data 10 (byte 0x42)) ;; 'B'
+    (aset data 11 (byte 0x50)) ;; 'P'
+    data))
+
+(defn- write-data-to-tempfile
+  "Write byte array to a tempfile and return the Path."
+  [^bytes data suffix]
+  (let [path (tmp/tempfile :prefix "test-sanitize." :suffix suffix)]
+    (nio/write-bytes path data)
+    path))
+
+;; ----------------------------------------------------------------
+;; Tests with crafted data
+;; ----------------------------------------------------------------
+
+(t/deftest png-with-appended-secret-truncated
+  (let [secret (.getBytes "SECRET_DATA_HERE")
+        data   (make-png secret)
+        path   (write-data-to-tempfile data ".png")
+        _      (t/is (= (alength data) (alength (nio/read-bytes path))))
+        new-size (sanitize/truncate-after-eof path "image/png")]
+    (t/is (= new-size (+ (alength png-signature)
+                         (alength png-ihdr-chunk)
+                         (alength png-iend-chunk))))
+    (t/is (= new-size (alength (nio/read-bytes path))))
+    (let [expected (make-png nil)
+          actual   (nio/read-bytes path)]
+      (t/is (java.util.Arrays/equals expected actual)))))
+
+(t/deftest png-clean-not-truncated
+  (let [data (make-png nil)
+        path (write-data-to-tempfile data ".png")]
+    (t/is (= (alength data) (sanitize/truncate-after-eof path "image/png")))
+    (t/is (= (alength data) (alength (nio/read-bytes path))))))
+
+(t/deftest jpeg-with-appended-secret-truncated
+  (let [secret (.getBytes "\u0000\u0000SECRET")
+        data   (make-jpeg secret)
+        path   (write-data-to-tempfile data ".jpg")
+        _      (t/is (= (alength data) (alength (nio/read-bytes path))))
+        new-size (sanitize/truncate-after-eof path "image/jpeg")]
+    (t/is (= new-size (+ (alength jpeg-soi) (alength jpeg-eoi))))
+    (let [expected (make-jpeg nil)
+          actual   (nio/read-bytes path)]
+      (t/is (java.util.Arrays/equals expected actual)))))
+
+(t/deftest jpeg-clean-not-truncated
+  (let [data (make-jpeg nil)
+        path (write-data-to-tempfile data ".jpg")]
+    (t/is (= (alength data) (sanitize/truncate-after-eof path "image/jpeg")))
+    (t/is (= (alength data) (alength (nio/read-bytes path))))))
+
+(t/deftest gif-trailer-already-correct
+  (let [parts  [gif-header gif-trailer]
+        total  (reduce + 0 (map alength parts))
+        data   (byte-array total)
+        offset (volatile! 0)]
+    (doseq [part parts]
+      (System/arraycopy part 0 data @offset (alength part))
+      (vswap! offset + (alength part)))
+    (let [path (write-data-to-tempfile data ".gif")]
+      (t/is (= total (sanitize/truncate-after-eof path "image/gif")))
+      (t/is (= total (alength (nio/read-bytes path)))))))
+
+(t/deftest webp-declared-size-honored
+  (let [total-size 24
+        data      (make-webp total-size)
+        extra     (byte-array 10 (byte 0x42))
+        full-data (byte-array (+ total-size 10))]
+    (System/arraycopy data 0 full-data 0 total-size)
+    (System/arraycopy extra 0 full-data total-size 10)
+    (let [path (write-data-to-tempfile full-data ".webp")]
+      (t/is (= total-size (sanitize/truncate-after-eof path "image/webp")))
+      (t/is (= total-size (alength (nio/read-bytes path)))))))
+
+(t/deftest webp-clean-not-truncated
+  (let [data (make-webp 24)
+        path (write-data-to-tempfile data ".webp")]
+    (t/is (= 24 (sanitize/truncate-after-eof path "image/webp")))
+    (t/is (= 24 (alength (nio/read-bytes path))))))
+
+(t/deftest non-webp-riff-rejected-as-invalid-image
+  ;; A RIFF file whose FourCC is not 'WEBP' (e.g. a WAV file) must be
+  ;; rejected so it cannot bypass sanitization by pretending to be WebP.
+  (let [data (byte-array 24)]
+    ;; Write RIFF magic
+    (aset data 0 (byte 0x52)) ;; 'R'
+    (aset data 1 (byte 0x49)) ;; 'I'
+    (aset data 2 (byte 0x46)) ;; 'F'
+    (aset data 3 (byte 0x46)) ;; 'F'
+    ;; RIFF size = 16 (total 24 - 8)
+    (aset data 4 (byte 16))
+    ;; FourCC = 'WAVE' (not 'WEBP')
+    (aset data 8  (byte 0x57)) ;; 'W'
+    (aset data 9  (byte 0x41)) ;; 'A'
+    (aset data 10 (byte 0x56)) ;; 'V'
+    (aset data 11 (byte 0x45)) ;; 'E'
+    (let [path (write-data-to-tempfile data ".webp")]
+      (try
+        (sanitize/truncate-after-eof path "image/webp")
+        (t/is false "should have thrown")
+        (catch Exception e
+          (t/is (= :invalid-image (:code (ex-data e)))))))))
+
+(t/deftest svg-is-no-op
+  (let [data (.getBytes "<svg><rect/></svg>")
+        path (write-data-to-tempfile data ".svg")]
+    (t/is (= (alength data) (sanitize/truncate-after-eof path "image/svg+xml")))
+    (t/is (= (alength data) (alength (nio/read-bytes path))))))
+
+(t/deftest unknown-mtype-is-no-op
+  (let [data (.getBytes "some binary data")
+        path (write-data-to-tempfile data ".bin")]
+    (t/is (= (alength data) (sanitize/truncate-after-eof path "application/octet-stream")))
+    (t/is (= (alength data) (alength (nio/read-bytes path))))))
+
+(t/deftest png-missing-iend-raises-error
+  (let [data (byte-array [0x89 0x50 0x4E 0x47 0x0D 0x0A 0x1A 0x0A
+                          0x00 0x00 0x00 0x0D 0x49 0x48 0x44 0x52
+                          0x00 0x00 0x00 0x01 0x00 0x00 0x00 0x01
+                          0x08 0x02 0x00 0x00 0x00 0x90 0x77 0x53 0xDE])
+        path (write-data-to-tempfile data ".png")]
+    (try
+      (sanitize/truncate-after-eof path "image/png")
+      (t/is false "should have thrown")
+      (catch Exception e
+        (t/is (= :validation (:type (ex-data e))))
+        (t/is (= :invalid-image (:code (ex-data e))))))))
+
+;; ----------------------------------------------------------------
+;; Tests with real files from test_files/
+;; ----------------------------------------------------------------
+
+(t/deftest real-png-clean-not-truncated
+  (let [path     (copy-resource-to-tempfile "sample.png" ".png")
+        original (nio/read-bytes path)
+        size     (sanitize/truncate-after-eof path "image/png")]
+    (t/is (= (alength original) size))
+    (t/is (= (alength original) (alength (nio/read-bytes path))))))
+
+(t/deftest real-png-with-appended-secret-truncated
+  (let [path      (copy-resource-to-tempfile "sample.png" ".png")
+        original  (nio/read-bytes path)
+        orig-size (alength original)
+        secret    (.getBytes "EXFILTRATED_SECRET_DATA_12345")
+        _         (nio/append-bytes path secret)
+        _         (t/is (= (+ orig-size (alength secret))
+                           (alength (nio/read-bytes path))))
+        new-size  (sanitize/truncate-after-eof path "image/png")]
+    (t/is (= orig-size new-size))
+    (t/is (= orig-size (alength (nio/read-bytes path))))
+    (t/is (java.util.Arrays/equals original (nio/read-bytes path)))))
+
+(t/deftest real-jpg-clean-not-truncated
+  (let [path     (copy-resource-to-tempfile "sample.jpg" ".jpg")
+        original (nio/read-bytes path)
+        size     (sanitize/truncate-after-eof path "image/jpeg")]
+    (t/is (= (alength original) size))
+    (t/is (= (alength original) (alength (nio/read-bytes path))))))
+
+(t/deftest real-jpg-with-appended-secret-truncated
+  (let [path      (copy-resource-to-tempfile "sample.jpg" ".jpg")
+        original  (nio/read-bytes path)
+        orig-size (alength original)
+        secret    (.getBytes "EXFILTRATED_SECRET_DATA_12345")
+        _         (nio/append-bytes path secret)
+        _         (t/is (= (+ orig-size (alength secret))
+                           (alength (nio/read-bytes path))))
+        new-size  (sanitize/truncate-after-eof path "image/jpeg")]
+    (t/is (= orig-size new-size))
+    (t/is (= orig-size (alength (nio/read-bytes path))))
+    (t/is (java.util.Arrays/equals original (nio/read-bytes path)))))
+
+(t/deftest real-webp-clean-not-truncated
+  (let [path     (copy-resource-to-tempfile "sample.webp" ".webp")
+        original (nio/read-bytes path)
+        size     (sanitize/truncate-after-eof path "image/webp")]
+    (t/is (= (alength original) size))
+    (t/is (= (alength original) (alength (nio/read-bytes path))))))
+
+(t/deftest real-webp-with-appended-secret-truncated
+  (let [path      (copy-resource-to-tempfile "sample.webp" ".webp")
+        original  (nio/read-bytes path)
+        orig-size (alength original)
+        secret    (.getBytes "EXFILTRATED_SECRET_DATA_12345")
+        _         (nio/append-bytes path secret)
+        _         (t/is (= (+ orig-size (alength secret))
+                           (alength (nio/read-bytes path))))
+        new-size  (sanitize/truncate-after-eof path "image/webp")]
+    (t/is (= orig-size new-size))
+    (t/is (= orig-size (alength (nio/read-bytes path))))
+    (t/is (java.util.Arrays/equals original (nio/read-bytes path)))))
+
+;; ----------------------------------------------------------------
+;; Edge cases and boundary conditions
+;; ----------------------------------------------------------------
+
+(t/deftest empty-file-returns-zero
+  (let [path (write-data-to-tempfile (byte-array 0) ".png")]
+    (t/is (zero? (sanitize/truncate-after-eof path "image/png")))))
+
+(t/deftest png-signature-only-no-iend
+  ;; Just the 8-byte PNG signature, no chunks at all
+  (let [path (write-data-to-tempfile png-signature ".png")]
+    (try
+      (sanitize/truncate-after-eof path "image/png")
+      (t/is false "should have thrown")
+      (catch Exception e
+        (t/is (= :invalid-image (:code (ex-data e))))))))
+
+(t/deftest jpeg-soi-only-no-eoi
+  ;; Just the 2-byte SOI marker, no EOI
+  (let [path (write-data-to-tempfile jpeg-soi ".jpg")]
+    (try
+      (sanitize/truncate-after-eof path "image/jpeg")
+      (t/is false "should have thrown")
+      (catch Exception e
+        (t/is (= :invalid-image (:code (ex-data e))))))))
+
+(t/deftest jpeg-multiple-eoi-uses-last
+  ;; Progressive JPEGs can have multiple EOI markers; we want the last one
+  (let [data (byte-array (concat [0xFF 0xD8]           ;; SOI
+                                 [0x00 0x01 0x02]       ;; some data
+                                 [0xFF 0xD9]            ;; first EOI
+                                 [0x03 0x04 0x05]       ;; more data
+                                 [0xFF 0xD9]            ;; second (last) EOI
+                                 [0xDE 0xAD]))          ;; secret
+        path (write-data-to-tempfile data ".jpg")
+        new-size (sanitize/truncate-after-eof path "image/jpeg")]
+    ;; Should truncate at the last EOI (position 12: 2 + 3 + 2 + 3 + 2)
+    (t/is (= 12 new-size))
+    (let [result (nio/read-bytes path)]
+      (t/is (= 12 (alength result)))
+      ;; Verify it ends with the second FFD9
+      (t/is (= (unchecked-byte 0xFF) (aget result 10)))
+      (t/is (= (unchecked-byte 0xD9) (aget result 11))))))
+
+(t/deftest png-iend-with-nonzero-length-rejected
+  ;; IEND chunk with non-zero length field (malformed)
+  (let [bad-iend (byte-array [0x00 0x00 0x00 0x05  ;; length=5 (should be 0)
+                              0x49 0x45 0x4E 0x44   ;; "IEND"
+                              0xAE 0x42 0x60 0x82])  ;; CRC
+        data     (byte-array (concat png-signature png-ihdr-chunk bad-iend))
+        path     (write-data-to-tempfile data ".png")]
+    (try
+      (sanitize/truncate-after-eof path "image/png")
+      (t/is false "should have thrown")
+      (catch Exception e
+        (t/is (= :invalid-image (:code (ex-data e))))))))
+
+(t/deftest png-iend-length-read-as-big-endian
+  ;; Verify the IEND length field is interpreted as big-endian (PNG spec).
+  ;; Craft an IEND with length bytes [0x00 0x00 0x01 0x00]:
+  ;;   big-endian = 256 (non-zero → rejected)
+  ;;   little-endian = 65536 (also non-zero, but the code must still use BE)
+  ;; We additionally verify that a length of [0x00 0x01 0x00 0x00] is correctly
+  ;; read as 65536 in BE (not 256 as LE would give).
+  (let [be-iend (byte-array [0x00 0x01 0x00 0x00  ;; length=65536 BE (256 LE)
+                             0x49 0x45 0x4E 0x44   ;; "IEND"
+                             0xAE 0x42 0x60 0x82])  ;; CRC
+        data    (byte-array (concat png-signature png-ihdr-chunk be-iend))
+        path    (write-data-to-tempfile data ".png")]
+    (try
+      (sanitize/truncate-after-eof path "image/png")
+      (t/is false "should have thrown")
+      (catch Exception e
+        (t/is (= :invalid-image (:code (ex-data e))))))))
+
+(t/deftest png-iend-in-chunk-data-not-falsely-matched
+  ;; When "IEND" bytes appear inside chunk data (not as a chunk type),
+  ;; the scanner must not falsely match them as the IEND chunk.
+  ;; Build a PNG where the IHDR data contains "IEND" bytes, followed
+  ;; by a legitimate IEND chunk.
+  (let [ihdr-with-iend-in-data
+        (byte-array [0x00 0x00 0x00 0x0D  ;; length=13
+                     0x49 0x48 0x44 0x52   ;; "IHDR"
+                     0x00 0x00 0x00 0x01   ;; width=1
+                     0x49 0x45 0x4E 0x44   ;; "IEND" embedded in data (bytes 8-11 of payload)
+                     0x00 0x00 0x01        ;; remaining IHDR data bytes
+                     0x90 0x77 0x53 0xDE]) ;; CRC
+
+        valid-iend png-iend-chunk
+        data       (byte-array (concat png-signature ihdr-with-iend-in-data valid-iend))
+        path       (write-data-to-tempfile data ".png")
+        expected-size (+ (alength png-signature)
+                         (alength ihdr-with-iend-in-data)
+                         (alength valid-iend))]
+    ;; Should succeed and return the full size (no truncation needed)
+    (t/is (= expected-size (sanitize/truncate-after-eof path "image/png")))))
+
+(t/deftest png-iend-correct-offset-returned
+  ;; Verify that truncate-after-eof returns the exact byte offset of the
+  ;; end of the IEND chunk for a minimal valid PNG.
+  (let [data (make-png nil)
+        path (write-data-to-tempfile data ".png")
+        expected (+ (alength png-signature)
+                    (alength png-ihdr-chunk)
+                    (alength png-iend-chunk))]
+    (t/is (= expected (sanitize/truncate-after-eof path "image/png")))
+    (t/is (= expected (alength (nio/read-bytes path))))))
+
+(t/deftest gif-with-appended-data-truncated
+  ;; Appended bytes after trailer must be stripped even when they don't end in 0x3B.
+  (let [valid-size (+ (alength gif-header) (alength gif-trailer))
+        parts      [gif-header gif-trailer (byte-array [0x01 0x02 0x03])]
+        total      (reduce + 0 (map alength parts))
+        data       (byte-array total)
+        offset     (volatile! 0)]
+    (doseq [part parts]
+      (System/arraycopy part 0 data @offset (alength part))
+      (vswap! offset + (alength part)))
+    (let [path     (write-data-to-tempfile data ".gif")
+          new-size (sanitize/truncate-after-eof path "image/gif")]
+      (t/is (= valid-size new-size))
+      (t/is (= valid-size (alength (nio/read-bytes path)))))))
+
+(t/deftest gif-with-appended-data-ending-in-trailer-byte-truncated
+  ;; Security case: appended garbage that ends with 0x3B must NOT bypass the sanitizer.
+  ;; scan-backwards finds the rightmost 0x3B, which is the one in the appended payload;
+  ;; since that byte is AFTER the real trailer the truncation still drops the garbage.
+  ;; Actually the scan finds the last 0x3B overall — if the appended section ends
+  ;; with 0x3B we still truncate at that position, keeping only bytes up to the last 0x3B.
+  ;; The real trailer 0x3B is within the kept portion, so the GIF remains valid.
+  (let [valid-size (+ (alength gif-header) (alength gif-trailer))
+        ;; Append garbage: [0x01 0x02 0x3B] — ends with 0x3B
+        parts      [gif-header gif-trailer (byte-array [0x01 0x02 (unchecked-byte 0x3B)])]
+        total      (reduce + 0 (map alength parts))
+        data       (byte-array total)
+        offset     (volatile! 0)]
+    (doseq [part parts]
+      (System/arraycopy part 0 data @offset (alength part))
+      (vswap! offset + (alength part)))
+    (let [path     (write-data-to-tempfile data ".gif")
+          new-size (sanitize/truncate-after-eof path "image/gif")]
+      ;; The last 0x3B is at position total-1; scan finds it and returns total.
+      ;; No truncation occurs but the 0x01 0x02 garbage bytes still remain.
+      ;; This is an inherent limitation of the single-byte marker approach for GIF;
+      ;; the test documents the known behaviour.
+      (t/is (= total new-size)))))
+
+(t/deftest webp-riff-size-larger-than-file
+  ;; RIFF declares size larger than actual file - should return declared end
+  ;; even if it's beyond file size (FileChannel.truncate is a no-op for size >= file)
+  (let [data (make-webp 24)]
+    ;; Manually set RIFF size to 100 (so declared end = 108)
+    (aset data 4 (byte 100))
+    (aset data 5 (byte 0))
+    (aset data 6 (byte 0))
+    (aset data 7 (byte 0))
+    (let [path (write-data-to-tempfile data ".webp")
+          result (sanitize/truncate-after-eof path "image/webp")]
+      ;; Returns 108 (100 + 8), but file is only 24 bytes
+      ;; truncate is no-op when target >= size
+      (t/is (= 108 result))
+      (t/is (= 24 (alength (nio/read-bytes path)))))))
+
+(t/deftest webp-with-large-appended-data
+  (let [total-size 32
+        data       (make-webp total-size)
+        ;; Append 10000 bytes of secret
+        secret     (byte-array 10000 (byte 0x42))
+        full-data  (byte-array (+ total-size 10000))]
+    (System/arraycopy data 0 full-data 0 total-size)
+    (System/arraycopy secret 0 full-data total-size 10000)
+    (let [path    (write-data-to-tempfile full-data ".webp")
+          new-size (sanitize/truncate-after-eof path "image/webp")]
+      (t/is (= total-size new-size))
+      (t/is (= total-size (alength (nio/read-bytes path)))))))
+
+(t/deftest png-with-large-appended-secret
+  (let [data   (make-png nil)
+        ;; Append 1MB of secret data
+        secret (byte-array (* 1024 1024) (byte 0x42))
+        full   (byte-array (+ (alength data) (alength secret)))]
+    (System/arraycopy data 0 full 0 (alength data))
+    (System/arraycopy secret 0 full (alength data) (alength secret))
+    (let [path    (write-data-to-tempfile full ".png")
+          new-size (sanitize/truncate-after-eof path "image/png")]
+      (t/is (= (alength data) new-size))
+      (t/is (= (alength data) (alength (nio/read-bytes path)))))))
+
+(t/deftest jpeg-with-large-appended-secret
+  (let [data   (make-jpeg nil)
+        secret (byte-array (* 1024 1024) (byte 0x42))
+        full   (byte-array (+ (alength data) (alength secret)))]
+    (System/arraycopy data 0 full 0 (alength data))
+    (System/arraycopy secret 0 full (alength data) (alength secret))
+    (let [path    (write-data-to-tempfile full ".jpg")
+          new-size (sanitize/truncate-after-eof path "image/jpeg")]
+      (t/is (= (alength data) new-size))
+      (t/is (= (alength data) (alength (nio/read-bytes path)))))))
+
+(t/deftest png-with-appended-png-signature
+  ;; Appended data contains PNG signature bytes - should still find IEND
+  (let [extra  (byte-array (concat [0x89 0x50 0x4E 0x47] ;; PNG sig fragment
+                                   [0xDE 0xAD 0xBE 0xEF]))
+        data   (make-png extra)
+        path   (write-data-to-tempfile data ".png")
+        new-size (sanitize/truncate-after-eof path "image/png")]
+    (t/is (= (+ (alength png-signature)
+                (alength png-ihdr-chunk)
+                (alength png-iend-chunk)) new-size))))
+
+(t/deftest svg-with-trailing-data-is-no-op
+  ;; SVG is text format, no EOF truncation
+  (let [data (.getBytes "<svg><rect/></svg><!-- secret -->")
+        path (write-data-to-tempfile data ".svg")]
+    (t/is (= (alength data) (sanitize/truncate-after-eof path "image/svg+xml")))
+    (t/is (= (alength data) (alength (nio/read-bytes path))))))

backend/test/backend_tests/rpc_access_tokens_test.clj

@@ -107,4 +107,18 @@
         ;; (th/print-result! out)
         (t/is (nil? (:error out)))
         (let [results (:result out)]
-          (t/is (= 2 (count results))))))))
+          (t/is (= 2 (count results))))))
+
+    (t/testing "get mcp token"
+      (let [_ (th/command! {::th/type :create-access-token
+                            ::rpc/profile-id (:id prof)
+                            :type "mcp"
+                            :name "token 1"
+                            :perms ["get-profile"]})
+            {:keys [error result]}
+            (th/command! {::th/type :get-current-mcp-token
+                          ::rpc/profile-id (:id prof)})]
+        ;; (th/print-result! result)
+        (t/is (nil? error))
+        (t/is (string? (:token result)))))))
+

backend/test/backend_tests/rpc_audit_test.clj

@@ -9,7 +9,9 @@
    [app.common.pprint :as pp]
    [app.common.time :as ct]
    [app.common.uuid :as uuid]
+   [app.config :as cf]
    [app.db :as db]
+   [app.loggers.audit :as audit]
    [app.rpc :as-alias rpc]
    [backend-tests.helpers :as th]
    [clojure.test :as t]
@@ -96,4 +98,403 @@
         (t/is (= "navigate" (:name row)))
         (t/is (= "frontend" (:source row)))))))
 
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; TELEMETRY MODE (frontend ingest)
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 
+(t/deftest push-events-telemetry-mode-stores-anonymized-row
+  ;; When telemetry is enabled and audit-log is NOT, frontend events
+  ;; must be stored with source="telemetry", empty props, zeroed ip,
+  ;; and context filtered to safe keys only.
+  (with-redefs [cf/flags #{:telemetry}]
+    (let [prof    (th/create-profile* 1 {:is-active true})
+          team-id (:default-team-id prof)
+          proj-id (:default-project-id prof)
+
+          params  {::th/type :push-audit-events
+                   ::rpc/profile-id (:id prof)
+                   :events [{:name "navigate"
+                             :props {:project-id (str proj-id)
+                                     :team-id (str team-id)
+                                     :route "dashboard-files"}
+                             :context {:browser "Chrome"
+                                       :browser-version "120.0"
+                                       :os "Linux"
+                                       :version "2.0.0"
+                                       :session "should-be-stripped"
+                                       :external-session-id "also-stripped"
+                                       :initiator "app"}
+                             :timestamp (ct/now)
+                             :type "action"}]}
+
+          params  (with-meta params
+                    {:app.http/request http-request})
+          out     (th/command! params)]
+
+      (t/is (nil? (:error out)))
+      (t/is (nil? (:result out)))
+
+      (let [[row :as rows] (->> (th/db-exec! ["select * from audit_log"])
+                                (mapv decode-row))]
+        (t/is (= 1 (count rows)))
+        ;; source is telemetry:frontend, not frontend
+        (t/is (= "telemetry:frontend" (:source row)))
+        ;; profile-id preserved
+        (t/is (= (:id prof) (:profile-id row)))
+        ;; event name preserved
+        (t/is (= "navigate" (:name row)))
+        ;; navigate events keep route and team-id; other keys stripped
+        (t/is (= {:route "dashboard-files"
+                  :team-id (str team-id)}
+                 (:props row)))
+        ;; ip zeroed
+        (t/is (= "0.0.0.0" (str (:ip-addr row))))
+        ;; timestamps truncated to day precision
+        (let [day-now (ct/truncate (ct/now) :days)]
+          (t/is (= day-now (:created-at row)))
+          (t/is (= day-now (:tracked-at row))))
+        ;; context only contains safe keys
+        (let [ctx (:context row)]
+          (t/is (contains? ctx :browser))
+          (t/is (= "Chrome" (:browser ctx)))
+          (t/is (contains? ctx :os))
+          (t/is (= "Linux" (:os ctx)))
+          ;; session-linking keys stripped
+          (t/is (not (contains? ctx :session)))
+          (t/is (not (contains? ctx :external-session-id))))))))
+
+(t/deftest push-events-both-flags-creates-two-rows
+  ;; When both :audit-log and :telemetry flags are active, two rows
+  ;; should be stored: one full audit entry and one telemetry entry.
+  (with-redefs [cf/flags #{:audit-log :telemetry}]
+    (let [prof   (th/create-profile* 1 {:is-active true})
+          params {::th/type :push-audit-events
+                  ::rpc/profile-id (:id prof)
+                  :events [{:name "navigate"
+                            :props {:route "dashboard"}
+                            :context {:browser "Chrome"
+                                      :version "2.0.0"
+                                      :initiator "app"}
+                            :timestamp (ct/now)
+                            :type "action"}]}
+          params (with-meta params
+                   {:app.http/request http-request})
+          out    (th/command! params)]
+
+      (t/is (nil? (:error out)))
+
+      (let [[row1 row2 :as rows] (->> (th/db-exec! ["select * from audit_log order by source"])
+                                      (mapv decode-row))]
+        (t/is (= 2 (count rows)))
+        ;; First row: full audit-log entry
+        (t/is (= "frontend" (:source row1)))
+        (t/is (contains? (:props row1) :route))
+        (t/is (not= "0.0.0.0" (str (:ip-addr row1))))
+        ;; Second row: telemetry entry
+        (t/is (= "telemetry:frontend" (:source row2)))
+        (t/is (= "0.0.0.0" (str (:ip-addr row2))))
+        (let [day-now (ct/truncate (ct/now) :days)]
+          (t/is (= day-now (:created-at row2)))
+          (t/is (= day-now (:tracked-at row2))))))))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; BACKEND PROCESS-EVENT PATH (RPC commands)
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(t/deftest backend-process-event-only-audit-log
+  (with-redefs [cf/flags #{:audit-log}]
+    (let [prof  (th/create-profile* 1 {:is-active true})
+          event {:id (uuid/next)
+                 :type "action"
+                 :name "test-cmd"
+                 :profile-id (:id prof)
+                 :props {:full-key "full-val"}
+                 :context {:version "2.0.0" :initiator "app"}
+                 :tracked-at (ct/now)
+                 :created-at (ct/now)
+                 :source "backend"}]
+      (audit/submit* th/*system* event)
+      (let [[row :as rows] (->> (th/db-exec! ["select * from audit_log"])
+                                (mapv decode-row))]
+        (t/is (= 1 (count rows)))
+        (t/is (= "backend" (:source row)))
+        (t/is (= "full-val" (get-in row [:props :full-key])))
+        (t/is (not= "0.0.0.0" (str (:ip-addr row))))))))
+
+(t/deftest backend-process-event-only-telemetry
+  (with-redefs [cf/flags #{:telemetry}]
+    (let [prof  (th/create-profile* 1 {:is-active true})
+          event {:id (uuid/next)
+                 :type "action"
+                 :name "test-cmd"
+                 :profile-id (:id prof)
+                 :props {:full-key "full-val"}
+                 :context {:version "2.0.0" :initiator "app"}
+                 :tracked-at (ct/now)
+                 :created-at (ct/now)
+                 :source "backend"}]
+      (audit/submit* th/*system* event)
+      (let [[row :as rows] (->> (th/db-exec! ["select * from audit_log"])
+                                (mapv decode-row))]
+        (t/is (= 1 (count rows)))
+        (t/is (= "telemetry:backend" (:source row)))
+        (t/is (= "0.0.0.0" (str (:ip-addr row))))))))
+
+(t/deftest backend-process-event-both-flags-creates-two-rows
+  ;; When both :audit-log and :telemetry are active, the backend
+  ;; process-event must store two rows: one full audit entry and one
+  ;; telemetry entry.
+  (with-redefs [cf/flags #{:audit-log :telemetry}]
+    (let [prof  (th/create-profile* 1 {:is-active true})
+          event {:id (uuid/next)
+                 :type "action"
+                 :name "test-cmd"
+                 :profile-id (:id prof)
+                 :props {:keep-me "important"}
+                 :context {:version "2.0.0" :initiator "app"}
+                 :tracked-at (ct/now)
+                 :created-at (ct/now)
+                 :source "backend"}]
+      (audit/submit* th/*system* event)
+      (let [[row1 row2 :as rows] (->> (th/db-exec! ["select * from audit_log order by source"])
+                                      (mapv decode-row))]
+        (t/is (= 2 (count rows)))
+        ;; First row: full audit-log entry
+        (t/is (= "backend" (:source row1)))
+        (t/is (= "important" (get-in row1 [:props :keep-me])))
+        (t/is (not= "0.0.0.0" (str (:ip-addr row1))))
+        ;; Second row: telemetry entry
+        (t/is (= "telemetry:backend" (:source row2)))
+        (t/is (= "0.0.0.0" (str (:ip-addr row2))))
+        (let [day-now (ct/truncate (ct/now) :days)]
+          (t/is (= day-now (:created-at row2)))
+          (t/is (= day-now (:tracked-at row2))))))))
+
+(t/deftest push-events-disabled-when-no-flags-and-no-telemetry
+  ;; When neither audit-log nor telemetry is enabled, no rows should
+  ;; be stored.
+  (with-redefs [cf/flags #{}]
+    (let [prof   (th/create-profile* 1 {:is-active true})
+          params {::th/type :push-audit-events
+                  ::rpc/profile-id (:id prof)
+                  :events [{:name "navigate"
+                            :props {:route "dashboard"}
+                            :timestamp (ct/now)
+                            :type "action"}]}
+          params (with-meta params
+                   {:app.http/request http-request})
+          out    (th/command! params)]
+
+      (t/is (nil? (:error out)))
+      (t/is (= 0 (count (th/db-exec! ["select * from audit_log"])))))))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; PURE HELPER UNIT TESTS
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(t/deftest extract-utm-params-utm
+  ;; UTM params are namespaced under :penpot
+  (let [result (audit/extract-utm-params {:utm_source "google"
+                                          :utm_medium "cpc"
+                                          :utm_campaign "spring"
+                                          :other "ignored"})]
+    (t/is (= "google" (:penpot/utm-source result)))
+    (t/is (= "cpc" (:penpot/utm-medium result)))
+    (t/is (= "spring" (:penpot/utm-campaign result)))
+    (t/is (not (contains? result :other)))))
+
+(t/deftest extract-utm-params-mtm
+  ;; MTM params are also namespaced under :penpot
+  (let [result (audit/extract-utm-params {:mtm_source "newsletter"
+                                          :mtm_medium "email"})]
+    (t/is (= "newsletter" (:penpot/mtm-source result)))
+    (t/is (= "email" (:penpot/mtm-medium result)))))
+
+(t/deftest extract-utm-params-empty
+  (t/is (= {} (audit/extract-utm-params {})))
+  (t/is (= {} (audit/extract-utm-params {:foo "bar" :baz 42}))))
+
+(t/deftest profile->props-selects-and-merges
+  ;; Selects profile-props keys and merges with (:props profile)
+  (let [profile {:id (uuid/next)
+                 :fullname "John"
+                 :email "john@example.com"
+                 :is-active true
+                 :lang "en"
+                 :deleted-field "gone"
+                 :props {:custom-key "custom-val"
+                         :newsletter-updates true}}
+        result  (audit/profile->props profile)]
+    ;; Selected keys from profile
+    (t/is (= "John" (:fullname result)))
+    (t/is (= "john@example.com" (:email result)))
+    (t/is (true? (:is-active result)))
+    (t/is (= "en" (:lang result)))
+    ;; Merged from (:props profile)
+    (t/is (= "custom-val" (:custom-key result)))
+    (t/is (true? (:newsletter-updates result)))
+    ;; Keys not in profile-props are excluded
+    (t/is (not (contains? result :deleted-field)))))
+
+(t/deftest profile->props-removes-nils
+  (let [profile {:id (uuid/next) :fullname nil :email "a@b.com"}
+        result  (audit/profile->props profile)]
+    (t/is (not (contains? result :fullname)))
+    (t/is (= "a@b.com" (:email result)))))
+
+(t/deftest clean-props-removes-reserved
+  ;; Reserved props (:session-id, :password, :old-password, :token) are stripped
+  (let [props {:name "test"
+               :session-id "sess-123"
+               :password "secret"
+               :old-password "old-secret"
+               :token "tok-456"
+               :valid-key "kept"}
+        result (audit/clean-props props)]
+    (t/is (= "test" (:name result)))
+    (t/is (= "kept" (:valid-key result)))
+    (t/is (not (contains? result :session-id)))
+    (t/is (not (contains? result :password)))
+    (t/is (not (contains? result :old-password)))
+    (t/is (not (contains? result :token)))))
+
+(t/deftest clean-props-removes-qualified-keys
+  ;; Qualified keywords (namespaced) are stripped
+  (let [props {:simple "kept"
+               ::namespaced "stripped"
+               :app.rpc/also-stripped true}
+        result (audit/clean-props props)]
+    (t/is (= "kept" (:simple result)))
+    (t/is (not (contains? result ::namespaced)))
+    (t/is (not (contains? result :app.rpc/also-stripped)))))
+
+(t/deftest clean-props-removes-nils
+  (let [props {:a nil :b "val" :c nil}
+        result (audit/clean-props props)]
+    (t/is (= "val" (:b result)))
+    (t/is (not (contains? result :a)))
+    (t/is (not (contains? result :c)))))
+
+(t/deftest get-external-session-id-valid
+  (let [request (reify yetti.request/IRequest
+                  (get-header [_ name]
+                    (case name "x-external-session-id" "abc-123")))]
+    (t/is (= "abc-123" (audit/get-external-session-id request)))))
+
+(t/deftest get-external-session-id-nil-when-missing
+  (let [request (reify yetti.request/IRequest
+                  (get-header [_ _] nil))]
+    (t/is (nil? (audit/get-external-session-id request)))))
+
+(t/deftest get-external-session-id-nil-when-null-string
+  (let [request (reify yetti.request/IRequest
+                  (get-header [_ name]
+                    (case name "x-external-session-id" "null")))]
+    (t/is (nil? (audit/get-external-session-id request)))))
+
+(t/deftest get-external-session-id-nil-when-blank
+  (let [request (reify yetti.request/IRequest
+                  (get-header [_ name]
+                    (case name "x-external-session-id" "   ")))]
+    (t/is (nil? (audit/get-external-session-id request)))))
+
+(t/deftest get-external-session-id-nil-when-too-long
+  (let [long-id (apply str (repeat 300 "x"))
+        request (reify yetti.request/IRequest
+                  (get-header [_ name]
+                    (case name "x-external-session-id" long-id)))]
+    (t/is (nil? (audit/get-external-session-id request)))))
+
+(t/deftest get-client-user-agent-valid
+  (let [request (reify yetti.request/IRequest
+                  (get-header [_ name]
+                    (case name "user-agent" "Mozilla/5.0 (Test)")))]
+    (t/is (= "Mozilla/5.0 (Test)" (audit/get-client-user-agent request)))))
+
+(t/deftest get-client-user-agent-nil-when-missing
+  (let [request (reify yetti.request/IRequest
+                  (get-header [_ _] nil))]
+    (t/is (nil? (audit/get-client-user-agent request)))))
+
+(t/deftest get-client-user-agent-truncates-long
+  (let [long-ua (apply str (repeat 600 "x"))
+        request (reify yetti.request/IRequest
+                  (get-header [_ name]
+                    (case name "user-agent" long-ua)))]
+    (t/is (<= (count (audit/get-client-user-agent request)) 500))))
+
+(t/deftest get-client-event-origin-valid
+  (let [get-client-event-origin (ns-resolve 'app.loggers.audit 'get-client-event-origin)
+        request (reify yetti.request/IRequest
+                  (get-header [_ name]
+                    (case name "x-event-origin" "workspace")))]
+    (t/is (= "workspace" (get-client-event-origin request)))))
+
+(t/deftest get-client-event-origin-nil-when-null
+  (let [get-client-event-origin (ns-resolve 'app.loggers.audit 'get-client-event-origin)
+        request (reify yetti.request/IRequest
+                  (get-header [_ name]
+                    (case name "x-event-origin" "null")))]
+    (t/is (nil? (get-client-event-origin request)))))
+
+(t/deftest get-client-event-origin-nil-when-blank
+  (let [get-client-event-origin (ns-resolve 'app.loggers.audit 'get-client-event-origin)
+        request (reify yetti.request/IRequest
+                  (get-header [_ name]
+                    (case name "x-event-origin" "  ")))]
+    (t/is (nil? (get-client-event-origin request)))))
+
+(t/deftest get-client-event-origin-truncates-long
+  (let [get-client-event-origin (ns-resolve 'app.loggers.audit 'get-client-event-origin)
+        long-origin (apply str (repeat 300 "a"))
+        request (reify yetti.request/IRequest
+                  (get-header [_ name]
+                    (case name "x-event-origin" long-origin)))]
+    (t/is (<= (count (get-client-event-origin request)) 200))))
+
+(t/deftest get-client-version-valid
+  (let [get-client-version (ns-resolve 'app.loggers.audit 'get-client-version)
+        request (reify yetti.request/IRequest
+                  (get-header [_ name]
+                    (case name "x-frontend-version" "2.0.0")))]
+    (t/is (= "2.0.0" (get-client-version request)))))
+
+(t/deftest get-client-version-nil-when-null
+  (let [get-client-version (ns-resolve 'app.loggers.audit 'get-client-version)
+        request (reify yetti.request/IRequest
+                  (get-header [_ name]
+                    (case name "x-frontend-version" "null")))]
+    (t/is (nil? (get-client-version request)))))
+
+(t/deftest get-client-version-nil-when-blank
+  (let [get-client-version (ns-resolve 'app.loggers.audit 'get-client-version)
+        request (reify yetti.request/IRequest
+                  (get-header [_ name]
+                    (case name "x-frontend-version" "  ")))]
+    (t/is (nil? (get-client-version request)))))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; INSERT DEFAULTS
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(t/deftest insert-only-runs-with-audit-log-flag
+  ;; insert must be a no-op when :audit-log flag is not set
+  (with-redefs [app.config/flags #{:telemetry}]
+    (audit/insert th/*system* {:name "test" :type "action"})
+    (t/is (= 0 (count (th/db-exec! ["select * from audit_log"]))))))
+
+(t/deftest insert-sets-defaults
+  ;; insert must set defaults and persist when :audit-log is set
+  (with-redefs [app.config/flags #{:audit-log}]
+    (audit/insert th/*system* {:name "test-action" :type "action"})
+    (let [[row] (->> (th/db-exec! ["select * from audit_log"])
+                     (mapv decode-row))]
+      (t/is (some? row))
+      (t/is (= "test-action" (:name row)))
+      (t/is (= "action" (:type row)))
+      (t/is (= "backend" (:source row)))
+      (t/is (some? (:id row)))
+      (t/is (some? (:created-at row)))
+      (t/is (some? (:tracked-at row)))
+      (t/is (= {} (:props row)))
+      (t/is (= {} (:context row))))))

backend/test/backend_tests/rpc_file_test.clj

@@ -312,7 +312,8 @@
       ;; freeze because of the deduplication (we have uploaded 2 times
       ;; the same files).
 
-      (let [res (th/run-task! :storage-gc-touched {})]
+      (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                  (th/run-task! :storage-gc-touched {}))]
         (t/is (= 2 (:freeze res)))
         (t/is (= 0 (:delete res))))
 
@@ -386,7 +387,8 @@
       ;; Now that file-gc have deleted the file-media-object usage,
       ;; lets execute the touched-gc task, we should see that two of
       ;; them are marked to be deleted
-      (let [res (th/run-task! :storage-gc-touched {})]
+      (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                  (th/run-task! :storage-gc-touched {}))]
         (t/is (= 0 (:freeze res)))
         (t/is (= 2 (:delete res))))
 
@@ -571,7 +573,8 @@
       ;; Now that file-gc have deleted the file-media-object usage,
       ;; lets execute the touched-gc task, we should see that two of
       ;; them are marked to be deleted.
-      (let [res (th/run-task! :storage-gc-touched {})]
+      (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                  (th/run-task! :storage-gc-touched {}))]
         (t/is (= 0 (:freeze res)))
         (t/is (= 2 (:delete res))))
 
@@ -664,7 +667,8 @@
       ;; because of the deduplication (we have uploaded 2 times the
       ;; same files).
 
-      (let [res (th/run-task! :storage-gc-touched {})]
+      (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                  (th/run-task! :storage-gc-touched {}))]
         (t/is (= 1 (:freeze res)))
         (t/is (= 0 (:delete res))))
 
@@ -714,7 +718,8 @@
 
       ;; Now that objects-gc have deleted the object thumbnail lets
       ;; execute the touched-gc task
-      (let [res (th/run-task! "storage-gc-touched" {})]
+      (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                  (th/run-task! "storage-gc-touched" {}))]
         (t/is (= 1 (:freeze res))))
 
       ;; check file media objects
@@ -749,7 +754,8 @@
 
       ;; Now that file-gc have deleted the object thumbnail lets
       ;; execute the touched-gc task
-      (let [res (th/run-task! :storage-gc-touched {})]
+      (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                  (th/run-task! :storage-gc-touched {}))]
         (t/is (= 1 (:delete res))))
 
       ;; check file media objects
@@ -1319,7 +1325,8 @@
     ;; The FileGC task will schedule an inner taskq
     (th/run-pending-tasks!)
 
-    (let [res (th/run-task! :storage-gc-touched {})]
+    (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                (th/run-task! :storage-gc-touched {}))]
       (t/is (= 2 (:freeze res)))
       (t/is (= 0 (:delete res))))
 
@@ -1413,7 +1420,8 @@
 
     ;; we ensure that once object-gc is passed and marked two storage
     ;; objects to delete
-    (let [res (th/run-task! :storage-gc-touched {})]
+    (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                (th/run-task! :storage-gc-touched {}))]
       (t/is (= 0 (:freeze res)))
       (t/is (= 2 (:delete res))))
 

backend/test/backend_tests/rpc_file_thumbnails_test.clj

@@ -85,7 +85,7 @@
       (t/is (map? (:result out))))
 
     ;; run the task again
-    (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:minutes 31}))]
+    (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
                 (th/run-task! "storage-gc-touched" {}))]
       (t/is (= 2 (:freeze res))))
 
@@ -136,7 +136,7 @@
       (t/is (some? (sto/get-object storage (:media-id row2))))
 
       ;; run the task again
-      (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:minutes 31}))]
+      (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
                   (th/run-task! :storage-gc-touched {}))]
         (t/is (= 1 (:delete res)))
         (t/is (= 0 (:freeze res))))
@@ -235,7 +235,8 @@
         (t/is (= (:object-id data1) (:object-id row)))
         (t/is (uuid? (:media-id row1))))
 
-      (let [result (th/run-task! :storage-gc-touched {})]
+      (let [result (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                     (th/run-task! :storage-gc-touched {}))]
         (t/is (= 1 (:delete result))))
 
       ;; Check if storage objects still exists after file-gc

backend/test/backend_tests/rpc_font_test.clj

@@ -17,7 +17,9 @@
    [clojure.test :as t]
    [datoteka.fs :as fs]
    [datoteka.io :as io]
-   [mockery.core :refer [with-mocks]]))
+   [mockery.core :refer [with-mocks]])
+  (:import
+   java.io.RandomAccessFile))
 
 (t/use-fixtures :once th/state-init)
 (t/use-fixtures :each th/database-reset)
@@ -130,7 +132,8 @@
       ;; (th/print-result! out)
       (t/is (nil? (:error out))))
 
-    (let [res (th/run-task! :storage-gc-touched {})]
+    (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                (th/run-task! :storage-gc-touched {}))]
       (t/is (= 6 (:freeze res))))
 
     (let [params {::th/type :delete-font
@@ -142,14 +145,16 @@
       (t/is (nil? (:error out)))
       (t/is (nil? (:result out))))
 
-    (let [res (th/run-task! :storage-gc-touched {})]
+    (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                (th/run-task! :storage-gc-touched {}))]
       (t/is (= 0 (:freeze res)))
       (t/is (= 0 (:delete res))))
 
     (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:days 8}))]
       (let [res (th/run-task! :objects-gc {})]
-        (t/is (= 2 (:processed res))))
+        (t/is (= 2 (:processed res)))))
 
+    (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:days 8 :hours 3}))]
       (let [res (th/run-task! :storage-gc-touched {})]
         (t/is (= 0 (:freeze res)))
         (t/is (= 6 (:delete res)))))))
@@ -191,7 +196,8 @@
       ;; (th/print-result! out)
       (t/is (nil? (:error out))))
 
-    (let [res (th/run-task! :storage-gc-touched {})]
+    (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                (th/run-task! :storage-gc-touched {}))]
       (t/is (= 6 (:freeze res))))
 
     (let [params {::th/type :delete-font
@@ -203,14 +209,16 @@
       (t/is (nil? (:error out)))
       (t/is (nil? (:result out))))
 
-    (let [res (th/run-task! :storage-gc-touched {})]
+    (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                (th/run-task! :storage-gc-touched {}))]
       (t/is (= 0 (:freeze res)))
       (t/is (= 0 (:delete res))))
 
     (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:days 8}))]
       (let [res (th/run-task! :objects-gc {})]
-        (t/is (= 1 (:processed res))))
+        (t/is (= 1 (:processed res)))))
 
+    (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:days 8 :hours 3}))]
       (let [res (th/run-task! :storage-gc-touched {})]
         (t/is (= 0 (:freeze res)))
         (t/is (= 3 (:delete res)))))))
@@ -220,57 +228,42 @@
         team-id (:default-team-id prof)
         proj-id (:default-project-id prof)
         font-id (uuid/custom 10 1)
-
-        data1   (-> (io/resource "backend_tests/test_files/font-1.woff")
-                    (io/read*))
-
-        data2   (-> (io/resource "backend_tests/test_files/font-2.woff")
-                    (io/read*))
-        params1 {::th/type :create-font-variant
-                 ::rpc/profile-id (:id prof)
-                 :team-id team-id
-                 :font-id font-id
-                 :font-family "somefont"
-                 :font-weight 400
-                 :font-style "normal"
-                 :data {"font/woff" data1}}
-
-        params2 {::th/type :create-font-variant
-                 ::rpc/profile-id (:id prof)
-                 :team-id team-id
-                 :font-id font-id
-                 :font-family "somefont"
-                 :font-weight 500
-                 :font-style "normal"
-                 :data {"font/woff" data2}}
-
+        data1   (-> (io/resource "backend_tests/test_files/font-1.woff") (io/read*))
+        data2   (-> (io/resource "backend_tests/test_files/font-2.woff") (io/read*))
+        params1 {::th/type :create-font-variant ::rpc/profile-id (:id prof)
+                 :team-id team-id :font-id font-id :font-family "somefont"
+                 :font-weight 400 :font-style "normal" :data {"font/woff" data1}}
+        params2 {::th/type :create-font-variant ::rpc/profile-id (:id prof)
+                 :team-id team-id :font-id font-id :font-family "somefont"
+                 :font-weight 500 :font-style "normal" :data {"font/woff" data2}}
         out1    (th/command! params1)
         out2    (th/command! params2)]
-
-    ;; (th/print-result! out1)
     (t/is (nil? (:error out1)))
     (t/is (nil? (:error out2)))
 
-    (let [res (th/run-task! :storage-gc-touched {})]
+    ;; freeze with hours 3 clock
+    (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                (th/run-task! :storage-gc-touched {}))]
       (t/is (= 6 (:freeze res))))
 
-    (let [params {::th/type :delete-font-variant
-                  ::rpc/profile-id (:id prof)
-                  :team-id team-id
-                  :id (-> out1 :result :id)}
+    (let [params {::th/type :delete-font-variant ::rpc/profile-id (:id prof)
+                  :team-id team-id :id (-> out1 :result :id)}
           out    (th/command! params)]
-      ;; (th/print-result! out)
       (t/is (nil? (:error out)))
       (t/is (nil? (:result out))))
 
-    (let [res (th/run-task! :storage-gc-touched {})]
+    ;; no-op with hours 3 clock (nothing touched yet)
+    (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                (th/run-task! :storage-gc-touched {}))]
       (t/is (= 0 (:freeze res)))
       (t/is (= 0 (:delete res))))
 
+    ;; objects-gc at days 8, then storage-gc-touched at days 8 + 3h
     (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:days 8}))]
       (let [res (th/run-task! :objects-gc {})]
-        (t/is (= 1 (:processed res))))
+        (t/is (= 1 (:processed res)))))
 
+    (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:days 8 :hours 3}))]
       (let [res (th/run-task! :storage-gc-touched {})]
         (t/is (= 0 (:freeze res)))
         (t/is (= 3 (:delete res)))))))
@@ -301,3 +294,499 @@
       (let [error      (:error out)
             error-data (ex-data error)]
         (t/is (th/ex-info? error))))))
+
+;; -----------------------------------------------------------------------
+;; Helpers for chunked-upload font tests
+;; -----------------------------------------------------------------------
+
+(defn- split-bytes-into-chunks
+  "Splits `data` (byte array) into chunks of at most `chunk-size` bytes.
+  Returns a vector of byte arrays."
+  [^bytes data chunk-size]
+  (let [length (alength data)]
+    (loop [offset 0 chunks []]
+      (if (>= offset length)
+        chunks
+        (let [remaining (- length offset)
+              size      (min chunk-size remaining)
+              buf       (byte-array size)]
+          (System/arraycopy data offset buf 0 size)
+          (recur (+ offset size) (conj chunks buf)))))))
+
+(defn- make-chunk-mfile
+  "Writes `data` (byte array) to a tempfile and returns a map
+  compatible with the upload-chunk :content parameter."
+  [^bytes data mtype]
+  (let [tmp (fs/create-tempfile :dir "/tmp/penpot" :prefix "test-font-chunk-")]
+    (io/write* tmp data)
+    {:filename "chunk"
+     :path     tmp
+     :mtype    mtype
+     :size     (alength data)}))
+
+(defn- create-upload-session!
+  "Creates an upload session for `prof` with `total-chunks`. Returns the session-id UUID."
+  [prof total-chunks]
+  (let [out (th/command! {::th/type        :create-upload-session
+                          ::rpc/profile-id (:id prof)
+                          :total-chunks    total-chunks})]
+    (t/is (nil? (:error out)))
+    (:session-id (:result out))))
+
+(defn- upload-font-chunked!
+  "Splits `font-bytes` into chunks of `chunk-size` bytes, creates an upload
+  session, uploads all chunks, and returns the session-id UUID."
+  [prof ^bytes font-bytes mtype chunk-size]
+  (let [chunks     (split-bytes-into-chunks font-bytes chunk-size)
+        session-id (create-upload-session! prof (count chunks))]
+    (doseq [[idx chunk-data] (map-indexed vector chunks)]
+      (let [mfile (make-chunk-mfile chunk-data mtype)
+            out   (th/command! {::th/type        :upload-chunk
+                                ::rpc/profile-id (:id prof)
+                                :session-id      session-id
+                                :index           idx
+                                :content         mfile})]
+        (t/is (nil? (:error out)))))
+    session-id))
+
+(defn- assert-font-variant-result
+  "Checks that a successful create-font-variant result has valid UUIDs and
+  the expected scalar fields matching `params`."
+  [params result]
+  (t/is (uuid? (:id result)))
+  (t/is (uuid? (:ttf-file-id result)))
+  (t/is (uuid? (:otf-file-id result)))
+  (t/is (uuid? (:woff1-file-id result)))
+  (t/are [k] (= (get params k) (get result k))
+    :team-id
+    :font-id
+    :font-family
+    :font-weight
+    :font-style))
+
+;; -----------------------------------------------------------------------
+;; Path 1 – Normal (direct :data bytes)
+;; -----------------------------------------------------------------------
+
+(t/deftest create-font-variant-normal-ttf
+  (with-mocks [mock {:target 'app.rpc.quotes/check! :return nil}]
+    (let [prof    (th/create-profile* 1 {:is-active true})
+          team-id (:default-team-id prof)
+          font-id (uuid/custom 10 10)
+          data    (-> (io/resource "backend_tests/test_files/font-1.ttf") (io/read*))
+          params  {::th/type    :create-font-variant
+                   ::rpc/profile-id (:id prof)
+                   :team-id     team-id
+                   :font-id     font-id
+                   :font-family "chunked-test"
+                   :font-weight 400
+                   :font-style  "normal"
+                   :data        {"font/ttf" data}}
+          out     (th/command! params)]
+      (t/is (= 1 (:call-count @mock)))
+      (t/is (nil? (:error out)))
+      (assert-font-variant-result params (:result out)))))
+
+(t/deftest create-font-variant-normal-otf
+  (with-mocks [mock {:target 'app.rpc.quotes/check! :return nil}]
+    (let [prof    (th/create-profile* 1 {:is-active true})
+          team-id (:default-team-id prof)
+          font-id (uuid/custom 10 11)
+          data    (-> (io/resource "backend_tests/test_files/font-1.otf") (io/read*))
+          params  {::th/type    :create-font-variant
+                   ::rpc/profile-id (:id prof)
+                   :team-id     team-id
+                   :font-id     font-id
+                   :font-family "chunked-test"
+                   :font-weight 400
+                   :font-style  "normal"
+                   :data        {"font/otf" data}}
+          out     (th/command! params)]
+      (t/is (= 1 (:call-count @mock)))
+      (t/is (nil? (:error out)))
+      (assert-font-variant-result params (:result out)))))
+
+(t/deftest create-font-variant-normal-woff
+  (with-mocks [mock {:target 'app.rpc.quotes/check! :return nil}]
+    (let [prof    (th/create-profile* 1 {:is-active true})
+          team-id (:default-team-id prof)
+          font-id (uuid/custom 10 12)
+          data    (-> (io/resource "backend_tests/test_files/font-1.woff") (io/read*))
+          params  {::th/type    :create-font-variant
+                   ::rpc/profile-id (:id prof)
+                   :team-id     team-id
+                   :font-id     font-id
+                   :font-family "chunked-test"
+                   :font-weight 400
+                   :font-style  "normal"
+                   :data        {"font/woff" data}}
+          out     (th/command! params)]
+      (t/is (= 1 (:call-count @mock)))
+      (t/is (nil? (:error out)))
+      (assert-font-variant-result params (:result out)))))
+
+;; -----------------------------------------------------------------------
+;; Path 2 – Legacy chunking (:data with vector of byte-arrays per mtype)
+;; -----------------------------------------------------------------------
+
+(t/deftest create-font-variant-legacy-chunked-ttf
+  "Upload a TTF via the legacy :data path where each mtype value is a
+   vector of byte-array chunks (4 MiB each) instead of a single byte-array."
+  (with-mocks [mock {:target 'app.rpc.quotes/check! :return nil}]
+    (let [prof       (th/create-profile* 1 {:is-active true})
+          team-id    (:default-team-id prof)
+          font-id    (uuid/custom 10 20)
+          full-bytes (-> (io/resource "backend_tests/test_files/font-1.ttf") (io/read*))
+          ;; Simulate 4 MiB legacy chunks – font is small so a single chunk suffices
+          chunks     (split-bytes-into-chunks full-bytes (* 4 1024 1024))
+          params     {::th/type    :create-font-variant
+                      ::rpc/profile-id (:id prof)
+                      :team-id     team-id
+                      :font-id     font-id
+                      :font-family "legacy-chunked"
+                      :font-weight 700
+                      :font-style  "italic"
+                      :data        {"font/ttf" (vec chunks)}}
+          out        (th/command! params)]
+      (t/is (= 1 (:call-count @mock)))
+      (t/is (nil? (:error out)))
+      (assert-font-variant-result params (:result out)))))
+
+(t/deftest create-font-variant-legacy-chunked-woff
+  "Upload a WOFF via the legacy :data path with multiple sub-4 KiB chunks
+   to exercise the SequenceInputStream concatenation path."
+  (with-mocks [mock {:target 'app.rpc.quotes/check! :return nil}]
+    (let [prof       (th/create-profile* 1 {:is-active true})
+          team-id    (:default-team-id prof)
+          font-id    (uuid/custom 10 21)
+          full-bytes (-> (io/resource "backend_tests/test_files/font-1.woff") (io/read*))
+          ;; Split into small chunks to exercise the SequenceInputStream path
+          chunks     (split-bytes-into-chunks full-bytes 512)
+          params     {::th/type    :create-font-variant
+                      ::rpc/profile-id (:id prof)
+                      :team-id     team-id
+                      :font-id     font-id
+                      :font-family "legacy-chunked-woff"
+                      :font-weight 400
+                      :font-style  "normal"
+                      :data        {"font/woff" (vec chunks)}}
+          out        (th/command! params)]
+      (t/is (= 1 (:call-count @mock)))
+      (t/is (nil? (:error out)))
+      (assert-font-variant-result params (:result out)))))
+
+;; -----------------------------------------------------------------------
+;; Path 3 – New standardized chunked upload (:uploads map)
+;; -----------------------------------------------------------------------
+
+(t/deftest create-font-variant-chunked-upload-ttf
+  "Upload a TTF via the new :uploads path (chunked-upload API)."
+  (with-mocks [mock {:target 'app.rpc.quotes/check! :return nil}]
+    (let [prof       (th/create-profile* 1 {:is-active true})
+          team-id    (:default-team-id prof)
+          font-id    (uuid/custom 10 30)
+          font-bytes (-> (io/resource "backend_tests/test_files/font-1.ttf") (io/read*))
+          session-id (upload-font-chunked! prof font-bytes "font/ttf" (* 4 1024 1024))
+          params     {::th/type    :create-font-variant
+                      ::rpc/profile-id (:id prof)
+                      :team-id     team-id
+                      :font-id     font-id
+                      :font-family "new-chunked"
+                      :font-weight 400
+                      :font-style  "normal"
+                      :uploads     {"font/ttf" session-id}}
+          out        (th/command! params)]
+      ;; quotes/check! is called at least once (for the font-variant quota) plus
+      ;; once during session creation — assert it fired at least once.
+      (t/is (>= (:call-count @mock) 1))
+      (t/is (nil? (:error out)))
+      (assert-font-variant-result params (:result out)))))
+
+(t/deftest create-font-variant-chunked-upload-otf
+  "Upload an OTF via the new :uploads path."
+  (with-mocks [mock {:target 'app.rpc.quotes/check! :return nil}]
+    (let [prof       (th/create-profile* 1 {:is-active true})
+          team-id    (:default-team-id prof)
+          font-id    (uuid/custom 10 31)
+          font-bytes (-> (io/resource "backend_tests/test_files/font-1.otf") (io/read*))
+          session-id (upload-font-chunked! prof font-bytes "font/otf" (* 4 1024 1024))
+          params     {::th/type    :create-font-variant
+                      ::rpc/profile-id (:id prof)
+                      :team-id     team-id
+                      :font-id     font-id
+                      :font-family "new-chunked-otf"
+                      :font-weight 400
+                      :font-style  "normal"
+                      :uploads     {"font/otf" session-id}}
+          out        (th/command! params)]
+      (t/is (>= (:call-count @mock) 1))
+      (t/is (nil? (:error out)))
+      (assert-font-variant-result params (:result out)))))
+
+(t/deftest create-font-variant-chunked-upload-woff
+  "Upload a WOFF via the new :uploads path."
+  (with-mocks [mock {:target 'app.rpc.quotes/check! :return nil}]
+    (let [prof       (th/create-profile* 1 {:is-active true})
+          team-id    (:default-team-id prof)
+          font-id    (uuid/custom 10 32)
+          font-bytes (-> (io/resource "backend_tests/test_files/font-1.woff") (io/read*))
+          session-id (upload-font-chunked! prof font-bytes "font/woff" (* 4 1024 1024))
+          params     {::th/type    :create-font-variant
+                      ::rpc/profile-id (:id prof)
+                      :team-id     team-id
+                      :font-id     font-id
+                      :font-family "new-chunked-woff"
+                      :font-weight 400
+                      :font-style  "normal"
+                      :uploads     {"font/woff" session-id}}
+          out        (th/command! params)]
+      (t/is (>= (:call-count @mock) 1))
+      (t/is (nil? (:error out)))
+      (assert-font-variant-result params (:result out)))))
+
+(t/deftest create-font-variant-chunked-upload-multi-chunk
+  "Upload a WOFF split into many small chunks to exercise multi-chunk assembly."
+  (with-mocks [mock {:target 'app.rpc.quotes/check! :return nil}]
+    (let [prof       (th/create-profile* 1 {:is-active true})
+          team-id    (:default-team-id prof)
+          font-id    (uuid/custom 10 33)
+          font-bytes (-> (io/resource "backend_tests/test_files/font-1.woff") (io/read*))
+          ;; Use a chunk-size smaller than 4 MiB to force multiple chunks while
+          ;; staying within the 20-chunk-per-session quota limit (29836 / 2000 = ~15 chunks).
+          session-id (upload-font-chunked! prof font-bytes "font/woff" 2000)
+          params     {::th/type    :create-font-variant
+                      ::rpc/profile-id (:id prof)
+                      :team-id     team-id
+                      :font-id     font-id
+                      :font-family "multi-chunk-woff"
+                      :font-weight 400
+                      :font-style  "normal"
+                      :uploads     {"font/woff" session-id}}
+          out        (th/command! params)]
+      (t/is (>= (:call-count @mock) 1))
+      (t/is (nil? (:error out)))
+      (assert-font-variant-result params (:result out)))))
+
+;; -----------------------------------------------------------------------
+;; Error cases
+;; -----------------------------------------------------------------------
+
+(t/deftest create-font-variant-missing-data-and-uploads
+  "Neither :data nor :uploads is present — schema validation must reject it."
+  (let [prof    (th/create-profile* 1 {:is-active true})
+        team-id (:default-team-id prof)
+        font-id (uuid/custom 10 40)
+        params  {::th/type    :create-font-variant
+                 ::rpc/profile-id (:id prof)
+                 :team-id     team-id
+                 :font-id     font-id
+                 :font-family "bad"
+                 :font-weight 400
+                 :font-style  "normal"}
+        out     (th/command! params)]
+    (t/is (some? (:error out)))
+    (t/is (= :validation (-> out :error ex-data :type)))))
+
+(t/deftest create-font-variant-chunked-upload-missing-chunks
+  "When only some chunks are uploaded the assembly step must fail."
+  (with-mocks [_mock {:target 'app.rpc.quotes/check! :return nil}]
+    (let [prof       (th/create-profile* 1 {:is-active true})
+          team-id    (:default-team-id prof)
+          font-id    (uuid/custom 10 41)
+          font-bytes (-> (io/resource "backend_tests/test_files/font-1.ttf") (io/read*))
+          ;; 5000-byte chunks → 68640/5000 = 14 chunks; declare 15 but only upload 13
+          chunks     (split-bytes-into-chunks font-bytes 5000)
+          ;; Declare one extra chunk so assembly will fail (not all chunks present)
+          session-id (create-upload-session! prof (inc (count chunks)))]
+
+      ;; Upload all real chunks except the last one (omit it so the session is incomplete)
+      (doseq [[idx chunk-data] (map-indexed vector (butlast chunks))]
+        (let [mfile (make-chunk-mfile chunk-data "font/ttf")
+              out   (th/command! {::th/type        :upload-chunk
+                                  ::rpc/profile-id (:id prof)
+                                  :session-id      session-id
+                                  :index           idx
+                                  :content         mfile})]
+          (t/is (nil? (:error out)))))
+
+      (let [out (th/command! {::th/type    :create-font-variant
+                              ::rpc/profile-id (:id prof)
+                              :team-id     team-id
+                              :font-id     font-id
+                              :font-family "missing-chunks"
+                              :font-weight 400
+                              :font-style  "normal"
+                              :uploads     {"font/ttf" session-id}})]
+        (t/is (some? (:error out)))))))
+
+(t/deftest create-font-variant-chunked-upload-invalid-session
+  "Passing a non-existent session-id must fail at assembly time."
+  (with-mocks [_mock {:target 'app.rpc.quotes/check! :return nil}]
+    (let [prof    (th/create-profile* 1 {:is-active true})
+          team-id (:default-team-id prof)
+          font-id (uuid/custom 10 42)
+          out     (th/command! {::th/type    :create-font-variant
+                                ::rpc/profile-id (:id prof)
+                                :team-id     team-id
+                                :font-id     font-id
+                                :font-family "bad-session"
+                                :font-weight 400
+                                :font-style  "normal"
+                                :uploads     {"font/ttf" (uuid/next)}})]
+      (t/is (some? (:error out))))))
+
+;; -----------------------------------------------------------------------
+;; Font size validation tests
+;; -----------------------------------------------------------------------
+
+(t/deftest create-font-variant-size-exceeded-normal
+  "Direct :data upload exceeding font-max-file-size must be rejected."
+  (with-mocks [_mock {:target 'app.rpc.quotes/check! :return nil}]
+    (with-redefs [app.config/config (assoc app.config/config :font-max-file-size 1)]
+      (let [prof    (th/create-profile* 1 {:is-active true})
+            team-id (:default-team-id prof)
+            font-id (uuid/custom 10 50)
+            data    (-> (io/resource "backend_tests/test_files/font-1.ttf") (io/read*))
+            params  {::th/type    :create-font-variant
+                     ::rpc/profile-id (:id prof)
+                     :team-id     team-id
+                     :font-id     font-id
+                     :font-family "size-exceeded"
+                     :font-weight 400
+                     :font-style  "normal"
+                     :data        {"font/ttf" data}}
+            out     (th/command! params)]
+        (t/is (some? (:error out)))
+        (t/is (= :restriction (-> out :error ex-data :type)))
+        (t/is (= :font-max-file-size-reached (-> out :error ex-data :code)))))))
+
+(t/deftest create-font-variant-size-exceeded-legacy-chunked
+  "Legacy :data chunk-vector upload exceeding font-max-file-size must be rejected."
+  (with-mocks [_mock {:target 'app.rpc.quotes/check! :return nil}]
+    (with-redefs [app.config/config (assoc app.config/config :font-max-file-size 1)]
+      (let [prof       (th/create-profile* 1 {:is-active true})
+            team-id    (:default-team-id prof)
+            font-id    (uuid/custom 10 51)
+            full-bytes (-> (io/resource "backend_tests/test_files/font-1.woff") (io/read*))
+            chunks     (split-bytes-into-chunks full-bytes (* 4 1024 1024))
+            params     {::th/type    :create-font-variant
+                        ::rpc/profile-id (:id prof)
+                        :team-id     team-id
+                        :font-id     font-id
+                        :font-family "size-exceeded-legacy"
+                        :font-weight 400
+                        :font-style  "normal"
+                        :data        {"font/woff" (vec chunks)}}
+            out        (th/command! params)]
+        (t/is (some? (:error out)))
+        (t/is (= :restriction (-> out :error ex-data :type)))
+        (t/is (= :font-max-file-size-reached (-> out :error ex-data :code)))))))
+
+(t/deftest create-font-variant-size-exceeded-chunked-upload
+  "New :uploads path exceeding font-max-file-size must be rejected after assembly."
+  (with-mocks [_mock {:target 'app.rpc.quotes/check! :return nil}]
+    (let [prof       (th/create-profile* 1 {:is-active true})
+          team-id    (:default-team-id prof)
+          font-id    (uuid/custom 10 52)
+          font-bytes (-> (io/resource "backend_tests/test_files/font-1.ttf") (io/read*))
+          session-id (upload-font-chunked! prof font-bytes "font/ttf" (* 4 1024 1024))]
+      (with-redefs [app.config/config (assoc app.config/config :font-max-file-size 1)]
+        (let [out (th/command! {::th/type    :create-font-variant
+                                ::rpc/profile-id (:id prof)
+                                :team-id     team-id
+                                :font-id     font-id
+                                :font-family "size-exceeded-chunked"
+                                :font-weight 400
+                                :font-style  "normal"
+                                :uploads     {"font/ttf" session-id}})]
+          (t/is (some? (:error out)))
+          (t/is (= :restriction (-> out :error ex-data :type)))
+          (t/is (= :font-max-file-size-reached (-> out :error ex-data :code))))))))
+
+(t/deftest create-font-variant-size-within-limit
+  "Upload exactly at the limit must succeed."
+  (with-mocks [_mock {:target 'app.rpc.quotes/check! :return nil}]
+    (let [prof       (th/create-profile* 1 {:is-active true})
+          team-id    (:default-team-id prof)
+          font-id    (uuid/custom 10 53)
+          font-bytes (-> (io/resource "backend_tests/test_files/font-1.ttf") (io/read*))
+          font-size  (alength ^bytes font-bytes)]
+      (with-redefs [app.config/config (assoc app.config/config :font-max-file-size font-size)]
+        (let [params {::th/type    :create-font-variant
+                      ::rpc/profile-id (:id prof)
+                      :team-id     team-id
+                      :font-id     font-id
+                      :font-family "size-at-limit"
+                      :font-weight 400
+                      :font-style  "normal"
+                      :data        {"font/ttf" font-bytes}}
+              out    (th/command! params)]
+          (t/is (nil? (:error out)))
+          (assert-font-variant-result params (:result out)))))))
+
+;; -----------------------------------------------------------------------
+;; Font media-type validation tests
+;; -----------------------------------------------------------------------
+
+(t/deftest create-font-variant-invalid-type-normal
+  "Direct :data upload with a disallowed mtype must be rejected."
+  (with-mocks [_mock {:target 'app.rpc.quotes/check! :return nil}]
+    (let [prof    (th/create-profile* 1 {:is-active true})
+          team-id (:default-team-id prof)
+          font-id (uuid/custom 10 60)
+          data    (-> (io/resource "backend_tests/test_files/font-1.ttf") (io/read*))
+          params  {::th/type    :create-font-variant
+                   ::rpc/profile-id (:id prof)
+                   :team-id     team-id
+                   :font-id     font-id
+                   :font-family "invalid-type"
+                   :font-weight 400
+                   :font-style  "normal"
+                   :data        {"application/octet-stream" data}}
+          out     (th/command! params)]
+      (t/is (some? (:error out)))
+      (t/is (= :validation (-> out :error ex-data :type)))
+      (t/is (= :media-type-not-allowed (-> out :error ex-data :code))))))
+
+(t/deftest create-font-variant-invalid-type-legacy-chunked
+  "Legacy :data chunk-vector upload with a disallowed mtype must be rejected."
+  (with-mocks [_mock {:target 'app.rpc.quotes/check! :return nil}]
+    (let [prof       (th/create-profile* 1 {:is-active true})
+          team-id    (:default-team-id prof)
+          font-id    (uuid/custom 10 61)
+          full-bytes (-> (io/resource "backend_tests/test_files/font-1.woff") (io/read*))
+          chunks     (split-bytes-into-chunks full-bytes (* 4 1024 1024))
+          params     {::th/type    :create-font-variant
+                      ::rpc/profile-id (:id prof)
+                      :team-id     team-id
+                      :font-id     font-id
+                      :font-family "invalid-type-legacy"
+                      :font-weight 400
+                      :font-style  "normal"
+                      :data        {"image/png" (vec chunks)}}
+          out        (th/command! params)]
+      (t/is (some? (:error out)))
+      (t/is (= :validation (-> out :error ex-data :type)))
+      (t/is (= :media-type-not-allowed (-> out :error ex-data :code))))))
+
+(t/deftest create-font-variant-invalid-type-chunked-upload
+  "New :uploads path with a disallowed mtype must be rejected after assembly."
+  (with-mocks [_mock {:target 'app.rpc.quotes/check! :return nil}]
+    (let [prof       (th/create-profile* 1 {:is-active true})
+          team-id    (:default-team-id prof)
+          font-id    (uuid/custom 10 62)
+          font-bytes (-> (io/resource "backend_tests/test_files/font-1.ttf") (io/read*))
+          ;; Upload the bytes under a valid session but lie about the mtype
+          ;; when calling create-font-variant.
+          session-id (upload-font-chunked! prof font-bytes "font/ttf" (* 4 1024 1024))
+          out        (th/command! {::th/type    :create-font-variant
+                                   ::rpc/profile-id (:id prof)
+                                   :team-id     team-id
+                                   :font-id     font-id
+                                   :font-family "invalid-type-chunked"
+                                   :font-weight 400
+                                   :font-style  "normal"
+                                   :uploads     {"image/jpeg" session-id}})]
+      (t/is (some? (:error out)))
+      (t/is (= :validation (-> out :error ex-data :type)))
+      (t/is (= :media-type-not-allowed (-> out :error ex-data :code))))))

backend/test/backend_tests/rpc_media_test.clj

@@ -6,9 +6,7 @@
 
 (ns backend-tests.rpc-media-test
   (:require
-   [app.common.time :as ct]
    [app.common.uuid :as uuid]
-   [app.db :as db]
    [app.http.client :as http]
    [app.media :as media]
    [app.rpc :as-alias rpc]
@@ -16,7 +14,10 @@
    [backend-tests.helpers :as th]
    [clojure.test :as t]
    [datoteka.fs :as fs]
-   [mockery.core :refer [with-mocks]]))
+   [datoteka.io :as io]
+   [mockery.core :refer [with-mocks]])
+  (:import
+   java.io.RandomAccessFile))
 
 (t/use-fixtures :once th/state-init)
 (t/use-fixtures :each th/database-reset)
@@ -260,7 +261,7 @@
                                    :is-shared false})
 
         _      (th/db-update! :file
-                              {:deleted-at (ct/now)}
+                              {:deleted-at (app.common.time/now)}
                               {:id (:id file)})
 
         mfile  {:filename "sample.jpg"
@@ -285,11 +286,11 @@
 
 (t/deftest download-image-connection-error
   (t/testing "connection refused raises validation error"
-    (with-mocks [http-mock {:target 'app.http.client/req!
+    (with-mocks [http-mock {:target 'app.http.client/req-with-redirects
                             :throw (java.net.ConnectException. "Connection refused")}]
       (let [cfg {::http/client :mock-client}
             err (try
-                  (media/download-image cfg "http://unreachable.invalid/image.png")
+                  (media/download-image cfg "https://example.com/image.png")
                   nil
                   (catch clojure.lang.ExceptionInfo e e))]
         (t/is (some? err))
@@ -297,11 +298,11 @@
         (t/is (= :unable-to-download-image (:code (ex-data err)))))))
 
   (t/testing "connection timeout raises validation error"
-    (with-mocks [http-mock {:target 'app.http.client/req!
+    (with-mocks [http-mock {:target 'app.http.client/req-with-redirects
                             :throw (java.net.http.HttpConnectTimeoutException. "Connect timed out")}]
       (let [cfg {::http/client :mock-client}
             err (try
-                  (media/download-image cfg "http://unreachable.invalid/image.png")
+                  (media/download-image cfg "https://example.com/image.png")
                   nil
                   (catch clojure.lang.ExceptionInfo e e))]
         (t/is (some? err))
@@ -309,11 +310,11 @@
         (t/is (= :unable-to-download-image (:code (ex-data err)))))))
 
   (t/testing "request timeout raises validation error"
-    (with-mocks [http-mock {:target 'app.http.client/req!
+    (with-mocks [http-mock {:target 'app.http.client/req-with-redirects
                             :throw (java.net.http.HttpTimeoutException. "Request timed out")}]
       (let [cfg {::http/client :mock-client}
             err (try
-                  (media/download-image cfg "http://unreachable.invalid/image.png")
+                  (media/download-image cfg "https://example.com/image.png")
                   nil
                   (catch clojure.lang.ExceptionInfo e e))]
         (t/is (some? err))
@@ -321,11 +322,11 @@
         (t/is (= :unable-to-download-image (:code (ex-data err)))))))
 
   (t/testing "I/O error raises validation error"
-    (with-mocks [http-mock {:target 'app.http.client/req!
+    (with-mocks [http-mock {:target 'app.http.client/req-with-redirects
                             :throw (java.io.IOException. "Stream closed")}]
       (let [cfg {::http/client :mock-client}
             err (try
-                  (media/download-image cfg "http://unreachable.invalid/image.png")
+                  (media/download-image cfg "https://example.com/image.png")
                   nil
                   (catch clojure.lang.ExceptionInfo e e))]
         (t/is (some? err))
@@ -335,14 +336,14 @@
 
 (t/deftest download-image-status-code-error
   (t/testing "404 status raises validation error"
-    (with-mocks [http-mock {:target 'app.http.client/req!
+    (with-mocks [http-mock {:target 'app.http.client/req-with-redirects
                             :return {:status 404
                                      :headers {"content-type" "text/html"
                                                "content-length" "0"}
                                      :body nil}}]
       (let [cfg {::http/client :mock-client}
             err (try
-                  (media/download-image cfg "http://example.com/not-found.png")
+                  (media/download-image cfg "https://example.com/not-found.png")
                   nil
                   (catch clojure.lang.ExceptionInfo e e))]
         (t/is (some? err))
@@ -350,14 +351,14 @@
         (t/is (= :unable-to-download-image (:code (ex-data err)))))))
 
   (t/testing "500 status raises validation error"
-    (with-mocks [http-mock {:target 'app.http.client/req!
+    (with-mocks [http-mock {:target 'app.http.client/req-with-redirects
                             :return {:status 500
                                      :headers {"content-type" "text/html"
                                                "content-length" "0"}
                                      :body nil}}]
       (let [cfg {::http/client :mock-client}
             err (try
-                  (media/download-image cfg "http://example.com/server-error.png")
+                  (media/download-image cfg "https://example.com/server-error.png")
                   nil
                   (catch clojure.lang.ExceptionInfo e e))]
         (t/is (some? err))
@@ -365,16 +366,338 @@
         (t/is (= :unable-to-download-image (:code (ex-data err)))))))
 
   (t/testing "302 status raises validation error"
-    (with-mocks [http-mock {:target 'app.http.client/req!
+    (with-mocks [http-mock {:target 'app.http.client/req-with-redirects
                             :return {:status 302
                                      :headers {"content-type" "text/html"
                                                "content-length" "0"}
                                      :body nil}}]
       (let [cfg {::http/client :mock-client}
             err (try
-                  (media/download-image cfg "http://example.com/redirect.png")
+                  (media/download-image cfg "https://example.com/redirect.png")
                   nil
                   (catch clojure.lang.ExceptionInfo e e))]
         (t/is (some? err))
         (t/is (= :validation (:type (ex-data err))))
         (t/is (= :unable-to-download-image (:code (ex-data err))))))))
+
+;; --------------------------------------------------------------------
+;; Helpers for chunked-upload tests
+;; --------------------------------------------------------------------
+
+(defn- split-file-into-chunks
+  "Splits the file at `path` into byte-array chunks of at most
+  `chunk-size` bytes. Returns a vector of byte arrays."
+  [path chunk-size]
+  (let [file   (RandomAccessFile. (str path) "r")
+        length (.length file)]
+    (try
+      (loop [offset 0 chunks []]
+        (if (>= offset length)
+          chunks
+          (let [remaining (- length offset)
+                size      (min chunk-size remaining)
+                buf       (byte-array size)]
+            (.seek file offset)
+            (.readFully file buf)
+            (recur (+ offset size) (conj chunks buf)))))
+      (finally
+        (.close file)))))
+
+(defn- make-chunk-mfile
+  "Writes `data` (byte array) to a tempfile and returns a map
+  compatible with `media/schema:upload`."
+  [data mtype]
+  (let [tmp (fs/create-tempfile :dir "/tmp/penpot" :prefix "test-chunk-")]
+    (io/write* tmp data)
+    {:filename "chunk"
+     :path     tmp
+     :mtype    mtype
+     :size     (alength data)}))
+
+;; --------------------------------------------------------------------
+;; Chunked-upload tests
+;; --------------------------------------------------------------------
+
+(defn- create-session!
+  "Creates an upload session for `prof` with `total-chunks`. Returns the session-id UUID."
+  [prof total-chunks]
+  (let [out (th/command! {::th/type        :create-upload-session
+                          ::rpc/profile-id (:id prof)
+                          :total-chunks    total-chunks})]
+    (t/is (nil? (:error out)))
+    (:session-id (:result out))))
+
+(t/deftest chunked-upload-happy-path
+  (let [prof       (th/create-profile* 1)
+        _          (th/create-project* 1 {:profile-id (:id prof)
+                                          :team-id (:default-team-id prof)})
+        file       (th/create-file* 1 {:profile-id (:id prof)
+                                       :project-id (:default-project-id prof)
+                                       :is-shared false})
+        source-path (th/tempfile "backend_tests/test_files/sample.jpg")
+        chunks      (split-file-into-chunks source-path 110000) ; ~107 KB each
+        mtype       "image/jpeg"
+        total-size  (reduce + (map alength chunks))
+        session-id  (create-session! prof (count chunks))]
+
+    (t/is (= 3 (count chunks)))
+
+    ;; --- 1. Upload chunks ---
+    (doseq [[idx chunk-data] (map-indexed vector chunks)]
+      (let [mfile (make-chunk-mfile chunk-data mtype)
+            out   (th/command! {::th/type        :upload-chunk
+                                ::rpc/profile-id (:id prof)
+                                :session-id      session-id
+                                :index           idx
+                                :content         mfile})]
+        (t/is (nil? (:error out)))
+        (t/is (= session-id (:session-id (:result out))))
+        (t/is (= idx (:index (:result out))))))
+
+    ;; --- 2. Assemble ---
+    (let [assemble-out (th/command! {::th/type        :assemble-file-media-object
+                                     ::rpc/profile-id (:id prof)
+                                     :session-id      session-id
+                                     :file-id         (:id file)
+                                     :is-local        true
+                                     :name            "assembled-image"
+                                     :mtype           mtype})]
+
+      (t/is (nil? (:error assemble-out)))
+      (let [{:keys [media-id thumbnail-id] :as result} (:result assemble-out)]
+        (t/is (= (:id file) (:file-id result)))
+        (t/is (= 800 (:width result)))
+        (t/is (= 800 (:height result)))
+        (t/is (= mtype (:mtype result)))
+        (t/is (uuid? media-id))
+        (t/is (uuid? thumbnail-id))
+
+        (let [storage (:app.storage/storage th/*system*)
+              mobj1   (sto/get-object storage media-id)
+              mobj2   (sto/get-object storage thumbnail-id)]
+          (t/is (sto/object? mobj1))
+          (t/is (sto/object? mobj2))
+          (t/is (= total-size (:size mobj1))))))))
+
+(t/deftest chunked-upload-idempotency
+  (let [prof       (th/create-profile* 1)
+        _          (th/create-project* 1 {:profile-id (:id prof)
+                                          :team-id (:default-team-id prof)})
+        file       (th/create-file* 1 {:profile-id (:id prof)
+                                       :project-id (:default-project-id prof)
+                                       :is-shared false})
+        media-id   (uuid/next)
+        source-path (th/tempfile "backend_tests/test_files/sample.jpg")
+        chunks      (split-file-into-chunks source-path 312043) ; single chunk = whole file
+        mtype       "image/jpeg"
+        mfile       (make-chunk-mfile (first chunks) mtype)
+        session-id  (create-session! prof 1)]
+
+    (th/command! {::th/type        :upload-chunk
+                  ::rpc/profile-id (:id prof)
+                  :session-id      session-id
+                  :index           0
+                  :content         mfile})
+
+    ;; First assemble succeeds; session row is deleted afterwards
+    (let [out1 (th/command! {::th/type        :assemble-file-media-object
+                             ::rpc/profile-id (:id prof)
+                             :session-id      session-id
+                             :file-id         (:id file)
+                             :is-local        true
+                             :name            "sample"
+                             :mtype           mtype
+                             :id              media-id})]
+      (t/is (nil? (:error out1)))
+      (t/is (= media-id (:id (:result out1)))))
+
+    ;; Second assemble with the same session-id must fail because the
+    ;; session row has been deleted after the first assembly
+    (let [out2 (th/command! {::th/type        :assemble-file-media-object
+                             ::rpc/profile-id (:id prof)
+                             :session-id      session-id
+                             :file-id         (:id file)
+                             :is-local        true
+                             :name            "sample"
+                             :mtype           mtype
+                             :id              media-id})]
+      (t/is (some? (:error out2)))
+      (t/is (= :not-found (-> out2 :error ex-data :type)))
+      (t/is (= :object-not-found (-> out2 :error ex-data :code))))))
+
+(t/deftest chunked-upload-no-permission
+  ;; A second profile must not be able to upload chunks into a session
+  ;; that belongs to another profile: the DB lookup includes profile-id,
+  ;; so the session will not be found.
+  (let [prof1      (th/create-profile* 1)
+        prof2      (th/create-profile* 2)
+        session-id (create-session! prof1 1)
+        source-path (th/tempfile "backend_tests/test_files/sample.jpg")
+        mfile       {:filename "sample.jpg"
+                     :path     source-path
+                     :mtype    "image/jpeg"
+                     :size     312043}
+
+        ;; prof2 tries to upload a chunk into prof1's session
+        out (th/command! {::th/type        :upload-chunk
+                          ::rpc/profile-id (:id prof2)
+                          :session-id      session-id
+                          :index           0
+                          :content         mfile})]
+
+    (t/is (some? (:error out)))
+    (t/is (= :not-found (-> out :error ex-data :type)))))
+
+(t/deftest chunked-upload-invalid-media-type
+  (let [prof       (th/create-profile* 1)
+        _          (th/create-project* 1 {:profile-id (:id prof)
+                                          :team-id (:default-team-id prof)})
+        file       (th/create-file* 1 {:profile-id (:id prof)
+                                       :project-id (:default-project-id prof)
+                                       :is-shared false})
+        session-id (create-session! prof 1)
+        source-path (th/tempfile "backend_tests/test_files/sample.jpg")
+        mfile       {:filename "sample.jpg"
+                     :path     source-path
+                     :mtype    "image/jpeg"
+                     :size     312043}]
+
+    (th/command! {::th/type        :upload-chunk
+                  ::rpc/profile-id (:id prof)
+                  :session-id      session-id
+                  :index           0
+                  :content         mfile})
+
+    ;; Assemble with a wrong mtype should fail validation
+    (let [out (th/command! {::th/type        :assemble-file-media-object
+                            ::rpc/profile-id (:id prof)
+                            :session-id      session-id
+                            :file-id         (:id file)
+                            :is-local        true
+                            :name            "bad-type"
+                            :mtype           "application/octet-stream"})]
+      (t/is (some? (:error out)))
+      (t/is (= :validation (-> out :error ex-data :type))))))
+
+(t/deftest chunked-upload-missing-chunks
+  (let [prof       (th/create-profile* 1)
+        _          (th/create-project* 1 {:profile-id (:id prof)
+                                          :team-id (:default-team-id prof)})
+        file       (th/create-file* 1 {:profile-id (:id prof)
+                                       :project-id (:default-project-id prof)
+                                       :is-shared false})
+        ;; Session expects 3 chunks
+        session-id (create-session! prof 3)
+        source-path (th/tempfile "backend_tests/test_files/sample.jpg")
+        mfile       {:filename "sample.jpg"
+                     :path     source-path
+                     :mtype    "image/jpeg"
+                     :size     312043}]
+
+    ;; Upload only 1 chunk
+    (th/command! {::th/type        :upload-chunk
+                  ::rpc/profile-id (:id prof)
+                  :session-id      session-id
+                  :index           0
+                  :content         mfile})
+
+    ;; Assemble: session says 3 expected, only 1 stored → :missing-chunks
+    (let [out (th/command! {::th/type        :assemble-file-media-object
+                            ::rpc/profile-id (:id prof)
+                            :session-id      session-id
+                            :file-id         (:id file)
+                            :is-local        true
+                            :name            "incomplete"
+                            :mtype           "image/jpeg"})]
+      (t/is (some? (:error out)))
+      (t/is (= :validation (-> out :error ex-data :type)))
+      (t/is (= :missing-chunks (-> out :error ex-data :code))))))
+
+(t/deftest chunked-upload-session-not-found
+  (let [prof       (th/create-profile* 1)
+        _          (th/create-project* 1 {:profile-id (:id prof)
+                                          :team-id (:default-team-id prof)})
+        file       (th/create-file* 1 {:profile-id (:id prof)
+                                       :project-id (:default-project-id prof)
+                                       :is-shared false})
+        bogus-id   (uuid/next)]
+
+    ;; Assemble with a session-id that was never created
+    (let [out (th/command! {::th/type        :assemble-file-media-object
+                            ::rpc/profile-id (:id prof)
+                            :session-id      bogus-id
+                            :file-id         (:id file)
+                            :is-local        true
+                            :name            "ghost"
+                            :mtype           "image/jpeg"})]
+      (t/is (some? (:error out)))
+      (t/is (= :not-found (-> out :error ex-data :type)))
+      (t/is (= :object-not-found (-> out :error ex-data :code))))))
+
+(t/deftest chunked-upload-over-chunk-limit
+  ;; Verify that requesting more chunks than the configured maximum
+  ;; (quotes-upload-chunks-per-session) raises a :restriction error.
+  (with-mocks [mock {:target 'app.config/get
+                     :return (th/config-get-mock
+                              {:quotes-upload-chunks-per-session 3})}]
+    (let [prof (th/create-profile* 1)
+          out  (th/command! {::th/type        :create-upload-session
+                             ::rpc/profile-id (:id prof)
+                             :total-chunks    4})]
+
+      (t/is (some? (:error out)))
+      (t/is (= :restriction (-> out :error ex-data :type)))
+      (t/is (= :max-quote-reached (-> out :error ex-data :code)))
+      (t/is (= "upload-chunks-per-session" (-> out :error ex-data :target))))))
+
+(t/deftest chunked-upload-invalid-chunk-index
+  ;; Both a negative index and an index >= total-chunks must be
+  ;; rejected with a :validation / :invalid-chunk-index error.
+  (let [prof (th/create-profile* 1)
+        session-id   (create-session! prof 2)
+        source-path  (th/tempfile "backend_tests/test_files/sample.jpg")
+        mfile        {:filename "sample.jpg"
+                      :path     source-path
+                      :mtype    "image/jpeg"
+                      :size     312043}]
+
+    ;; index == total-chunks (out of range)
+    (let [out (th/command! {::th/type        :upload-chunk
+                            ::rpc/profile-id (:id prof)
+                            :session-id      session-id
+                            :index           2
+                            :content         mfile})]
+      (t/is (some? (:error out)))
+      (t/is (= :validation (-> out :error ex-data :type)))
+      (t/is (= :invalid-chunk-index (-> out :error ex-data :code))))
+
+    ;; negative index
+    (let [out (th/command! {::th/type        :upload-chunk
+                            ::rpc/profile-id (:id prof)
+                            :session-id      session-id
+                            :index           -1
+                            :content         mfile})]
+      (t/is (some? (:error out)))
+      (t/is (= :validation (-> out :error ex-data :type)))
+      (t/is (= :invalid-chunk-index (-> out :error ex-data :code))))))
+
+(t/deftest chunked-upload-sessions-per-profile-quota
+  ;; With the session limit set to 2, creating a third session for the
+  ;; same profile must fail with :restriction / :max-quote-reached.
+  ;; The :quotes flag is already enabled by the test fixture.
+  (with-mocks [mock {:target 'app.config/get
+                     :return (th/config-get-mock
+                              {:quotes-upload-sessions-per-profile 2})}]
+    (let [prof (th/create-profile* 1)]
+
+      ;; First two sessions succeed
+      (create-session! prof 1)
+      (create-session! prof 1)
+
+      ;; Third session must be rejected
+      (let [out (th/command! {::th/type        :create-upload-session
+                              ::rpc/profile-id (:id prof)
+                              :total-chunks    1})]
+        (t/is (some? (:error out)))
+        (t/is (= :restriction (-> out :error ex-data :type)))
+        (t/is (= :max-quote-reached (-> out :error ex-data :code)))))))

backend/test/backend_tests/rpc_profile_test.clj

@@ -380,7 +380,9 @@
   (let [data  {::th/type :prepare-register-profile
                :email "user@example.com"
                :fullname "foobar"
-               :password "foobar"}
+               :password "foobar"
+               :utm_campaign "utma"
+               :mtm_campaign "mtma"}
         out   (th/command! data)
         token (get-in out [:result :token])]
     (t/is (string? token))
@@ -396,11 +398,9 @@
 
     ;; try correct register
     (let [data  {::th/type :register-profile
-                 :token token
-                 :utm_campaign "utma"
-                 :mtm_campaign "mtma"}]
-      (let [{:keys [result error]} (th/command! data)]
-        (t/is (nil? error))))
+                 :token token}
+          out   (th/command! data)]
+      (t/is (nil? (:error out))))
 
     (let [profile (some-> (th/db-get :profile {:email "user@example.com"})
                           (profile/decode-row))]
@@ -514,32 +514,89 @@
           (t/is (= 0 (:call-count @mock))))))))
 
 (t/deftest prepare-and-register-with-invitation-and-enabled-registration-1
-  (let [itoken (tokens/generate th/*system*
-                                {:iss :team-invitation
-                                 :exp (ct/in-future "48h")
-                                 :role :editor
-                                 :team-id uuid/zero
-                                 :member-email "user@example.com"})
-        data  {::th/type :prepare-register-profile
-               :invitation-token itoken
-               :fullname "foobar"
-               :email "user@example.com"
-               :password "foobar"}
+  ;; With email-verification ENABLED (the default), a brand-new
+  ;; profile created via the invitation flow is NOT active yet, so
+  ;; `register-profile` must NOT mint a session and must NOT echo
+  ;; back the invitation token. Instead it must dispatch the
+  ;; verify-email mail with the invitation token EMBEDDED into the
+  ;; verify-email JWE (so the team-invitation flow can resume after
+  ;; the user clicks the email link).
+  (with-mocks [mock {:target 'app.email/send! :return nil}]
+    (let [itoken (tokens/generate th/*system*
+                                  {:iss :team-invitation
+                                   :exp (ct/in-future "48h")
+                                   :role :editor
+                                   :team-id uuid/zero
+                                   :member-email "user@example.com"})
+          prep-data {::th/type :prepare-register-profile
+                     :invitation-token itoken
+                     :fullname "foobar"
+                     :email "user@example.com"
+                     :password "foobar"}
 
-        {:keys [result error] :as out} (th/command! data)]
-    (t/is (nil? error))
-    (t/is (map? result))
-    (t/is (string? (:token result)))
+          {prep-result :result prep-error :error} (th/command! prep-data)]
+      (t/is (nil? prep-error))
+      (t/is (map? prep-result))
+      (t/is (string? (:token prep-result)))
 
-    (let [rtoken (:token result)
-          data   {::th/type :register-profile
-                  :token rtoken}
+      (let [reg-data {::th/type :register-profile
+                      :token (:token prep-result)}
 
-          {:keys [result error] :as out} (th/command! data)]
-      ;; (th/print-result! out)
-      (t/is (nil? error))
-      (t/is (map? result))
-      (t/is (string? (:invitation-token result))))))
+            {reg-result :result reg-error :error} (th/command! reg-data)
+            mdata    (meta reg-result)]
+        (t/is (nil? reg-error))
+        (t/is (map? reg-result))
+
+        ;; No invitation token echoed back, no session minted.
+        (t/is (nil? (:invitation-token reg-result)))
+        (t/is (empty? (:app.rpc/response-transform-fns mdata)))
+
+        ;; The verify-email mail was dispatched, and its token claims
+        ;; carry the invitation-token through to the verification step.
+        (t/is (= 1 (:call-count @mock)))
+        (let [send-args   (-> @mock :call-args)
+              email-token (->> send-args (some (fn [m] (when (map? m) (:token m)))))
+              vclaims     (tokens/decode th/*system* email-token)]
+          (t/is (= :verify-email (:iss vclaims)))
+          (t/is (= itoken (:invitation-token vclaims))))))))
+
+(t/deftest prepare-and-register-with-invitation-and-enabled-registration-1b
+  ;; With email-verification DISABLED, the brand-new profile is
+  ;; immediately active, so `register-profile` mints a session and
+  ;; returns the regenerated invitation token in the body — the
+  ;; frontend then redirects to :auth-verify-token to complete the
+  ;; team-invitation flow.
+  (with-redefs [app.config/flags #{:registration :login-with-password}]
+    (let [itoken (tokens/generate th/*system*
+                                  {:iss :team-invitation
+                                   :exp (ct/in-future "48h")
+                                   :role :editor
+                                   :team-id uuid/zero
+                                   :member-email "user@example.com"})
+          prep-data {::th/type :prepare-register-profile
+                     :invitation-token itoken
+                     :fullname "foobar"
+                     :email "user@example.com"
+                     :password "foobar"}
+
+          {prep-result :result prep-error :error} (th/command! prep-data)]
+      (t/is (nil? prep-error))
+      (t/is (string? (:token prep-result)))
+
+      (let [reg-data {::th/type :register-profile
+                      :token (:token prep-result)}
+
+            {reg-result :result reg-error :error} (th/command! reg-data)
+            mdata    (meta reg-result)]
+        (t/is (nil? reg-error))
+        (t/is (map? reg-result))
+
+        ;; Active branch: invitation-token is echoed back and a session
+        ;; is minted via `session/create-fn`.
+        (t/is (string? (:invitation-token reg-result)))
+        (t/is (seq (:app.rpc/response-transform-fns mdata)))
+        (t/is (= "accept-invitation"
+                 (get-in mdata [:app.loggers.audit/context :action])))))))
 
 (t/deftest prepare-and-register-with-invitation-and-enabled-registration-2
   (let [itoken (tokens/generate th/*system*
@@ -692,6 +749,188 @@
       (t/is (= :validation (:type edata)))
       (t/is (= :email-as-password (:code edata))))))
 
+(t/deftest prepare-register-rejects-active-profile-email
+  ;; SECURITY: `prepare-register` must reject any attempt to prepare a
+  ;; registration for an email that already belongs to an *active*
+  ;; profile, regardless of whether an invitation token is supplied.
+  ;; Active profiles must use the standard login flow.
+  (let [_victim (th/create-profile* 1 {:is-active true
+                                       :email "victim@corp.tld"})]
+
+    ;; Without invitation token.
+    (let [out (th/command! {::th/type :prepare-register-profile
+                            :fullname "Mallory"
+                            :email "victim@corp.tld"
+                            :password "Whatever1!"})]
+      (t/is (not (th/success? out)))
+      (let [edata (-> out :error ex-data)]
+        (t/is (= :validation (:type edata)))
+        (t/is (= :email-already-exists (:code edata)))))
+
+    ;; With invitation token (the GHSA-4937-35vc-hqjj exploit shape).
+    (let [itoken (tokens/generate th/*system*
+                                  {:iss :team-invitation
+                                   :exp (ct/in-future "48h")
+                                   :role :editor
+                                   :team-id uuid/zero
+                                   :member-email "victim@corp.tld"})
+          out    (th/command! {::th/type :prepare-register-profile
+                               :invitation-token itoken
+                               :fullname "Mallory"
+                               :email "victim@corp.tld"
+                               :password "Whatever1!"})]
+      (t/is (not (th/success? out)))
+      (let [edata (-> out :error ex-data)]
+        (t/is (= :validation (:type edata)))
+        (t/is (= :email-already-exists (:code edata)))))))
+
+(t/deftest prepare-register-must-not-leak-existing-profile-id
+  ;; Victim is a pre-existing profile that has not yet activated (e.g.
+  ;; freshly registered, has not clicked the email verification link).
+  ;; `prepare-register` allows the call (no active profile exists), but
+  ;; the issued JWE must NOT carry the existing profile's id.
+  (let [_victim (th/create-profile* 1 {:is-active false
+                                       :email "victim@corp.tld"})
+
+        ;; Attacker holds a cryptographically valid `:team-invitation` JWE
+        ;; for the victim's email. (In a real exploit this is obtained
+        ;; from `create-team-invitations` or `get-team-invitation-token`
+        ;; on a team the attacker owns.)
+        itoken (tokens/generate th/*system*
+                                {:iss :team-invitation
+                                 :exp (ct/in-future "48h")
+                                 :role :editor
+                                 :team-id uuid/zero
+                                 :member-email "victim@corp.tld"})
+
+        ;; Anonymous request — no ::rpc/profile-id.
+        data   {::th/type :prepare-register-profile
+                :invitation-token itoken
+                :fullname "Mallory"
+                :email "victim@corp.tld"
+                :password "Whatever1!"}
+
+        out    (th/command! data)]
+
+    ;; The current behaviour either returns a token or rejects the request;
+    ;; what MUST hold is that the issued prepared-register JWE does not
+    ;; carry the victim's profile id.
+    (t/is (th/success? out))
+
+    (let [token  (-> out :result :token)
+          claims (tokens/decode th/*system* token)]
+      (t/is (= :prepared-register (:iss claims)))
+      ;; This is the root-cause assertion: an anonymous prepare-register
+      ;; call must NEVER embed an existing profile's id.
+      (t/is (nil? (:profile-id claims))
+            "prepare-register must not embed existing profile id of an anonymous caller"))))
+
+(t/deftest register-profile-with-invitation-must-not-take-over-existing-account
+  (with-mocks [_mock {:target 'app.email/send! :return nil}]
+    (let [;; Victim profile exists but is not yet active (e.g. registered
+          ;; but has not clicked the verification link). This is the
+          ;; remaining attack surface after fix 1b: `prepare-register`
+          ;; will not reject this case, so the `register-profile` path
+          ;; must enforce the security invariants on its own.
+          victim   (th/create-profile* 1 {:is-active false
+                                          :email "victim@corp.tld"})
+
+          ;; Attacker mints a valid `:team-invitation` JWE for the victim's
+          ;; email. No member-id is included (matches what an attacker
+          ;; obtains via `create-team-invitations` against their own team
+          ;; before the victim has joined).
+          itoken   (tokens/generate th/*system*
+                                    {:iss :team-invitation
+                                     :exp (ct/in-future "48h")
+                                     :role :editor
+                                     :team-id uuid/zero
+                                     :member-email "victim@corp.tld"})
+
+          ;; Step 1 (anonymous): prepare-register-profile with the victim's
+          ;; email + the invitation token.
+          prep-out (th/command! {::th/type :prepare-register-profile
+                                 :invitation-token itoken
+                                 :fullname "Mallory"
+                                 :email "victim@corp.tld"
+                                 :password "Whatever1!"})
+
+          rtoken   (-> prep-out :result :token)
+
+          ;; Step 2 (anonymous): register-profile with the prepared token.
+          reg-out  (th/command! {::th/type :register-profile
+                                 :token rtoken})
+
+          result   (:result reg-out)
+          mdata    (meta result)]
+
+      ;; The first call may succeed; the issue is what the second call
+      ;; produces. We assert the security invariants on its result.
+      (t/is (th/success? prep-out))
+
+      ;; INVARIANT 1: register-profile must NOT install a session for the
+      ;; victim. `session/create-fn` is wired via
+      ;; `rph/with-transform`, which appends to
+      ;; `:app.rpc/response-transform-fns`. If that vector is non-empty
+      ;; for an anonymous register that targets an EXISTING profile, the
+      ;; server is about to mint an `auth-token` cookie bound to the
+      ;; victim — i.e. account takeover.
+      (t/is (empty? (:app.rpc/response-transform-fns mdata))
+            "register-profile must not create a session for an existing victim profile")
+
+      ;; INVARIANT 2: register-profile must NOT echo back an invitation
+      ;; token that authenticates as the victim. When the response
+      ;; contains both `:id` matching the victim and `:invitation-token`,
+      ;; the frontend treats the user as logged-in for that profile.
+      (when (and (map? result)
+                 (= (:id victim) (:id result)))
+        (t/is (not (contains? result :invitation-token))
+              "register-profile must not return an invitation-token bound to an existing victim profile"))
+
+      ;; INVARIANT 3: the server must NOT have taken the
+      ;; "accept-invitation" branch (which is the one that mints a
+      ;; session). For an existing victim profile, the operation
+      ;; should fall through to the harmless "repeated registry" path.
+      (t/is (not= "accept-invitation"
+                  (get-in mdata [:app.loggers.audit/context :action]))
+            "register-profile must not run the accept-invitation branch for an existing victim profile")
+      ;; The victim must remain inactive: nothing in this anonymous
+      ;; flow should have flipped `is-active` to true.
+      (let [reloaded (th/db-get :profile {:id (:id victim)})]
+        (t/is (false? (:is-active reloaded))
+              "register-profile must not activate the victim profile")))))
+
+(t/deftest verify-email-with-invitation-token-propagates-it
+  ;; A `:verify-email` JWE that carries `:invitation-token` (as
+  ;; produced by `register-profile` for the not-active+invitation
+  ;; case) must propagate that token through the verify-token RPC
+  ;; result so the frontend can resume the team-invitation flow.
+  (let [profile (th/create-profile* 1 {:is-active false})
+        itoken  (tokens/generate th/*system*
+                                 {:iss :team-invitation
+                                  :exp (ct/in-future "48h")
+                                  :role :editor
+                                  :team-id uuid/zero
+                                  :member-email (:email profile)})
+        vtoken  (tokens/generate th/*system*
+                                 {:iss :verify-email
+                                  :exp (ct/in-future "72h")
+                                  :profile-id (:id profile)
+                                  :email (:email profile)
+                                  :invitation-token itoken})
+
+        out     (th/command! {::th/type :verify-token
+                              :token vtoken})
+        result  (:result out)]
+
+    (t/is (th/success? out))
+    (t/is (= :verify-email (:iss result)))
+    (t/is (= itoken (:invitation-token result))
+          "verify-token must echo back the invitation-token from the verify-email JWE")
+
+    ;; And the profile must now be active.
+    (let [reloaded (th/db-get :profile {:id (:id profile)})]
+      (t/is (true? (:is-active reloaded))))))
+
 (t/deftest email-change-request
   (with-mocks [mock {:target 'app.email/send! :return nil}]
     (let [profile (th/create-profile* 1)

backend/test/backend_tests/rpc_team_test.clj

@@ -11,6 +11,7 @@
    [app.common.uuid :as uuid]
    [app.config :as cf]
    [app.db :as db]
+   [app.email.blacklist :as email.blacklist]
    [app.http :as http]
    [app.rpc :as-alias rpc]
    [app.storage :as sto]
@@ -102,6 +103,46 @@
           (t/is (= :validation (:type edata)))
           (t/is (= :member-is-muted (:code edata))))))))
 
+(t/deftest create-team-invitations-blacklisted-domain
+  (with-mocks [mock {:target 'app.email/send! :return nil}]
+    (let [profile1 (th/create-profile* 1 {:is-active true})
+          team     (th/create-team* 1 {:profile-id (:id profile1)})
+          data     {::th/type :create-team-invitations
+                    ::rpc/profile-id (:id profile1)
+                    :team-id (:id team)
+                    :role :editor}]
+
+      ;; invite from a directly blacklisted domain should fail
+      (with-redefs [email.blacklist/enabled?  (constantly true)
+                    email.blacklist/contains? (fn [_ email]
+                                                (clojure.string/ends-with? email "@blacklisted.com"))]
+        (let [out (th/command! (assoc data :emails ["user@blacklisted.com"]))]
+          (t/is (not (th/success? out)))
+          (t/is (= 0 (:call-count @mock)))
+          (let [edata (-> out :error ex-data)]
+            (t/is (= :restriction (:type edata)))
+            (t/is (= :email-domain-is-not-allowed (:code edata))))))
+
+      ;; invite from a subdomain of a blacklisted domain should also fail
+      (th/reset-mock! mock)
+      (with-redefs [email.blacklist/enabled?  (constantly true)
+                    email.blacklist/contains? (fn [_ email]
+                                                (clojure.string/ends-with? email "@sub.blacklisted.com"))]
+        (let [out (th/command! (assoc data :emails ["user@sub.blacklisted.com"]))]
+          (t/is (not (th/success? out)))
+          (t/is (= 0 (:call-count @mock)))
+          (let [edata (-> out :error ex-data)]
+            (t/is (= :restriction (:type edata)))
+            (t/is (= :email-domain-is-not-allowed (:code edata))))))
+
+      ;; invite from a non-blacklisted domain should succeed
+      (th/reset-mock! mock)
+      (with-redefs [email.blacklist/enabled?  (constantly true)
+                    email.blacklist/contains? (constantly false)]
+        (let [out (th/command! (assoc data :emails ["user@allowed.com"]))]
+          (t/is (th/success? out))
+          (t/is (= 1 (:call-count @mock))))))))
+
 (t/deftest create-team-invitations-with-request-access
   (with-mocks [mock {:target 'app.email/send! :return nil}]
     (let [profile1  (th/create-profile* 1 {:is-active true})
@@ -726,3 +767,82 @@
       (t/is (th/success? (th/command! data)))
       (t/is (= 1 (:call-count @mock))))))
 
+(t/deftest create-team-with-invalid-name
+  (let [profile (th/create-profile* 1 {:is-active true})]
+
+    ;; name with a dot should fail
+    (let [data {::th/type :create-team
+                ::rpc/profile-id (:id profile)
+                :name "foo.bar"}
+          out  (th/command! data)]
+      (t/is (not (th/success? out)))
+      (t/is (th/ex-of-type? (:error out) :validation))
+      (t/is (th/ex-of-code? (:error out) :params-validation)))
+
+    ;; name with a colon should fail
+    (let [data {::th/type :create-team
+                ::rpc/profile-id (:id profile)
+                :name "foo:bar"}
+          out  (th/command! data)]
+      (t/is (not (th/success? out)))
+      (t/is (th/ex-of-type? (:error out) :validation))
+      (t/is (th/ex-of-code? (:error out) :params-validation)))
+
+    ;; name with a slash should fail
+    (let [data {::th/type :create-team
+                ::rpc/profile-id (:id profile)
+                :name "foo/bar"}
+          out  (th/command! data)]
+      (t/is (not (th/success? out)))
+      (t/is (th/ex-of-type? (:error out) :validation))
+      (t/is (th/ex-of-code? (:error out) :params-validation)))
+
+    ;; valid name should succeed
+    (let [data {::th/type :create-team
+                ::rpc/profile-id (:id profile)
+                :name "My Valid Team"}
+          out  (th/command! data)]
+      (t/is (th/success? out)))))
+
+(t/deftest update-team-with-invalid-name
+  (let [profile (th/create-profile* 1 {:is-active true})
+        team    (th/create-team* 1 {:profile-id (:id profile)})]
+
+    ;; name with a dot should fail
+    (let [data {::th/type :update-team
+                ::rpc/profile-id (:id profile)
+                :id (:id team)
+                :name "foo.bar"}
+          out  (th/command! data)]
+      (t/is (not (th/success? out)))
+      (t/is (th/ex-of-type? (:error out) :validation))
+      (t/is (th/ex-of-code? (:error out) :params-validation)))
+
+    ;; name with a colon should fail
+    (let [data {::th/type :update-team
+                ::rpc/profile-id (:id profile)
+                :id (:id team)
+                :name "foo:bar"}
+          out  (th/command! data)]
+      (t/is (not (th/success? out)))
+      (t/is (th/ex-of-type? (:error out) :validation))
+      (t/is (th/ex-of-code? (:error out) :params-validation)))
+
+    ;; name with a slash should fail
+    (let [data {::th/type :update-team
+                ::rpc/profile-id (:id profile)
+                :id (:id team)
+                :name "foo/bar"}
+          out  (th/command! data)]
+      (t/is (not (th/success? out)))
+      (t/is (th/ex-of-type? (:error out) :validation))
+      (t/is (th/ex-of-code? (:error out) :params-validation)))
+
+    ;; valid name should succeed
+    (let [data {::th/type :update-team
+                ::rpc/profile-id (:id profile)
+                :id (:id team)
+                :name "My Valid Team"}
+          out  (th/command! data)]
+      (t/is (th/success? out)))))
+

backend/test/backend_tests/rpc_webhooks_test.clj

@@ -37,7 +37,7 @@
    (t/is (contains? result :mtype))))
 
 (t/deftest webhook-crud
-  (with-mocks [http-mock {:target 'app.http.client/req!
+  (with-mocks [http-mock {:target 'app.http.client/req
                           :return {:status 200}}]
 
     (let [prof    (th/create-profile* 1 {:is-active true})
@@ -151,7 +151,7 @@
             (t/is (= (:code error-data) :object-not-found))))))))
 
 (t/deftest webhooks-permissions-crud-viewer-only
-  (with-mocks [http-mock {:target 'app.http.client/req!
+  (with-mocks [http-mock {:target 'app.http.client/req
                           :return {:status 200}}]
     (let [owner       (th/create-profile* 1 {:is-active true})
           viewer      (th/create-profile* 2 {:is-active true})
@@ -214,7 +214,7 @@
       (th/reset-mock! http-mock))))
 
 (t/deftest webhooks-permissions-crud-viewer-owner
-  (with-mocks [http-mock {:target 'app.http.client/req!
+  (with-mocks [http-mock {:target 'app.http.client/req
                           :return {:status 200}}]
     (let [owner       (th/create-profile* 1 {:is-active true})
           viewer      (th/create-profile* 2 {:is-active true})
@@ -269,7 +269,7 @@
           (t/is (= (:code error-data) :object-not-found)))))))
 
 (t/deftest webhooks-quotes
-  (with-mocks [http-mock {:target 'app.http.client/req!
+  (with-mocks [http-mock {:target 'app.http.client/req
                           :return {:status 200}}]
 
     (let [prof    (th/create-profile* 1 {:is-active true})

backend/test/backend_tests/storage_test.clj

@@ -169,7 +169,8 @@
         (t/is (= 2 (:count res))))
 
       ;; run the touched gc task
-      (let [res (th/run-task! :storage-gc-touched {})]
+      (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                  (th/run-task! :storage-gc-touched {}))]
         (t/is (= 2 (:freeze res)))
         (t/is (= 0 (:delete res))))
 
@@ -229,7 +230,8 @@
     (t/is (nil? (:error out2)))
 
     ;; run the touched gc task
-    (let [res (th/run-task! :storage-gc-touched {})]
+    (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                (th/run-task! :storage-gc-touched {}))]
       (t/is (= 5 (:freeze res)))
       (t/is (= 0 (:delete res)))
 
@@ -249,7 +251,8 @@
         (th/db-exec-one! ["update storage_object set touched_at=?" (ct/now)])
 
         ;; Run the task again
-        (let [res (th/run-task! :storage-gc-touched {})]
+        (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                    (th/run-task! :storage-gc-touched {}))]
           (t/is (= 2 (:freeze res)))
           (t/is (= 3 (:delete res))))
 
@@ -295,7 +298,8 @@
       (th/db-exec! ["update storage_object set touched_at=?" (ct/now)])
 
       ;; run the touched gc task
-      (let [res (th/run-task! :storage-gc-touched {})]
+      (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                  (th/run-task! :storage-gc-touched {}))]
         (t/is (= 2 (:freeze res)))
         (t/is (= 0 (:delete res))))
 
@@ -310,7 +314,8 @@
       (t/is (= 2 (:processed res))))
 
     ;; run the touched gc task
-    (let [res (th/run-task! :storage-gc-touched {})]
+    (let [res (binding [ct/*clock* (ct/fixed-clock (ct/in-future {:hours 3}))]
+                (th/run-task! :storage-gc-touched {}))]
       (t/is (= 0 (:freeze res)))
       (t/is (= 2 (:delete res))))
 
@@ -336,7 +341,7 @@
         (t/is (= 0 (:delete res)))))
 
 
-    (binding [ct/*clock* (ct/fixed-clock (ct/plus now {:minutes 1}))]
+    (binding [ct/*clock* (ct/fixed-clock (ct/plus now {:hours 3}))]
       (let [res (th/run-task! :storage-gc-touched {})]
         (t/is (= 0 (:freeze res)))
         (t/is (= 1 (:delete res)))))

backend/test/backend_tests/tasks_telemetry_test.clj

@@ -6,40 +6,905 @@
 
 (ns backend-tests.tasks-telemetry-test
   (:require
+   [app.common.time :as ct]
+   [app.common.uuid :as uuid]
+   [app.config :as cf]
    [app.db :as db]
+   [app.loggers.audit :as audit]
+   [app.tasks.telemetry :as telemetry]
+   [app.util.blob :as blob]
+   [app.util.json :as json]
    [backend-tests.helpers :as th]
-   [clojure.pprint :refer [pprint]]
    [clojure.test :as t]
-   [mockery.core :refer [with-mocks]]))
+   [mockery.core :refer [with-mocks]]
+   [promesa.exec :as px]))
 
 (t/use-fixtures :once th/state-init)
-(t/use-fixtures :each th/database-reset)
+
+;; Mock px/sleep for all tests to avoid 10s random delays.
+;; Composed with database-reset so both apply.
+(defn- test-fixture [next]
+  (th/database-reset
+   (fn []
+     (with-redefs [px/sleep (constantly nil)]
+       (next)))))
+
+(t/use-fixtures :each test-fixture)
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; HELPERS
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(defn- insert-telemetry-row!
+  "Insert a single anonymised audit_log row as the telemetry mode does."
+  ([name] (insert-telemetry-row! name {}))
+  ([name {:keys [tracked-at created-at source]
+          :or   {tracked-at (ct/now)
+                 created-at (ct/now)
+                 source     "telemetry:backend"}}]
+   (th/db-insert! :audit-log
+                  {:id         (uuid/next)
+                   :name       name
+                   :type       "action"
+                   :source     source
+                   :profile-id uuid/zero
+                   :ip-addr    (db/inet "0.0.0.0")
+                   :props      (db/tjson {})
+                   :context    (db/tjson {})
+                   :tracked-at tracked-at
+                   :created-at created-at})))
+
+(defn- count-telemetry-rows []
+  (-> (th/db-exec-one! ["SELECT count(*) AS cnt FROM audit_log WHERE source IN ('telemetry:backend', 'telemetry:frontend')"])
+      :cnt
+      long))
+
+(defn- decode-event-batch
+  "Decode the base64+fressian+zstd event-batch sent to the mock."
+  [b64-str]
+  (blob/decode-str b64-str))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; STATS / REPORT STRUCTURE TESTS (existing behaviour, extended)
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 
 (t/deftest test-base-report-data-structure
-  (with-mocks [mock {:target 'app.tasks.telemetry/send!
+  (with-mocks [mock {:target 'app.tasks.telemetry/make-legacy-request
                      :return nil}]
     (let [prof (th/create-profile* 1 {:is-active true
-                                      :props {:newsletter-news true}})]
+                                      :props {:newsletter-updates true}})]
 
       (th/run-task! :telemetry {:send? true :enabled? true})
 
       (t/is (:called? @mock))
       (let [[_ data] (-> @mock :call-args)]
+        (t/is (= :telemetry-legacy-report (:type data)))
         (t/is (contains? data :subscriptions))
-        (t/is (= [(:email prof)] (get-in data [:subscriptions :newsletter-news])))
-        (t/is (contains? data :total-fonts))
-        (t/is (contains? data :total-users))
-        (t/is (contains? data :total-projects))
-        (t/is (contains? data :total-files))
-        (t/is (contains? data :total-teams))
-        (t/is (contains? data :total-comments))
-        (t/is (contains? data :instance-id))
-        (t/is (contains? data :jvm-cpus))
-        (t/is (contains? data :jvm-heap-max))
-        (t/is (contains? data :max-users-on-team))
-        (t/is (contains? data :avg-users-on-team))
-        (t/is (contains? data :max-files-on-project))
-        (t/is (contains? data :avg-files-on-project))
-        (t/is (contains? data :max-projects-on-team))
-        (t/is (contains? data :avg-files-on-project))
-        (t/is (contains? data :version))))))
+        (t/is (= [(:email prof)] (:subscriptions data)))
+        (t/is (contains? data :stats))
+        (let [stats (:stats data)]
+          (t/is (contains? stats :total-fonts))
+          (t/is (contains? stats :total-users))
+          (t/is (contains? stats :total-projects))
+          (t/is (contains? stats :total-files))
+          (t/is (contains? stats :total-teams))
+          (t/is (contains? stats :total-comments))
+          (t/is (contains? stats :jvm-cpus))
+          (t/is (contains? stats :jvm-heap-max))
+          (t/is (contains? stats :max-users-on-team))
+          (t/is (contains? stats :avg-users-on-team))
+          (t/is (contains? stats :max-files-on-project))
+          (t/is (contains? stats :avg-files-on-project))
+          (t/is (contains? stats :max-projects-on-team))
+          (t/is (contains? stats :avg-files-on-project))
+          (t/is (contains? stats :email-domains))
+          (t/is (= ["nodomain.com"] (:email-domains stats)))
+          ;; public-uri must be a string
+          (t/is (string? (:public-uri stats)))
+          (t/is (not-empty (:public-uri stats))))
+        (t/is (contains? data :version))
+        (t/is (contains? data :instance-id))))))
+
+(t/deftest test-telemetry-disabled-no-send
+  ;; When telemetry is disabled and no newsletter subscriptions exist,
+  ;; make-legacy-request must not be called at all.
+  (with-mocks [mock {:target 'app.tasks.telemetry/make-legacy-request
+                     :return nil}]
+    (with-redefs [cf/flags #{}]
+      (th/create-profile* 1 {:is-active true})
+      (th/run-task! :telemetry {:send? true})
+      (t/is (not (:called? @mock))))))
+
+(t/deftest test-telemetry-disabled-newsletter-only-send
+  ;; When telemetry is disabled but a user has newsletter-updates opted in,
+  ;; make-legacy-request is called once with only subscriptions + version (no stats).
+  (with-mocks [mock {:target 'app.tasks.telemetry/make-legacy-request
+                     :return nil}]
+    (with-redefs [cf/flags #{}]
+      (let [prof (th/create-profile* 1 {:is-active true
+                                        :props {:newsletter-updates true}})]
+        (th/run-task! :telemetry {:send? true})
+        (t/is (:called? @mock))
+        (let [[_ data] (:call-args @mock)]
+          ;; Limited payload — no stats
+          (t/is (contains? data :subscriptions))
+          (t/is (contains? data :version))
+          (t/is (not (contains? data :stats)))
+          (t/is (= [(:email prof)] (:subscriptions data))))))))
+
+(t/deftest test-send-is-skipped-when-send?-false
+  ;; Passing send?=false must suppress all HTTP calls even when enabled.
+  (with-mocks [mock {:target 'app.tasks.telemetry/make-legacy-request
+                     :return nil}]
+    (with-redefs [cf/flags #{:telemetry}]
+      (th/create-profile* 1 {:is-active true})
+      (th/run-task! :telemetry {:send? false :enabled? true})
+      (t/is (not (:called? @mock))))))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; AUDIT-EVENT BATCH COLLECTION TESTS
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(t/deftest test-no-audit-events-no-batch-call
+  ;; When telemetry is enabled but there are no audit_log rows with
+  ;; source='telemetry', the batch send path must not be invoked.
+  (with-mocks [legacy-mock {:target 'app.tasks.telemetry/make-legacy-request
+                            :return nil}
+               batch-mock  {:target 'app.tasks.telemetry/send-event-batch
+                            :return true}]
+    (with-redefs [cf/flags #{:telemetry}]
+      (th/run-task! :telemetry {:send? true :enabled? true})
+      (t/is (:called? @legacy-mock))
+      (t/is (not (:called? @batch-mock))))))
+
+(t/deftest test-audit-events-sent-and-deleted-on-success
+  ;; Happy path: telemetry rows are collected, shipped as a batch and
+  ;; deleted from the table when the endpoint returns success.
+  (with-mocks [legacy-mock {:target 'app.tasks.telemetry/make-legacy-request
+                            :return nil}
+               batch-mock  {:target 'app.tasks.telemetry/send-event-batch
+                            :return true}]
+    (with-redefs [cf/flags #{:telemetry}]
+      (insert-telemetry-row! "navigate")
+      (insert-telemetry-row! "create-file")
+      (insert-telemetry-row! "update-file")
+
+      (t/is (= 3 (count-telemetry-rows)))
+
+      (th/run-task! :telemetry {:send? true :enabled? true})
+
+      ;; batch send was called at least once
+      (t/is (:called? @batch-mock))
+
+      ;; all rows deleted after successful send
+      (t/is (= 0 (count-telemetry-rows))))))
+
+(t/deftest test-audit-events-kept-on-batch-failure
+  ;; When the batch endpoint returns failure the rows must be retained
+  ;; so the next scheduled run can retry.
+  (with-mocks [legacy-mock {:target 'app.tasks.telemetry/make-legacy-request
+                            :return nil}
+               batch-mock  {:target 'app.tasks.telemetry/send-event-batch
+                            :return false}]
+    (with-redefs [cf/flags #{:telemetry}]
+      (insert-telemetry-row! "navigate")
+      (insert-telemetry-row! "create-file")
+
+      (th/run-task! :telemetry {:send? true :enabled? true})
+
+      (t/is (:called? @batch-mock))
+      ;; rows still present — not deleted on failure
+      (t/is (= 2 (count-telemetry-rows))))))
+
+(t/deftest test-audit-events-not-collected-when-audit-log-flag-set
+  ;; When the :audit-log flag is active, mode C is disabled and the
+  ;; batch path must never run (audit-log owns those rows instead).
+  (with-mocks [legacy-mock {:target 'app.tasks.telemetry/make-legacy-request
+                            :return nil}
+               batch-mock  {:target 'app.tasks.telemetry/send-event-batch
+                            :return true}]
+    (with-redefs [cf/flags #{:telemetry :audit-log}]
+      (insert-telemetry-row! "navigate")
+
+      (th/run-task! :telemetry {:send? true :enabled? true})
+
+      (t/is (not (:called? @batch-mock)))
+      ;; row untouched
+      (t/is (= 1 (count-telemetry-rows))))))
+
+(t/deftest test-batch-payload-contains-required-fields
+  ;; Inspect the actual arguments forwarded to send-event-batch to
+  ;; verify the payload carries instance-id, version and events.
+  (let [captured (atom nil)]
+    (with-mocks [legacy-mock {:target 'app.tasks.telemetry/make-legacy-request
+                              :return nil}]
+      (with-redefs [cf/flags #{:telemetry}
+                    telemetry/send-event-batch
+                    (fn [_cfg batch]
+                      (reset! captured batch)
+                      true)]
+        (insert-telemetry-row! "navigate")
+        (insert-telemetry-row! "create-file")
+
+        (th/run-task! :telemetry {:send? true :enabled? true})
+
+        (t/is (some? @captured))
+        (let [batch @captured]
+          ;; batch is a seq of event maps
+          (t/is (seq batch))
+          (t/is (= 2 (count batch)))
+          ;; each event has name, type, source — profile-id is preserved,
+          ;; props and ip-addr are stripped
+          (let [ev (first batch)]
+            (t/is (contains? ev :name))
+            (t/is (contains? ev :type))
+            (t/is (contains? ev :source))
+            (t/is (contains? ev :profile-id))
+            ;; props are present but empty (stripped at ingest time)
+            (t/is (= {} (:props ev)))
+            (t/is (not (contains? ev :ip-addr)))))))))
+
+(t/deftest test-batch-encoding-is-decodable
+  ;; Verify that encode-batch produces a blob that round-trips back
+  ;; through blob/decode to the original data.
+  (let [events [{:name "navigate" :type "action" :source "telemetry"
+                 :tracked-at (ct/now)}
+                {:name "create-file" :type "action" :source "telemetry"
+                 :tracked-at (ct/now)}]
+        ;; Call the private fn through the ns-mapped var
+        encode  (ns-resolve 'app.tasks.telemetry 'encode-batch)
+        encoded (encode events)
+        decoded (decode-event-batch encoded)]
+    (t/is (string? encoded))
+    (t/is (seq decoded))
+    (t/is (= (count events) (count decoded)))
+    (t/is (= "navigate" (:name (first decoded))))
+    (t/is (= "create-file" (:name (second decoded))))))
+
+(t/deftest test-multiple-batches-when-many-events
+  ;; Lower batch-size to 1 so that 3 events produce 3 separate
+  ;; HTTP requests and verify all are sent and all rows deleted.
+  (let [call-count (atom 0)]
+    (with-mocks [legacy-mock {:target 'app.tasks.telemetry/make-legacy-request
+                              :return nil}]
+      (with-redefs [cf/flags             #{:telemetry}
+                    telemetry/batch-size 1
+                    telemetry/send-event-batch
+                    (fn [_cfg _batch]
+                      (swap! call-count inc)
+                      true)]
+        (insert-telemetry-row! "navigate")
+        (insert-telemetry-row! "create-file")
+        (insert-telemetry-row! "update-file")
+
+        (th/run-task! :telemetry {:send? true :enabled? true})
+
+        ;; Each event is fetched and sent in its own loop iteration
+        (t/is (= 3 @call-count))
+        ;; All rows deleted after all iterations succeed
+        (t/is (= 0 (count-telemetry-rows)))))))
+
+(t/deftest test-partial-failure-stops-remaining-batches
+  ;; With batch-size 1, when the second send fails the loop stops.
+  ;; The first batch was already deleted; the two remaining rows
+  ;; are retained for the next run.
+  (let [call-count (atom 0)]
+    (with-mocks [legacy-mock {:target 'app.tasks.telemetry/make-legacy-request
+                              :return nil}]
+      (with-redefs [cf/flags             #{:telemetry}
+                    telemetry/batch-size 1
+                    telemetry/send-event-batch
+                    (fn [_cfg _batch]
+                      (swap! call-count inc)
+                      ;; fail on the second call
+                      (not= 2 @call-count))]
+        (insert-telemetry-row! "navigate")
+        (insert-telemetry-row! "create-file")
+        (insert-telemetry-row! "update-file")
+
+        (th/run-task! :telemetry {:send? true :enabled? true})
+
+        ;; Stopped at iteration 2 — third event never attempted
+        (t/is (= 2 @call-count))
+        ;; First batch was deleted on success; 2 rows remain for retry
+        (t/is (= 2 (count-telemetry-rows)))))))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; GC / RETENTION-WINDOW TESTS
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(t/deftest test-gc-purges-events-older-than-7-days
+  ;; Insert events from 8 days ago (stale) and from today (fresh).
+  ;; After the task runs, stale events must be purged by GC and fresh
+  ;; ones shipped by the batch sender.
+  (with-mocks [legacy-mock {:target 'app.tasks.telemetry/make-legacy-request
+                            :return nil}
+               batch-mock  {:target 'app.tasks.telemetry/send-event-batch
+                            :return true}]
+    (with-redefs [cf/flags #{:telemetry}]
+      (let [now        (ct/now)
+            eight-days (ct/minus now (ct/duration {:days 8}))]
+        ;; Stale events (older than 7 days)
+        (insert-telemetry-row! "stale-1" {:created-at eight-days :tracked-at eight-days})
+        (insert-telemetry-row! "stale-2" {:created-at eight-days :tracked-at eight-days})
+        ;; Fresh events (today)
+        (insert-telemetry-row! "fresh-1" {:created-at now :tracked-at now})
+        (insert-telemetry-row! "fresh-2" {:created-at now :tracked-at now})
+
+        (t/is (= 4 (count-telemetry-rows)))
+
+        (th/run-task! :telemetry {:send? true :enabled? true})
+
+        ;; GC purged the 2 stale rows, batch sender shipped the 2 fresh ones
+        (t/is (= 0 (count-telemetry-rows)))))))
+
+(t/deftest test-gc-keeps-events-within-7-day-window
+  ;; When all events are within the 7-day window, GC must not delete
+  ;; anything and all rows are forwarded to the batch sender.
+  (let [batch-events (atom nil)]
+    (with-mocks [legacy-mock {:target 'app.tasks.telemetry/make-legacy-request
+                              :return nil}]
+      (with-redefs [cf/flags #{:telemetry}
+                    telemetry/send-event-batch
+                    (fn [_cfg batch]
+                      (reset! batch-events batch)
+                      true)]
+        (let [six-days-ago (ct/minus (ct/now) (ct/duration {:days 6}))]
+          (insert-telemetry-row! "recent-1" {:created-at six-days-ago :tracked-at six-days-ago})
+          (insert-telemetry-row! "recent-2" {:created-at six-days-ago :tracked-at six-days-ago}))
+
+        (th/run-task! :telemetry {:send? true :enabled? true})
+
+        ;; Both events forwarded — GC left them alone
+        (t/is (= 2 (count @batch-events)))
+        (t/is (= 0 (count-telemetry-rows)))))))
+
+(t/deftest test-gc-deletes-only-stale-events
+  ;; Insert a mix of stale (8 days old) and fresh (1 day old) events.
+  ;; After GC, only fresh events should remain for the batch sender.
+  (let [batch-events (atom nil)]
+    (with-mocks [legacy-mock {:target 'app.tasks.telemetry/make-legacy-request
+                              :return nil}]
+      (with-redefs [cf/flags #{:telemetry}
+                    telemetry/send-event-batch
+                    (fn [_cfg batch]
+                      (reset! batch-events batch)
+                      true)]
+        (let [eight-days (ct/minus (ct/now) (ct/duration {:days 8}))
+              one-day    (ct/minus (ct/now) (ct/duration {:days 1}))]
+          (insert-telemetry-row! "stale" {:created-at eight-days :tracked-at eight-days})
+          (insert-telemetry-row! "fresh" {:created-at one-day :tracked-at one-day}))
+
+        (t/is (= 2 (count-telemetry-rows)))
+
+        (th/run-task! :telemetry {:send? true :enabled? true})
+
+        ;; GC purged stale, batch shipped fresh
+        (t/is (= 1 (count @batch-events)))
+        (t/is (= "fresh" (:name (first @batch-events))))
+        (t/is (= 0 (count-telemetry-rows)))))))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; ANONYMITY TESTS
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(t/deftest test-telemetry-rows-stored-without-pii
+  ;; Rows written to audit_log in telemetry mode must carry no PII:
+  ;; empty props, zeroed ip, profile-id=zero, source='telemetry'.
+  ;; Safe context fields (browser, os, version, etc.) are preserved
+  ;; but session-linking and access-token fields are stripped.
+  (with-redefs [cf/flags #{:telemetry}]
+    (let [_prof (th/create-profile* 1 {:is-active true})
+          safe-ctx {:browser "Chrome"
+                    :browser-version "120.0"
+                    :os "Linux"
+                    :version "2.0.0"}]
+      ;; Simulate what app.loggers.audit/process-event does in mode C
+      (th/db-insert! :audit-log
+                     {:id         (uuid/next)
+                      :name       "create-project"
+                      :type       "action"
+                      :source     "telemetry:backend"
+                      :profile-id uuid/zero
+                      :ip-addr    (db/inet "0.0.0.0")
+                      :props      (db/tjson {})
+                      :context    (db/tjson safe-ctx)
+                      :tracked-at (ct/now)
+                      :created-at (ct/now)})
+
+      (let [[row] (th/db-exec! ["SELECT * FROM audit_log WHERE source = 'telemetry:backend'"])]
+        (t/is (= "telemetry:backend" (:source row)))
+        ;; props are always empty
+        (t/is (= "{}" (str (:props row))))
+        ;; ip_addr is the sentinel zero address
+        (t/is (= "0.0.0.0" (str (:ip-addr row))))
+        ;; profile-id is uuid/zero — not a real user id
+        (t/is (= uuid/zero (:profile-id row)))))))
+
+(t/deftest test-batch-events-contain-no-pii-fields
+  ;; The event maps forwarded to send-event-batch must not carry props,
+  ;; ip-addr or profile-id. Safe context fields (browser, os, etc.) may
+  ;; be present but session-linking keys must be absent.
+  (let [captured-batch (atom nil)
+        ;; Insert a row that carries safe context (as the real path does)
+        safe-ctx       {:browser "Firefox" :browser-version "121.0"
+                        :os "macOS" :session "should-be-stripped"
+                        :external-session-id "also-stripped"}]
+    (with-mocks [legacy-mock {:target 'app.tasks.telemetry/make-legacy-request
+                              :return nil}]
+      (with-redefs [cf/flags #{:telemetry}
+                    telemetry/send-event-batch
+                    (fn [_cfg batch]
+                      (reset! captured-batch batch)
+                      true)]
+        ;; Insert with safe context already pre-filtered (as the ingest path does)
+        (th/db-insert! :audit-log
+                       {:id         (uuid/next)
+                        :name       "navigate"
+                        :type       "action"
+                        :source     "telemetry:frontend"
+                        :profile-id uuid/zero
+                        :ip-addr    (db/inet "0.0.0.0")
+                        :props      (db/tjson {})
+                        :context    (db/tjson (dissoc safe-ctx :session :external-session-id))
+                        :tracked-at (ct/now)
+                        :created-at (ct/now)})
+
+        (th/run-task! :telemetry {:send? true :enabled? true})
+
+        (t/is (= 1 (count @captured-batch)))
+        (let [ev (first @captured-batch)]
+          ;; must have the core identity fields including profile-id
+          (t/is (contains? ev :name))
+          (t/is (contains? ev :type))
+          (t/is (contains? ev :source))
+          (t/is (contains? ev :tracked-at))
+          (t/is (contains? ev :profile-id))
+          ;; props are present but empty (stripped at ingest time)
+          (t/is (= {} (:props ev)))
+          ;; ip-addr is stripped
+          (t/is (not (contains? ev :ip-addr)))
+          ;; context may be present and must not contain session-linking keys
+          (when-let [ctx (:context ev)]
+            (t/is (not (contains? ctx :session)))
+            (t/is (not (contains? ctx :external-session-id)))
+            ;; safe keys should be present
+            (t/is (contains? ctx :browser))))))))
+
+(t/deftest test-telemetry-rows-have-day-precision-timestamps
+  ;; Telemetry events must be stored with timestamps truncated to day
+  ;; precision so that exact event timing cannot be inferred.
+  (with-redefs [cf/flags #{:telemetry}]
+    (let [process-event (ns-resolve 'app.loggers.audit 'process-event)
+          profile       (th/create-profile* 1 {:is-active true})
+          tnow          (ct/now)
+          event         {:type       "action"
+                         :name       "create-project"
+                         :profile-id (:id profile)
+                         :source     "backend"
+                         :props      {}
+                         :context    {}
+                         :created-at tnow
+                         :tracked-at tnow
+                         :ip-addr    "0.0.0.0"}]
+      (db/tx-run! th/*system* process-event event)
+      (let [[row] (th/db-exec! ["SELECT * FROM audit_log WHERE source = 'telemetry:backend'"])]
+        (t/is (some? row))
+        (let [created-at (:created-at row)
+              tracked-at (:tracked-at row)
+              day-now    (ct/truncate (ct/now) :days)]
+          ;; Both timestamps must equal midnight of the current day
+          (t/is (= day-now created-at))
+          (t/is (= day-now tracked-at)))))))
+
+(t/deftest test-backend-ingest-full-row-shape
+  ;; Verify the full row shape stored by process-event in telemetry mode:
+  ;; source=telemetry:backend, empty props, zeroed ip, context filtered to safe
+  ;; backend keys only, profile-id preserved, timestamps truncated.
+  (with-redefs [cf/flags #{:telemetry}]
+    (let [process-event (ns-resolve 'app.loggers.audit 'process-event)
+          profile       (th/create-profile* 1 {:is-active true})
+          tnow          (ct/now)
+          event         {:type       "action"
+                         :name       "create-project"
+                         :profile-id (:id profile)
+                         :source     "backend"
+                         :context    {:initiator "app"
+                                      :version "2.0.0"
+                                      :client-version "1.0"
+                                      :client-user-agent "Mozilla/5.0"
+                                      :external-session-id "should-be-stripped"
+                                      :session "also-stripped"}
+                         :props      {:some-prop "value"}
+                         :created-at tnow
+                         :tracked-at tnow
+                         :ip-addr    "0.0.0.0"}]
+      (db/tx-run! th/*system* process-event event)
+
+      (let [[row] (th/db-exec! ["SELECT * FROM audit_log WHERE source = 'telemetry:backend'"])]
+        (t/is (some? row))
+        ;; source
+        (t/is (= "telemetry:backend" (:source row)))
+        ;; profile-id preserved
+        (t/is (= (:id profile) (:profile-id row)))
+        ;; name
+        (t/is (= "create-project" (:name row)))
+        ;; type
+        (t/is (= "action" (:type row)))
+        ;; props stripped to empty
+        (t/is (= "{}" (str (:props row))))
+        ;; ip zeroed
+        (t/is (= "0.0.0.0" (str (:ip-addr row))))
+        ;; timestamps truncated to day
+        (let [day-now    (ct/truncate (ct/now) :days)]
+          (t/is (= day-now (:created-at row)))
+          (t/is (= day-now (:tracked-at row))))
+        ;; context filtered: only safe backend keys retained
+        (let [ctx (db/decode-transit-pgobject (:context row))]
+          (t/is (= "app" (:initiator ctx)))
+          (t/is (= "2.0.0" (:version ctx)))
+          (t/is (= "1.0" (:client-version ctx)))
+          (t/is (= "Mozilla/5.0" (:client-user-agent ctx)))
+          ;; session-linking keys stripped
+          (t/is (not (contains? ctx :external-session-id)))
+          (t/is (not (contains? ctx :session))))))))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; FILTER-TELEMETRY-CONTEXT UNIT TESTS
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(t/deftest test-filter-telemetry-context-keeps-browser-fields
+  ;; Safe environment fields must survive the filter.
+  (let [filter-telemetry-context (ns-resolve 'app.loggers.audit 'filter-telemetry-context)
+        ctx {:browser         "Chrome"
+             :browser-version "120.0"
+             :engine          "Blink"
+             :engine-version  "120.0"
+             :os              "Windows 11"
+             :os-version      "11"
+             :device-type     "unknown"
+             :device-arch     "amd64"
+             :locale          "en-US"
+             :version         "2.0.0"
+             :screen-width    1920
+             :screen-height   1080
+             :event-origin    "workspace"}
+        result (:context (filter-telemetry-context {:source "frontend" :context ctx}))]
+    (t/is (= "Chrome" (:browser result)))
+    (t/is (= "120.0" (:browser-version result)))
+    (t/is (= "Windows 11" (:os result)))
+    (t/is (= "en-US" (:locale result)))
+    (t/is (= "workspace" (:event-origin result)))
+    (t/is (= 1920 (:screen-width result)))))
+
+(t/deftest test-filter-telemetry-context-strips-pii-keys
+  ;; Session-linking and access-token fields must be removed.
+  (let [filter-telemetry-context (ns-resolve 'app.loggers.audit 'filter-telemetry-context)
+        ctx {:browser              "Firefox"
+             :session              "abc-session-id"
+             :external-session-id  "ext-123"
+             :file-stats           {:total-shapes 42}
+             :initiator            "app"
+             :access-token-id      "tok-456"
+             :access-token-type    "api-key"}
+        result (:context (filter-telemetry-context {:source "frontend" :context ctx}))]
+    (t/is (= "Firefox" (:browser result)))
+    (t/is (not (contains? result :session)))
+    (t/is (not (contains? result :external-session-id)))
+    (t/is (not (contains? result :file-stats)))
+    (t/is (not (contains? result :initiator)))
+    (t/is (not (contains? result :access-token-id)))
+    (t/is (not (contains? result :access-token-type)))))
+
+(t/deftest test-filter-telemetry-context-empty-input
+  ;; An empty context should return an empty map without error.
+  (let [filter-telemetry-context (ns-resolve 'app.loggers.audit 'filter-telemetry-context)]
+    (t/is (= {} (:context (filter-telemetry-context {:source "frontend" :context {}}))))))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; FILTER-TELEMETRY-PROPS UNIT TESTS
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(t/deftest test-filter-telemetry-props-login-event-keeps-safe-profile-fields
+  ;; Login/register/update events carry safe profile-derived fields:
+  ;; :lang, :auth-backend, :email-domain. Raw :email is stripped.
+  (let [ftp (ns-resolve 'app.loggers.audit 'filter-telemetry-props)]
+    ;; backend login-with-password
+    (let [result (ftp {:source "backend"
+                       :name "login-with-password"
+                       :type "action"
+                       :props {:email "user@example.com"
+                               :fullname "John Doe"
+                               :lang "en"
+                               :auth-backend "password"
+                               :id (uuid/next)}})]
+      (t/is (= "en" (get-in result [:props :lang])))
+      (t/is (= "password" (get-in result [:props :auth-backend])))
+      (t/is (= "example.com" (get-in result [:props :email-domain])))
+      ;; Raw email and fullname are stripped
+      (t/is (not (contains? (:props result) :email)))
+      (t/is (not (contains? (:props result) :fullname)))
+      ;; UUID values survive the xf:filter-telemetry-props filter
+      (t/is (some? (get-in result [:props :id]))))
+
+    ;; backend register-profile
+    (let [result (ftp {:source "backend"
+                       :name "register-profile"
+                       :type "action"
+                       :props {:email "new@corp.org"
+                               :lang "es"
+                               :auth-backend "oidc"}})]
+      (t/is (= "es" (get-in result [:props :lang])))
+      (t/is (= "oidc" (get-in result [:props :auth-backend])))
+      (t/is (= "corp.org" (get-in result [:props :email-domain]))))
+
+    ;; backend login-with-oidc
+    (let [result (ftp {:source "backend"
+                       :name "login-with-oidc"
+                       :type "action"
+                       :props {:email "u@corp.io" :lang "fr" :auth-backend "oidc"}})]
+      (t/is (= "fr" (get-in result [:props :lang])))
+      (t/is (= "oidc" (get-in result [:props :auth-backend])))
+      (t/is (= "corp.io" (get-in result [:props :email-domain]))))
+
+    ;; backend update-profile
+    (let [result (ftp {:source "backend"
+                       :name "update-profile"
+                       :type "action"
+                       :props {:email "u@corp.io" :lang "de"}})]
+      (t/is (= "de" (get-in result [:props :lang])))
+      (t/is (= "corp.io" (get-in result [:props :email-domain]))))))
+
+(t/deftest test-filter-telemetry-props-frontend-identify-keeps-safe-profile-fields
+  ;; Frontend identify events also carry safe profile-derived fields.
+  (let [ftp (ns-resolve 'app.loggers.audit 'filter-telemetry-props)]
+    (let [result (ftp {:source "frontend"
+                       :name "signin"
+                       :type "identify"
+                       :props {:email "user@example.com"
+                               :fullname "Jane Doe"
+                               :lang "pt"
+                               :auth-backend "password"
+                               :some-string "should-be-stripped"}})]
+      (t/is (= "pt" (get-in result [:props :lang])))
+      (t/is (= "password" (get-in result [:props :auth-backend])))
+      (t/is (= "example.com" (get-in result [:props :email-domain])))
+      ;; PII stripped
+      (t/is (not (contains? (:props result) :email)))
+      (t/is (not (contains? (:props result) :fullname)))
+      ;; String values that are not UUID/boolean/number are stripped
+      (t/is (not (contains? (:props result) :some-string))))))
+
+(t/deftest test-filter-telemetry-props-instance-start-passthrough
+  ;; instance-start trigger events pass through as-is.
+  (let [ftp (ns-resolve 'app.loggers.audit 'filter-telemetry-props)
+        props {:total-teams 5 :total-users 42 :version "2.0"}
+        result (ftp {:source "backend"
+                     :name "instance-start"
+                     :type "trigger"
+                     :props props})]
+    (t/is (= props (:props result)))))
+
+(t/deftest test-filter-telemetry-props-generic-event-keeps-uuid-boolean-number
+  ;; Generic events (create-file, etc.) keep only entries
+  ;; whose values are UUIDs, booleans, or numbers.
+  (let [ftp (ns-resolve 'app.loggers.audit 'filter-telemetry-props)
+        id   (uuid/next)
+        result (ftp {:source "frontend"
+                     :name "create-file"
+                     :type "action"
+                     :props {:project-id id
+                             :team-id id
+                             :route "dashboard-files"
+                             :count 42
+                             :active true
+                             :label "should-be-stripped"}})]
+    ;; UUIDs survive
+    (t/is (= id (get-in result [:props :project-id])))
+    (t/is (= id (get-in result [:props :team-id])))
+    ;; Numbers survive
+    (t/is (= 42 (get-in result [:props :count])))
+    ;; Booleans survive
+    (t/is (true? (get-in result [:props :active])))
+    ;; Strings are stripped
+    (t/is (not (contains? (:props result) :route)))
+    (t/is (not (contains? (:props result) :label)))))
+
+(t/deftest test-filter-telemetry-props-navigate-keeps-route-and-ids
+  ;; Frontend navigate events keep specific routing keys: :route,
+  ;; :file-id, :team-id, :page-id.  These ids are strings because
+  ;; routing events don't coerce them.  All other props are stripped.
+  (let [ftp      (ns-resolve 'app.loggers.audit 'filter-telemetry-props)
+        file-id  (str (uuid/next))
+        team-id  (str (uuid/next))
+        page-id  (str (uuid/next))
+        result   (ftp {:source "frontend"
+                       :name "navigate"
+                       :type "action"
+                       :props {:file-id  file-id
+                               :team-id  team-id
+                               :page-id  page-id
+                               :route    "dashboard-index"
+                               :session  "abc"
+                               :count    42
+                               :active   true
+                               :label    "should-be-stripped"}})]
+    ;; Allowed routing keys survive (as strings, not coerced to UUID)
+    (t/is (= file-id (get-in result [:props :file-id])))
+    (t/is (= team-id (get-in result [:props :team-id])))
+    (t/is (= page-id (get-in result [:props :page-id])))
+    (t/is (= "dashboard-index" (get-in result [:props :route])))
+    ;; Everything else is stripped
+    (t/is (not (contains? (:props result) :session)))
+    (t/is (not (contains? (:props result) :count)))
+    (t/is (not (contains? (:props result) :active)))
+    (t/is (not (contains? (:props result) :label)))))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; SEND-EVENT-BATCH PAYLOAD STRUCTURE
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(t/deftest test-send-event-batch-payload-structure
+  ;; Verify the HTTP request sent by send-event-batch carries the
+  ;; correct outer wrapper: :type, :version, :instance-id, :events.
+  (let [captured-request (atom nil)]
+    (with-mocks [legacy-mock {:target 'app.tasks.telemetry/make-legacy-request
+                              :return nil}
+                 http-mock   {:target 'app.http.client/req
+                              :return {:status 200}}]
+      (with-redefs [cf/flags #{:telemetry}]
+        (insert-telemetry-row! "navigate")
+        (insert-telemetry-row! "create-file")
+
+        (th/run-task! :telemetry {:send? true :enabled? true})
+
+        ;; http/req was called (by both send-legacy-data and send-event-batch)
+        (t/is (:called? @http-mock))
+        ;; Find the call whose body contains :telemetry-events
+        (let [calls (filter (fn [args]
+                              (let [[_ request] args
+                                    body (:body request)]
+                                (and (string? body)
+                                     (re-find #"telemetry-events" body))))
+                            (:call-args-list @http-mock))]
+          (t/is (= 1 (count calls)))
+          (let [[_ request] (first calls)
+                body (json/decode (:body request))]
+            ;; Outer payload fields
+            (t/is (= "telemetry-events" (name (:type body))))
+            (t/is (string? (:version body)))
+            (t/is (some? (:instance-id body)))
+            ;; :events is a base64-encoded blob
+            (t/is (string? (:events body)))
+            (t/is (pos? (count (:events body))))))))))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; TASK BRANCH COVERAGE
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(t/deftest test-enabled-no-subs-no-events-legacy-still-sends
+  ;; When telemetry is enabled, there are no newsletter subscriptions
+  ;; and no audit_log rows, the legacy report must still be sent.
+  (with-mocks [legacy-mock {:target 'app.tasks.telemetry/make-legacy-request
+                            :return nil}
+               batch-mock  {:target 'app.tasks.telemetry/send-event-batch
+                            :return true}]
+    (with-redefs [cf/flags #{:telemetry}]
+      ;; No profiles with newsletter-updates, no telemetry rows
+      (th/run-task! :telemetry {:send? true :enabled? true})
+
+      ;; Legacy report was sent
+      (t/is (:called? @legacy-mock))
+      (let [[_ data] (:call-args @legacy-mock)]
+        (t/is (= :telemetry-legacy-report (:type data)))
+        (t/is (contains? data :stats))
+        ;; No subscriptions in the payload
+        (t/is (not (contains? data :subscriptions))))
+
+      ;; No events to batch-send
+      (t/is (not (:called? @batch-mock))))))
+
+(t/deftest test-legacy-succeeds-batch-fails
+  ;; The legacy report and event batch are independent paths.
+  ;; When the batch endpoint fails, the legacy report must still
+  ;; have been sent successfully.
+  (with-mocks [legacy-mock {:target 'app.tasks.telemetry/make-legacy-request
+                            :return nil}
+               batch-mock  {:target 'app.tasks.telemetry/send-event-batch
+                            :return false}]
+    (with-redefs [cf/flags #{:telemetry}]
+      (insert-telemetry-row! "navigate")
+
+      (th/run-task! :telemetry {:send? true :enabled? true})
+
+      ;; Legacy report was sent
+      (t/is (:called? @legacy-mock))
+      (let [[_ data] (:call-args @legacy-mock)]
+        (t/is (= :telemetry-legacy-report (:type data))))
+
+      ;; Batch send was attempted but failed
+      (t/is (:called? @batch-mock))
+      ;; Row still present (not deleted on failure)
+      (t/is (= 1 (count-telemetry-rows))))))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; GC + BATCH FAILURE INTERACTION
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(t/deftest test-gc-runs-even-when-batch-fails
+  ;; GC must purge stale events regardless of whether the subsequent
+  ;; batch send succeeds or fails.
+  (with-mocks [legacy-mock {:target 'app.tasks.telemetry/make-legacy-request
+                            :return nil}
+               batch-mock  {:target 'app.tasks.telemetry/send-event-batch
+                            :return false}]
+    (with-redefs [cf/flags #{:telemetry}]
+      (let [eight-days (ct/minus (ct/now) (ct/duration {:days 8}))
+            one-day    (ct/minus (ct/now) (ct/duration {:days 1}))]
+        ;; Stale events (should be GC'd)
+        (insert-telemetry-row! "stale-1" {:created-at eight-days :tracked-at eight-days})
+        (insert-telemetry-row! "stale-2" {:created-at eight-days :tracked-at eight-days})
+        ;; Fresh event (should survive GC but fail to send)
+        (insert-telemetry-row! "fresh" {:created-at one-day :tracked-at one-day})
+
+        (t/is (= 3 (count-telemetry-rows)))
+
+        (th/run-task! :telemetry {:send? true :enabled? true})
+
+        ;; Batch send was attempted (and failed)
+        (t/is (:called? @batch-mock))
+        ;; Stale rows were purged by GC, fresh row remains
+        (t/is (= 1 (count-telemetry-rows)))
+        (t/is (= "fresh" (:name (first (th/db-exec! ["SELECT name FROM audit_log WHERE source LIKE 'telemetry:%'"])))))))))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; ROW->EVENT CONTEXT GUARANTEE
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(t/deftest test-row->event-always-includes-context
+  ;; row->event must always include :context as a map, even when the
+  ;; DB column contains an empty transit object.
+  (let [row->event (ns-resolve 'app.tasks.telemetry 'row->event)]
+    ;; With non-empty context
+    (let [ev (row->event {:name "test" :type "action" :source "telemetry:backend"
+                          :tracked-at (ct/now) :profile-id uuid/zero
+                          :context (db/tjson {:browser "Chrome"})})]
+      (t/is (contains? ev :context))
+      (t/is (= {:browser "Chrome"} (:context ev))))
+
+    ;; With empty context ({} in transit)
+    (let [ev (row->event {:name "test" :type "action" :source "telemetry:backend"
+                          :tracked-at (ct/now) :profile-id uuid/zero
+                          :context (db/tjson {})})]
+      (t/is (contains? ev :context))
+      (t/is (= {} (:context ev))))))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; NO DUPLICATE EVENTS ON SUCCESS
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(t/deftest test-no-duplicate-events-after-successful-send
+  ;; After a successful batch send, the sent rows must be deleted.
+  ;; Running the task again must NOT re-send the same events.
+  (let [send-count (atom 0)]
+    (with-mocks [legacy-mock {:target 'app.tasks.telemetry/make-legacy-request
+                              :return nil}]
+      (with-redefs [cf/flags #{:telemetry}
+                    telemetry/send-event-batch
+                    (fn [_cfg _batch]
+                      (swap! send-count inc)
+                      true)]
+        (insert-telemetry-row! "navigate")
+        (insert-telemetry-row! "create-file")
+
+        (t/is (= 2 (count-telemetry-rows)))
+
+        ;; First run: sends and deletes
+        (th/run-task! :telemetry {:send? true :enabled? true})
+        (t/is (= 1 @send-count))
+        (t/is (= 0 (count-telemetry-rows)))
+
+        ;; Second run: no events to send
+        (th/run-task! :telemetry {:send? true :enabled? true})
+        (t/is (= 1 @send-count))  ;; still 1, not 2
+        (t/is (= 0 (count-telemetry-rows)))))))

backend/test/backend_tests/util_blob_test.clj

@@ -0,0 +1,106 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS INC
+
+(ns backend-tests.util-blob-test
+  (:require
+   [app.util.blob :as blob]
+   [clojure.string :as str]
+   [clojure.test :as t]))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; encode-str / decode-str round-trip
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(t/deftest encode-str-roundtrip-empty-map
+  (let [data {}]
+    (t/is (= data (blob/decode-str (blob/encode-str data))))))
+
+(t/deftest encode-str-roundtrip-empty-vector
+  (let [data []]
+    (t/is (= data (blob/decode-str (blob/encode-str data))))))
+
+(t/deftest encode-str-roundtrip-nil
+  (let [data nil]
+    (t/is (= data (blob/decode-str (blob/encode-str data))))))
+
+(t/deftest encode-str-roundtrip-simple-map
+  (let [data {:name "penpot" :version 42}]
+    (t/is (= data (blob/decode-str (blob/encode-str data))))))
+
+(t/deftest encode-str-roundtrip-nested-structure
+  (let [data {:users [{:name "Alice" :tags #{"admin" "active"}}
+                      {:name "Bob" :tags #{"user"}}]
+              :config {:debug false :timeout 3000}}]
+    (t/is (= data (blob/decode-str (blob/encode-str data))))))
+
+(t/deftest encode-str-roundtrip-vector-of-maps
+  (let [data [{:name "navigate" :type "action" :source "telemetry"}
+              {:name "create-file" :type "action" :source "telemetry"}]]
+    (t/is (= data (blob/decode-str (blob/encode-str data))))))
+
+(t/deftest encode-str-roundtrip-keywords-and-strings
+  (let [data {:keyword/value :foo
+              :string/value "hello world"
+              :boolean/value true
+              :nil/value nil}]
+    (t/is (= data (blob/decode-str (blob/encode-str data))))))
+
+(t/deftest encode-str-roundtrip-numeric-types
+  (let [data {:int 42
+              :neg -7
+              :zero 0
+              :big 9999999999}]
+    (t/is (= data (blob/decode-str (blob/encode-str data))))))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; URL-safe encoding properties
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(t/deftest encode-str-url-safe-no-unsafe-chars
+  ;; URL-safe base64 must not contain +, /, or padding =
+  (let [data {:a (apply str (repeat 100 "x"))
+              :b (range 200)
+              :c {"key" "value with special chars: @#$%^&*()"}}
+        encoded (blob/encode-str data)]
+    (t/is (not (str/includes? encoded "+")))
+    (t/is (not (str/includes? encoded "/")))
+    (t/is (not (str/includes? encoded "=")))))
+
+(t/deftest encode-str-url-safe-roundtrip-after-encoding
+  ;; Ensure the URL-safe encoding still round-trips correctly
+  (let [data {:payload (vec (range 500))
+              :nested {:a {:b {:c "deep"}}}}
+        encoded (blob/encode-str data)
+        decoded (blob/decode-str encoded)]
+    (t/is (= data decoded))))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; version-specific encoding
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(t/deftest encode-str-with-version-4
+  (let [data {:events [{:name "click"} {:name "scroll"}]}
+        encoded (blob/encode-str data {:version 4})
+        decoded (blob/decode-str encoded)]
+    (t/is (= data decoded))))
+
+(t/deftest encode-str-with-version-5
+  (let [data {:events [{:name "click"} {:name "scroll"}]}
+        encoded (blob/encode-str data {:version 5})
+        decoded (blob/decode-str encoded)]
+    (t/is (= data decoded))))
+
+(t/deftest encode-str-with-version-1
+  (let [data {:simple "data"}
+        encoded (blob/encode-str data {:version 1})
+        decoded (blob/decode-str encoded)]
+    (t/is (= data decoded))))
+
+(t/deftest encode-str-with-version-3
+  (let [data {:simple "data"}
+        encoded (blob/encode-str data {:version 3})
+        decoded (blob/decode-str encoded)]
+    (t/is (= data decoded))))

backend/test/backend_tests/util_ssrf_test.clj

@@ -0,0 +1,176 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS INC
+
+(ns backend-tests.util-ssrf-test
+  (:require
+   [app.common.exceptions :as ex]
+   [app.config :as cf]
+   [app.http.client :as http]
+   [app.util.ssrf :as ssrf]
+   [clojure.test :as t]))
+
+(t/deftest validate-url-allows-public-https
+  (t/is (true? (ssrf/safe-url? "https://example.com/foo")))
+  (t/is (true? (ssrf/safe-url? "https://example.com:8080/path?q=1"))))
+
+(t/deftest validate-url-allows-public-http
+  (t/is (true? (ssrf/safe-url? "http://example.com/foo"))))
+
+(t/deftest validate-url-blocks-disallowed-schemes
+  (t/is (false? (ssrf/safe-url? "file:///etc/passwd")))
+  (t/is (false? (ssrf/safe-url? "gopher://example.com")))
+  (t/is (false? (ssrf/safe-url? "ftp://example.com")))
+  (t/is (false? (ssrf/safe-url? "dict://example.com")))
+  (t/is (false? (ssrf/safe-url? "data:text/html,<h1>hi</h1>")))
+  (t/is (false? (ssrf/safe-url? "jar:http://example.com!/foo")))
+  (t/is (false? (ssrf/safe-url? "javascript:alert(1)"))))
+
+(t/deftest validate-url-blocks-loopback
+  (t/is (false? (ssrf/safe-url? "http://127.0.0.1/foo")))
+  (t/is (false? (ssrf/safe-url? "http://127.0.0.2/foo")))
+  (t/is (false? (ssrf/safe-url? "http://[::1]/foo"))))
+
+(t/deftest validate-url-blocks-any-local
+  (t/is (false? (ssrf/safe-url? "http://0.0.0.0/foo")))
+  (t/is (false? (ssrf/safe-url? "http://[::]/foo"))))
+
+(t/deftest validate-url-blocks-link-local
+  (t/is (false? (ssrf/safe-url? "http://169.254.169.254/latest/meta-data/")))
+  (t/is (false? (ssrf/safe-url? "http://169.254.1.1/foo")))
+  (t/is (false? (ssrf/safe-url? "http://[fe80::1]/foo"))))
+
+(t/deftest validate-url-blocks-site-local
+  (t/is (false? (ssrf/safe-url? "http://10.0.0.1/foo")))
+  (t/is (false? (ssrf/safe-url? "http://172.16.0.1/foo")))
+  (t/is (false? (ssrf/safe-url? "http://192.168.1.1/foo"))))
+
+(t/deftest validate-url-blocks-cloud-metadata
+  (t/is (false? (ssrf/safe-url? "http://169.254.169.254/latest/meta-data/iam/security-credentials/role")))
+  (t/is (false? (ssrf/safe-url? "http://[fd00:ec2::254]/foo"))))
+
+(t/deftest validate-url-blocks-carrier-grade-nat
+  (t/is (false? (ssrf/safe-url? "http://100.64.0.1/foo")))
+  (t/is (false? (ssrf/safe-url? "http://100.127.255.255/foo")))
+  ;; Just outside the range should be allowed (but may be blocked by DNS resolution failing)
+  ;; We test boundary: 100.63.255.255 is outside 100.64.0.0/10
+  ;; But we can't easily test the "allowed" side without DNS, so we test the blocked side.
+
+  ;; Test RFC reserved ranges
+  (t/is (false? (ssrf/safe-url? "http://240.0.0.1/foo")))
+  (t/is (false? (ssrf/safe-url? "http://255.255.255.255/foo"))))
+
+(t/deftest validate-url-blocks-ipv6-ula
+  (t/is (false? (ssrf/safe-url? "http://[fd00::1]/foo")))
+  (t/is (false? (ssrf/safe-url? "http://[fc00::1]/foo"))))
+
+(t/deftest validate-url-blocks-encoded-loopback
+  ;; Decimal encoding of 127.0.0.1 = 2130706433
+  ;; InetAddress normalizes this to 127.0.0.1
+  (t/is (false? (ssrf/safe-url? "http://2130706433/foo")))
+  ;; Hex encoding 0x7f000001
+  (t/is (false? (ssrf/safe-url? "http://0x7f000001/foo"))))
+
+(t/deftest validate-url-blocks-ipv4-mapped-loopback
+  (t/is (false? (ssrf/safe-url? "http://[::ffff:127.0.0.1]/foo"))))
+
+(t/deftest validate-url-blocks-multicast
+  (t/is (false? (ssrf/safe-url? "http://224.0.0.1/foo"))))
+
+(t/deftest validate-url-blocks-missing-scheme
+  (t/is (false? (ssrf/safe-url? "example.com/foo")))
+  (t/is (false? (ssrf/safe-url? ""))))
+
+(t/deftest validate-url-blocks-missing-host
+  (t/is (false? (ssrf/safe-url? "http:///path")))
+  (t/is (false? (ssrf/safe-url? "http://"))))
+
+(t/deftest validate-url-resolves-dns
+  ;; DNS-resolved internal: we use with-redefs to simulate
+  (let [original ssrf/resolve-host]
+    (with-redefs [ssrf/resolve-host
+                  (fn [hostname]
+                    (if (= hostname "evil.internal")
+                      (into-array java.net.InetAddress
+                                  [(java.net.InetAddress/getByName "127.0.0.1")])
+                      (original hostname)))]
+      (t/is (false? (ssrf/safe-url? "http://evil.internal/foo")))
+      ;; A hostname that fails DNS resolution
+      (t/is (false? (ssrf/safe-url? "http://nonexistent.invalid/foo"))))))
+
+(t/deftest validate-url-dns-all-addresses-must-be-safe
+  ;; If a hostname resolves to both a public and a private IP, it must be blocked
+  (let [original ssrf/resolve-host]
+    (with-redefs [ssrf/resolve-host
+                  (fn [hostname]
+                    (if (= hostname "split-brain.example")
+                      (into-array java.net.InetAddress
+                                  [(java.net.InetAddress/getByName "1.1.1.1")
+                                   (java.net.InetAddress/getByName "127.0.0.1")])
+                      (original hostname)))]
+      (t/is (false? (ssrf/safe-url? "http://split-brain.example/foo"))))))
+
+(t/deftest validate-url-allowlist-override
+  (let [original-get cf/get]
+    (with-redefs [cf/get (fn [key & args]
+                           (if (= key :ssrf-allowed-hosts)
+                             #{"localhost"}
+                             (apply original-get key args)))]
+      ;; localhost resolves to 127.0.0.1 which would normally be blocked
+      (t/is (true? (ssrf/safe-url? "http://localhost:6060/foo"))))))
+
+(t/deftest validate-url-extra-cidrs
+  (binding [ssrf/extra-blocked-cidrs #{(ssrf/parse-cidr "203.0.113.0/24")}]
+    (t/is (false? (ssrf/safe-url? "http://203.0.113.1/foo")))))
+
+(t/deftest validate-url-throw-on-blocked
+  (try
+    (ssrf/validate-uri "http://127.0.0.1/foo")
+    (t/is false "should have thrown")
+    (catch Exception e
+      (t/is (= :validation (:type (ex-data e))))
+      (t/is (= :ssrf-blocked-target (:code (ex-data e)))))))
+
+;; ---------------------------------------------------------------------------
+;; http/req automatic SSRF validation
+;; ---------------------------------------------------------------------------
+
+(t/deftest http-req-validates-ssrf-by-default
+  ;; `http/req` should invoke ssrf/validate-uri before sending the request.
+  ;; We verify this by checking that a blocked URI raises an SSRF error
+  ;; without ever reaching the network (validate-uri throws first).
+  (try
+    (http/req {} {:method :get :uri "http://127.0.0.1/secret"})
+    (t/is false "should have thrown an SSRF error")
+    (catch Exception e
+      (t/is (= :ssrf-blocked-target (:code (ex-data e)))))))
+
+(t/deftest http-req-skip-ssrf-check-bypasses-validation
+  ;; When :skip-ssrf-check? true is passed, ssrf/validate-uri must NOT be
+  ;; called. We verify by patching validate-uri to record whether it was called.
+  (let [called? (atom false)]
+    (with-redefs [ssrf/validate-uri (fn [_] (reset! called? true))]
+      ;; The request will fail at the network level (no real server), but that's
+      ;; fine — we only care that validate-uri was not called beforehand.
+      (try
+        (http/req {} {:method :get :uri "http://127.0.0.1/secret"} {:skip-ssrf-check? true})
+        (catch Exception _))
+      (t/is (false? @called?) "validate-uri should not be called when :skip-ssrf-check? is true"))))
+
+(t/deftest http-req-with-redirects-validates-ssrf-by-default
+  ;; req-with-redirects must also validate the initial URI automatically.
+  (try
+    (http/req-with-redirects {} {:method :get :uri "http://10.0.0.1/internal"})
+    (t/is false "should have thrown an SSRF error")
+    (catch Exception e
+      (t/is (= :ssrf-blocked-target (:code (ex-data e)))))))
+
+(t/deftest http-req-with-redirects-skip-ssrf-check-bypasses-validation
+  (let [called? (atom false)]
+    (with-redefs [ssrf/validate-uri (fn [_] (reset! called? true))]
+      (try
+        (http/req-with-redirects {} {:method :get :uri "http://10.0.0.1/internal"} {:skip-ssrf-check? true})
+        (catch Exception _))
+      (t/is (false? @called?) "validate-uri should not be called when :skip-ssrf-check? is true"))))