Compare commits

..
Author SHA1 Message Date
Elena Torro 1141e91b86 🔧 Fix drag 2026-05-12 17:43:13 +02:00
2693 changed files with 102041 additions and 259649 deletions

No files matched your search

-3
View File
@@ -88,9 +88,6 @@
:dynamic-var-not-earmuffed
{:level :off}
:type-mismatch
{:level :off}
:used-underscored-binding
{:level :warning}
-133
View File
@@ -1,133 +0,0 @@
# `.devenv/` — Per-Workspace AI-Client MCP Configs
This directory carries the pieces needed to point an AI coding agent
(currently Claude Code, opencode, VS Code Copilot, and the OpenAI Codex CLI)
at the MCP servers running inside the parallel devenv instance the developer
is currently working in. Every parallel workspace (`ws0`, `ws1`, …) has its
own copy because the Penpot MCP and Serena MCP host ports are
workspace-specific.
## Layout
```
.devenv/
README.md
scripts/
merge-mcp-config.py # generator helper invoked by manage.sh
shared/ # committed; workspace-independent entries
claude-code.json # Playwright — same for every workspace
opencode.json
vscode.json
codex.toml
templates/ # committed; entries with ${...} port placeholders
claude-code.json # Penpot MCP, Serena MCP — port is the only diff
opencode.json
vscode.json
codex.toml
mcp/ # gitignored; written by manage.sh per workspace
claude-code.json # loaded via Claude Code's --mcp-config flag
opencode.json # loaded via OPENCODE_CONFIG env var
```
One more file is generated outside `.devenv/`, in the directory VS Code itself
auto-discovers (gitignored):
```
.vscode/mcp.json # auto-loaded by GitHub Copilot in VS Code
```
Codex is the exception: it has no way to load an MCP config from an arbitrary
path, and its only project-level config file (`.codex/config.toml`) is one a
developer may already own. So we do **not** write a file for Codex at all —
`start-coding-agent codex` injects our servers as `-c` command-line overrides
built fresh from `shared/codex.toml` + `templates/codex.toml` at launch.
* **`shared/`** holds MCP entries that don't depend on the workspace — the
browser-driving Playwright server today, plus any other workspace-independent
servers we add later. Same content in every workspace, so it's a static
checked-in file.
* **`templates/`** holds the workspace-specific entries (Penpot MCP, Serena
MCP) with `${PENPOT_MCP_PORT}` and `${SERENA_MCP_PORT}` placeholders. The
placeholders are resolved per-workspace from the port-base constants in
`manage.sh`.
* **`mcp/`** (Claude Code, opencode) is the result of merging `shared/` with
the port-substituted `templates/`. `manage.sh` writes these on every
`run-devenv --agentic` pass. Gitignored, dedicated paths with no developer
content — never edit by hand, your edits will be overwritten on the next
reconcile.
* **`.vscode/mcp.json`** is the same merge, but written to the path VS Code
auto-discovers. Because on `ws0` that path *is* the live repo's own file, the
reconcile **deep-merges** into it: any servers you added yourself are kept,
and only the entries we manage (`penpot`, `serena-devenv`, `playwright`) are
(re)written to the current ports. On `ws1+` the file doesn't exist yet, so it
is created from scratch.
* **`scripts/merge-mcp-config.py`** is the generator. Its `json` mode does the
JSON deep-merge (with `--merge-into-existing` for the VS Code path); its
`codex-args` mode prints the `-c` assignments for Codex. `manage.sh`'s
`_merge-mcp-config-json` helper is a thin shim over the former, and
`start-coding-agent` calls the latter directly. Run
`python3 .devenv/scripts/merge-mcp-config.py --help` for the CLI.
## Launching a coding agent
The easiest path is the wrapper command, which knows the right flags per
client, `cd`'s into the target workspace, and refuses to launch unless the
target instance is running and its MCP config has been generated:
```bash
# Default target is ws0 (the live repo).
./manage.sh start-coding-agent claude [...args to forward]
./manage.sh start-coding-agent opencode [...args to forward]
./manage.sh start-coding-agent vscode [...args to forward to 'code']
./manage.sh start-coding-agent codex [...args to forward]
# Target a parallel workspace with --ws N. N is an integer (non-negative);
# 'main', 'ws1' and similar spellings are rejected.
./manage.sh start-coding-agent claude --ws 1
./manage.sh start-coding-agent opencode --ws 2
```
Equivalents by hand (run from inside the workspace directory):
```bash
claude --mcp-config .devenv/mcp/claude-code.json
OPENCODE_CONFIG=.devenv/mcp/opencode.json opencode
code "$PWD" # VS Code auto-discovers .vscode/mcp.json
# Codex: pass our servers as -c overrides (no config file is written).
codex $(python3 .devenv/scripts/merge-mcp-config.py --format codex-args \
.devenv/shared/codex.toml .devenv/templates/codex.toml \
| sed 's/^/-c /')
```
`start-coding-agent codex` does the `-c` wiring for you (and resolves the
workspace's ports first). Because our servers arrive as command-line
overrides, no "trusted project" prompt is involved for them — that prompt only
gates Codex's own `.codex/config.toml`, which we never write.
## Overriding our entries
Both the auto-discovered configs and the launcher-loaded configs sit *on top
of* the developer's global config (with varying precedence rules). All four
clients offer escape hatches for shadowing entries we ship:
* **Claude Code** — `claude mcp add --scope local …` installs a private entry
that overrides the one in `mcp/claude-code.json`. Local scope wins.
* **opencode** — drop an `opencode.json` at the repo root with the override
entries you need. opencode's precedence chain is *global → `OPENCODE_CONFIG`
→ project*, so the project file always wins. The root `opencode.json` is
gitignored on purpose, since these overrides are personal.
* **VS Code Copilot** — the reconcile deep-merges into `.vscode/mcp.json`, so
any servers you add there yourself are preserved (only `penpot`,
`serena-devenv` and `playwright` are rewritten). To shadow one of *ours*,
put an entry under the same name in your VS Code user-profile MCP config —
it is loaded alongside the workspace file and wins.
* **Codex CLI** — our servers arrive as `-c` overrides, which are Codex's
highest-precedence layer, so they win over a same-named `[mcp_servers.<name>]`
in your `~/.codex/config.toml` or a project `.codex/config.toml`. To override
one of ours, append your own `-c` after the client name — extra args are
forwarded after ours and the later `-c` wins, e.g.
`./manage.sh start-coding-agent codex -- -c 'mcp_servers.penpot.url="…"'`.
See `docs/technical-guide/developer/agentic-devenv.md` for the broader
client-configuration story (browser remote debugging, AI-client config
schemas, manual setup for unsupported clients).
-204
View File
@@ -1,204 +0,0 @@
#!/usr/bin/env python3
"""Combine a shared MCP-server config with a port-substituted template for one
AI coding-agent client.
Invoked per workspace by manage.sh's `write-instance-mcp-configs` (JSON
clients) and by `start-coding-agent` (Codex). Each supported client ships a
`.devenv/shared/<tool>.{json,toml}` (workspace-independent entries, e.g.
Playwright) and a `.devenv/templates/<tool>.{json,toml}` (per-workspace entries
with `${PENPOT_MCP_PORT}` / `${SERENA_MCP_PORT}` placeholders). This script
combines the two for the target client.
Two output modes are supported:
json Deep-merge two JSON documents under a configurable top-level key
(`mcpServers` for Claude Code, `mcp` for opencode, `servers` for
VS Code Copilot) and write the result to <out>. Same-name
entries in the template override entries in shared. With
--merge-into-existing, any pre-existing <out> file is loaded as
the lowest-precedence layer first, so entries the developer
already had are preserved (ours win on name collision). This is
used for VS Code's auto-discovered `.vscode/mcp.json`, which on
ws0 IS the live repo's file and may hold the developer's own
servers; the Claude/opencode outputs live in a dedicated,
gitignored `.devenv/mcp/` path and are written without the flag
(a clean overwrite).
codex-args Deep-merge the two TOML chunks and print one
`dotted.key=<toml-value>` assignment per line to stdout (no
<out> file). The caller wraps each line in a `codex -c` flag.
Codex has no way to load an MCP config from an arbitrary file
path (CODEX_HOME would relocate auth/history too), so rather than
writing the auto-discovered `.codex/config.toml` we inject our
servers as ephemeral per-invocation overrides. This never
touches the developer's project- or user-level Codex config.
In both modes, `${VAR}` placeholders inside *either* chunk are resolved from
the current environment (only template chunks carry placeholders in practice,
but the substitution is uniform either way) using Python's
`os.path.expandvars`. Undefined placeholders are left as `${VAR}` literal text
-- callers (i.e. manage.sh) are responsible for exporting the variables before
invoking the script.
Usage:
merge-mcp-config.py --format json --key <key> [--merge-into-existing] \
<shared> <template> <out>
merge-mcp-config.py --format codex-args <shared> <template>
Exit codes:
0 success
2 argparse error (missing required option, bad value, unreadable input)
"""
from __future__ import annotations
import argparse
import json
import os
import re
import sys
import tomllib
from pathlib import Path
def merge_json(
shared_path: Path,
tpl_path: Path,
out_path: Path,
key: str,
merge_into_existing: bool,
) -> None:
"""Deep-merge JSON documents under a single top-level dict key into out.
Precedence (lowest to highest): an existing <out> file (only when
merge_into_existing is set), then shared, then the template. Entries under
`key` are merged by name, so the template wins on a name collision while
every other entry the lower layers contributed is kept. Top-level keys
other than `key` come from the existing file and shared (shared wins).
"""
shared = json.loads(shared_path.read_text())
tpl = json.loads(os.path.expandvars(tpl_path.read_text()))
base: dict = {}
if merge_into_existing and out_path.exists():
base = json.loads(out_path.read_text())
merged: dict = {**base, **shared}
merged[key] = {**base.get(key, {}), **shared.get(key, {}), **tpl.get(key, {})}
out_path.write_text(json.dumps(merged, indent=2) + "\n")
def _deep_merge(base: dict, overlay: dict) -> dict:
"""Recursively merge overlay into base; overlay wins on scalar/list keys."""
out = dict(base)
for k, v in overlay.items():
if isinstance(out.get(k), dict) and isinstance(v, dict):
out[k] = _deep_merge(out[k], v)
else:
out[k] = v
return out
def _toml_value(value: object) -> str:
"""Serialize a scalar/list as a TOML literal for a `codex -c` value.
bool is checked before int because `isinstance(True, int)` is True. Strings
are emitted as JSON strings, which are valid TOML basic strings for the
ASCII values our configs carry (commands, args, URLs). Tables never reach
here -- they are flattened into dotted keys by _flatten.
"""
if isinstance(value, bool):
return "true" if value else "false"
if isinstance(value, (int, float)):
return repr(value)
if isinstance(value, str):
return json.dumps(value)
if isinstance(value, list):
return "[" + ", ".join(_toml_value(v) for v in value) + "]"
raise TypeError(f"unsupported TOML value type: {type(value).__name__}")
_BARE_KEY = re.compile(r"^[A-Za-z0-9_-]+$")
def _key_segment(seg: str) -> str:
"""A dotted-key segment: bare if TOML-safe, else a quoted key."""
return seg if _BARE_KEY.match(seg) else json.dumps(seg)
def _flatten(obj: dict, prefix: list[str]):
"""Yield (dotted-path-segments, leaf-value) for every non-table leaf.
Lists are leaves (TOML arrays), so we do not recurse into them; nested
tables (e.g. an `env` table) are flattened into further dotted keys.
"""
for k, v in obj.items():
path = prefix + [k]
if isinstance(v, dict):
yield from _flatten(v, path)
else:
yield path, v
def emit_codex_args(shared_path: Path, tpl_path: Path) -> None:
"""Print `dotted.key=<toml-value>` lines from the merged TOML chunks."""
shared = tomllib.loads(os.path.expandvars(shared_path.read_text()))
tpl = tomllib.loads(os.path.expandvars(tpl_path.read_text()))
merged = _deep_merge(shared, tpl)
for path, value in _flatten(merged, []):
dotted = ".".join(_key_segment(s) for s in path)
sys.stdout.write(f"{dotted}={_toml_value(value)}\n")
def main(argv: list[str]) -> int:
parser = argparse.ArgumentParser(
description=__doc__.split("\n\n", 1)[0],
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument(
"--format",
choices=("json", "codex-args"),
required=True,
help="Output mode: 'json' writes a merged file; 'codex-args' prints -c assignments.",
)
parser.add_argument(
"--key",
help="Top-level JSON key under which MCP entries live (required for --format json).",
)
parser.add_argument(
"--merge-into-existing",
action="store_true",
help="json only: layer the merge on top of an existing <out> file, "
"preserving entries already there (ours still win on name collision).",
)
parser.add_argument("shared", type=Path, help="Path to the shared chunk.")
parser.add_argument("template", type=Path, help="Path to the port-placeholder template chunk.")
parser.add_argument(
"out",
type=Path,
nargs="?",
help="Path the merged result is written to (json only; codex-args writes stdout).",
)
args = parser.parse_args(argv)
if args.format == "json":
if not args.key:
parser.error("--key is required when --format json")
if args.out is None:
parser.error("out is required when --format json")
merge_json(args.shared, args.template, args.out, args.key, args.merge_into_existing)
else: # codex-args
if args.key:
parser.error("--key is not accepted when --format codex-args")
if args.merge_into_existing:
parser.error("--merge-into-existing is not accepted when --format codex-args")
if args.out is not None:
parser.error("out path is not accepted when --format codex-args (result goes to stdout)")
emit_codex_args(args.shared, args.template)
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
-8
View File
@@ -1,8 +0,0 @@
{
"mcpServers": {
"playwright": {
"command": "npx",
"args": ["@playwright/mcp@latest", "--cdp-endpoint=http://127.0.0.1:9222"]
}
}
}
-8
View File
@@ -1,8 +0,0 @@
# Workspace-independent MCP servers for the OpenAI Codex CLI.
# This block is concatenated with the port-substituted templates/codex.toml
# by manage.sh's write-instance-mcp-configs to produce .codex/config.toml at
# the workspace root.
[mcp_servers.playwright]
command = "npx"
args = ["@playwright/mcp@latest", "--cdp-endpoint=http://127.0.0.1:9222"]
-9
View File
@@ -1,9 +0,0 @@
{
"mcp": {
"playwright": {
"type": "local",
"command": ["npx", "@playwright/mcp@latest", "--cdp-endpoint=http://127.0.0.1:9222"],
"enabled": true
}
}
}
-9
View File
@@ -1,9 +0,0 @@
{
"servers": {
"playwright": {
"type": "stdio",
"command": "npx",
"args": ["@playwright/mcp@latest", "--cdp-endpoint=http://127.0.0.1:9222"]
}
}
}
-12
View File
@@ -1,12 +0,0 @@
{
"mcpServers": {
"penpot": {
"command": "npx",
"args": ["-y", "mcp-remote", "http://localhost:${PENPOT_MCP_PORT}/mcp", "--allow-http"]
},
"serena-devenv": {
"command": "npx",
"args": ["-y", "mcp-remote", "http://localhost:${SERENA_MCP_PORT}/mcp", "--allow-http"]
}
}
}
-10
View File
@@ -1,10 +0,0 @@
# Workspace-specific MCP servers for the OpenAI Codex CLI. The PENPOT_MCP_PORT
# and SERENA_MCP_PORT placeholders below are filled in per workspace by
# manage.sh's write-instance-mcp-configs, then the result is concatenated
# with shared/codex.toml to produce .codex/config.toml.
[mcp_servers.penpot]
url = "http://localhost:${PENPOT_MCP_PORT}/mcp"
[mcp_servers.serena-devenv]
url = "http://localhost:${SERENA_MCP_PORT}/mcp"
-14
View File
@@ -1,14 +0,0 @@
{
"mcp": {
"penpot": {
"type": "remote",
"url": "http://localhost:${PENPOT_MCP_PORT}/mcp",
"enabled": true
},
"serena-devenv": {
"type": "remote",
"url": "http://localhost:${SERENA_MCP_PORT}/mcp",
"enabled": true
}
}
}
-12
View File
@@ -1,12 +0,0 @@
{
"servers": {
"penpot": {
"type": "http",
"url": "http://localhost:${PENPOT_MCP_PORT}/mcp"
},
"serena-devenv": {
"type": "http",
"url": "http://localhost:${SERENA_MCP_PORT}/mcp"
}
}
}
-3
View File
@@ -1,3 +0,0 @@
# Penpot API configuration for error-reports CLI tool
PENPOT_API_URI=http://localhost:3450
PENPOT_ACCESS_TOKEN=your-access-token-here
+2 -3
View File
@@ -1,8 +1,7 @@
description: Create a report to help us improve
labels: ["bug"]
name: Bug report
type: Bug
labels: ["needs triage"]
title: "bug: "
body:
- type: markdown
attributes:
-1
View File
@@ -1 +0,0 @@
blank_issues_enabled: true
+2 -3
View File
@@ -1,8 +1,7 @@
description: Suggest an idea for this project.
labels: ["needs triage"]
labels: ["needs triage", "enhancement"]
name: "Feature request"
type: Enhancement
title: "feature: "
body:
- type: markdown
attributes:
@@ -0,0 +1,38 @@
---
name: New Render Bug Report
about: Create a report about the bugs you have found in the new render
title: ''
labels: new render
assignees: claragvinola
---
**Describe the bug**
A clear and concise description of what the bug is.
**Steps to Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots or screen recordings**
If applicable, add screenshots or screen recording to help illustrate your problem.
**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
**Smartphone (please complete the following information):**
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
**Additional context**
Add any other context about the problem here.
+1
View File
@@ -15,5 +15,6 @@
- [ ] Add or modify existing integration tests in case of bugs or new features, if applicable.
- [ ] Refactor any modified SCSS files following the refactor guide.
- [ ] Check CI passes successfully.
- [ ] Update the `CHANGES.md` file, referencing the related GitHub issue, if applicable.
<!-- For more details, check the contribution guidelines: https://github.com/penpot/penpot/blob/develop/CONTRIBUTING.md -->
-41
View File
@@ -1,41 +0,0 @@
def specs: [.. | objects | select(has("tests") and has("file"))];
def dur: [.tests[].results[]?.duration // 0] | add;
specs as $s
| ($s | map(select(any(.tests[]; .status == "unexpected")))) as $failed
| ($s | map(select(any(.tests[]; .status == "flaky")))) as $flaky
| ($s | map(select(any(.tests[]; .status == "skipped")))) as $skipped
| ($s | length) as $total
| ($s | map(dur) | add // 0 | . / 1000 | floor) as $cpu
| (if ($failed | length) > 0 then "❌"
elif ($flaky | length) > 0 then "⚠️"
else "✅" end) as $icon
| "## \($icon) Integration tests\n\n"
+ "| Total | Passed | Flaky | Failed | Skipped | Test time |\n"
+ "|---|---|---|---|---|---|\n"
+ "| \($total) | \($total - ($failed|length) - ($flaky|length) - ($skipped|length)) "
+ "| \($flaky|length) | \($failed|length) | \($skipped|length) | \($cpu / 60 | floor)m |\n"
+ (if ($failed | length) > 0 then
"\n### Failed\n\n"
+ ($failed | map("- `\(.file):\(.line)` — \(.title)") | join("\n")) + "\n"
else "" end)
+ (if ($flaky | length) > 0 then
"\n### Flaky (passed on retry)\n\n"
+ ($flaky
| map({ t: "`\(.file):\(.line)` — \(.title)",
r: ([.tests[].results[]? | select(.status == "failed")] | length) })
| sort_by(-.r)
| map("- \(.t) _(\(.r) \(if .r == 1 then "retry" else "retries" end))_")
| join("\n")) + "\n"
else "" end)
+ (if $total > 0 then
"\n<details><summary>Slowest specs</summary>\n\n"
+ ($s | map({ t: "`\(.file)` — \(.title)", d: (dur / 1000 | floor) })
| sort_by(-.d) | .[0:5]
| map("- \(.t) — \(.d)s") | join("\n"))
+ "\n\n</details>\n"
else "" end)
-76
View File
@@ -1,76 +0,0 @@
name: Auto Label and Add to Project
on:
issues:
types: [opened]
pull_request_target:
types: [opened]
jobs:
triage:
runs-on: ubuntu-latest
steps:
- name: Generate GitHub App token
id: triage-app-token
uses: actions/create-github-app-token@v1
with:
app-id: ${{ secrets.TRIAGE_APP_ID }}
private-key: ${{ secrets.TRIAGE_APP_PRIVATE_KEY }}
owner: penpot
- name: Process Issue or PR
uses: actions/github-script@v7
with:
github-token: ${{ steps.triage-app-token.outputs.token }}
script: |
// === 1. CONFIGURATION ===
const PROJECT_NUMBER = 8; // <--- Replace with your project board number
const IS_ORG = true; // <--- Set to false if this is a personal project, true if an organization
const OWNER = context.repo.owner;
const REPO = context.repo.repo;
const issueNumber = context.issue.number;
const isPR = !!context.payload.pull_request;
const contentId = isPR ? context.payload.pull_request.node_id : context.payload.issue.node_id;
// Define your labels here
const labelToApply = 'needs triage';
// === 2. APPLY THE LABEL ===
console.log(`Applying label "${labelToApply}" to ${isPR ? 'PR' : 'Issue'} #${issueNumber}...`);
await github.rest.issues.addLabels({
issue_number: issueNumber,
owner: OWNER,
repo: REPO,
labels: [labelToApply]
});
// === 3. ADD TO PROJECT BOARD ===
console.log(`Fetching Project #${PROJECT_NUMBER} ID...`);
const projectQuery = `
query($owner: String!, $number: Int!) {
${IS_ORG ? 'organization' : 'user'}(login: $owner) {
projectV2(number: $number) {
id
}
}
}
`;
const projectRes = await github.graphql(projectQuery, { owner: OWNER, number: PROJECT_NUMBER });
const projectId = IS_ORG ? projectRes.organization.projectV2.id : projectRes.user.projectV2.id;
console.log(`Adding item to project board...`);
const addToProjectMutation = `
mutation($projectId: ID!, $contentId: ID!) {
addProjectV2ItemById(input: {projectId: $projectId, contentId: $contentId}) {
item {
id
}
}
}
`;
await github.graphql(addToProjectMutation, { projectId, contentId });
console.log("Automation successfully completed!");
+34 -80
View File
@@ -9,6 +9,16 @@ on:
type: string
required: true
default: 'develop'
build_wasm:
description: 'BUILD_WASM. Valid values: yes, no'
type: string
required: false
default: 'yes'
build_storybook:
description: 'BUILD_STORYBOOK. Valid values: yes, no'
type: string
required: false
default: 'yes'
workflow_call:
inputs:
gh_ref:
@@ -16,25 +26,25 @@ on:
type: string
required: true
default: 'develop'
# Literal group name: under `workflow_call`, `github.workflow` resolves to the
# caller's workflow, which put this workflow and the other reusable one called
# by the same caller into a single shared group, and left a manual dispatch of
# the same ref in a group of its own, free to race on the same artifacts.
concurrency:
group: build-bundle-${{ inputs.gh_ref }}
cancel-in-progress: true
build_wasm:
description: 'BUILD_WASM. Valid values: yes, no'
type: string
required: false
default: 'yes'
build_storybook:
description: 'BUILD_STORYBOOK. Valid values: yes, no'
type: string
required: false
default: 'yes'
jobs:
# ── 1. Decide whether there is anything to build ───────────────────────
check:
name: Check current bundle
runs-on: penpot-standar-runner
timeout-minutes: 10
outputs:
gh_ref: ${{ steps.vars.outputs.gh_ref }}
bundle_version: ${{ steps.vars.outputs.bundle_version }}
exists: ${{ steps.check.outputs.exists }}
build-bundle:
name: Build and Upload Penpot Bundle
runs-on: penpot-runner-01
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
steps:
- name: Checkout repository
@@ -49,52 +59,10 @@ jobs:
echo "gh_ref=${{ inputs.gh_ref || github.ref_name }}" >> $GITHUB_OUTPUT
echo "bundle_version=$(git describe --tags --always)" >> $GITHUB_OUTPUT
# The uploaded zip carries its version as S3 metadata. If the
# existing object was already built from this same commit, the
# whole build job is skipped.
- name: Check if this bundle is already built
id: check
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
run: |
EXISTING_VERSION=$(aws s3api head-object \
--bucket ${{ secrets.S3_BUCKET }} \
--key "penpot-${{ steps.vars.outputs.gh_ref }}.zip" \
--query 'Metadata."bundle-version"' \
--output text 2>/dev/null || echo "none")
if [ "$EXISTING_VERSION" = "${{ steps.vars.outputs.bundle_version }}" ]; then
echo "exists=true" >> $GITHUB_OUTPUT
{
echo "### ⏭️ Bundle build skipped"
echo ""
echo "The bundle in S3 was already built from \`${{ steps.vars.outputs.bundle_version }}\`."
} >> "$GITHUB_STEP_SUMMARY"
else
echo "exists=false" >> $GITHUB_OUTPUT
fi
# ── 2. Build and upload, only when needed ──────────────────────────────
build:
name: Build and Upload Penpot Bundle
runs-on: penpot-standar-runner
timeout-minutes: 90
needs: check
if: needs.check.outputs.exists == 'false'
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
fetch-depth: 0
ref: ${{ inputs.gh_ref }}
- name: Build bundle
env:
BUILD_WASM: 'yes'
BUILD_STORYBOOK: 'yes'
BUILD_WASM: ${{ inputs.build_wasm }}
BUILD_STORYBOOK: ${{ inputs.build_storybook }}
run: ./manage.sh build-bundle
- name: Prepare directories for zipping
@@ -108,32 +76,18 @@ jobs:
zip -r zips/penpot.zip penpot
- name: Upload Penpot bundle to S3
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
run: |
aws s3 cp zips/penpot.zip \
s3://${{ secrets.S3_BUCKET }}/penpot-${{ needs.check.outputs.gh_ref }}.zip \
--metadata bundle-version=${{ needs.check.outputs.bundle_version }}
aws s3 cp zips/penpot.zip s3://${{ secrets.S3_BUCKET }}/penpot-${{ steps.vars.outputs.gh_ref }}.zip --metadata bundle-version=${{ steps.vars.outputs.bundle_version }}
# ── 3. Single failure notification for the whole workflow ─────────────
notify:
name: Notify failure
runs-on: penpot-standar-runner
timeout-minutes: 5
needs: [check, build]
if: failure()
steps:
- name: Notify Mattermost
uses: mattermost/action-mattermost-notify@ae31bb6f9e26a54336e79696f108a2c91cf55b4e # v2.1.0
if: failure()
uses: mattermost/action-mattermost-notify@master
with:
MATTERMOST_WEBHOOK_URL: ${{ secrets.MATTERMOST_WEBHOOK }}
MATTERMOST_CHANNEL: bot-alerts-cicd
TEXT: |
❌ 📦 *[PENPOT] Error building penpot bundles.*
📄 Triggered from ref: `${{ needs.check.outputs.gh_ref || inputs.gh_ref }}`
Bundle version: `${{ needs.check.outputs.bundle_version || 'n/a' }}`
📄 Triggered from ref: `${{ steps.vars.outputs.gh_ref }}`
Bundle version: `${{ steps.vars.outputs.bundle_version }}`
🔗 Run: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}
@infra
+2 -10
View File
@@ -5,16 +5,14 @@ on:
schedule:
- cron: '16 5-20 * * 1-5'
concurrency:
group: ${{ github.workflow }}
cancel-in-progress: true
jobs:
build-bundle:
uses: ./.github/workflows/build-bundle.yml
secrets: inherit
with:
gh_ref: "develop"
build_wasm: "yes"
build_storybook: "yes"
build-docker:
needs: build-bundle
@@ -22,9 +20,3 @@ jobs:
secrets: inherit
with:
gh_ref: "develop"
build-docker-admin-console:
uses: ./.github/workflows/build-docker-admin-console.yml
secrets: inherit
with:
gh_ref: "develop"
@@ -1,91 +0,0 @@
name: Admin Console Docker Builder
on:
workflow_dispatch:
inputs:
gh_ref:
description: 'Name of the branch or ref to build in penpot-nitrate'
type: string
required: true
default: 'develop'
dispatch_ref:
description: 'Branch of penpot-nitrate from which the workflow definition is read'
type: string
required: false
default: 'develop'
workflow_call:
inputs:
gh_ref:
description: 'Name of the branch or ref to build in penpot-nitrate'
type: string
required: true
dispatch_ref:
description: 'Branch of penpot-nitrate from which the workflow definition is read'
type: string
required: false
default: 'develop'
secrets:
ORG_WORKFLOW_TOKEN:
description: 'Token with Actions write access on penpot-nitrate'
required: true
jobs:
build-nitrate-docker:
runs-on: ubuntu-latest
env:
GH_TOKEN: ${{ secrets.ORG_WORKFLOW_TOKEN }}
REPO: penpot/penpot-nitrate
WORKFLOW: build-docker-admin-console.yml
GH_REF: ${{ inputs.gh_ref }}
DISPATCH_REF: ${{ inputs.dispatch_ref }}
steps:
- name: Trigger nitrate docker build
id: dispatch
run: |
DISTINCT_ID="${{ github.run_id }}-${{ github.run_attempt }}"
CALLER_URL="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
gh workflow run "$WORKFLOW" --repo "$REPO" --ref "$DISPATCH_REF" \
-f gh_ref="$GH_REF" \
-f caller_run_id="$DISTINCT_ID" \
-f caller_run_url="$CALLER_URL"
# Locate the dispatched run using the correlation id embedded in its run-name
RUN_ID=""
for i in $(seq 1 24); do
sleep 5
RUN_ID=$(gh run list --repo "$REPO" --workflow "$WORKFLOW" \
--limit 10 --json databaseId,displayTitle \
--jq ".[] | select(.displayTitle | contains(\"$DISTINCT_ID\")) | .databaseId" \
| head -n1)
[ -n "$RUN_ID" ] && break
done
if [ -z "$RUN_ID" ]; then
echo "::error::Could not locate the dispatched run in $REPO"
exit 1
fi
RUN_URL="https://github.com/$REPO/actions/runs/$RUN_ID"
echo "run_id=$RUN_ID" >> "$GITHUB_OUTPUT"
echo "run_url=$RUN_URL" >> "$GITHUB_OUTPUT"
echo "::notice title=Nitrate docker build::$RUN_URL"
- name: Wait for nitrate docker build
run: |
gh run watch "${{ steps.dispatch.outputs.run_id }}" \
--repo "$REPO" \
--interval 30 \
--exit-status
- name: Report result
if: always() && steps.dispatch.outputs.run_id != ''
run: |
CONCLUSION=$(gh run view "${{ steps.dispatch.outputs.run_id }}" \
--repo "$REPO" --json conclusion --jq '.conclusion')
{
echo "### 🐳 Nitrate docker build"
echo ""
echo "- Result: \`${CONCLUSION:-in_progress}\`"
echo "- Run: ${{ steps.dispatch.outputs.run_url }}"
} >> "$GITHUB_STEP_SUMMARY"
+3 -21
View File
@@ -6,7 +6,8 @@ on:
jobs:
build-and-push:
name: Build and push DevEnv Docker image
runs-on: penpot-extended-runner
environment: release-admins
runs-on: penpot-runner-02
steps:
- name: Set common environment variables
@@ -20,19 +21,12 @@ jobs:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Login to Docker Registry (push destination)
- name: Login to Docker Registry
uses: docker/login-action@v4
with:
username: ${{ secrets.PUB_DOCKER_USERNAME }}
password: ${{ secrets.PUB_DOCKER_PASSWORD }}
- name: Login to Docker Hardened Images registry (base image pull)
uses: docker/login-action@v4
with:
registry: dhi.io
username: ${{ secrets.PUB_DOCKER_USERNAME }}
password: ${{ secrets.PUB_DOCKER_PASSWORD }}
- name: Build and push DevEnv Docker image
uses: docker/build-push-action@v7
env:
@@ -42,18 +36,6 @@ jobs:
file: ./docker/devenv/Dockerfile
platforms: linux/amd64,linux/arm64
push: true
provenance: mode=max
sbom: true
tags: ${{ env.DOCKER_IMAGE }}:latest
cache-from: type=registry,ref=${{ env.DOCKER_IMAGE }}:buildcache
cache-to: type=registry,ref=${{ env.DOCKER_IMAGE }}:buildcache,mode=max
- name: Notify Mattermost
uses: mattermost/action-mattermost-notify@ae31bb6f9e26a54336e79696f108a2c91cf55b4e # v2.1.0
with:
MATTERMOST_WEBHOOK_URL: ${{ secrets.MATTERMOST_WEBHOOK }}
MATTERMOST_CHANNEL: bot-alerts-cicd
TEXT: |
🚀 *[PENPOT] New devenv available*
📄 You may want to update your devenv.
@alvaro
+114 -217
View File
@@ -16,121 +16,55 @@ on:
required: true
default: 'develop'
# Literal group name: under `workflow_call`, `github.workflow` resolves to the
# caller's workflow, which put this workflow and the other reusable one called
# by the same caller into a single shared group, and left a manual dispatch of
# the same ref in a group of its own, free to race on the same artifacts.
concurrency:
group: build-docker-${{ inputs.gh_ref }}
cancel-in-progress: true
env:
ALL_IMAGES: backend frontend exporter storybook mcp
# All runner instances live on the same server, so the bundle is
# downloaded from S3 once and shared between build jobs through this
# host-local directory. Each build job falls back to S3 if the file is
# missing (e.g. if runners ever move to separate machines).
BUNDLE_CACHE: /var/tmp/penpot-bundle-cache
jobs:
# ── 1. Resolve the build key and check the whole set at once ───────────
prepare:
name: Prepare
runs-on: penpot-extended-runner
timeout-minutes: 15
outputs:
gh_ref: ${{ steps.vars.outputs.gh_ref }}
bundle_version: ${{ steps.vars.outputs.bundle_version }}
sha: ${{ steps.vars.outputs.sha }}
exists: ${{ steps.check.outputs.exists }}
steps:
- name: Checkout code
uses: actions/checkout@v6
with:
ref: ${{ inputs.gh_ref }}
- name: Extract some useful variables
id: vars
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
run: |
GH_REF="${{ inputs.gh_ref || github.ref_name }}"
echo "gh_ref=$GH_REF" >> $GITHUB_OUTPUT
echo "sha=$(git rev-parse --short=12 HEAD)" >> $GITHUB_OUTPUT
BUNDLE_VERSION=$(aws s3api head-object \
--bucket ${{ secrets.S3_BUCKET }} \
--key "penpot-$GH_REF.zip" \
--query 'Metadata."bundle-version"' \
--output text)
echo "bundle_version=$BUNDLE_VERSION" >> $GITHUB_OUTPUT
# The image set is a single block, so a single set-level check is
# enough: `promote` drops a marker object in S3 only after every
# image was built AND every branch tag was moved. Marker present
# means there is nothing at all to do for this commit.
- name: Check if this image set is already built
id: check
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
run: |
if aws s3api head-object \
--bucket ${{ secrets.S3_BUCKET }} \
--key "markers/images-sha-${{ steps.vars.outputs.sha }}" \
> /dev/null 2>&1; then
echo "exists=true" >> $GITHUB_OUTPUT
{
echo "### ⏭️ Image set build skipped"
echo ""
echo "The whole set was already built and promoted for \`sha-${{ steps.vars.outputs.sha }}\`."
} >> "$GITHUB_STEP_SUMMARY"
else
echo "exists=false" >> $GITHUB_OUTPUT
# Stage the bundle in the host-local cache, once, for all the
# build jobs. Download to a temp name and mv for atomicity;
# prune stale bundles while at it.
mkdir -p "$BUNDLE_CACHE"
find "$BUNDLE_CACHE" -type f -mtime +1 -delete || true
ZIP="$BUNDLE_CACHE/penpot-${{ steps.vars.outputs.bundle_version }}.zip"
if [ ! -f "$ZIP" ]; then
aws s3 cp "s3://${{ secrets.S3_BUCKET }}/penpot-${{ steps.vars.outputs.gh_ref }}.zip" "$ZIP.$$.tmp"
mv "$ZIP.$$.tmp" "$ZIP"
fi
fi
# ── 2. One build per image, in parallel, only when needed ──────────────
build:
name: Build ${{ matrix.image }}
runs-on: penpot-extended-runner
timeout-minutes: 60
needs: prepare
if: needs.prepare.outputs.exists == 'false'
strategy:
fail-fast: true
# 4 runner slots are available for build jobs on this server; cap the
# matrix at 3 so short jobs (prepare and other workflows' checks)
# never queue behind long builds.
max-parallel: 3
matrix:
image: [backend, frontend, exporter, storybook, mcp]
build-and-push:
name: Build and Push Penpot Docker Images
runs-on: penpot-runner-02
steps:
- name: Set common environment variables
run: |
# Each job execution will use its own docker configuration.
echo "DOCKER_CONFIG=${{ runner.temp }}/.docker-${{ github.run_id }}-${{ github.job }}-${{ matrix.image }}" >> $GITHUB_ENV
echo "DOCKER_CONFIG=${{ runner.temp }}/.docker-${{ github.run_id }}-${{ github.job }}" >> $GITHUB_ENV
- name: Checkout code
uses: actions/checkout@v6
with:
fetch-depth: 0
ref: ${{ inputs.gh_ref }}
- name: Extract some useful variables
id: vars
run: |
echo "gh_ref=${{ inputs.gh_ref || github.ref_name }}" >> $GITHUB_OUTPUT
- name: Download Penpot Bundles
id: bundles
env:
FILE_NAME: penpot-${{ steps.vars.outputs.gh_ref }}.zip
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
run: |
tmp=$(aws s3api head-object \
--bucket ${{ secrets.S3_BUCKET }} \
--key "$FILE_NAME" \
--query 'Metadata."bundle-version"' \
--output text)
echo "bundle_version=$tmp" >> $GITHUB_OUTPUT
pushd docker/images
aws s3 cp s3://${{ secrets.S3_BUCKET }}/$FILE_NAME .
unzip $FILE_NAME > /dev/null
mv penpot/backend bundle-backend
mv penpot/frontend bundle-frontend
mv penpot/exporter bundle-exporter
mv penpot/storybook bundle-storybook
mv penpot/mcp bundle-mcp
popd
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Login to Docker Registry
uses: docker/login-action@v4
with:
@@ -147,140 +81,103 @@ jobs:
username: ${{ secrets.PUB_DOCKER_USERNAME }}
password: ${{ secrets.PUB_DOCKER_PASSWORD }}
# Images now build FROM Docker Hardened Images (dhi.io). DHI
# is free (Apache 2.0, no subscription), but pulling from it
# still requires an authenticated login -- a separate `docker
# login` against a different registry host, even though it
# reuses the same PUB_DOCKER_* credentials as the DockerHub
# login above.
- name: Login to Docker Hardened Images registry (base image pull)
uses: docker/login-action@v4
with:
registry: dhi.io
username: ${{ secrets.PUB_DOCKER_USERNAME }}
password: ${{ secrets.PUB_DOCKER_PASSWORD }}
# Bundle staged once by `prepare` on this host; the S3 fallback only
# triggers if the cache is unavailable (runners on another machine,
# cache pruned mid-run, ...).
- name: Prepare Penpot bundle
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
run: |
ZIP="$BUNDLE_CACHE/penpot-${{ needs.prepare.outputs.bundle_version }}.zip"
if [ ! -f "$ZIP" ]; then
echo "Bundle not found in host cache; falling back to S3."
mkdir -p "$BUNDLE_CACHE"
aws s3 cp "s3://${{ secrets.S3_BUCKET }}/penpot-${{ needs.prepare.outputs.gh_ref }}.zip" "$ZIP.$$.tmp"
mv "$ZIP.$$.tmp" "$ZIP"
fi
# Extract only the bundle this job needs.
pushd docker/images
unzip -q "$ZIP" "penpot/${{ matrix.image }}/*"
mv "penpot/${{ matrix.image }}" "bundle-${{ matrix.image }}"
popd
- name: Set up QEMU (stable)
uses: docker/setup-qemu-action@v4
with:
platforms: linux/amd64,linux/arm64
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Extract metadata (tags, labels)
id: meta
uses: docker/metadata-action@v6
with:
images: ${{ matrix.image }}
images:
frontend
backend
exporter
storybook
mcp
labels: |
bundle_version=${{ needs.prepare.outputs.bundle_version }}
bundle_version=${{ steps.bundles.outputs.bundle_version }}
- name: Build and push Docker image
- name: Build and push Backend Docker image
uses: docker/build-push-action@v7
env:
DOCKER_IMAGE: 'backend'
BUNDLE_PATH: './bundle-backend'
with:
context: ./docker/images/
file: ./docker/images/Dockerfile.${{ matrix.image }}
file: ./docker/images/Dockerfile.backend
platforms: linux/amd64,linux/arm64
push: true
provenance: mode=max
sbom: true
# Immutable tag only; branch tags are moved atomically for the
# whole image set by the `promote` job.
tags: ${{ secrets.DOCKER_REGISTRY }}/${{ matrix.image }}:sha-${{ needs.prepare.outputs.sha }}
tags: ${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:${{ steps.vars.outputs.gh_ref }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ matrix.image }}:buildcache
cache-to: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ matrix.image }}:buildcache,mode=max
cache-from: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache
cache-to: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache,mode=max
# ── 3. Move the branch tags of ALL images together ─────────────────────
# Runs only when every build succeeded (default `needs` semantics); if
# the set was already complete, `build` is skipped and so is this job —
# the S3 marker guarantees the branch tags were already moved.
promote:
name: Promote image set
runs-on: penpot-extended-runner
timeout-minutes: 10
needs: [prepare, build]
steps:
- name: Set common environment variables
run: |
echo "DOCKER_CONFIG=${{ runner.temp }}/.docker-${{ github.run_id }}-${{ github.job }}" >> $GITHUB_ENV
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Login to Docker Registry
uses: docker/login-action@v4
with:
registry: ${{ secrets.DOCKER_REGISTRY }}
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Point branch tags to the new build key
run: |
set -e
for image in $ALL_IMAGES; do
docker buildx imagetools create \
-t "${{ secrets.DOCKER_REGISTRY }}/$image:${{ needs.prepare.outputs.gh_ref }}" \
"${{ secrets.DOCKER_REGISTRY }}/$image:sha-${{ needs.prepare.outputs.sha }}"
done
# The marker is written LAST: its presence certifies that all five
# images exist and all branch tags point to this build key.
- name: Write set-completed marker
- name: Build and push Frontend Docker image
uses: docker/build-push-action@v7
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
run: |
echo "${{ github.run_id }}" | aws s3 cp - \
"s3://${{ secrets.S3_BUCKET }}/markers/images-sha-${{ needs.prepare.outputs.sha }}"
{
echo "### ✅ Image set promoted"
echo ""
echo "All \`:${{ needs.prepare.outputs.gh_ref }}\` tags now point to \`sha-${{ needs.prepare.outputs.sha }}\`."
} >> "$GITHUB_STEP_SUMMARY"
DOCKER_IMAGE: 'frontend'
BUNDLE_PATH: './bundle-frontend'
with:
context: ./docker/images/
file: ./docker/images/Dockerfile.frontend
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:${{ steps.vars.outputs.gh_ref }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache
cache-to: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache,mode=max
# ── 4. Single failure notification for the whole workflow ─────────────
notify:
name: Notify failure
runs-on: penpot-extended-runner
timeout-minutes: 5
needs: [prepare, build, promote]
if: failure()
- name: Build and push Exporter Docker image
uses: docker/build-push-action@v7
env:
DOCKER_IMAGE: 'exporter'
BUNDLE_PATH: './bundle-exporter'
with:
context: ./docker/images/
file: ./docker/images/Dockerfile.exporter
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:${{ steps.vars.outputs.gh_ref }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache
cache-to: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache,mode=max
- name: Build and push Storybook Docker image
uses: docker/build-push-action@v7
env:
DOCKER_IMAGE: 'storybook'
BUNDLE_PATH: './bundle-storybook'
with:
context: ./docker/images/
file: ./docker/images/Dockerfile.storybook
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:${{ steps.vars.outputs.gh_ref }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache
cache-to: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache,mode=max
- name: Build and push MCP Docker image
uses: docker/build-push-action@v7
env:
DOCKER_IMAGE: 'mcp'
BUNDLE_PATH: './bundle-mcp'
with:
context: ./docker/images/
file: ./docker/images/Dockerfile.mcp
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:${{ steps.vars.outputs.gh_ref }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache
cache-to: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache,mode=max
steps:
- name: Notify Mattermost
uses: mattermost/action-mattermost-notify@ae31bb6f9e26a54336e79696f108a2c91cf55b4e # v2.1.0
if: failure()
uses: mattermost/action-mattermost-notify@master
with:
MATTERMOST_WEBHOOK_URL: ${{ secrets.MATTERMOST_WEBHOOK }}
MATTERMOST_CHANNEL: bot-alerts-cicd
TEXT: |
❌ 🐳 *[PENPOT] Error building/promoting the penpot docker image set.*
📄 Triggered from ref: `${{ needs.prepare.outputs.gh_ref || inputs.gh_ref }}`
📦 Bundle: `${{ needs.prepare.outputs.bundle_version || 'n/a' }}`
❌ 🐳 *[PENPOT] Error building penpot docker images.*
📄 Triggered from ref: `${{ steps.vars.outputs.gh_ref }}`
📦 Bundle: `${{ steps.bundles.outputs.bundle_version }}`
🔗 Run: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}
@infra
@@ -1,24 +1,22 @@
name: _TMP TOKENS
name: _MAIN-STAGING
on:
workflow_dispatch:
schedule:
- cron: '46 5-20 * * 1-5'
concurrency:
group: ${{ github.workflow }}
cancel-in-progress: true
- cron: '26 5-20 * * 1-5'
jobs:
build-bundle:
uses: ./.github/workflows/build-bundle.yml
secrets: inherit
with:
gh_ref: "hiru-tokens-in-libs"
gh_ref: "main-staging"
build_wasm: "yes"
build_storybook: "yes"
build-docker:
needs: build-bundle
uses: ./.github/workflows/build-docker.yml
secrets: inherit
with:
gh_ref: "hiru-tokens-in-libs"
gh_ref: "main-staging"
+2 -10
View File
@@ -5,16 +5,14 @@ on:
schedule:
- cron: '36 5-20 * * 1-5'
concurrency:
group: ${{ github.workflow }}
cancel-in-progress: true
jobs:
build-bundle:
uses: ./.github/workflows/build-bundle.yml
secrets: inherit
with:
gh_ref: "staging"
build_wasm: "yes"
build_storybook: "yes"
build-docker:
needs: build-bundle
@@ -22,9 +20,3 @@ jobs:
secrets: inherit
with:
gh_ref: "staging"
build-docker-admin-console:
uses: ./.github/workflows/build-docker-admin-console.yml
secrets: inherit
with:
gh_ref: "staging"
+6 -19
View File
@@ -6,18 +6,14 @@ on:
tags:
- '*'
# Keyed by ref and never cancelling: pushing 2.17.2 shortly after 2.17.2-RC1
# must not abort the release already in flight.
concurrency:
group: ${{ github.workflow }}-${{ github.ref_name }}
cancel-in-progress: false
jobs:
build-bundle:
uses: ./.github/workflows/build-bundle.yml
secrets: inherit
with:
gh_ref: ${{ github.ref_name }}
build_wasm: "yes"
build_storybook: "yes"
build-docker:
needs: build-bundle
@@ -26,21 +22,14 @@ jobs:
with:
gh_ref: ${{ github.ref_name }}
build-docker-admin-console:
uses: ./.github/workflows/build-docker-admin-console.yml
secrets: inherit
with:
gh_ref: ${{ github.ref_name }}
notify:
name: Notifications
runs-on: ubuntu-24.04
needs:
- build-docker
- build-docker-admin-console
needs: build-docker
steps:
- name: Notify Mattermost
uses: mattermost/action-mattermost-notify@ae31bb6f9e26a54336e79696f108a2c91cf55b4e # v2.1.0
uses: mattermost/action-mattermost-notify@master
with:
MATTERMOST_WEBHOOK_URL: ${{ secrets.MATTERMOST_WEBHOOK }}
MATTERMOST_CHANNEL: bot-alerts-cicd
@@ -51,9 +40,7 @@ jobs:
publish-final-tag:
if: ${{ !contains(github.ref_name, '-RC') && !contains(github.ref_name, '-alpha') && !contains(github.ref_name, '-beta') && contains(github.ref_name, '.') }}
needs:
- build-docker
- build-docker-admin-console
needs: build-docker
uses: ./.github/workflows/release.yml
secrets: inherit
with:
+1 -1
View File
@@ -131,7 +131,7 @@ jobs:
- name: Notify Mattermost
if: failure()
uses: mattermost/action-mattermost-notify@ae31bb6f9e26a54336e79696f108a2c91cf55b4e # v2.1.0
uses: mattermost/action-mattermost-notify@master
with:
MATTERMOST_WEBHOOK_URL: ${{ secrets.MATTERMOST_WEBHOOK }}
MATTERMOST_CHANNEL: bot-alerts-cicd
+2 -2
View File
@@ -34,7 +34,7 @@ permissions:
jobs:
deploy:
runs-on: penpot-standar-runner
runs-on: penpot-runner-01
steps:
- name: Checkout
uses: actions/checkout@v6
@@ -114,7 +114,7 @@ jobs:
- name: Notify Mattermost
if: failure()
uses: mattermost/action-mattermost-notify@ae31bb6f9e26a54336e79696f108a2c91cf55b4e # v2.1.0
uses: mattermost/action-mattermost-notify@master
with:
MATTERMOST_WEBHOOK_URL: ${{ secrets.MATTERMOST_WEBHOOK }}
MATTERMOST_CHANNEL: bot-alerts-cicd
@@ -129,7 +129,7 @@ jobs:
- name: Notify Mattermost
if: failure()
uses: mattermost/action-mattermost-notify@ae31bb6f9e26a54336e79696f108a2c91cf55b4e # v2.1.0
uses: mattermost/action-mattermost-notify@master
with:
MATTERMOST_WEBHOOK_URL: ${{ secrets.MATTERMOST_WEBHOOK }}
MATTERMOST_CHANNEL: bot-alerts-cicd
+3 -2
View File
@@ -19,6 +19,7 @@ permissions:
jobs:
release:
environment: release-admins
runs-on: ubuntu-24.04
outputs:
version: ${{ steps.vars.outputs.gh_ref }}
@@ -62,7 +63,7 @@ jobs:
echo "$PUB_DOCKER_PASSWORD" | skopeo login --username "$PUB_DOCKER_USERNAME" --password-stdin docker.io
IMAGES=("frontend" "backend" "exporter" "mcp" "storybook")
IMAGES=("frontend" "backend" "exporter" "storybook")
SHORT_TAG=${TAG%.*}
for image in "${IMAGES[@]}"; do
@@ -103,7 +104,7 @@ jobs:
- name: Notify Mattermost
if: failure()
uses: mattermost/action-mattermost-notify@ae31bb6f9e26a54336e79696f108a2c91cf55b4e # v2.1.0
uses: mattermost/action-mattermost-notify@master
with:
MATTERMOST_WEBHOOK_URL: ${{ secrets.MATTERMOST_WEBHOOK }}
MATTERMOST_CHANNEL: bot-alerts-cicd
-85
View File
@@ -1,85 +0,0 @@
name: "CI: Backend"
defaults:
run:
shell: bash
on:
pull_request:
paths:
- 'backend/**'
- 'common/**'
types:
- opened
- synchronize
- ready_for_review
push:
branches:
- develop
- staging
paths:
- 'backend/**'
- 'common/**'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
test-backend:
if: ${{ !github.event.pull_request.draft }}
name: "Backend Tests"
runs-on: penpot-extended-runner
container:
image: penpotapp/devenv:latest
volumes:
- /var/cache/github-runner/m2:/root/.m2
- /var/cache/github-runner/gitlib:/root/.gitlibs
services:
postgres:
image: postgres:17
# Provide the password for postgres
env:
POSTGRES_USER: penpot_test
POSTGRES_PASSWORD: penpot_test
POSTGRES_DB: penpot_test
# Set health checks to wait until postgres has started
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
redis:
image: valkey/valkey:9
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Fmt
working-directory: ./backend
run: |
cljfmt check --parallel=true src/ test/
- name: Lint
working-directory: ./backend
run: |
clj-kondo --parallel --lint ../common/src/ src/
- name: Tests
working-directory: ./backend
env:
PENPOT_TEST_DATABASE_URI: "postgresql://postgres/penpot_test"
PENPOT_TEST_DATABASE_USERNAME: penpot_test
PENPOT_TEST_DATABASE_PASSWORD: penpot_test
PENPOT_TEST_REDIS_URI: "redis://redis/1"
run: |
mkdir -p /tmp/penpot;
clojure -M:dev:test
-57
View File
@@ -1,57 +0,0 @@
name: "CI: Common"
defaults:
run:
shell: bash
on:
pull_request:
paths:
- 'common/**'
types:
- opened
- synchronize
- ready_for_review
push:
branches:
- develop
- staging
paths:
- 'common/**'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
test-common:
if: ${{ !github.event.pull_request.draft }}
name: "Common Tests"
runs-on: penpot-extended-runner
container:
image: penpotapp/devenv:latest
volumes:
- /var/cache/github-runner/m2:/root/.m2
- /var/cache/github-runner/gitlib:/root/.gitlibs
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Lint
working-directory: ./common
run: |
corepack enable;
corepack install;
pnpm install;
pnpm run check-fmt:clj
pnpm run check-fmt:js
pnpm run lint:clj
- name: Tests
working-directory: ./common
run: |
./scripts/test
@@ -1,69 +0,0 @@
name: "CI: Composable Test Suite"
# Runs the composable component test suite (it exercises component semantics
# through the real Plugin API against the full frontend, so it needs the
# frontend bundle + the plugin runtime, but no backend): the driver serves the
# prebuilt frontend bundle and intercepts every backend RPC with Playwright
# fixtures. See plugins/apps/composable-test-suite/README.md ("Running in CI").
defaults:
run:
shell: bash
on:
pull_request:
paths:
- 'plugins/**'
- 'frontend/**'
- 'common/**'
types:
- opened
- synchronize
- ready_for_review
push:
branches:
- develop
- staging
paths:
- 'plugins/**'
- 'frontend/**'
- 'common/**'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
composable-test-suite:
if: ${{ !github.event.pull_request.draft }}
name: "Run composable test suite (mocked backend)"
runs-on: penpot-extended-runner
container:
image: penpotapp/devenv:latest
volumes:
- /var/cache/github-runner/m2:/root/.m2
- /var/cache/github-runner/gitlib:/root/.gitlibs
steps:
- uses: actions/checkout@v6
# The driver serves the prebuilt bundle from frontend/resources/public.
- name: Build frontend bundle
working-directory: ./frontend
run: ./scripts/build
- name: Install deps
working-directory: ./plugins
run: |
corepack enable;
corepack install;
pnpm install;
- name: Install Playwright Chromium
working-directory: ./plugins
run: pnpm --filter composable-test-suite exec playwright install --with-deps chromium
- name: Run composable test suite (mocked)
working-directory: ./plugins
run: pnpm --filter composable-test-suite run test:ci
-58
View File
@@ -1,58 +0,0 @@
name: "CI: Exporter"
defaults:
run:
shell: bash
on:
pull_request:
paths:
- 'exporter/**'
- 'common/**'
types:
- opened
- synchronize
- ready_for_review
push:
branches:
- develop
- staging
paths:
- 'exporter/**'
- 'common/**'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
test-exporter:
if: ${{ !github.event.pull_request.draft }}
name: "Exporter Tests"
runs-on: penpot-runner-02
container:
image: penpotapp/devenv:latest
volumes:
- /var/cache/github-runner/m2:/root/.m2
- /var/cache/github-runner/gitlib:/root/.gitlibs
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Lint
working-directory: ./exporter
run: |
corepack enable;
corepack install;
pnpm install;
pnpm run check-fmt:clj
pnpm run lint:clj
- name: Tests
working-directory: ./exporter
run: |
./scripts/test
-71
View File
@@ -1,71 +0,0 @@
name: "CI: Frontend"
defaults:
run:
shell: bash
on:
pull_request:
paths:
- 'frontend/**'
- 'common/**'
- 'render-wasm/**'
types:
- opened
- synchronize
- ready_for_review
push:
branches:
- develop
- staging
paths:
- 'frontend/**'
- 'common/**'
- 'render-wasm/**'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
test-frontend:
if: ${{ !github.event.pull_request.draft }}
name: "Frontend Tests"
runs-on: penpot-extended-runner
container:
image: penpotapp/devenv:latest
volumes:
- /var/cache/github-runner/m2:/root/.m2
- /var/cache/github-runner/gitlib:/root/.gitlibs
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Lint
working-directory: ./frontend
run: |
corepack enable;
corepack install;
pnpm install;
pnpm run check-fmt:js
pnpm run check-fmt:clj
pnpm run check-fmt:scss
pnpm run lint:clj
pnpm run lint:js
pnpm run lint:scss
- name: Unit Tests
working-directory: ./frontend
run: |
./scripts/test
- name: Component Tests
working-directory: ./frontend
env:
VITEST_BROWSER_TIMEOUT: 120000
run: |
./scripts/test-components
-250
View File
@@ -1,250 +0,0 @@
name: "CI: Integration"
defaults:
run:
shell: bash
on:
workflow_dispatch:
inputs:
gh_ref:
description: 'Name of the branch or ref'
type: string
required: true
default: 'develop'
shards:
description: 'Shard layout (JSON array)'
type: choice
required: true
default: '[1, 2, 3, 4]'
options:
- '[1, 2, 3, 4]'
- '[1, 2, 3, 4, 5, 6]'
- '[1, 2]'
- '[1]'
workers:
description: 'Playwright workers per shard'
type: string
required: true
default: '2'
pull_request:
paths:
- 'frontend/**'
- 'common/**'
- 'render-wasm/**'
- '.github/workflows/tests-integration.yml'
types:
- opened
- synchronize
- ready_for_review
push:
branches:
- develop
- staging
paths:
- 'frontend/**'
- 'common/**'
- 'render-wasm/**'
- '.github/workflows/tests-integration.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || inputs.gh_ref || github.ref }}
cancel-in-progress: true
jobs:
build-integration:
if: ${{ !github.event.pull_request.draft }}
name: "Build Integration Bundle"
runs-on: penpot-extended-runner
timeout-minutes: 30
container:
image: penpotapp/devenv:latest
volumes:
- /var/cache/github-runner/m2:/root/.m2
- /var/cache/github-runner/gitlib:/root/.gitlibs
outputs:
bundle_key: ${{ steps.vars.outputs.bundle_key }}
steps:
# An empty `ref` makes checkout fall back to its default (the PR merge
# ref on pull_request, the pushed ref on push).
- name: Checkout repository
uses: actions/checkout@v6
with:
ref: ${{ inputs.gh_ref }}
# The cache key must come from the SHA actually checked out: on a manual
# run `github.sha` points at the dispatching ref, not at `gh_ref`.
- name: Extract cache key
id: vars
run: |
echo "bundle_key=integration-bundle-$(git rev-parse HEAD)" >> $GITHUB_OUTPUT
- name: Build Bundle
working-directory: ./frontend
run: |
./scripts/build
- name: Store Bundle Cache
uses: actions/cache@v5
with:
key: ${{ steps.vars.outputs.bundle_key }}
path: frontend/resources/public
test-integration:
if: ${{ !github.event.pull_request.draft }}
name: "Integration Tests (${{ matrix.shard }})"
runs-on: penpot-extended-runner
timeout-minutes: 40
needs: build-integration
strategy:
fail-fast: false
matrix:
shard: ${{ fromJSON(inputs.shards || '[1, 2, 3, 4]') }}
container:
image: penpotapp/devenv:latest
volumes:
- /var/cache/github-runner/m2:/root/.m2
- /var/cache/github-runner/gitlib:/root/.gitlibs
- /var/cache/github-runner/ms-playwright:/ms-playwright
env:
PLAYWRIGHT_BROWSERS_PATH: /ms-playwright
steps:
- name: Checkout Repository
uses: actions/checkout@v6
with:
ref: ${{ inputs.gh_ref }}
- name: Restore Cache
uses: actions/cache/restore@v5
with:
key: ${{ needs.build-integration.outputs.bundle_key }}
path: frontend/resources/public
- name: Install deps
working-directory: ./frontend
run: |
corepack enable;
corepack install;
pnpm install --frozen-lockfile;
# No-op once the shared volume is warm; keeps the first run working.
- name: Install Playwright Chromium
working-directory: ./frontend
run: pnpm exec playwright install chromium
# `strategy.job-total` is the matrix size, so the shard denominator
# follows the `shards` input without being hardcoded.
- name: Run Tests
working-directory: ./frontend
env:
WORKERS: ${{ inputs.workers }}
run: |
WORKERS=${WORKERS:-2}
echo "Running shard ${{ matrix.shard }}/${{ strategy.job-total }} with $WORKERS workers"
pnpm exec playwright test --project default \
--workers="$WORKERS" \
--shard=${{ matrix.shard }}/${{ strategy.job-total }} \
--reporter=blob
- name: Upload blob report
uses: actions/upload-artifact@v7
if: always()
with:
name: integration-blob-report-${{ matrix.shard }}
path: frontend/blob-report/
overwrite: true
retention-days: 3
- name: Upload test result
uses: actions/upload-artifact@v7
if: always()
with:
name: integration-tests-result-${{ matrix.shard }}
path: frontend/test-results/
overwrite: true
if-no-files-found: ignore
retention-days: 3
merge-reports:
if: ${{ always() && !github.event.pull_request.draft && needs.test-integration.result != 'skipped' }}
name: "Merge Integration Reports"
runs-on: penpot-extended-runner
timeout-minutes: 15
needs: test-integration
container:
image: penpotapp/devenv:latest
volumes:
- /var/cache/github-runner/m2:/root/.m2
- /var/cache/github-runner/gitlib:/root/.gitlibs
steps:
- name: Checkout Repository
uses: actions/checkout@v6
with:
ref: ${{ inputs.gh_ref }}
- name: Install deps
working-directory: ./frontend
run: |
corepack enable;
corepack install;
pnpm install --frozen-lockfile;
- name: Download blob reports
uses: actions/download-artifact@v7
with:
path: frontend/all-blob-reports
pattern: integration-blob-report-*
merge-multiple: true
- name: Merge into HTML report
working-directory: ./frontend
env:
PLAYWRIGHT_JSON_OUTPUT_NAME: report.json
run: |
pnpm exec playwright merge-reports \
--reporter=html,json,list ./all-blob-reports
- name: Test summary
if: always()
working-directory: ./frontend
run: |
if [ ! -f report.json ]; then
echo "No report produced (all shards failed early)." >> "$GITHUB_STEP_SUMMARY"
exit 0
fi
jq -r -f ../.github/scripts/playwright-summary.jq report.json >> "$GITHUB_STEP_SUMMARY"
# Kept for 30 days so flakiness rates can be aggregated across runs
# without scraping job logs.
- name: Upload JSON report
uses: actions/upload-artifact@v7
if: always()
with:
name: integration-json-report
path: frontend/report.json
overwrite: true
if-no-files-found: ignore
retention-days: 30
- name: Upload HTML report
uses: actions/upload-artifact@v7
with:
name: integration-html-report
path: frontend/playwright-report/
overwrite: true
retention-days: 7
-58
View File
@@ -1,58 +0,0 @@
name: "CI: Library"
defaults:
run:
shell: bash
on:
pull_request:
paths:
- 'common/**'
- 'library/**'
types:
- opened
- synchronize
- ready_for_review
push:
branches:
- develop
- staging
paths:
- 'common/**'
- 'library/**'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
test-library:
if: ${{ !github.event.pull_request.draft }}
name: "Library Tests"
runs-on: penpot-extended-runner
container:
image: penpotapp/devenv:latest
volumes:
- /var/cache/github-runner/m2:/root/.m2
- /var/cache/github-runner/gitlib:/root/.gitlibs
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Lint
working-directory: ./library
run: |
corepack enable;
corepack install;
pnpm install;
pnpm run check-fmt
pnpm run lint
- name: Tests
working-directory: ./library
run: |
./scripts/test
+2 -7
View File
@@ -1,4 +1,4 @@
name: "CI: MCP"
name: "MCP CI"
on:
pull_request:
@@ -28,7 +28,7 @@ jobs:
test-mcp:
if: ${{ !github.event.pull_request.draft }}
name: "Test MCP"
runs-on: penpot-extended-runner
runs-on: penpot-runner-02
container: penpotapp/devenv:latest
steps:
@@ -45,8 +45,3 @@ jobs:
pnpm run fmt:check;
pnpm -r run build;
pnpm -r run types:check;
- name: Tests
working-directory: ./mcp
run: |
pnpm run test;
@@ -1,133 +0,0 @@
name: "CI: Plugin API Test Suite"
# Runs the Plugin API Test Suite (it exercises the real Penpot Plugin API, so it
# needs a running frontend + the plugin runtime). Two jobs:
#
# - api-test-suite-mocked (pull_request / push): the per-PR gate. Serves the
# prebuilt frontend bundle and intercepts every backend RPC with Playwright
# (MOCK_BACKEND=1). No backend / no login. Validates the frontend Plugin API
# binding + in-memory store; backend-result-dependent tests are skipped via the
# `skipIfMocked` tag. See plugins/apps/plugin-api-test-suite/README.md.
#
# - api-test-suite-live (workflow_dispatch): true end-to-end against a LIVE
# instance. Point PENPOT_BASE_URL at a reachable instance and provide login
# credentials via repo secrets. Manual because the CI runner has no Docker to
# stand up a full stack.
defaults:
run:
shell: bash
on:
workflow_dispatch:
inputs:
base_url:
description: "Penpot base URL (e.g. https://localhost:3449)"
required: false
default: "https://localhost:3449"
pull_request:
paths:
- 'plugins/**'
- 'frontend/**'
- 'common/**'
types:
- opened
- synchronize
- ready_for_review
push:
branches:
- develop
- staging
paths:
- 'plugins/**'
- 'frontend/src/app/plugins/**'
- 'common/**'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
api-test-suite-mocked:
if: ${{ github.event_name != 'workflow_dispatch' && !github.event.pull_request.draft }}
name: "Run Plugin API Test Suite (mocked)"
runs-on: penpot-extended-runner
container:
image: penpotapp/devenv:latest
volumes:
- /var/cache/github-runner/m2:/root/.m2
- /var/cache/github-runner/gitlib:/root/.gitlibs
steps:
- uses: actions/checkout@v6
# Mocked mode serves the prebuilt bundle from frontend/resources/public.
- name: Build frontend bundle
working-directory: ./frontend
run: ./scripts/build
- name: Install deps
working-directory: ./plugins
run: |
corepack enable;
corepack install;
pnpm install;
- name: Install Playwright Chromium
working-directory: ./plugins
run: pnpm --filter plugin-api-test-suite exec playwright install --with-deps chromium
- name: Generate API surface
working-directory: ./plugins
run: pnpm --filter plugin-api-test-suite run gen:api
- name: Run API test suite (mocked)
working-directory: ./plugins
env:
MOCK_BACKEND: "1"
run: pnpm --filter plugin-api-test-suite run test:ci
## The following job will launch the whole suite of tests but we need
## to have a full environment in the CI for this to work.
# api-test-suite-live:
# if: ${{ github.event_name == 'workflow_dispatch' }}
# name: Run Plugin API Test Suite (live)
# runs-on: penpot-extended-runner
# container:
# image: penpotapp/devenv:latest
#
# env:
# PENPOT_BASE_URL: ${{ github.event.inputs.base_url }}
# E2E_LOGIN_EMAIL: ${{ secrets.E2E_LOGIN_EMAIL }}
# E2E_LOGIN_PASSWORD: ${{ secrets.E2E_LOGIN_PASSWORD }}
#
# steps:
# - uses: actions/checkout@v6
#
# - name: Setup Node
# uses: actions/setup-node@v6
# with:
# node-version-file: .nvmrc
#
# - name: Install deps
# working-directory: ./plugins
# run: |
# corepack enable;
# corepack install;
# pnpm install;
#
# - name: Install Playwright Chromium
# working-directory: ./plugins
# run: pnpm --filter plugin-api-test-suite exec playwright install --with-deps chromium
#
# - name: Generate API surface
# working-directory: ./plugins
# run: pnpm --filter plugin-api-test-suite run gen:api
#
# # Note: requires a running Penpot instance reachable at PENPOT_BASE_URL.
# - name: Run API test suite
# working-directory: ./plugins
# run: pnpm --filter plugin-api-test-suite run test:ci
-77
View File
@@ -1,77 +0,0 @@
name: "CI: Plugins"
defaults:
run:
shell: bash
on:
pull_request:
paths:
- 'plugins/**'
types:
- opened
- synchronize
- ready_for_review
push:
branches:
- develop
- staging
paths:
- 'plugins/**'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
test-plugins:
if: ${{ !github.event.pull_request.draft }}
name: Plugins Runtime Linter & Tests
runs-on: penpot-extended-runner
container:
image: penpotapp/devenv:latest
volumes:
- /var/cache/github-runner/m2:/root/.m2
- /var/cache/github-runner/gitlib:/root/.gitlibs
steps:
- uses: actions/checkout@v6
- name: Install deps
working-directory: ./plugins
shell: bash
run: |
corepack enable;
corepack install;
pnpm install -r;
- name: Run Lint
working-directory: ./plugins
run: pnpm run lint
- name: Run Format Check
working-directory: ./plugins
run: pnpm run format:check
- name: Run Test
working-directory: ./plugins
run: pnpm run test
- name: Build runtime
working-directory: ./plugins
run: pnpm run build:runtime
- name: Build doc
working-directory: ./plugins
run: pnpm run build:doc
- name: Build plugins
working-directory: ./plugins
run: pnpm run build:plugins
- name: Build styles
working-directory: ./plugins
run: pnpm run build:styles-example
-57
View File
@@ -1,57 +0,0 @@
name: "CI: WASM"
defaults:
run:
shell: bash
on:
pull_request:
paths:
- 'render-wasm/**'
types:
- opened
- synchronize
- ready_for_review
push:
branches:
- develop
- staging
paths:
- 'render-wasm/**'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
test-render-wasm:
if: ${{ !github.event.pull_request.draft }}
name: "Render WASM Tests"
runs-on: penpot-extended-runner
container:
image: penpotapp/devenv:latest
volumes:
- /var/cache/github-runner/m2:/root/.m2
- /var/cache/github-runner/gitlib:/root/.gitlibs
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Format
working-directory: ./render-wasm
run: |
cargo fmt --check
- name: Lint
working-directory: ./render-wasm
run: |
./lint
- name: Test
working-directory: ./render-wasm
run: |
./test
+363
View File
@@ -0,0 +1,363 @@
name: "CI"
defaults:
run:
shell: bash
on:
pull_request:
types:
- opened
- synchronize
- ready_for_review
push:
branches:
- develop
- staging
concurrency:
group: ${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
lint:
if: ${{ !github.event.pull_request.draft }}
name: "Linter"
runs-on: penpot-runner-02
container: penpotapp/devenv:latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Lint Common
working-directory: ./common
run: |
corepack enable;
corepack install;
pnpm install;
pnpm run check-fmt:clj
pnpm run check-fmt:js
pnpm run lint:clj
- name: Lint Frontend
working-directory: ./frontend
run: |
corepack enable;
corepack install;
pnpm install;
pnpm run check-fmt:js
pnpm run check-fmt:clj
pnpm run check-fmt:scss
pnpm run lint:clj
pnpm run lint:js
pnpm run lint:scss
- name: Lint Backend
working-directory: ./backend
run: |
corepack enable;
corepack install;
pnpm install;
pnpm run check-fmt
pnpm run lint
- name: Lint Exporter
working-directory: ./exporter
run: |
corepack enable;
corepack install;
pnpm install;
pnpm run check-fmt
pnpm run lint
- name: Lint Library
working-directory: ./library
run: |
corepack enable;
corepack install;
pnpm install;
pnpm run check-fmt
pnpm run lint
test-common:
if: ${{ !github.event.pull_request.draft }}
name: "Common Tests"
runs-on: penpot-runner-02
container: penpotapp/devenv:latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Run tests
working-directory: ./common
run: |
./scripts/test
test-plugins:
if: ${{ !github.event.pull_request.draft }}
name: Plugins Runtime Linter & Tests
runs-on: penpot-runner-02
container: penpotapp/devenv:latest
steps:
- uses: actions/checkout@v6
- name: Setup Node
id: setup-node
uses: actions/setup-node@v6
with:
node-version-file: .nvmrc
- name: Install deps
working-directory: ./plugins
shell: bash
run: |
corepack enable;
corepack install;
pnpm install;
- name: Run Lint
working-directory: ./plugins
run: pnpm run lint
- name: Run Format Check
working-directory: ./plugins
run: pnpm run format:check
- name: Run Test
working-directory: ./plugins
run: pnpm run test
- name: Build runtime
working-directory: ./plugins
run: pnpm run build:runtime
- name: Build doc
working-directory: ./plugins
run: pnpm run build:doc
- name: Build plugins
working-directory: ./plugins
run: pnpm run build:plugins
- name: Build styles
working-directory: ./plugins
run: pnpm run build:styles-example
test-frontend:
if: ${{ !github.event.pull_request.draft }}
name: "Frontend Tests"
runs-on: penpot-runner-02
container: penpotapp/devenv:latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Unit Tests
working-directory: ./frontend
run: |
./scripts/test
- name: Component Tests
working-directory: ./frontend
env:
VITEST_BROWSER_TIMEOUT: 120000
run: |
./scripts/test-components
test-render-wasm:
if: ${{ !github.event.pull_request.draft }}
name: "Render WASM Tests"
runs-on: penpot-runner-02
container: penpotapp/devenv:latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Format
working-directory: ./render-wasm
run: |
cargo fmt --check
- name: Lint
working-directory: ./render-wasm
run: |
./lint
- name: Test
working-directory: ./render-wasm
run: |
./test
test-backend:
if: ${{ !github.event.pull_request.draft }}
name: "Backend Tests"
runs-on: penpot-runner-02
container: penpotapp/devenv:latest
services:
postgres:
image: postgres:17
# Provide the password for postgres
env:
POSTGRES_USER: penpot_test
POSTGRES_PASSWORD: penpot_test
POSTGRES_DB: penpot_test
# Set health checks to wait until postgres has started
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
redis:
image: valkey/valkey:9
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Run tests
working-directory: ./backend
env:
PENPOT_TEST_DATABASE_URI: "postgresql://postgres/penpot_test"
PENPOT_TEST_DATABASE_USERNAME: penpot_test
PENPOT_TEST_DATABASE_PASSWORD: penpot_test
PENPOT_TEST_REDIS_URI: "redis://redis/1"
run: |
clojure -M:dev:test --reporter kaocha.report/documentation
test-library:
if: ${{ !github.event.pull_request.draft }}
name: "Library Tests"
runs-on: penpot-runner-02
container: penpotapp/devenv:latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Run tests
working-directory: ./library
run: |
./scripts/test
build-integration:
if: ${{ !github.event.pull_request.draft }}
name: "Build Integration Bundle"
runs-on: penpot-runner-02
container: penpotapp/devenv:latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Build Bundle
working-directory: ./frontend
run: |
./scripts/build
- name: Store Bundle Cache
uses: actions/cache@v5
with:
key: "integration-bundle-${{ github.sha }}"
path: frontend/resources/public
test-integration-1:
if: ${{ !github.event.pull_request.draft }}
name: "Integration Tests 1/3"
runs-on: penpot-runner-02
container: penpotapp/devenv:latest
needs: build-integration
steps:
- name: Checkout Repository
uses: actions/checkout@v6
- name: Restore Cache
uses: actions/cache/restore@v5
with:
key: "integration-bundle-${{ github.sha }}"
path: frontend/resources/public
- name: Run Tests
working-directory: ./frontend
run: |
./scripts/test-e2e --shard="1/3";
- name: Upload test result
uses: actions/upload-artifact@v7
if: always()
with:
name: integration-tests-result-1
path: frontend/test-results/
overwrite: true
retention-days: 3
test-integration-2:
if: ${{ !github.event.pull_request.draft }}
name: "Integration Tests 2/3"
runs-on: penpot-runner-02
container: penpotapp/devenv:latest
needs: build-integration
steps:
- name: Checkout Repository
uses: actions/checkout@v6
- name: Restore Cache
uses: actions/cache/restore@v5
with:
key: "integration-bundle-${{ github.sha }}"
path: frontend/resources/public
- name: Run Tests
working-directory: ./frontend
run: |
./scripts/test-e2e --shard="2/3";
- name: Upload test result
uses: actions/upload-artifact@v7
if: always()
with:
name: integration-tests-result-2
path: frontend/test-results/
overwrite: true
retention-days: 3
test-integration-3:
if: ${{ !github.event.pull_request.draft }}
name: "Integration Tests 3/3"
runs-on: penpot-runner-02
container: penpotapp/devenv:latest
needs: build-integration
steps:
- name: Checkout Repository
uses: actions/checkout@v6
- name: Restore Cache
uses: actions/cache/restore@v5
with:
key: "integration-bundle-${{ github.sha }}"
path: frontend/resources/public
- name: Run Tests
working-directory: ./frontend
run: |
./scripts/test-e2e --shard="3/3";
- name: Upload test result
uses: actions/upload-artifact@v7
if: always()
with:
name: integration-tests-result-3
path: frontend/test-results/
overwrite: true
retention-days: 3
+3 -26
View File
@@ -9,22 +9,12 @@
.clj-kondo
.cpcache
.lsp
.env
.nrepl-port
.nyc_output
.rebel_readline_history
.repl
opencode.json
.opencode/package-lock.json
/*.jpg
/*.md
!CHANGES.md
!CONTRIBUTING.md
!README.md
!AGENTS.md
!CODE_OF_CONDUCT.md
!SECURITY.md
!HIGHLIGHTS.md
/*.png
/*.svg
/*.sql
@@ -34,6 +24,7 @@ opencode.json
/.clj-kondo/.cache
/_dump
/notes
/.opencode/package-lock.json
/plans
/prompts
/playground/
@@ -59,8 +50,6 @@ opencode.json
/docker/images/bundle*
/exporter/target
/exporter/.shadow-cljs
/exporter/resources/wasm/
/exporter/src/app/wasm/shared.js
/frontend/.storybook/preview-body.html
/frontend/.storybook/preview-head.html
/frontend/playwright-report/
@@ -77,6 +66,8 @@ opencode.json
/frontend/target/
/frontend/test-results/
/frontend/.shadow-cljs
/other/
/scripts/
/nexus/
/tmp/
/vendor/**/target
@@ -91,21 +82,7 @@ opencode.json
/blob-report/
/playwright/.cache/
/render-wasm/target/
/media-processor/dist/
/**/node_modules
/**/.yarn/*
/.pnpm-store
/.vscode
/.idea
*.iml
/.claude
/CLAUDE.md
/.playwright-mcp
/.devenv/mcp/
/opencode.json
/.opencode/plans
/.opencode/reports
/.opencode/prompts
/.ci-logs
/.codex/
/tools/__pycache__
+1 -1
View File
@@ -1 +1 @@
v24.19.0
v22.22.0
+33
View File
@@ -0,0 +1,33 @@
---
name: commiter
description: Git commit assistant following CONTRIBUTING.md commit rules
mode: all
---
## Role
You are responsible for creating git commits for Penpot and must
follow the repository commit-format rules exactly. It should have
concise title and clear summary of changes in the description,
including the rationale if proceed.
## Requirements
* Override your internal commit rules when the user explicitly requests
something that conflicts with them.
* Read `CONTRIBUTING.md` before creating any commit and follow the
commit guidelines strictly.
* Use commit messages in the form `:emoji: <imperative subject>`.
* Keep the subject capitalized, concise, 70 characters or fewer, and
without a trailing period.
* Keep the description (commit body) with maximum line length of 80
characters. Use manual line breaks to wrap text before it exceeds
this limit.
* Separate the subject from the body with a blank line.
* Write a clear and concise body when needed.
* Use `git commit -s` so the commit includes the required
`Signed-off-by` line.
* Do not guess or hallucinate git author information (Name or
Email). Never include the `--author` flag in git commands unless
specifically instructed by the user for a unique case; assume the
local environment is already configured.
+37
View File
@@ -0,0 +1,37 @@
---
name: Penpot Engineer
description: Senior Full-Stack Software Engineer
mode: primary
---
Role: You are a high-autonomy Senior Full-Stack Software Engineer working on
Penpot, an open-source design tool. You have full permission to navigate the
codebase, modify files, and execute commands to fulfill your tasks. Your goal is
to solve complex technical tasks with high precision while maintaining a strong
focus on maintainability and performance.
Tech stack: Clojure (backend), ClojureScript (frontend/exporter), Rust/WASM
(render-wasm), TypeScript (plugins/mcp), SCSS.
Requirements:
* Read the root `AGENTS.md` to understand the repository and application
architecture. Then read the `AGENTS.md` **only** for each affected module.
Not all modules have one — verify before reading.
* Before writing code, analyze the task in depth and describe your plan. If the
task is complex, break it down into atomic steps.
* When searching code, prefer `ripgrep` (`rg`) over `grep` — it respects
`.gitignore` by default.
* Do **not** touch unrelated modules unless the task explicitly requires it.
* Only reference functions, namespaces, or APIs that actually exist in the
codebase. Verify their existence before citing them. If unsure, search first.
* Be concise and autonomous — avoid unnecessary explanations.
* After making changes, run the applicable lint and format checks for the
affected module before considering the work done (see module `AGENTS.md` for
exact commands).
* Make small and logical commits following the commit guideline described in
`CONTRIBUTING.md`. Commit only when explicitly asked.
- Do not guess or hallucinate git author information (Name or Email). Never include the
`--author` flag in git commands unless specifically instructed by the user for a unique
case; assume the local environment is already configured. Allow git commit to
automatically pull the identity from the local git config `user.name` and `user.email`.
+64
View File
@@ -0,0 +1,64 @@
---
name: Penpot Planner
description: Software architect for planning and analysis only
mode: primary
permission:
edit: ask
---
# Penpot Planner
## Role
You are a Senior Software Architect working on Penpot, an open-source design
tool. Your sole responsibility is planning and analysis — you do NOT write,
modify any code.
You help users understand the codebase, design solutions, and create detailed
implementation plans that other agents or developers can execute. Document
everything they need to know: which files to touch for each task, code, testing,
docs they might need to check, how to test it. Give them the whole plan as
bite-sized tasks. DRY. YAGNI. TDD. Frequent commits.
Do **not** suggest commit messages or commit names anywhere in your plans or
responses — committing is the developer's responsibility.
Assume they are a skilled developer, but know almost nothing about our toolset
or problem domain. Assume they don't know good test design very well.
## Requirements
* Analyze the codebase architecture and identify affected modules.
* Read `AGENTS.md` files (root and per-module) to understand structure and
conventions.
* Search code using `ripgrep` skill (`rg`) to trace dependencies, find patterns,
and understand existing implementations.
* Break down complex features or bugs into atomic, actionable steps.
* Propose solutions with clear rationale, trade-offs, and sequencing.
* Identify risks, edge cases, and testing considerations.
Save plans to: plans/YYYY-MM-DD-<plan-one-line-title>.md
## Constraints
* You are **read-only** — never create, edit, or delete files.
* You do **not** run builds, tests, linters, or any commands that modify state.
* You do **not** create git commits or interact with version control.
* You do **not** execute shell commands beyond read-only searches (`rg`, `ls`,
`find`, `cat`).
* Your output is a structured plan or analysis, ready for handoff to an
engineer agent or developer.
## Output format
When producing a plan, structure it as:
1. **Context** — What is the problem or feature request?
2. **Affected modules** — Which parts of the codebase are involved?
3. **Approach** — Step-by-step implementation plan with file paths and
function names where applicable.
4. **Risks & considerations** — Edge cases, performance implications, breaking
changes.
5. **Testing strategy** — How to verify the implementation works correctly.
+59
View File
@@ -0,0 +1,59 @@
---
name: Prompt Assistant
description: Refines and improves prompts for maximum clarity and effectiveness
mode: all
---
# Prompt Assistant
## Role
You are an expert Prompt Engineer with strong knowledge of
penpot. Your sole responsibility is to take a prompt provided by the
user and transform it into the most effective, clear, and
well-structured version possible — ready to be used with any AI model.
## Requirements
* You do NOT execute tasks. You do NOT write code. You only design and
refine prompts
* Read the root `AGENTS.md` to understand the repository and application
architecture. Then read the `AGENTS.md` **only** for each affected module.
* Analyze the original prompt: identify its intent, target audience,
ambiguities, missing context, and structural weaknesses
* Ask clarifying questions if the intent is unclear or if critical
information is missing (e.g. target model, expected output format,
tone, constraints). Keep questions concise and grouped
* Rewrite the prompt using prompt engineering best practices
## Prompt Engineering Principles
Apply these techniques when refining prompts:
- **Be specific and explicit**: Replace vague instructions with precise ones.
- **Set the context**: Include background information the model needs to
perform well.
- **Specify the output format**: State the desired structure, length, tone,
or format (e.g. bullet list, JSON, step-by-step).
- **Add constraints**: Include what the model should avoid or not do.
- **Use examples** (few-shot): When applicable, suggest adding examples to
anchor the model's behaviour.
- **Break down complexity**: Split multi-step tasks into clear numbered steps.
- **Avoid ambiguity**: Remove pronouns and references that could be
misinterpreted.
- **Chain of thought**: For reasoning tasks, include "Think step by step."
## Constraints
- Do NOT execute the prompt yourself.
- Do NOT answer the question inside the prompt.
- Do NOT add unnecessary verbosity — prompts should be as short as they can
be while remaining complete.
- Always preserve the user's original intent.
## Output
Refined Prompt: The improved, ready-to-use prompt. Print it for
immediate use and save it to
prompts/YYYY-MM-DD-N-<prompt-one-line-title>.md for future use.
-39
View File
@@ -1,39 +0,0 @@
---
description: Execute a ready plan end-to-end — create a GitHub issue, branch issue-NNNN, implement the plan, then commit via the create-commit skill
agent: build
---
This command is run once a plan is ready (for example, from plan mode). Execute
the plan already prepared in the current session context. Follow these steps in order.
## 1. Create the issue
Use the **`create-issue`** skill, following the *Creating Issues from Draft Body*
flow in `mem:workflow/creating-issues`. Derive the issue title and body from the
plan. Capture the new issue's number — call it **NNNN** (needed for the branch
name and the commit reference).
## 2. Create the branch
Create and switch to a branch named after the issue:
```
git checkout -b issue-NNNN
```
(Replace NNNN with the issue number from step 1.)
## 3. Execute the plan
Implement the prepared plan from the session context. Work methodically, keeping
changes focused on what the issue requires. Do not commit — the commit happens in
step 4.
## 4. Commit with the create-commit skill
After the implementation is complete, load the **`create-commit`** skill and
follow its workflow to commit the changes. Provide a brief summary of what was
implemented and why, the issue reference (`issue-NNNN`), and the model name you
are running as so the `AI-assisted-by` trailer is set correctly.
Do not push. Pushing is handled separately by the user.
@@ -1,40 +0,0 @@
---
description: Resolve local git conflicts and stage the resolved files with git add — never continues the rebase
agent: build
---
# Fix Git Conflicts
Resolve conflicts in the local repository. The user handles finishing the
rebase themselves — you must **never** run `git rebase --continue`,
`git rebase --skip`, `git merge --continue`, or anything similar.
## Phase 1 — Understand the problem (read-only)
1. Run `git status` to detect the conflict state (rebase, merge, cherry-pick, etc.) and list conflicted files.
2. For each conflicted (unmerged) file, understand the situation **without modifying anything**:
- Read the file and identify the conflict markers (`<<<<<<<`, `=======`, `>>>>>>>`).
- Inspect both sides — `git show <ours>:<file>` and `git show <theirs>:<file>` — plus `git log`/`git show` on the commits involved to understand intent.
- Identify what each side changed and why, and how they should be combined.
## Phase 2 — Present the resolution plan
3. **Present a clear plan to the user before touching any file.** For each conflicted file, state:
- What each side changed and why.
- Your proposed resolution and the reasoning behind it.
- How the two sides are combined (both additive → merge; both modify the same code → keep the semantically correct version, merging intent from both sides when clear from code and context).
4. **Ask the user only when genuinely unclear.** Do not ask about anything you can determine yourself from the code, commit messages, or context. Only decisions that are not determinable and change the outcome (e.g. conflicting product decisions, which side to discard) warrant a question. **Collect all such questions together in an "Open Questions" section at the end of the plan**, so the user has full context to answer them properly.
5. **Wait for the user to accept the plan** (and answer any open questions) before editing, staging, or otherwise modifying anything.
## Phase 3 — Execute
6. Resolve each conflicted file by editing the file to the agreed merged content and removing all conflict markers.
## Phase 4 — Stage and verify
7. **Stage every resolved file** with `git add <file>`. Do not stage unrelated untracked files unless clearly part of the resolution.
8. Verify no conflict markers remain (search for `<<<<<<<` / `>>>>>>>` in resolved files) and that `git status` shows no unmerged paths.
## Phase 5 — Report
9. Briefly report the conflict state, how each conflicted file was resolved (and any answers received to open questions), and stop — do **not** run `git rebase --continue` or any other continuation command.
-299
View File
@@ -1,299 +0,0 @@
import { tool } from "@opencode-ai/plugin"
import path from "path"
import { spawn } from "child_process"
const penpotPsqlTool = tool({
description:
"Execute a SQL command against the Penpot database. Uses the defaults from scripts/psql.",
args: {
sql: tool.schema
.string()
.describe("SQL command to execute"),
test: tool.schema
.boolean()
.describe("Use the penpot_test database")
.optional(),
},
async execute(args, context) {
const host = process.env.PENPOT_DB_HOST || "postgres"
const user = process.env.PENPOT_DB_USER || "penpot"
const db = args.test
? "penpot_test"
: process.env.PENPOT_DB_NAME || "penpot"
const password = process.env.PENPOT_DB_PASSWORD || "penpot"
const psqlArgs = ["-h", host, "-U", user, "-d", db, "-c", args.sql]
return new Promise((resolve) => {
let stdout = ""
let stderr = ""
const proc = spawn("psql", psqlArgs, {
cwd: context.worktree,
env: { ...process.env, PGPASSWORD: password },
})
proc.stdout.on("data", (data) => {
stdout += data.toString()
})
proc.stderr.on("data", (data) => {
stderr += data.toString()
})
proc.on("error", (error) => {
resolve(`Error: ${error.message}`)
})
proc.on("close", (exitCode) => {
const output =
exitCode === 0
? stdout.trim() || "Query executed successfully"
: `Error (exit ${exitCode}): ${
(stderr || stdout).trim() || "No error output"
}`
resolve(output)
})
})
},
})
const parenRepairTool = tool({
description:
"Fix mismatched parentheses/braces in Clojure files (.clj, .cljs, .cljc) then reformat with cljfmt.",
args: {
// A string is used instead of an array so OpenCode displays it
// in the generic tool invocation.
files: tool.schema
.string()
.describe(
"Comma-separated file paths to fix, for example: frontend/src/app/config.cljs, backend/src/core.clj",
)
.optional(),
code: tool.schema
.string()
.describe("Code string to fix via stdin")
.optional(),
},
async execute(args, context) {
const script = path.join(context.worktree, "scripts/paren-repair")
const files = args.files
? args.files
.split(",")
.map((file) => file.trim())
.filter(Boolean)
: []
const paramInfo =
files.length > 0
? `files=[${files.join(", ")}]`
: args.code !== undefined
? `code=(${args.code.length} chars)`
: "none"
return new Promise((resolve) => {
const childArgs =
files.length > 0
? [script, ...files]
: [script]
const proc = spawn("bb", childArgs, {
cwd: context.worktree,
})
let stdout = ""
let stderr = ""
proc.stdout.on("data", (data) => {
stdout += data.toString()
})
proc.stderr.on("data", (data) => {
stderr += data.toString()
})
proc.on("error", (error) => {
resolve(`Error: ${error.message}`)
})
proc.on("close", (exitCode) => {
const output =
exitCode === 0
? stdout.trim() || "No changes needed"
: `Error (exit ${exitCode}): ${
(stderr || stdout).trim() || "No error output"
}`
resolve(output)
})
// Close stdin in all cases so the process cannot wait indefinitely.
if (args.code !== undefined) {
proc.stdin.end(args.code)
} else {
proc.stdin.end()
}
})
},
})
export default async function plugin() {
return {
tool: {
"paren-repair": parenRepairTool,
"penpot-psql": penpotPsqlTool,
},
}
}
// import { tool } from "@opencode-ai/plugin"
// import path from "path"
// import { spawn } from "child_process"
// function formatFiles(files) {
// if (files.length === 0) return "stdin"
// // Keep the visible tool title reasonably short.
// if (files.length <= 3) return files.join(", ")
// return `${files.slice(0, 3).join(", ")} (+${files.length - 3} more)`
// }
// const parenRepairTool = tool({
// description:
// "Fix mismatched parentheses/braces in Clojure files, then reformat with cljfmt.",
// args: {
// files: tool.schema
// .array(tool.schema.string())
// .describe("Array of file paths to fix")
// .optional(),
// code: tool.schema
// .string()
// .describe("Code string to fix via stdin")
// .optional(),
// },
// async execute(args, context) {
// const script = path.join(context.worktree, "scripts/paren-repair")
// const files = (args.files ?? []).map((file) => {
// const absolute = path.isAbsolute(file)
// ? file
// : path.resolve(context.worktree, file)
// return path.relative(context.worktree, absolute)
// })
// const targetSummary =
// files.length > 0
// ? formatFiles(files)
// : args.code !== undefined
// ? `stdin (${args.code.length} chars)`
// : "no input"
// // This updates the tool-call title immediately, while it is running.
// await context.metadata({
// title: `Paren repair: ${targetSummary}`,
// metadata: {
// files,
// codeChars: args.code?.length,
// },
// })
// const childArgs =
// args.files && args.files.length > 0
// ? [script, ...args.files]
// : [script]
// return new Promise((resolve) => {
// const proc = spawn("bb", childArgs, {
// cwd: context.worktree,
// })
// let stdout = ""
// let stderr = ""
// if (args.code !== undefined) {
// proc.stdin.end(args.code)
// }
// proc.stdout.on("data", (data) => {
// stdout += data.toString()
// })
// proc.stderr.on("data", (data) => {
// stderr += data.toString()
// })
// proc.on("close", (exitCode) => {
// const successful = exitCode === 0
// const commandOutput = successful
// ? stdout.trim() || "No changes needed"
// : `Error (exit ${exitCode}): ${(stderr || stdout).trim()}`
// const parameterOutput =
// files.length > 0
// ? `Files passed:\n${files.map((file) => `- ${file}`).join("\n")}`
// : args.code !== undefined
// ? `Input passed through stdin: ${args.code.length} characters`
// : "No files or stdin input were passed"
// resolve({
// title: `Paren repair: ${targetSummary}`,
// output: `${parameterOutput}\n\n${commandOutput}`,
// metadata: {
// files,
// codeChars: args.code?.length,
// exitCode,
// successful,
// },
// })
// })
// proc.on("error", (error) => {
// resolve({
// title: `Paren repair failed: ${targetSummary}`,
// output: [
// files.length > 0
// ? `Files passed:\n${files.map((file) => `- ${file}`).join("\n")}`
// : `Input: ${targetSummary}`,
// `Failed to start bb: ${error.message}`,
// ].join("\n\n"),
// metadata: {
// files,
// codeChars: args.code?.length,
// successful: false,
// },
// })
// })
// })
// },
// })
// export default async function plugin() {
// return {
// tool: {
// "paren-repair": parenRepairTool,
// },
// }
// }
+90
View File
@@ -0,0 +1,90 @@
---
name: backport-commit
description: Port changes from a specific Git commit to the current branch by manually applying the diff, avoiding cherry-pick when it would introduce complex conflicts.
---
# Backport Commit
Port changes from a specific Git commit to the current branch by manually
applying the diff, avoiding `git cherry-pick` when it would introduce
complex conflicts.
## When to Use
Use this skill whenever the user asks to backport a commit, especially when:
- The commit touches multiple modules or files with significant divergence
- `git cherry-pick` is explicitly ruled out ("do not use cherry-pick")
- The target commit is old enough that conflicts are likely
- The commit introduces both source changes AND new files (tests, etc.)
- You need full control over how each hunk is applied
## Workflow
### 1. Identify the target commit
```bash
# Verify the commit exists and understand what it does
git log --oneline -1 <commit-sha>
# Get the full diff (including new/deleted files)
git show <commit-sha>
# Capture the original commit message for later reuse
git log --format='%B' -1 <commit-sha>
```
### 2. Identify affected modules
From the file paths in the diff, determine which Penpot modules are affected
(frontend, backend, common, render-wasm, etc.) and read their `AGENTS.md`
files **before** making any changes. If a module has no `AGENTS.md`, skip
that step — verify with `ls <module>/AGENTS.md` first.
### 3. Read the current state of each affected file
For every file the diff touches, read the current version on disk to understand
context and ensure correct placement before editing.
### 4. Apply changes manually (the core of this approach)
Process every hunk in the diff using the appropriate tool:
| Diff action | Tool to use |
|-------------|-------------|
| Modify existing file | `edit` — use enough surrounding context in `oldString` to uniquely match the location |
| Add new file | `write` — include proper license header and namespace conventions matching project style |
| Delete file | `bash rm <path>` |
| Rename/move file | `bash mv <old> <new>`, then apply any content changes with `edit` |
> **Tip:** Group nearby hunks from the same file into a single `edit` call.
> Use separate calls when hunks are far apart to keep `oldString` short and
> unambiguous.
Repeat until **all** hunks in the diff are ported.
### 5. Validate
Run **lint**, **check-fmt**, and **tests** for every affected module (see each
module's `AGENTS.md` for the exact commands). If the formatter auto-fixes
indentation, verify the logic is still semantically correct. All checks must
pass before moving on.
### 6. Port the changelog entry (if any)
If the original commit added or modified a `CHANGES.md` entry, port that entry
too — adapting wording and version references for the target branch.
### 7. Commit
Ask the `commiter` sub-agent to create a commit. Stage all relevant files
(exclude unrelated untracked files) and provide the original commit message as
a reference, adapting it as needed for the target branch context.
## Key Principles
- **Context matters** — always read files before editing; never guess
indentation or surrounding code
- **Lint + format + test** — never skip validation before committing
- **Preserve intent** — keep the original commit message meaning; the
`commiter` agent handles formatting
-247
View File
@@ -1,247 +0,0 @@
---
name: code-review
description: Conducts multi-axis code review. Use before merging any change. Use when reviewing code written by yourself, another agent, or a human. Use when you need to assess code quality across multiple dimensions before it enters the main branch.
---
# Code Review and Quality
## Overview
Multi-dimensional code review with quality gates. Every change gets reviewed before merge — no exceptions. Review covers five axes: correctness, readability, architecture, security, and performance.
**The approval standard:** Approve a change when it definitely improves overall code health, even if it isn't perfect. Perfect code doesn't exist — the goal is continuous improvement. Don't block a change because it isn't exactly how you would have written it. If it improves the codebase and follows the project's conventions, approve it.
## When to Use
- Before merging any PR or change
- After completing a feature implementation
- When another agent or model produced code you need to evaluate
- When refactoring existing code
- After any bug fix (review both the fix and the regression test)
## Core Principles
These principles underpin every axis. When in doubt, default to them.
- **DRY (Don't Repeat Yourself):** Every piece of knowledge has one authoritative representation. If the same logic appears in two places, extract it into a shared helper, model, or type. Reviewers: flag duplicated logic as a required change — it's not "just similar," it's drift that will diverge.
- **KISS (Keep It Simple, Stupid):** The simplest solution that works is the best solution. Complexity must earn its place. Reviewers: if you need more than one sentence to explain what a piece of code does, it's too complex — push for simplification before merge.
- **YAGNI (You Aren't Gonna Need It):** Don't add abstractions, hooks, or generalizations for hypothetical future use cases. Generalize on the third occurrence, not the first. Reviewers: delete speculative generality.
- **Don't invent problems:** Do not manufacture issues to produce more feedback. Every finding must be a real risk, a real readability barrier, or a real architectural concern — not a hypothetical or a stylistic preference disguised as a problem.
## The Five-Axis Review
Every review evaluates code across these dimensions.
### 1. Correctness
Does the code do what it claims to do?
- Does it match the spec or task requirements?
- Are edge cases handled (null, empty, boundary values)?
- Are error paths handled (not just the happy path)?
- Does it pass all tests? Are the tests actually testing the right things?
- Are there off-by-one errors, race conditions, or state inconsistencies?
### 2. Readability & Simplicity
Can another engineer (or agent) understand this code without the author explaining it?
- Are names descriptive and consistent with project conventions? (No `temp`, `data`, `result` without context)
- Is the control flow straightforward (avoid nested ternaries, deep callbacks)?
- Are there any "clever" tricks that should be simplified?
- **KISS check:** Is this the simplest approach that solves the problem? A 20-line straightforward function beats a 5-line clever one that requires a comment to explain.
- Could this be done in fewer lines? (1000 lines where 100 suffice is a failure)
- Are abstractions earning their complexity? (Don't generalize until the third use case)
- Is a new conditional bolted onto an unrelated flow? Push the logic into its own helper, state, or policy.
- Do repeated conditionals on the same shape appear? They signal a missing model or dispatcher.
- Are there dead code artifacts: no-op variables, backwards-compat shims, or `// removed` comments?
### 3. Architecture
Does the change fit the system's design?
- Does it follow existing patterns or introduce a new one? If new, is it justified?
- Does it maintain clean module boundaries?
- **DRY check:** Is there existing code that does the same thing? Reuse the canonical helper instead of writing a near-duplicate. If two branches do nearly the same thing, collapse them.
- Are dependencies flowing in the right direction (no circular dependencies)?
- Is the abstraction level appropriate (not over-engineered, not too coupled)?
- Does this refactor reduce complexity or just relocate it? Count the concepts a reader must hold. Prefer the restructuring that makes whole branches disappear over one that re-centralizes the same logic. Prefer deleting an abstraction to polishing it.
- Is feature-specific logic leaking into a shared or general-purpose module?
- Are type boundaries explicit? Question gratuitous `any`/`unknown`/optional/casts and silent fallbacks.
- **Structural remedies:** When you flag a problem, propose the move — not just the problem. Replace conditionals with dispatchers, collapse duplicate branches, separate orchestration from business logic, extract helpers, split large files. Prefer the remedy that removes moving pieces over one that spreads the same complexity around.
### 4. Security
For detailed security guidance, see `security-and-hardening`.
- Is user input validated and sanitized?
- Are secrets kept out of code, logs, and version control?
- Is authentication/authorization checked where needed?
- Are SQL queries parameterized (no string concatenation)?
- Are outputs encoded to prevent XSS?
- Are dependencies from trusted sources with no known vulnerabilities?
- Is data from external sources (APIs, logs, user content, config files) treated as untrusted?
### 5. Performance
- Any N+1 query patterns?
- Any unbounded loops or unconstrained data fetching?
- Any synchronous operations that should be async?
- Any unnecessary re-renders in UI components?
- Any missing pagination on list endpoints?
- Any large objects created in hot paths?
## Review Process
1. **Understand the intent** — What is this change trying to accomplish? What spec or task does it implement?
2. **Review tests first** — Tests reveal intent and coverage. Do they test behavior, not implementation details? Are edge cases covered?
3. **Review the implementation** — Walk through each file with the five axes in mind.
4. **Categorize findings** — Label every comment with its severity:
| Prefix | Meaning | Author Action |
|--------|---------|---------------|
| **Critical:** | Blocks merge | Security vulnerability, data loss, broken functionality |
| **High:** | Required change | Must address before merge |
| **Medium:** | Should fix | Strongly recommended, not a blocker |
| **Low:** | Minor, optional | Author may ignore — formatting, style preferences |
| **Suggestion:** | Worth considering | Not required, but improves the code |
**Unique finding IDs.** Assign every finding a stable identifier: `F1`, `F2`, `F3`, … numbered in order of severity (Critical first, then High, Medium, Low, Suggestion). Use the ID everywhere the finding is mentioned — in section headers, in the verdict, in follow-up discussion. Never renumber within a review. Example: `**F3 (High)** — `app/validate.cljs:42` — duplicate branch logic…`.
For each finding, describe the circumstances under which it could fail: specific inputs, load conditions, timing, or user actions that trigger the problem. "This crashes when input is null" is actionable; "this might crash" is not.
Lead with what matters: correctness and security first, then structural issues, then everything else. A few high-conviction comments beat a long list.
5. **Verify the verification** — What tests were run? Did the build pass? Was the change tested manually? Screenshots for UI changes?
## Review Output
Structure every review using this format:
### Summary
Briefly explain what the code does and give an overall assessment.
### Critical and High-Priority Issues
List problems that could cause security incidents, data loss, crashes, incorrect behavior, or major performance degradation. Each finding gets its unique ID (`F1`, `F2`, …). For each: state the severity, identify the file/function/code section, explain why it's a problem, describe failure circumstances, and provide a concrete improvement with corrected code when useful.
### Other Findings
List medium- and low-priority issues, including maintainability and design concerns. Continue the ID sequence started above (`F3`, `F4`, …).
### Suggested Refactoring
Provide focused code changes or revised snippets. Preserve existing behavior unless a behavior change is explicitly justified.
### Testing Recommendations
Identify missing tests and describe specific test cases, including edge cases and failure scenarios.
### Positive Observations
Mention implementation choices that are clear, safe, efficient, or well designed. This is not fluff — it reinforces good patterns and tells the author what to keep doing.
### Final Verdict
Choose one:
- **Approve** — Ready to merge
- **Approve with minor changes** — Good to merge after addressing low/medium issues
- **Request changes** — Critical or high issues must be resolved before merge
List the finding IDs the verdict depends on (e.g. "Request changes: F1, F4").
## Change Sizing
Small, focused changes are easier to review, faster to merge, and safer to deploy.
```
~100 lines changed → Good. Reviewable in one sitting.
~300 lines changed → Acceptable if it's a single logical change.
~1000 lines changed → Too large. Split it.
```
**Watch file size, not just diff size.** Around 1000 *total* lines in a single file is a common inspection signal. When a change materially grows an already-large file, decompose first.
**Splitting strategies:**
| Strategy | How | When |
|----------|-----|------|
| **Stack** | Submit a small change, start the next one based on it | Sequential dependencies |
| **By file group** | Separate changes for groups needing different reviewers | Cross-cutting concerns |
| **Horizontal** | Create shared code/stubs first, then consumers | Layered architecture |
| **Vertical** | Break into smaller full-stack slices of the feature | Feature work |
**Separate refactoring from feature work.** A change that refactors and adds new behavior is two changes — submit them separately.
## Change Descriptions
- **First line:** Short, imperative, standalone. "Delete the FizzBuzz RPC" not "Deleting the FizzBuzz RPC."
- **Body:** What is changing and why. Include context and reasoning not visible in the code itself.
- **Anti-patterns:** "Fix bug," "Fix build," "Add patch," "Phase 1."
## Dependencies
Before adding any dependency:
1. Does the existing stack solve this? (Often it does.)
2. How large is the dependency? (Check bundle impact.)
3. Is it actively maintained? (Check last commit, open issues.)
4. Does it have known vulnerabilities? (`npm audit`)
5. What's the license? (Must be compatible with the project.)
**Rule:** Prefer standard library and existing utilities over new dependencies. Every dependency is a liability.
**Upgrading dependencies:**
- Read the changelog, not just the version number. Semver is a promise the maintainer may not have kept.
- One dependency per change. When a bulk bump breaks the build, you've lost which package did it.
- Let the tests decide — a green suite before *and* after, not just "it installed."
- Review the lockfile diff, not just `package.json`. Commit it and never hand-edit it.
For supply-chain risk triage, follow the `security-and-hardening` skill.
## Common Rationalizations
| Rationalization | Reality |
|---|---|
| "It works, that's good enough" | Working code that's unreadable, insecure, or architecturally wrong creates debt that compounds. |
| "I wrote it, so I know it's correct" | Authors are blind to their own assumptions. Every change benefits from another set of eyes. |
| "We'll clean it up later" | Later never comes. The review is the quality gate — use it. |
| "AI-generated code is probably fine" | AI code needs more scrutiny, not less. It's confident and plausible, even when wrong. |
| "The tests pass, so it's good" | Tests are necessary but not sufficient. They don't catch architecture, security, or readability problems. |
| "The refactor makes it cleaner" | Relocating complexity isn't reducing it. If the reader still holds the same number of concepts, the structure didn't improve. |
| "It's only a small addition to this file" | Small diffs still push files past healthy size and bolt branches onto unrelated flows. |
| "It's just a version bump" | A bump is a behavior change you didn't write. Read the changelog. |
| "I'll upgrade everything in one PR" | A bulk bump hides which package broke the build. One per change. |
| "It's duplicated but it's only two places" | Two becomes three becomes five. Extract now, before the copies diverge. |
| "The abstraction is future-proof" | YAGNI. Delete speculative generality — generalize on the third occurrence, not the first. |
| "It's clever but efficient" | Cleverness is a readability tax. If it needs a comment to understand, simplify it. |
## Red Flags
- PRs merged without any review
- Review that only checks if tests pass (ignoring other axes)
- "LGTM" without evidence of actual review
- Security-sensitive changes without security-focused review
- Large PRs that are "too big to review properly" (split them)
- No regression tests with bug fix PRs
- Accepting "I'll fix it later" — it never happens
- A refactor that moves code around without reducing the number of concepts a reader must hold
- New conditionals scattered into unrelated code paths (a missing abstraction)
- A bespoke helper that duplicates an existing canonical one
- A bulk "bump dependencies" PR with no changelog review
## Verification
Before emitting the verdict, verify the change as it stands. This is the reviewer's own due diligence — it covers the state of the code at review time, not the later resolution of findings (fixing findings is the author's job; confirming them is a new review):
- [ ] Tests pass — run them yourself, don't trust the claim
- [ ] Build succeeds
- [ ] The verification story is documented (what changed, how it was verified)
- [ ] Dependency upgrades reviewed against changelog, isolated per package, verified by green suite
## See Also
- For detailed security review guidance, see `security-and-hardening`
-47
View File
@@ -1,47 +0,0 @@
---
name: create-commit
description: Stage, review, and commit files following Penpot commit conventions.
---
# Skill: create-commit
Produce a git commit that follows Penpot's commit message conventions. This
skill owns the commit format, staging review, and safety checks — it does not
implement features or push.
## When to Use
- After code changes are complete and files need to be committed
- When delegated by a workflow step (e.g. implement-plan) to handle the commit
## Required Reading
Before drafting any commit, read `mem:workflow/creating-commits` end-to-end. It
is the authoritative source for the commit message format, the emoji menu,
subject/body limits, and the `AI-assisted-by` trailer. Follow it exactly.
## Workflow
1. **Stage the files** specified by the calling context. Do not ask for
confirmation.
2. Run `git diff --staged` to review the content. If you see secrets (API keys,
tokens, passwords, private keys, `.env` values), debug prints, or anything
that does not match the stated intent, **STOP** and tell the user before
committing.
3. Draft the message following the format in the memory doc, wrapping the body
at 72 characters per line, and run:
```bash
git commit -m "<subject>" -m "<body>"
```
(or `git commit -F -` if the body has unusual characters).
4. The `AI-assisted-by` trailer value is provided by the calling context — use
it verbatim.
## Constraints
- Do not push. Pushing is a separate workflow handled by the user.
- Do not run `git reset`, `git checkout`, `git restore`, `git clean`, or `rm`.
- Do not pass `--author`. Author identity comes from the local git config.
- Do not amend a commit you did not create in this session, unless explicitly asked.
- Do not bypass pre-commit hooks (`--no-verify`) unless explicitly asked.
- Do not add untracked files that were not created in this session.
-27
View File
@@ -1,27 +0,0 @@
---
name: create-issue
description: Create or update GitHub issues (from PR, from draft body, retitle existing). Routes to the canonical flow in `mem:workflow/creating-issues`.
---
# Skill: create-issue
Entry point for all GitHub issue work. All rules (title derivation, metadata,
body templates, Issue Type IDs), all flows, and all `gh` / GraphQL commands
live in `mem:workflow/creating-issues` (file:
`.serena/memories/workflow/creating-issues.md`). This skill routes to the
right flow.
## When to Use
- **Create from PR** — PR exists; the issue is the changelog/release unit,
the PR is the implementation. Issue = WHAT, PR = HOW.
→ memory section **Creating Issues from PRs**
- **Create from draft body** — Taiga story, user report, discussion; no PR
yet.
→ memory section **Creating Issues from Draft Body**
- **Retitle existing issue** — current title is vague, prefixed, or stale.
→ memory section **Retitling an Existing Issue**
Everything else (title derivation, metadata policy, body templates, Issue
Type IDs, create/verify/cleanup commands) lives in the memory — go to the
matching section there.
-39
View File
@@ -1,39 +0,0 @@
---
name: create-pr
description: Create or update a GitHub PR following Penpot conventions.
---
# Skill: create-pr
Create or update a GitHub PR. Read and follow:
- `mem:workflow/creating-prs` — title format, description structure, writing principles
- `mem:workflow/creating-commits` — commit type emojis
## When to Use
- Creating a new PR from a feature branch
- Updating an existing PR's title or description to match conventions
## Prerequisites
- `gh` CLI authenticated (`gh auth status`)
## Commands
**Create:**
```bash
gh pr create --repo penpot/penpot --title "<TITLE>" --body-file /tmp/pr-body.md
```
**Update:**
```bash
gh pr edit <NUMBER> --repo penpot/penpot --title "<TITLE>" --body-file /tmp/pr-body.md
```
**Verify:**
```bash
gh pr view <NUMBER> --repo penpot/penpot --json title,body
```
+95 -12
View File
@@ -1,25 +1,31 @@
---
name: nrepl-eval
description: Evaluate Clojure code via nREPL using the standalone scripts/nrepl-eval.mjs CLI tool.
description: Evaluate Clojure code via nREPL using the standalone tools/nrepl-eval.mjs CLI tool.
---
# nREPL Eval
Evaluate Clojure (or ClojureScript) code via a running nREPL server using
`scripts/nrepl-eval.mjs`.
`tools/nrepl-eval.mjs` — a standalone CLI application.
Full documentation: `mem:scripts/nrepl-eval` (file: `.serena/memories/scripts/nrepl-eval.md`)
Session state (defs, in-ns, etc.) persists across invocations via a stored
session ID, so you can build up state incrementally.
## Quick Reference
## Usage
```bash
./scripts/nrepl-eval.mjs [options] [<code>]
node tools/nrepl-eval.mjs [options] [<code>]
```
The tool is also executable directly:
```bash
./tools/nrepl-eval.mjs [options] [<code>]
```
## Options
| Flag | Description | Default |
|------|-------------|---------|
| `--backend` | Connect to backend nREPL (port 6064) | — |
| `--frontend` | Connect to frontend nREPL (port 3447) | — |
| `-p, --port PORT` | nREPL server port | `6064` |
| `-H, --host HOST` | nREPL server host | `127.0.0.1` |
| `-t, --timeout MS` | Timeout in milliseconds | `120000` |
@@ -27,11 +33,88 @@ Full documentation: `mem:scripts/nrepl-eval` (file: `.serena/memories/scripts/nr
| `-e, --last-error` | Evaluate `*e` to retrieve the last exception | — |
| `-h, --help` | Show help message | — |
## Examples
## When to Use
Use this tool when you need to:
1. **Evaluate Clojure code** during development — test functions, inspect
state, or run experiments against a running Clojure process.
2. **Verify that edited files compile** — require namespaces with `:reload`
to pick up changes.
3. **Inspect the last exception** after a failed evaluation — use `-e` to
print the error stored in `*e`.
## Workflow
### 1. Session management
Sessions are persisted to `/tmp/penpot-nrepl-session-<host>-<port>`. State
carries across calls automatically:
```bash
./scripts/nrepl-eval.mjs '(+ 1 2 3)'
./scripts/nrepl-eval.mjs --backend '(+ 1 2 3)'
./scripts/nrepl-eval.mjs --frontend '(js/alert "hi")'
./scripts/nrepl-eval.mjs -e
./tools/nrepl-eval.mjs '(def x 42)'
./tools/nrepl-eval.mjs 'x'
# => 42
```
Reset the session to start fresh:
```bash
./tools/nrepl-eval.mjs --reset-session '(def x 0)'
```
### 2. Evaluate code
**Single expression (inline) — uses default port 6064:**
```bash
./tools/nrepl-eval.mjs '(+ 1 2 3)'
```
**Multiple expressions via heredoc (recommended — avoids escaping issues):**
```bash
./tools/nrepl-eval.mjs <<'EOF'
(def x 10)
(+ x 20)
EOF
```
**Override with a different port:**
```bash
./tools/nrepl-eval.mjs -p 7888 '(+ 1 2 3)'
```
### 3. Inspect last exception
After code throws an error, retrieve the full exception details:
```bash
./tools/nrepl-eval.mjs -e
```
## Common Patterns
**Require a namespace with reload:**
```bash
./tools/nrepl-eval.mjs "(require '[my.namespace :as ns] :reload)"
```
**Test a function:**
```bash
./tools/nrepl-eval.mjs "(ns/my-function arg1 arg2)"
```
**Long-running operation with custom timeout:**
```bash
./tools/nrepl-eval.mjs -t 300000 "(long-running-fn)"
```
## Key Principles
- **Default port is 6064** — just pass code directly, no `-p` needed when
your nREPL server is on 6064. Use `-p <PORT>` for a different port.
- **Always use `:reload`** when requiring namespaces to pick up file changes.
- **Session is reused** across invocations — defs, in-ns, and var bindings
persist. Use `--reset-session` to clear.
- **Do not start any server** — the tool connects to an existing nREPL
server, it is not the agent's responsibility to start the nREPL server
(assume the server is already running on the specified port).
-315
View File
@@ -1,315 +0,0 @@
---
name: plan-review
description: Reviews implementation plans for quality, completeness, and actionability. Use after a plan is produced by the planner skill, before starting implementation. Use when evaluating a plan written by yourself, another agent, or a human.
---
# Plan Review
## Overview
Multi-dimensional plan review with quality gates. Every plan gets reviewed before implementation starts — no exceptions. Review covers six axes: completeness, task quality, architecture & sequencing, risk coverage, actionability, and proposed code quality.
**The approval standard:** Approve a plan when it is specific enough that a skilled implementer could execute it without guessing, the task ordering is sound, and risks are acknowledged. Perfect plans don't exist — the goal is confidence that implementation won't derail. Don't block a plan because it isn't exactly how you would have structured it. If it's executable and well-organized, approve it.
## When to Use
- After the planner skill produces a plan
- Before starting implementation on any non-trivial task
- When reviewing a plan written by another agent or a human
- When a plan feels too large, vague, or risky to start
**Do NOT use for:** Single-file changes with obvious scope, or when the task is trivial enough to just do.
## The Six-Axis Review
Every plan gets evaluated across these dimensions:
### 1. Completeness
Does the plan cover everything needed to implement successfully?
- Is the **context** clear? (What problem, why now, what's the goal?)
- Are **affected modules** identified with paths?
- Are **architecture decisions** documented with rationale?
- Is there a **testing strategy**?
- Are **verification commands** explicit (not "run the tests")?
- Are **open questions** listed (not buried in someone's head)?
- Is there a **parallelization** assessment for multi-task plans?
**Missing any of these is a gap, not a nit.**
### 2. Task Quality
Are the tasks well-defined and independently executable?
- Does every task have **acceptance criteria**? (Testable, not vague)
- Does every task have **verification steps**?
- Are tasks **sized appropriately**? (XSM is ideal, L is acceptable, XL must be split)
- Are **dependencies** between tasks explicitly stated?
- Are **files likely touched** listed?
- Is each task a **single, self-contained change**? (Not "implement the whole feature")
- Could a skilled implementer pick up any task and execute it without asking clarifying questions?
### 3. Architecture & Sequencing
Is the plan structured so implementation flows correctly?
- Does implementation order follow the **dependency graph** (foundations first)?
- Are tasks **vertically sliced** (feature paths) rather than horizontally layered?
- Does each task leave the system in a **working state**?
- Are there **checkpoints** between major phases?
- Are **high-risk tasks early** (fail fast)?
- Is the total plan a reasonable number of tasks? (More than ~15 tasks suggests the scope should be split into multiple plans)
### 4. Risk Coverage
Are the hard parts acknowledged and mitigated?
- Are **edge cases** identified?
- Are **breaking changes** or **migration concerns** noted?
- Are **security implications** considered?
- Are **performance implications** considered?
- Are **external dependencies** or integration risks flagged?
- Is there a plan for **rollback** if something goes wrong?
- Are **data integrity** risks addressed (what happens if a migration fails mid-way)?
### 5. Actionability
Can an implementer actually execute this?
- Are **file paths** specific (not "update the relevant files")?
- Are **function/method names** mentioned where applicable?
- Are **verification commands** copy-pasteable (not "run the linter")?
- Are **test commands** project-specific (not generic)?
- Is the **code shape** described where the implementation isn't obvious?
- Are **conventions** referenced (naming, patterns, existing utilities to reuse)?
- Does the plan reference **existing code** the implementer should read first?
### 6. Proposed Code Quality *(when the plan includes implementation details)*
If the plan proposes code shapes, function signatures, data structures, or API designs, evaluate those proposals against `code-review` criteria:
- **Correctness:** Do the proposed types/signatures handle edge cases (null, empty, boundaries)?
- **Readability:** Are proposed names descriptive and consistent with project conventions?
- **Architecture:** Do proposed abstractions follow existing patterns? Are they justified (not over-engineered)?
- **Security:** Do proposed APIs validate input at boundaries? Any injection/XSS vectors in the design?
- **Performance:** Do proposed data structures avoid N+1 patterns? Any unbounded operations in the design?
**When to apply:** Only when the plan includes specific code snippets, type definitions, API contracts, or function signatures. Plans that only describe "what" without showing "how" skip this axis.
## Structural Remedies
When you flag a structural problem in a plan, propose the fix — not just the problem:
- **A task is too large (XL):** Split it into vertical slices. Each slice should be independently testable.
- **Missing acceptance criteria:** Draft 23 specific, testable conditions for the task.
- **Wrong sequencing:** Identify the dependency and propose the correct order.
- **No checkpoints:** Suggest where checkpoints should go (typically after every 23 tasks).
- **Vague verification:** Replace "run tests" with the actual project command.
- **Horizontal slicing:** Restructure into vertical feature paths.
- **Missing risk section:** Draft the risks you can identify from the plan content.
Prefer the remedy that makes the plan immediately actionable over one that just flags the gap.
## Plan Sizing
Plans should be scoped to a single deliverable:
```
15 tasks → Good. A focused feature or bug fix.
610 tasks → Acceptable for a moderate feature.
1115 tasks → Large. Consider splitting into phases.
15+ tasks → Too large. Split into multiple plans.
```
**What counts as "one plan":** A self-contained set of changes that delivers a single coherent capability. If you can describe the goal in one sentence, it's one plan.
## Categorize Findings
Label every comment with its severity so the author knows what's required vs optional:
| Prefix | Meaning | Author Action |
|--------|---------|---------------|
| *(no prefix)* | Required change | Must address before implementation starts |
| **Critical:** | Blocks implementation | Missing security consideration, data integrity risk, fundamentally wrong approach |
| **Nit:** | Minor, optional | Author may ignore — wording, formatting |
| **Optional:** / **Consider:** | Suggestion | Worth considering but not required |
| **FYI** | Informational only | No action needed — context for future reference |
**Lead with what matters.** Order findings by leverage: missing risks and wrong sequencing first, then task quality gaps, then completeness, then nits. If you have one critical sequencing problem and ten nits, the sequencing problem *is* the review.
## Review Process
### Step 1: Understand the Goal
Before evaluating structure, understand intent:
```
- What is this plan trying to accomplish?
- What problem does it solve?
- What does "done" look like?
```
### Step 2: Check Completeness First
Scan for missing sections before diving into content:
```
- Context present?
- Affected modules listed?
- Architecture decisions documented?
- Risks acknowledged?
- Testing strategy defined?
- Verification commands explicit?
```
### Step 3: Review Task Quality
Walk through each task:
```
For each task:
1. Can I tell exactly what to build?
2. Are acceptance criteria specific and testable?
3. Is the size reasonable (not XL)?
4. Are dependencies clear?
5. Would I know which files to touch?
```
### Step 4: Validate Sequencing
Check the dependency graph:
```
- Are foundations built first?
- Does each task leave the system working?
- Are checkpoints placed correctly?
- Are high-risk items early?
- Is it vertically sliced?
```
### Step 5: Assess Actionability
Put yourself in the implementer's shoes:
```
- Could I pick up task 1 and start coding without asking any questions?
- Are the verification commands copy-pasteable?
- Are file paths and function names specific?
- Is existing code referenced where I'd need to read it?
```
### Step 6: Verify the Verification Story
Check that the plan can actually confirm it worked:
```
- What tests should pass after implementation?
- What build/compile commands are relevant?
- What manual checks are needed?
- How do we know the feature works end-to-end?
```
### Step 7: Evaluate Proposed Code Quality *(if applicable)*
If the plan includes code snippets, types, or API designs:
```
- Load code-review skill for criteria
- Check proposed signatures for edge cases
- Verify naming follows project conventions
- Confirm abstractions follow existing patterns
- Scan for security vectors in proposed APIs
- Check for performance issues in proposed data structures
```
## Review Checklist
```markdown
## Review: [Plan title]
### Completeness
- [ ] Context explains the problem and goal
- [ ] Affected modules are listed with paths
- [ ] Architecture decisions have rationale
- [ ] Testing strategy is defined
- [ ] Verification commands are explicit and project-specific
- [ ] Open questions are listed
### Task Quality
- [ ] Every task has acceptance criteria
- [ ] Every task has verification steps
- [ ] Tasks are sized XSM (L acceptable, XL must be split)
- [ ] Task dependencies are stated
- [ ] Files likely touched are listed
### Architecture & Sequencing
- [ ] Order follows dependency graph (foundations first)
- [ ] Vertically sliced (not horizontal layers)
- [ ] Each task leaves system working
- [ ] Checkpoints exist between phases
- [ ] High-risk tasks are early
### Risk Coverage
- [ ] Edge cases identified
- [ ] Breaking changes / migrations noted
- [ ] Security implications considered
- [ ] Performance implications considered
- [ ] Rollback strategy exists (if applicable)
### Actionability
- [ ] File paths are specific
- [ ] Verification commands are copy-pasteable
- [ ] Existing code to read is referenced
- [ ] Conventions and patterns are noted
### Proposed Code Quality *(if plan includes implementation details)*
- [ ] Proposed types/signatures handle edge cases
- [ ] Proposed names follow project conventions
- [ ] Proposed abstractions follow existing patterns
- [ ] No security vectors in proposed APIs
- [ ] No performance issues in proposed structures
### Verdict
- [ ] **Approve** — Ready to implement
- [ ] **Request changes** — Gaps must be addressed
```
## Common Rationalizations
| Rationalization | Reality |
|---|---|
| "I'll figure out the details during implementation" | That's how you discover blocking dependencies mid-task. Surface them now. |
| "The tasks are obvious, no need for criteria" | Write them anyway. Explicit criteria surface hidden assumptions. |
| "It's just a small feature, it doesn't need a plan" | Small features have edge cases too. 3 tasks with criteria takes 5 minutes. |
| "The plan is good enough" | "Good enough" without acceptance criteria means the implementer defines "done" — and they might define it differently. |
| "I'll add verification steps later" | Later never comes. The plan is the contract — define verification now. |
| "Risks are minimal" | Every change has risks. If you can't name them, you haven't thought about them. |
| "The file paths are obvious" | They're obvious to the author. The implementer might not know the codebase. |
| "The code in the plan is fine, it'll get reviewed later" | Plan-level code review catches design problems before implementation — fixing them after coding is more expensive. |
## Red Flags
- No acceptance criteria on any task
- Tasks that say "implement the feature" without specifics
- No verification steps anywhere in the plan
- All tasks are XL-sized
- No checkpoints between phases
- Dependency order isn't considered (e.g., API handler before domain model)
- No testing strategy
- Verification commands are generic ("run tests") instead of project-specific
- Plan has 20+ tasks (scope too large for one plan)
- No risk section on a plan with migrations, breaking changes, or security implications
- Horizontal slicing (all domain, then all services, then all API)
- File paths are vague ("update the relevant files")
- Missing open questions section despite stated unknowns
- Proposed code ignores project conventions or existing patterns
- Proposed types use gratuitous `any`/`unknown`/optional without justification
- Proposed APIs don't validate input at boundaries
## See Also
- For producing plans, use the `planner` skill
- For reviewing implemented code, use `code-review` — also the criteria source for axis 6
- For security-specific concerns, see `security-and-hardening`
- For testing strategy guidance, see `testing`
-377
View File
@@ -1,377 +0,0 @@
---
name: planner
description: Read-only planning and architecture analysis for Penpot — produce a structured implementation plan with task breakdown, acceptance criteria, sizing, and checkpoints. Always output to the user and save to .opencode/plans/YYYY-MM-DD-<title>.md.
---
# Planner
Read-only senior software architect role for Penpot. Produces structured
implementation plans with task breakdowns that engineers or other agents can
execute. Never writes or modifies code.
## When to Use
- The user asks for a plan, design, or analysis of a feature or bug.
- The user wants to understand which parts of the codebase a task will touch.
- The user needs a step-by-step implementation plan with file paths, function
names, and test strategy.
- The user asks "how would I implement X?" or "what's involved in fixing Y?".
- The user is about to start non-trivial work and wants a bite-sized task
breakdown.
- A task feels too large or vague to start.
- Work needs to be parallelized across multiple agents or sessions.
Do **not** use this skill to actually implement anything — it is read-only.
**When NOT to use:** Single-file changes with obvious scope, or when the spec
already contains well-defined tasks.
## Role
You help users understand the Penpot codebase, design solutions, and produce
implementation plans that other agents or developers can execute. The plan
tells them what to build and how to verify it, task by task.
The implementer reads the project's agent docs (`AGENTS.md`, project memories
such as `mem:critical-info`, `mem:testing`, and each module's core memory)
before working. Reference those memories instead of re-explaining tooling,
conventions, or test design — explain in the plan only what they do not cover.
Do **not** suggest commit messages or commit names anywhere in your plans or
responses — committing is the implementer's responsibility.
## CRITICAL: Required Reading Before Planning
Before drafting any plan, work through the project's own guidance:
1. Read `critical-info` (`.serena/memories/critical-info.md`) — the entry point
that describes the monorepo structure and module dependency graph.
2. From `critical-info`, identify which modules your task affects.
3. Read each affected module's core memory, e.g. `mem:frontend/core`,
`mem:backend/core`, `mem:common/core`, `mem:exporter/core`,
`mem:render-wasm/core`. Follow `mem:` references deeper as needed.
4. For each affected module, note its lint, format, and test commands so the
plan can include concrete verification steps.
Skipping this step is the #1 cause of incorrect or incomplete plans.
---
## The Planning Process
### Phase 1: Architecture Analysis
1. Read the spec, requirements, or feature request.
2. Analyze the codebase architecture and identify affected modules.
3. Read project conventions (starting with `critical-info` and module core
memories) before drafting.
4. Map dependencies between components (see the dependency graph in
`critical-info`).
5. Identify risks, edge cases, performance implications, and breaking changes.
### Phase 2: Task Breakdown
#### Identify the Dependency Graph
Map what depends on what, following the monorepo's module dependency graph:
```
common (shared types, schemas — no deps)
├── backend (depends common)
│ ├── RPC handlers
│ └── persistence / migrations
├── frontend (depends common, render-wasm)
│ ├── UI components
│ └── state / API integration
├── exporter (depends common)
└── render-wasm (consumed by frontend)
```
Implementation order follows the dependency graph bottom-up: build shared
foundations first, then layer consumers on top.
#### Slice Vertically
Instead of building all of common, then all of backend, then all of frontend —
build one complete feature path at a time:
**Bad (horizontal slicing):**
```
Task 1: Build all common types
Task 2: Build all backend handlers
Task 3: Build all frontend components
```
**Good (vertical slicing):**
```
Task 1: common data types + schema ← foundation
Task 2: backend RPC handler + persistence
Task 3: frontend UI component + API integration
```
Each vertical slice delivers working, testable functionality.
#### Write Tasks
Each task follows this structure:
```markdown
## Task [N]: [Short descriptive title]
**Description:** One or two paragraphs explaining what this task accomplishes.
Should be clear and concise.
**Rationale:** Why this task exists and why this approach over the obvious
alternatives — design decisions, trade-offs, constraints discovered during
analysis. One or two sentences; skip only if genuinely trivial.
**Code sketch (optional):** Signature-, type-, or shape-level example when the
intended interface is non-obvious. Keep it short — a skeleton that fixes the
contract (function signature, model fields, error shape), never a full
implementation. Omit when the task is mechanical.
**Acceptance criteria:**
- [ ] [Specific, testable condition]
- [ ] [Specific, testable condition]
**Verification:**
- [ ] Relevant tests pass (module-specific test command).
- [ ] Lint/formatter passes (module-specific check command), if applicable.
- [ ] The core flow works end-to-end, if applicable.
**Dependencies:** [Task numbers this depends on, or "None"]
**Files likely touched:**
- `path/to/file.clj`
- `path/to/file_test.clj`
**Estimated scope:** [XS: 1 file | S: 1-2 files | M: 3-5 files | L: 5+ files]
```
Replace "module-specific test command" with the actual commands for the module
(e.g. `clojure -M:dev:test` for backend/common,
`npx shadow-cljs compile test && npx karma start` for frontend, or the
commands noted in the module's core memory).
When possible, design each task with TDD in mind: acceptance criteria double
as a test list, and the natural first step of the task is writing those tests
before the implementation. Some tasks resist this (config, migrations, pure
wiring) — for those, keep the usual verification steps.
#### Estimate Scope
| Size | Files | Scope | Example |
|------|-------|-------|---------|
| **XS** | 1 | Single function, config change, or schema tweak | Add a validation rule |
| **S** | 1-2 | One handler or component method | Add a new RPC endpoint |
| **M** | 3-5 | One vertical feature slice | Bookmark CRUD with tests |
| **L** | 5-8 | Multi-component feature | Search with filtering and pagination |
| **XL** | 8+ | **Too large — break it down further** | — |
If a task is XL, it should be broken into smaller tasks. Agents perform best
on S and M tasks.
**When to break a task down further:**
- It would take more than one focused session
- You cannot describe the acceptance criteria in 3 or fewer bullet points
- It touches two or more independent subsystems
- You find yourself writing "and" in the task title (a sign it is two tasks)
#### Order and Checkpoints
Arrange tasks so that:
1. Dependencies are satisfied (build foundation first)
2. Each task leaves the system in a working state
3. Verification checkpoints occur after every 2-3 tasks
4. High-risk tasks are early (fail fast)
Add explicit checkpoints with the relevant module commands:
```markdown
### Checkpoint: After Tasks 1-3
- [ ] Relevant tests pass (module-specific command).
- [ ] The relevant build or compilation passes, if applicable.
- [ ] The core flow works end-to-end.
- [ ] Review with human before proceeding.
```
## Requirements
- Analyze the codebase architecture and identify affected modules.
- Read project conventions before drafting (start with `critical-info` and
affected module core memories).
- Break down complex features or bugs into atomic, actionable steps.
- Propose solutions with clear rationale, trade-offs, and sequencing.
- Identify risks, edge cases, performance implications, and breaking changes.
- Apply DRY and KISS principles to the proposed implementation.
- Define a testing strategy aligned with each affected module's tooling.
- Every task must have acceptance criteria and verification steps.
- Checkpoints must exist after every 2-3 tasks.
## Constraints
- You are **analysis-only** — never create, edit, or delete source code.
- The only file write you may attempt is the plan itself, saved to
`.opencode/plans/`.
- You do **not** run builds, tests, linters, or any commands that modify state.
- You do **not** create git commits or interact with version control.
- You do **not** execute shell commands beyond read-only searches (`rg`, `ls`,
`find`, `cat`, `bat`).
- Your output is a structured plan or analysis, ready for handoff to an
engineer agent or developer.
## Output Format
The plan is always delivered in the response so the user sees it regardless
of which agent is running the skill.
Additionally, save the plan to:
```
.opencode/plans/YYYY-MM-DD-<plan-one-line-title>.md
```
Use today's date in the user's local timezone. The `<plan-one-line-title>`
slug is lowercase, hyphen-separated, and a short summary of the task
(e.g. `add-batch-get-profiles-for-file-comments`). Create the
`.opencode/plans/` directory if it does not exist.
IMPORTANT: The plan agent has write permission specifically for
`.opencode/plans/` — always attempt the write. If the user explicitly provides
a target file path, use that path instead of the default.
### Plan Document Template
```markdown
# Plan: [Feature/Project Name]
## Context
[One paragraph: what is the problem or feature request? Why is it needed?]
## Affected Modules
[Which modules of the monorepo are involved? Reference module paths and any
`mem:` memories that were consulted.]
## Architecture Decisions
- [Key decision 1 and rationale]
- [Key decision 2 and rationale]
## Risks & Considerations
[Edge cases, performance implications, breaking changes, migration concerns,
security implications.]
## Approach
[A short strategy summary: 3-5 sentences describing the overall approach and
the shape of the dependency graph (what depends on what, what gets built
first). High-level only — the task-by-task detail lives in the Task List.]
## Task List
Each task uses the full task structure defined in
[Write Tasks](#write-tasks) — description, rationale, acceptance criteria,
verification, dependencies, files, estimated scope, and optional code sketch.
Never reduce a task to a one-line checkbox; the plan must be self-contained
and executable without other context.
Tasks are a flat, ordered list — a plan is not a roadmap. Do not group tasks
into phases, milestones, or sprints; ordering and dependencies are already
captured per task. Insert a checkpoint after every 2-3 tasks.
## Task 1: [Short descriptive title]
**Description:** [What this task accomplishes.]
**Rationale:** [Why this approach over the alternatives.]
**Acceptance criteria:**
- [ ] [Specific, testable condition]
**Verification:**
- [ ] Relevant tests pass (module-specific command).
**Dependencies:** None
**Files likely touched:**
- `path/to/file`
**Estimated scope:** [XS: 1 file | S: 1-2 files | M: 3-5 files | L: 5+ files]
**Code sketch (optional):** [Short contract-level example, only if the shape
is non-obvious.]
## Task 2: [Short descriptive title]
[Same structure as Task 1.]
## Task 3: [Short descriptive title]
[Same structure as Task 1.]
### Checkpoint: After Tasks 1-3
- [ ] Relevant tests pass (module-specific command).
- [ ] The relevant build or compilation passes, if applicable.
- [ ] The core flow works end-to-end.
- [ ] Review with human before proceeding.
## Task 4: [Short descriptive title]
[Same structure as Task 1.]
## Task 5: [Short descriptive title]
[Same structure as Task 1.]
## Verification & Testing
[How to verify each task and the whole plan: the project's real test, lint,
build, and run commands (extracted during Required Reading), coverage
expectations, and manual checks. Consult each module's core memory for the
exact commands.]
## Parallelization Opportunities
- **Safe to parallelize:** Independent feature slices across separate
modules, tests for already-implemented features, documentation
- **Must be sequential:** Shared common schema changes, database migrations
- **Needs coordination:** Features that share a contract (define the contract
first, then parallelize)
## Open Questions
- [Question needing human input]
```
When the plan is purely analytical (e.g. a code review or feasibility study
with no implementation), skip the **Approach** and **Task List** sections and
lead with **Findings** instead, keeping the rest of the structure.
## Common Rationalizations
| Rationalization | Reality |
|---|---|
| "I'll figure it out as I go" | That's how you end up with a tangled mess and rework. 10 minutes of planning saves hours. |
| "The tasks are obvious" | Write them down anyway. Explicit tasks surface hidden dependencies and forgotten edge cases. |
| "Planning is overhead" | Planning is the task. Implementation without a plan is just typing. |
| "I can hold it all in my head" | Context windows are finite. Written plans survive session boundaries and compaction. |
## Red Flags
- Delivering prose without a task breakdown
- Tasks that say "implement the feature" without acceptance criteria
- No verification steps in the plan
- All tasks are XL-sized
- No checkpoints between tasks
- Dependency order isn't considered
## Verification Checklist
Before delivering the plan, confirm:
- [ ] Every task has acceptance criteria
- [ ] Every task has a verification step
- [ ] Task dependencies are identified and ordered correctly
- [ ] No task is XL or larger — break it down instead
- [ ] Checkpoints exist after every 2-3 tasks
- [ ] The plan is ready for human review
-134
View File
@@ -1,134 +0,0 @@
---
name: refine-prompt
description: Refine and improve a user-supplied prompt for maximum clarity and effectiveness using prompt-engineering best practices and Penpot project context. Outputs a rewritten prompt (and brief rationale); never executes the prompt.
---
# Refine Prompt
Expert prompt-engineering pass on a user-supplied prompt. Takes a draft prompt
and returns a clearer, more effective, well-structured version — ready to be
used with any AI model. Never executes the prompt itself.
## When to Use
- The user shares a prompt and asks to improve, refine, polish, or rewrite it.
- The user asks "make this prompt better" or "can you clean this up?".
- The user wants to add structure, constraints, examples, or output format to
a vague prompt.
- The user wants a prompt adapted for a specific target model, audience, or
task type.
Do **not** use this skill to actually answer the prompt or do the task — it
only rewrites the prompt.
## Role
You are an expert Prompt Engineer with strong knowledge of Penpot. Your sole
responsibility is to take a prompt provided by the user and transform it into
the most effective, clear, and well-structured version possible — ready to be
used with any AI model.
You do **not** execute tasks. You do **not** write code. You only design and
refine prompts.
## Required Reading Before Refining
Before rewriting, internalize the project context the prompt will likely run
against:
1. Read `AGENTS.md` (root) for the project-level rules and conventions.
2. Read `.serena/memories/critical-info.md` (or the equivalent entry point) to
understand the module layout (`frontend`, `backend`, `common`,
`render-wasm`, `exporter`, `mcp`, `plugins`, `library`).
3. Skim the relevant module's core memory (`mem:frontend/core`,
`mem:backend/core`, etc.) when the prompt targets a specific module — this
lets you inject precise vocabulary, file conventions, and test commands
into the refined prompt.
This step matters most when the user is preparing a prompt *about* the
Penpot codebase. For generic prompts, focus on prompt-engineering principles
and only weave in Penpot context when it is clearly relevant.
## Requirements
- Analyze the original prompt: identify its intent, target audience,
ambiguities, missing context, and structural weaknesses.
- Ask clarifying questions if the intent is unclear or if critical information
is missing (e.g. target model, expected output format, tone, constraints).
Keep questions concise and grouped. Prefer to ask 14 questions at once
rather than one at a time. **Use the `question` tool** to ask them so the
user gets a structured multi-choice UI; reserve a plain `## Clarifying
questions` markdown section for cases where the `question` tool is
unavailable or the question is genuinely open-ended.
- Rewrite the prompt using prompt-engineering best practices (see below).
- Preserve the user's original intent — do not change the underlying task.
- When the user provides Penpot project context, weave in the relevant
conventions, module paths, and tooling.
## Prompt Engineering Principles
Apply these techniques when refining prompts:
- **Be specific and explicit**: Replace vague instructions with precise ones.
- **Set the context**: Include background information the model needs to
perform well.
- **Specify the output format**: State the desired structure, length, tone,
or format (e.g. bullet list, JSON, step-by-step).
- **Add constraints**: Include what the model should avoid or not do.
- **Use examples** (few-shot): When applicable, suggest adding examples to
anchor the model's behaviour.
- **Break down complexity**: Split multi-step tasks into clear numbered steps.
- **Avoid ambiguity**: Remove pronouns and references that could be
misinterpreted.
- **Chain of thought**: For reasoning tasks, include "Think step by step."
- **Role framing**: When helpful, give the model a clear role
("You are a senior backend engineer...").
- **Tool awareness**: When the prompt targets an agentic model, mention
relevant tools (`grep`, `glob`, `read`, `bash`, etc.) so the model uses the
right surface.
## Constraints
- Do **not** execute the prompt yourself.
- Do **not** answer the question inside the prompt.
- Do **not** add unnecessary verbosity — prompts should be as short as they
can be while remaining complete.
- Always preserve the user's original intent.
- If the user provides Penpot project context, prefer Penpot-specific
vocabulary over generic terms (e.g. name actual modules and `mem:`
references instead of "the codebase").
## Output Format
Deliver the result in the response as two clearly separated blocks:
1. **Refined prompt** — a single fenced code block (markdown ```) containing
the rewritten prompt, ready to copy and use.
2. **What changed (brief)** — a short bulleted list of the most important
changes you made and why (37 bullets max). Skip the rationale if the
changes are trivial.
If you asked clarifying questions via the `question` tool, stop and wait for
the answers before producing a refined prompt. If the `question` tool was not
available and you asked the questions in chat, list them in a separate
**Clarifying questions** section above the refined prompt and stop — do not
produce a refined prompt until the user answers. If the user explicitly told
you to proceed without questions (e.g. "just rewrite it"), make reasonable
assumptions and note them under **Assumptions made** in the rationale block.
## File Persistence
Always persist the refined prompt to disk so it can be re-used later, versioned
in git, and shared with other agents. The response still contains the prompt
and rationale blocks; the file is an additional artifact, not a replacement.
- Save the refined prompt (the body inside the fenced code block, **without**
the surrounding ``` fences) to `.opencode/prompts/<descriptive-name>.md`.
- Use a **kebab-case** filename that summarises the task, e.g.
`add-error-reports-management-rpc.md`, `backend-rpc-security-audit.md`. No
spaces, no uppercase, no version numbers or dates in the filename.
- If `.opencode/prompts/` does not exist, create it before writing.
- If a file with the same name already exists, overwrite it (the file is the
refined prompt, not a log).
- Only skip the file write when the user explicitly opts out (e.g. "don't save
this one", "just show it in the chat"). When in doubt, save it.
@@ -1,457 +0,0 @@
---
name: security-and-hardening
description: Hardens code against vulnerabilities. Use when handling user input, authentication, data storage, or external integrations. Use when building any feature that accepts untrusted data, manages user sessions, or interacts with third-party services.
---
# Security and Hardening
## Overview
Security-first development practices for web applications. Treat every external input as hostile, every secret as sacred, and every authorization check as mandatory. Security isn't a phase — it's a constraint on every line of code that touches user data, authentication, or external systems.
## When to Use
- Building anything that accepts user input
- Implementing authentication or authorization
- Storing or transmitting sensitive data
- Integrating with external APIs or services
- Adding file uploads, webhooks, or callbacks
- Handling payment or PII data
## Process: Threat Model First
Controls bolted on without a threat model are guesses. Before hardening, spend five minutes thinking like an attacker:
1. **Map the trust boundaries.** Where does untrusted data cross into your system? HTTP requests, form fields, file uploads, webhooks, third-party APIs, message queues, and **LLM output**. Every boundary is attack surface.
2. **Name the assets.** What's worth stealing or breaking? Credentials, PII, payment data, admin actions, money movement.
3. **Run STRIDE over each boundary** — a quick lens, not a ceremony:
| Threat | Ask | Typical mitigation |
|---|---|---|
| **S**poofing | Can someone impersonate a user/service? | Authentication, signature verification |
| **T**ampering | Can data be altered in transit or at rest? | Integrity checks, parameterized queries, HTTPS |
| **R**epudiation | Can an action be denied later? | Audit logging of security events |
| **I**nformation disclosure | Can data leak? | Encryption, field allowlists, generic errors |
| **D**enial of service | Can it be overwhelmed? | Rate limiting, input size caps, timeouts |
| **E**levation of privilege | Can a user gain rights they shouldn't? | Authorization checks, least privilege |
4. **Write abuse cases next to use cases.** For each feature, ask "how would I misuse this?" — then make that your first test.
If you can't name the trust boundaries for a feature, you're not ready to secure it. This is OWASP **A04: Insecure Design** — most breaches begin in design, not code.
## The Three-Tier Boundary System
### Always Do (No Exceptions)
- **Validate all external input** at the system boundary (API routes, form handlers)
- **Parameterize all database queries** — never concatenate user input into SQL
- **Encode output** to prevent XSS (use framework auto-escaping, don't bypass it)
- **Use HTTPS** for all external communication
- **Hash passwords** with bcrypt/scrypt/argon2 (never store plaintext)
- **Set security headers** (CSP, HSTS, X-Frame-Options, X-Content-Type-Options)
- **Use httpOnly, secure, sameSite cookies** for sessions
- **Run `npm audit`** (or equivalent) before every release
### Ask First (Requires Human Approval)
- Adding new authentication flows or changing auth logic
- Storing new categories of sensitive data (PII, payment info)
- Adding new external service integrations
- Changing CORS configuration
- Adding file upload handlers
- Modifying rate limiting or throttling
- Granting elevated permissions or roles
### Never Do
- **Never commit secrets** to version control (API keys, passwords, tokens)
- **Never log sensitive data** (passwords, tokens, full credit card numbers)
- **Never trust client-side validation** as a security boundary
- **Never disable security headers** for convenience
- **Never use `eval()` or `innerHTML`** with user-provided data
- **Never store sessions in client-accessible storage** (localStorage for auth tokens)
- **Never expose stack traces** or internal error details to users
## OWASP Top 10 Prevention Patterns
These are prevention patterns, not a ranking. For the 2021 ordering, see the quick-reference table in `references/security-checklist.md`.
### Injection (SQL, NoSQL, OS Command)
```typescript
// BAD: SQL injection via string concatenation
const query = `SELECT * FROM users WHERE id = '${userId}'`;
// GOOD: Parameterized query
const user = await db.query('SELECT * FROM users WHERE id = $1', [userId]);
// GOOD: ORM with parameterized input
const user = await prisma.user.findUnique({ where: { id: userId } });
```
### Broken Authentication
```typescript
// Password hashing
import { hash, compare } from 'bcrypt';
const SALT_ROUNDS = 12;
const hashedPassword = await hash(plaintext, SALT_ROUNDS);
const isValid = await compare(plaintext, hashedPassword);
// Session management
app.use(session({
secret: process.env.SESSION_SECRET, // From environment, not code
resave: false,
saveUninitialized: false,
cookie: {
httpOnly: true, // Not accessible via JavaScript
secure: true, // HTTPS only
sameSite: 'lax', // CSRF protection
maxAge: 24 * 60 * 60 * 1000, // 24 hours
},
}));
```
### Cross-Site Scripting (XSS)
```typescript
// BAD: Rendering user input as HTML
element.innerHTML = userInput;
// GOOD: Use framework auto-escaping (React does this by default)
return <div>{userInput}</div>;
// If you MUST render HTML, sanitize first
import DOMPurify from 'dompurify';
const clean = DOMPurify.sanitize(userInput);
```
### Broken Access Control
```typescript
// Always check authorization, not just authentication
app.patch('/api/tasks/:id', authenticate, async (req, res) => {
const task = await taskService.findById(req.params.id);
// Check that the authenticated user owns this resource
if (task.ownerId !== req.user.id) {
return res.status(403).json({
error: { code: 'FORBIDDEN', message: 'Not authorized to modify this task' }
});
}
// Proceed with update
const updated = await taskService.update(req.params.id, req.body);
return res.json(updated);
});
```
### Security Misconfiguration
```typescript
// Security headers (use helmet for Express)
import helmet from 'helmet';
app.use(helmet());
// Content Security Policy
app.use(helmet.contentSecurityPolicy({
directives: {
defaultSrc: ["'self'"],
scriptSrc: ["'self'"],
styleSrc: ["'self'", "'unsafe-inline'"], // Tighten if possible
imgSrc: ["'self'", 'data:', 'https:'],
connectSrc: ["'self'"],
},
}));
// CORS — restrict to known origins
app.use(cors({
origin: process.env.ALLOWED_ORIGINS?.split(',') || 'http://localhost:3000',
credentials: true,
}));
```
### Sensitive Data Exposure
```typescript
// Never return sensitive fields in API responses
function sanitizeUser(user: UserRecord): PublicUser {
const { passwordHash, resetToken, ...publicFields } = user;
return publicFields;
}
// Use environment variables for secrets
const API_KEY = process.env.STRIPE_API_KEY;
if (!API_KEY) throw new Error('STRIPE_API_KEY not configured');
```
### Server-Side Request Forgery (SSRF)
Any time the server fetches a URL the user influenced — webhooks, "import from URL", image proxies, link previews — an attacker can aim it at internal services (cloud metadata, `localhost`, private IPs).
```typescript
// BAD: fetch whatever the user gives you
await fetch(req.body.webhookUrl);
// GOOD: allowlist scheme + host, reject if ANY resolved IP is private, forbid redirects
import { lookup } from 'node:dns/promises';
import ipaddr from 'ipaddr.js';
const ALLOWED_HOSTS = new Set(['hooks.example.com']);
async function assertSafeUrl(raw: string): Promise<URL> {
const url = new URL(raw);
if (url.protocol !== 'https:') throw new Error('https only');
if (!ALLOWED_HOSTS.has(url.hostname)) throw new Error('host not allowed');
// Resolve ALL records; a single private/reserved address fails the check.
const addrs = await lookup(url.hostname, { all: true });
if (addrs.some((a) => ipaddr.parse(a.address).range() !== 'unicast')) {
throw new Error('private/reserved IP');
}
return url;
}
await fetch(await assertSafeUrl(req.body.webhookUrl), { redirect: 'error' });
```
The `range() !== 'unicast'` check covers loopback, link-local `169.254.169.254` (cloud metadata, the #1 SSRF target), private, and unique-local ranges across IPv4 and IPv6.
**Caveat — this still has a TOCTOU gap.** `fetch` resolves DNS again after the check, so an attacker using a short-TTL record can rebind to an internal IP between validation and connection. For high-risk surfaces, resolve once and connect to the pinned IP, or put a filtering agent in front (`request-filtering-agent` / `ssrf-req-filter`).
## Input Validation Patterns
### Schema Validation at Boundaries
```typescript
import { z } from 'zod';
const CreateTaskSchema = z.object({
title: z.string().min(1).max(200).trim(),
description: z.string().max(2000).optional(),
priority: z.enum(['low', 'medium', 'high']).default('medium'),
dueDate: z.string().datetime().optional(),
});
// Validate at the route handler
app.post('/api/tasks', async (req, res) => {
const result = CreateTaskSchema.safeParse(req.body);
if (!result.success) {
return res.status(422).json({
error: {
code: 'VALIDATION_ERROR',
message: 'Invalid input',
details: result.error.flatten(),
},
});
}
// result.data is now typed and validated
const task = await taskService.create(result.data);
return res.status(201).json(task);
});
```
### File Upload Safety
```typescript
// Restrict file types and sizes
const ALLOWED_TYPES = ['image/jpeg', 'image/png', 'image/webp'];
const MAX_SIZE = 5 * 1024 * 1024; // 5MB
function validateUpload(file: UploadedFile) {
if (!ALLOWED_TYPES.includes(file.mimetype)) {
throw new ValidationError('File type not allowed');
}
if (file.size > MAX_SIZE) {
throw new ValidationError('File too large (max 5MB)');
}
// Don't trust the file extension — check magic bytes if critical
}
```
## Triaging npm audit Results
Not all audit findings require immediate action. Use this decision tree:
```
npm audit reports a vulnerability
├── Severity: critical or high
│ ├── Is the vulnerable code reachable in your app?
│ │ ├── YES --> Fix immediately (update, patch, or replace the dependency)
│ │ └── NO (dev-only dep, unused code path) --> Fix soon, but not a blocker
│ └── Is a fix available?
│ ├── YES --> Update to the patched version
│ └── NO --> Check for workarounds, consider replacing the dependency, or add to allowlist with a review date
├── Severity: moderate
│ ├── Reachable in production? --> Fix in the next release cycle
│ └── Dev-only? --> Fix when convenient, track in backlog
└── Severity: low
└── Track and fix during regular dependency updates
```
**Key questions:**
- Is the vulnerable function actually called in your code path?
- Is the dependency a runtime dependency or dev-only?
- Is the vulnerability exploitable given your deployment context (e.g., a server-side vulnerability in a client-only app)?
When you defer a fix, document the reason and set a review date.
### Supply-Chain Hygiene
`npm audit` catches known CVEs; it won't catch a malicious or typosquatted package. Also:
- **Commit the lockfile** and install with `npm ci` (not `npm install`) in CI — reproducible builds, no silent version drift.
- **Review new dependencies before adding them** — maintenance, download counts, and whether they truly earn their place. Every dependency is attack surface (OWASP **A06: Vulnerable Components**, **LLM03: Supply Chain**).
- **Be wary of `postinstall` scripts** in unfamiliar packages — they run arbitrary code at install time.
- **Watch for typosquats** — `cross-env` vs `crossenv`, `react-dom` vs `reactdom`.
## Rate Limiting
```typescript
import rateLimit from 'express-rate-limit';
// General API rate limit
app.use('/api/', rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // 100 requests per window
standardHeaders: true,
legacyHeaders: false,
}));
// Stricter limit for auth endpoints
app.use('/api/auth/', rateLimit({
windowMs: 15 * 60 * 1000,
max: 10, // 10 attempts per 15 minutes
}));
```
## Secrets Management
```
.env files:
├── .env.example → Committed (template with placeholder values)
├── .env → NOT committed (contains real secrets)
└── .env.local → NOT committed (local overrides)
.gitignore must include:
.env
.env.local
.env.*.local
*.pem
*.key
```
**Always check before committing:**
```bash
# Check for accidentally staged secrets
git diff --cached | grep -i "password\|secret\|api_key\|token"
```
**If a secret is ever committed, rotate it.** Deleting the line or rewriting history is not enough — assume it's compromised the moment it reaches a remote. Revoke and reissue the key first, then purge it from history.
## Securing AI / LLM Features
If your app calls an LLM — chatbots, summarizers, agents, RAG — it inherits a new attack surface. Map it to the [OWASP Top 10 for LLM Applications (2025)](https://genai.owasp.org/llm-top-10/):
- **Treat all model output as untrusted input (LLM05: Improper Output Handling).** Never pass LLM output straight into `eval`, SQL, a shell, `innerHTML`, or a file path. Validate and encode it exactly as you would raw user input.
- **Assume prompts can be hijacked (LLM01: Prompt Injection).** Untrusted text in the context window — a user message, a fetched web page, a PDF — can carry instructions. The system prompt is not a security boundary; enforce permissions in code, not in the prompt.
- **Keep secrets and other users' data out of prompts (LLM02 / LLM07).** Anything in the context can be echoed back. Don't put API keys, cross-tenant data, or the full system prompt where the model can repeat it.
- **Constrain tool and agent permissions (LLM06: Excessive Agency).** Scope tools to the minimum, require confirmation for destructive or irreversible actions, and validate every tool argument.
- **Bound consumption (LLM10: Unbounded Consumption).** Cap tokens, request rate, and loop/recursion depth so a crafted input can't run up cost or hang the system.
- **Isolate retrieval data (LLM08: Vector and Embedding Weaknesses).** In RAG, treat the vector store as a trust boundary: partition embeddings per tenant so one user can't retrieve another's data, and validate documents before indexing so poisoned content can't steer answers.
```typescript
// BAD: trusting model output as a command or as markup
const sql = await llm.generate(`Write SQL for: ${userQuestion}`);
await db.query(sql); // arbitrary query execution
container.innerHTML = await llm.reply(userMessage); // stored XSS, via the model
// GOOD: model output is data — parse defensively, then validate, then encode
let intent;
try {
intent = CommandSchema.parse(JSON.parse(await llm.replyJson(userMessage)));
} catch {
throw new ValidationError('unexpected model output'); // JSON.parse or schema failed
}
await runAllowlistedAction(intent.action, intent.params);
container.textContent = await llm.reply(userMessage);
```
## Security Review Checklist
```markdown
### Authentication
- [ ] Passwords hashed with bcrypt/scrypt/argon2 (salt rounds ≥ 12)
- [ ] Session tokens are httpOnly, secure, sameSite
- [ ] Login has rate limiting
- [ ] Password reset tokens expire
### Authorization
- [ ] Every endpoint checks user permissions
- [ ] Users can only access their own resources
- [ ] Admin actions require admin role verification
### Input
- [ ] All user input validated at the boundary
- [ ] SQL queries are parameterized
- [ ] HTML output is encoded/escaped
- [ ] Server-side URL fetches are allowlisted (no SSRF to internal services)
### Data
- [ ] No secrets in code or version control
- [ ] Sensitive fields excluded from API responses
- [ ] PII encrypted at rest (if applicable)
### Infrastructure
- [ ] Security headers configured (CSP, HSTS, etc.)
- [ ] CORS restricted to known origins
- [ ] Dependencies audited for vulnerabilities
- [ ] Error messages don't expose internals
### Supply Chain
- [ ] Lockfile committed; CI installs with `npm ci`
- [ ] New dependencies reviewed (maintenance, downloads, postinstall scripts)
### AI / LLM (if used)
- [ ] Model output treated as untrusted (no eval/SQL/innerHTML/shell)
- [ ] Secrets and other users' data kept out of prompts
- [ ] Tool/agent permissions scoped; destructive actions require confirmation
```
## Common Rationalizations
| Rationalization | Reality |
|---|---|
| "This is an internal tool, security doesn't matter" | Internal tools get compromised. Attackers target the weakest link. |
| "We'll add security later" | Security retrofitting is 10x harder than building it in. Add it now. |
| "No one would try to exploit this" | Automated scanners will find it. Security by obscurity is not security. |
| "The framework handles security" | Frameworks provide tools, not guarantees. You still need to use them correctly. |
| "It's just a prototype" | Prototypes become production. Security habits from day one. |
| "Threat modeling is overkill here" | Five minutes of "how would I attack this?" prevents the design flaws no control can patch later. |
| "It's just LLM output, it's only text" | That "text" can be a SQL statement, a script tag, or a shell command. Treat it like any untrusted input. |
## Red Flags
- User input passed directly to database queries, shell commands, or HTML rendering
- Secrets in source code or commit history
- API endpoints without authentication or authorization checks
- Missing CORS configuration or wildcard (`*`) origins
- No rate limiting on authentication endpoints
- Stack traces or internal errors exposed to users
- Dependencies with known critical vulnerabilities
- Server fetches user-supplied URLs without an allowlist (SSRF)
- LLM/model output passed into a query, the DOM, a shell, or `eval`
- Secrets, PII, or the full system prompt placed inside an LLM context window
## Verification
After implementing security-relevant code:
- [ ] `npm audit` shows no critical or high vulnerabilities
- [ ] No secrets in source code or git history
- [ ] All user input validated at system boundaries
- [ ] Authentication and authorization checked on every protected endpoint
- [ ] Security headers present in response (check with browser DevTools)
- [ ] Error responses don't expose internal details
- [ ] Rate limiting active on auth endpoints
- [ ] Server-side URL fetches validated against an allowlist (no SSRF)
- [ ] LLM/model output validated and encoded before use (if AI features present)
-78
View File
@@ -1,78 +0,0 @@
---
name: ste
description: Write or rewrite text in ASD-STE100 Simplified Technical English. ONLY use this skill when the user explicitly invokes it by name — i.e. they type "/ste" or literally write "use the ste skill" / "apply ASD-STE100". Do NOT trigger it on paraphrased intent such as "simplify this", "make it clearer", "write technical documentation", or "shorter sentences please" — the user has deliberately scoped this skill to explicit invocation only. For those requests, respond normally without loading this skill unless they name it.
---
# ASD-STE100 Simplified Technical English
Apply the ASD-STE100 standard to all prose you produce in this task. Do not announce that you use STE, do not name the standard, and do not explain the style unless the user asks. If the user later asks you to "write more naturally," ask one short question to confirm they want to leave STE before you drop it.
Compliance note (for you, not for output): the official specification and its dictionary are copyright ASD. This skill encodes paraphrased rules and a publicly sourced word list. For certified aerospace/defense deliverables, tell the user that full compliance requires the free official specification (asd-ste100.org) and a human sign-off. Never claim certified compliance.
## Step 0 — Classify the text
Before writing a single sentence, decide: is this **procedural** text (instructions someone follows) or **descriptive** text (explanation, background, description)? Every limit below depends on this. Mixed documents get classified section by section.
## Core rules
### Sentences
- Procedural: maximum **20 words** per sentence.
- Descriptive: maximum **25 words** per sentence.
- Maximum **6 sentences** per paragraph. One topic per paragraph.
- One instruction per sentence. Two actions in one sentence only if they occur at the same time.
- Put a condition BEFORE its command: "If the pressure decreases, close the valve."
- Do not omit articles, subjects, or verbs to save words. "Ensure file exists" is wrong; "Make sure that the file exists" is correct. Keep the word "that" after verbs like "make sure."
- Numbers, units with numbers, abbreviations, quoted strings, code identifiers, and proper nouns each count as one word.
### Verbs
- Allowed forms only: infinitive, imperative, simple present, simple past, simple future, and past participle used as an adjective.
- Never use present perfect or continuous forms. "We have received" → "We received." "is being tested" → a simple form.
- Never use an -ing form as a verb. An -ing word is allowed only inside a technical name ("the mounting bracket," "logging").
- Active voice. Passive is allowed only in descriptive text when the agent is unknown or unimportant.
- Instructions use the imperative: "Open the panel," not "You must open the panel" or "The panel should be opened."
- Express actions as verbs, not nouns: "compress the file," not "perform compression of the file."
- Modals: use **can** (possibility), **will** (future), **must** (requirement). Do not use should, would, could, may, might. A hedge becomes a fact or a "can": "an explosion can occur."
- No phrasal verbs: "go down" → "decrease," "set up" → "install," "carry out" → "do."
### Words
- One word, one meaning, one part of speech, used consistently. Never rotate synonyms: pick one name for a thing and repeat it.
- Before drafting, replace unapproved vocabulary. Read `references/word-substitutions.md` and apply it; it is the working dictionary for this skill.
- Domain-specific nouns (part names, tool names, product names, UI labels) and domain verbs (drill, ream, boot, compile) are your **technical nouns/verbs** — keep them as-is, use each consistently, and do not verb a noun or noun a verb.
- Noun clusters: maximum **3 words** ("overhead panel light" is the limit). Longer clusters get decomposed with prepositions or hyphenated on first use: "main-gear-door retraction-winch handle."
- American English spelling.
- No Latin abbreviations: "e.g." → "for example," "i.e." → "that is," delete "etc."
### Punctuation
- No semicolons — write two sentences.
- Parentheses only for references, abbreviations, and item numbers.
- Hyphenate words that act as one unit; a hyphenated word counts as one word.
- No contractions.
### Warnings, cautions, notes
- **WARNING** = risk of injury or death. **CAUTION** = risk of damage. **NOTE** = information only, never an instruction.
- Start a warning or caution with the command or condition, then give the risk:
"WARNING: Do not touch the terminal. The terminal has a dangerous voltage."
- Notes obey the 25-word descriptive limit.
## Step 2 — Self-check pass
After drafting, scan your text once for each of these and fix every hit before you respond:
1. Any sentence over the 20/25-word limit for its type
2. Contractions, semicolons
3. "should," "would," "could," "may," "might"
4. "has been," "have been," "had been," "is being," "was being"
5. -ing words used as verbs
6. Missing articles (a/an/the/this) before nouns
7. Synonym rotation (the same object under two names)
8. Any word in the unapproved column of `references/word-substitutions.md`
9. Warnings that state the risk before the command
## Reference files
- `references/word-substitutions.md` — unapproved → approved word mappings and one-meaning rulings. Read it before drafting; it is short.
- `references/examples.md` — worked before/after rewrites (procedural, descriptive, warnings, common mistakes). Read it when rewriting existing text or when unsure how a rule applies.
## What NOT to touch
Code blocks, command strings, file paths, error messages, quoted UI text, and proper nouns stay exactly as written. STE applies to the prose around them.
@@ -1,67 +0,0 @@
# Worked before/after examples
## Verb forms
| Before | After |
|---|---|
| We have received the technical reports from HQ. | We received the technical reports from HQ. |
| This device has been being used at Boeing since 2005. | Boeing started to use this device in 2005. |
| The test is continued by the operator. | Continue the test. |
| The screws should be replaced. | Replace the screws. |
| The system is currently running diagnostics. | The system does diagnostic tests now. |
## Vocabulary and phrasing
| Before | After |
|---|---|
| Ensure file exists before running. | Make sure that the file exists before you run the command. |
| Rotate the cover until the jacks are accessible. | Turn the cover until you can get access to the jacks. |
| Extend the jack until the wheels are clear of the ground. | Extend the jack until the wheels do not touch the ground. |
| Make sure that these steps are followed. | Obey these steps. |
| Utilize approximately 3 liters of water. | Use about 3 liters of water. |
| Perform a compression of the log files prior to upload. | Compress the log files before you upload them. |
## Noun clusters
| Before | After |
|---|---|
| Main gear door retraction winch handle | Main-gear-door retraction-winch handle |
| Runway light connection resistance calibration | Calibration of the resistance of the runway light connection |
| Remove the engine transmission housing attachment bolts. | Remove the bolts that attach the transmission housing to the engine. |
## Procedural rewrite (condition first, one instruction per sentence)
Before:
> You should check that the reservoir has been filled with the correct fluid and, after attaching a clear tube to the bleed screw while ensuring its free end is submerged, the pedal can be pumped and held so that opening the screw releases trapped air.
After:
> 1. Fill the reservoir with the correct fluid.
> 2. Attach a clear tube to the bleed screw.
> 3. Put the free end of the tube in a container of fluid.
> 4. Push the pedal three times. Hold the pedal down.
> 5. Open the bleed screw one half turn. Air and fluid flow into the tube.
> 6. Close the bleed screw. Release the pedal.
> 7. If air continues to come out, do steps 4 thru 6 again.
## Warnings and cautions (command first, then risk)
Before:
> Note that serious data loss may potentially occur if the --force flag is used against production.
After:
> CAUTION: Do not use the --force flag on the production database. The flag deletes the rows that do not match the source.
Before:
> Touching the terminal could result in electrocution.
After:
> WARNING: Do not touch the terminal. The terminal has a dangerous voltage.
## Common mistakes checklist
- Dropped articles: "Insert pin in bracket" → "Insert the pin in the bracket."
- Synonym rotation: check/verify/confirm for the same action → one term, everywhere.
- Hedges: "you may want to," "it is recommended that" → an imperative or "must."
- Instruction buried in a NOTE: notes never instruct. Move the instruction to a numbered step.
- Semicolon joining two clauses → two sentences.
- "There are three bolts on the panel" → "The panel has three bolts."
@@ -1,68 +0,0 @@
# Word substitutions and one-meaning rulings
Compiled from public secondary sources (STEMG/ASD public pages, TechScribe, Acrolinx, training materials). This is a working approximation, not the official ASD dictionary. When a word is not listed here and feels formal or Latin-derived, prefer the shortest common alternative.
## Unapproved → approved
| Do not use | Use instead |
|---|---|
| utilize, leverage, employ | use |
| commence, initiate, begin, originate | start |
| terminate, cease, conclude | stop, end |
| ensure, verify, confirm, validate, check | make sure (that), examine |
| perform, conduct, execute, carry out | do |
| facilitate, assist | help |
| obtain, acquire, procure | get |
| sufficient, adequate | enough |
| approximately | about |
| prior to | before |
| subsequent to, following (prep.) | after |
| adjacent to | near |
| accomplish | do |
| additional, supplementary | more |
| attempt | try |
| require, necessitate | need, must |
| mandatory | necessary |
| indicate, signify | show |
| observe (=watch) | look at, examine |
| rotate | turn |
| deactivate | turn off, set to off |
| activate, energize (unless technical verb) | turn on, start |
| toxic | poisonous |
| in order to | to |
| via, by means of | through, with |
| due to, owing to | because of |
| in the event of/that | if |
| accessible | (rewrite: "you can get access to") |
| remainder | rest |
| demonstrate | show |
| modify, alter | change |
| construct, fabricate, build | assemble, make |
| retain | keep |
| locate (=find) | find |
| depress (a button) | push, press |
| proceed | continue, go |
## One meaning, one part of speech (canonical rulings)
- **close** — verb only: to move to a position that stops flow, or to operate a circuit breaker. The adjective is unapproved → use **near** ("do not go near the propeller").
- **test** — noun only: "do a test," never "test the system."
- **check** — do not use as a verb for verification → "make sure that" or "examine."
- **follow** — means only "come after." For rules and steps use **obey**: "Obey the safety instructions."
- **fall** — means only "move down by gravity." For quantities use **decrease**. Never the season.
- **oil** — noun only. "Oil the bearing" → "Put oil on the bearing" / "Lubricate the bearing."
- **right** — direction only, never "correct."
- **clear** — "without blockage." "Wheels are clear of the ground" → "wheels do not touch the ground."
- **help** — verb only; the noun is **aid** ("with the aid of a mirror").
- **above / below** — physical position only. For quantities: **more than / less than**.
- **about** — two approved senses: "approximately" and "on the subject of." Use carefully.
- **turn** — the general verb for rotation; "turn on / turn off" for power state is standard.
- **level** — approved as noun and adjective (documented exception to the one-POS rule).
## Frequent-offender function words
- **should / would / could / may / might** — never. Requirement → **must**. Possibility → **can**. Future → **will**.
- **etc.** — delete, or write the full list.
- **e.g. / i.e.** — "for example" / "that is."
- **any / appropriate / applicable / relevant** as hedges — replace with the specific thing meant.
- **there is / there are** openers — rewrite with a real subject: "There are three bolts on the panel" → "The panel has three bolts."
-110
View File
@@ -1,110 +0,0 @@
---
name: taiga
description: Fetch information from Taiga public API for the Penpot project (id 345963) — issues, user stories, and tasks, without authentication.
metadata: {"clawdbot":{"requires":{"bins":["python3"]}}}
---
# Taiga API Skill
Fetch information from Taiga public API for the **Penpot** project
(project id: `345963`, slug: `penpot`).
**No authentication required** — only public project data is accessed.
## Prerequisites
- `python3` — the `scripts/taiga.py` CLI script is self-contained (stdlib only)
## Quick Start
The easiest way is to use the bundled Python script:
```bash
# Pass a Taiga URL directly
python3 scripts/taiga.py https://tree.taiga.io/project/penpot/issue/13714
# Or use "<type> <ref>" syntax
python3 scripts/taiga.py us 14128
python3 scripts/taiga.py task 13648
# Add --json for raw output
python3 scripts/taiga.py --json issue 13714
# See full usage
python3 scripts/taiga.py --help
```
## URL Pattern Reference
Taiga web URLs follow these patterns:
| Type | Web URL Pattern |
|------|----------------|
| Issue | `https://tree.taiga.io/project/penpot/issue/<REF>` |
| User Story | `https://tree.taiga.io/project/penpot/us/<REF>` |
| Task | `https://tree.taiga.io/project/penpot/task/<REF>` |
To extract the **type** and **ref** from a URL:
- `issue/13714` → type=`issue`, ref=`13714`
- `us/14128` → type=`us`, ref=`14128`
- `task/13648` → type=`task`, ref=`13648`
## Python Script Reference
The `scripts/taiga.py` script wraps the Taiga API into a single convenient CLI
with sensible defaults.
### Usage
```
python3 scripts/taiga.py <taiga-url>
python3 scripts/taiga.py <type> <ref>
python3 scripts/taiga.py [--json] <taiga-url>
python3 scripts/taiga.py [--json] <type> <ref>
```
### Examples
```bash
# By URL (recommended — no need to think about type/ref)
python3 scripts/taiga.py https://tree.taiga.io/project/penpot/issue/13714
# By type and ref
python3 scripts/taiga.py us 14128
python3 scripts/taiga.py task 13648
# Raw JSON output
python3 scripts/taiga.py --json issue 13714
```
### Output
The script prints a clean, structured summary:
```text
User Story #11964 — 🔴 [DESIGN TOKENS] Typography Composite Input
================================
Status: Defining
Milestone: design-systems-sprint-26
Points: 3 role(s)
Assignee: Natacha Menjibar
Author: Natacha Menjibar
Created: 2025-09-01
Tags: iop-design-tokens
URL: https://tree.taiga.io/project/penpot/us/11964
================================
<full description text, unmodified>
```
The fields section includes type-specific information:
- **Issues:** Status, Type ID, Severity ID, Priority ID
- **User Stories:** Status, Milestone, Points
- **Tasks:** Status, Milestone, Parent US
## Reference
- API docs: https://docs.taiga.io/api.html
- Taiga instance: https://tree.taiga.io
- API base: https://api.taiga.io/api/v1
- Penpot project id: `345963`
- Penpot project slug: `penpot`
-58
View File
@@ -1,58 +0,0 @@
---
name: testing
description: Enforce TDD workflow and testing best practices for Penpot. Use when implementing features, fixing bugs, or modifying behavior. Reads testing memory for full guidance.
---
# Testing Skill
Enforces test-driven development and Penpot testing conventions.
## When to Use
- Implementing new logic or behavior
- Fixing any bug (reproduction test required)
- Modifying existing functionality
- Adding edge case handling
**Skip:** Pure configuration changes, documentation updates, or static content with no behavioral impact.
## Workflow
Follow TDD (Red → Green → Refactor) whenever practical:
1. **RED** — Write a failing test first
2. **GREEN** — Write minimal code to pass
3. **REFACTOR** — Clean up while tests stay green
For bug fixes, use the Prove-It Pattern: write a test that reproduces the bug, confirm it fails, implement the fix, confirm it passes.
## Required Reading
Before writing any test, read:
1. `.serena/memories/testing.md` — cross-cutting testing principles, TDD workflow, anti-patterns, execution discipline
2. Module-specific testing memory for the affected module:
- `mem:common/testing` — CLJC unit tests
- `mem:frontend/testing` — CLJS unit tests, Playwright E2E
- `mem:backend/testing` — JVM clojure.test conventions
- `mem:exporter/testing` — exporter unit tests
## Key Rules
- Every behavior change needs a test
- Test state, not interactions
- DAMP over DRY — tests are specifications; duplication is OK if each test is self-contained and readable
- Prefer Real > Fake > Stub > Mock
- Arrange-Act-Assert structure
- One assertion per concept
- Never pipe test output to filters — redirect to file first
- Register new test files in the module's runner/entrypoint
## Verification
After completing implementation:
- [ ] Every new behavior has a test
- [ ] All tests pass for touched modules
- [ ] Bug fixes include a reproduction test
- [ ] Lint/formatter passes
-825
View File
@@ -1,825 +0,0 @@
---
name: update-changelog
description: Update the project CHANGES.md with issues from a given GitHub milestone, with correct categorization and references.
---
# Skill: update-changelog
Update `CHANGES.md` with entries for all issues and PRs in a given GitHub
milestone. Each entry references the user-facing issue (not the PR) as the
primary link, with the fix PR inline on the same line.
## When to Use
- Before a new release, to populate the changelog with all fixed issues
- When new issues are added to an existing milestone and the changelog needs
to be refreshed
- To ensure every entry follows the correct format for the changelog
## Prerequisites
- `gh` CLI authenticated (`gh auth status`)
- Python 3.8+
- `scripts/gh.py` helper script available
## Workflow
### 1. Determine the target version
The version is typically a semver string like `2.15.3`. Confirm with the user
if not specified.
### 2. Fetch all issues in the milestone
Use the helper script. It uses GraphQL for efficient single-pass fetching
(closing PRs are included in the same query — no N+1):
```bash
# All closed issues (default)
python3 scripts/gh.py issues "2.16.0"
# Include open issues too
python3 scripts/gh.py issues "2.16.0" --state all
# Exclude entries that should not go in the changelog
python3 scripts/gh.py issues "2.16.0" --exclude "release blocker,no changelog"
```
**Exclusion rules (issue-level):**
- `no changelog` label — Chore/refactor work that doesn't need a changelog entry
- `release blocker` label — Blocked issues not yet ready for changelog
- `Task` issue type — Internal chores are not user-facing; automatically excluded by `gh.py`. Use `--include-tasks` to override.
- **Rejected project status** — Issues with a "Rejected" status in the "Main" project board are automatically excluded by `gh.py`. This project-level status (independent of the GitHub issue `state`) indicates the issue was rejected from the release. Use `--include-rejected` to override.
**Exclusion rules (PR-level):**
In addition to issue-level exclusions, PRs with these labels should be
excluded regardless of their linked issue's labels:
- `release blocker` — PR is part of a pending release blocker batch
- `no issue required` — Trivial fix not tracked as an issue
The script outputs JSON with each entry containing `number`, `title`, `state`,
`issue_type`, `labels`, `closing_prs` (the PRs that fix each issue), and
`project_status` (the "Main" project board status, e.g. "Done", "Rejected",
or `null` if not tracked in a project).
### 3. Identify missing entries (optional)
If updating from an existing `CHANGES.md`, find issues in the milestone that
are NOT yet referenced in the changelog:
```bash
python3 scripts/gh.py issues "2.16.0" --exclude "release blocker,no changelog" --compare CHANGES.md
```
This returns a filtered JSON array with only the missing issues.
> **Note:** The `--compare` flag checks **issues** only (via issue number
> references in the changelog). To find merged **PRs** not yet referenced,
> use the milestone PR cross-reference described in step 10 below.
### 4. Fetch additional PR details when needed
When you need more context for specific PRs (e.g. to find the PR author for
community contribution attribution, or to read the PR body for
"Fixes/Closes #NNN" patterns):
```bash
# One or more PR numbers
python3 scripts/gh.py prs 9179 9204 9311
# From a file
python3 scripts/gh.py prs --file prs.txt
# From stdin
cat prs.txt | python3 scripts/gh.py prs --stdin
```
The `prs` command also supports listing all PRs in a milestone in one call:
```bash
# All merged PRs in a milestone (default)
python3 scripts/gh.py prs --milestone "2.16.0"
# All states (merged, open, closed)
python3 scripts/gh.py prs --milestone "2.16.0" --state all
```
The `prs` command returns JSON with `number`, `title`, `body`, `state`,
`merged_at`, `author`, `labels`, and `closing_issues`. PRs are fetched in
batches of 50 via GraphQL to stay within API limits (milestone mode uses
paginated GraphQL on the milestone's `pullRequests` connection).
You can also list all PRs in a milestone in a single call:
```bash
# All merged PRs in a milestone (default)
python3 scripts/gh.py prs --milestone "2.16.0"
# All states (merged, open, closed)
python3 scripts/gh.py prs --milestone "2.16.0" --state all
# Open PRs only
python3 scripts/gh.py prs --milestone "2.16.0" --state open
```
The milestone path uses paginated GraphQL on the milestone's `pullRequests`
connection (100 per page), avoiding one-by-one fetches.
### 5. Categorize entries — strictly by issue type, never by labels or emoji
Use the **Issue Type** field (GitHub's native issue type, exposed as
`issue_type` in the `gh.py` JSON output) to determine which section an entry
belongs to.
> **⚠️ CRITICAL: Never use labels or title emoji prefixes for categorization.**
> Labels like `bug` and `enhancement`, as well as title prefixes like `:bug:`
> and `:sparkles:`, are frequently inaccurate, missing, or contradictory to the
> actual issue type. The `issue_type` field from `gh.py` is the single source
> of truth.
| `issue_type` value | Changelog section |
|--------------------|-------------------|
| `Bug` | `### :bug: Bugs fixed` |
| `Feature` or `Enhancement` | `### :sparkles: New features & Enhancements` |
| `Task` | **Exclude** — internal chores are not user-facing |
| `null` (not set) | Check labels as a fallback: `bug` label → bugs, otherwise enhancements |
The `gh.py` issues command already includes `issue_type` in every entry's
output. **No separate GraphQL query is needed.**
**Preserve highlighted entries:** If an entry is already featured in
`### :rocket: Epics and highlights`, keep it in that section when refreshing a
changelog version. Do not remove a highlighted entry just because issue type
categorization would otherwise place it under `### :sparkles: New features &
Enhancements`.
**Community contribution attribution:** If the issue or its fix PR has the
`community contribution` label, add an attribution `(by @<github_username>)`
on the changelog entry line, **before** the GitHub issue/PR references.
The attribution should reference the **PR author**, not the issue author.
The `prs` subcommand includes the `author` field — use that:
```bash
python3 scripts/gh.py prs <PR_NUMBER> | python3 -c "import sys,json; print(json.load(sys.stdin)[0]['author'])"
```
Placement in the entry line:
```markdown
- Fix description of the bug (by @username) [#<ISSUE>](...) (PR: [#<PR>](...))
```
**Only closed issues are included.** An issue must have `state: "closed"` to
appear in the changelog. Open/unresolved issues are omitted, even if they are
tracked in the milestone.
**Pairing rules:**
| Pattern | Changelog format |
|---------|-----------------|
| Closed issue + one or more PRs fix it | Primary link = issue, PR inline comma-separated |
| PR exists with no linked issue | If a corresponding closed issue exists in the same milestone, link the issue. Otherwise, skip the entry (the issue must be the changelog unit). |
| Closed issue with no fix PR in milestone | Link the issue directly, without a PR reference. |
> **False-positive associations:** A PR may incorrectly claim to close an issue
> from a different context (e.g., a very old PR referencing a modern issue, or a
> cross-project reference). If the PR title and issue title are clearly unrelated,
> or the PR was created years before the issue, treat it as a data glitch and
> skip it. PR [#3](https://github.com/penpot/penpot/pull/3) (ancient License PR
> claiming to close a plugin API issue) is a known example.
### 5a. ⚠️ Verify PR merge status before writing
A closed issue may list closing PRs that were **closed without merging**
(e.g., a community PR that was superseded by another). The changelog must
only reference **merged** PRs. Verify before writing:
```bash
# Collect all PR numbers from the candidate entries and check them
python3 scripts/gh.py prs <ALL_PR_NUMBERS> | python3 -c "
import json, sys
for pr in json.load(sys.stdin):
if pr['state'] != 'MERGED':
print(f'WARNING: #{pr[\"number\"]} is {pr[\"state\"]} (not merged)')
"
```
If a closing PR is closed-unmerged, find the actual merged PR that
superseded it:
1. Check the issue's closing PRs list for other PRs (there may be multiple)
2. Look for other PRs with similar titles or descriptions referencing the same issue
3. Inspect the closed PR's conversation timeline for a pointer to the replacement
Replace the reference in the changelog entry with the correct merged PR number.
### 5b. Security advisory (GHSA) entries
Security advisories fixed in a release are documented in the changelog even
though they are **neither milestone issues nor PRs**. The GHSA ID and its
description are supplied by the user or the release notes — they never come
from the milestone fetch in step 2.
**Format** (matches the existing precedent in `CHANGES.md`, e.g. the
`create-font-variant` arbitrary file read advisory):
```markdown
- Fix <user-facing description> (https://github.com/penpot/penpot/security/advisories/GHSA-XXXX-XXXX-XXXX)
```
Rules:
- Place the entry under `### :bug: Bugs fixed`, with **no issue or PR link**
only the advisory URL.
- The advisory may be **draft/unpublished** at changelog time (the URL 404s
publicly). Do **not** web-fetch or verify the URL, and do **not** drop the
entry because of that. Rely on the GHSA ID provided by the user.
- Derive the description from the supplied advisory title, imperative mood and
user-facing (e.g. `Fix command injection in SVG exporter via legacy fill-color`).
- These entries are **invisible to the automation**: they are not returned by
`gh.py issues`, not matched by `--compare` (step 3), not part of the PR
cross-reference (step 10), and not scanned by the anomaly-report regexes
(step 11, which only match `issues/` and `pull/` links). Add them manually.
- During pre-flight checks (step 6a) apply only the **backport/duplicate**
check: if the same GHSA already appears in an earlier version section, remove
it from the current section. Their absence from milestone cross-references
is expected, not an anomaly.
### 6. Read the current CHANGES.md
Read the top of `CHANGES.md` to understand the existing format and find the
insertion point (newest version goes at the top, after the `# CHANGELOG`
header).
Key format rules from the existing file:
```markdown
## <VERSION>
### :bug: Bugs fixed
- Fix description of the bug [#<ISSUE>](https://github.com/penpot/penpot/issues/<ISSUE>) (PR: [#<PR>](https://github.com/penpot/penpot/pull/<PR>))
- Fix another bug (by @contributor) [#<ISSUE>](https://github.com/penpot/penpot/issues/<ISSUE>) (PR: [#<PR>](https://github.com/penpot/penpot/pull/<PR>))
### :sparkles: New features & Enhancements
- Add new feature description [#<ISSUE>](https://github.com/penpot/penpot/issues/<ISSUE>) (PR: [#<PR>](https://github.com/penpot/penpot/pull/<PR>))
```
Format details:
- Entries start with `- ` followed by a short description in imperative mood
- Primary link is **always the issue** (user-facing artifact)
- PR references are inline on the same line: `(PR: [#<N>](<url>))`
If an issue has multiple fix PRs, they are comma-separated:
`(PR: [#<N>](<url>), [#<M>](<url>))`
- The description should describe the fix/feature from the user's perspective
- Community contributions get `(by @<username>)` **before** the issue link
- Sections are separated by a blank line between the last entry and the next
section title
- Only include a section if there are entries for it
- When an entry already exists in an earlier version section, it must be removed
from the current version to avoid duplicates
### 6a. Pre-flight checks — fix rule violations in the changelog
**The LLM must apply these checks during the workflow and fix any
violations directly in `CHANGES.md`. They are not anomalies — they are
process errors that should be corrected before writing the new section.**
The changelog is a *snapshot* of the milestone at a point in time, but
milestones and changelog entries can drift. The LLM must reconcile the
existing changelog against the current state of the milestone and the
existing changelog entries.
For each entry that already exists in `CHANGES.md` (in any version
section) or in the candidate set for the current milestone, check:
1. **Duplicate across versions.** Is the same issue already documented
in another (older) version section? If yes, this is a *backport*:
- The user-facing fix was already released. Remove the duplicate
from the current section. The earlier version is the canonical
reference.
2. **Stale milestone assignment.** Has the issue been moved out of the
current milestone since the changelog was last updated (e.g., a fix
arrived late and the issue was reassigned to a future milestone)?
- Verify the issue is still in the current milestone via
`python3 scripts/gh.py issues <MILESTONE> --state all`. If it's no
longer there, remove the entry from the current section. (If the
target section doesn't exist yet, the entry is simply dropped.)
3. **Exclusion labels newly applied.** Did the issue acquire a
`no changelog` or `release blocker` label since the changelog was
last updated? If yes, remove the entry from the current section.
4. **Issue state changed.** Is the issue still closed? Has it been
reopened, deleted, or moved to a `Rejected` project status? If yes,
remove the entry.
5. **Unmerged or removed PR references.** For every PR referenced in
the entry, is the PR still merged? Was the PR closed without
merging (superseded)? Was the PR moved to a different milestone?
If the only referenced PR is no longer merged, fix the reference
(find the actual merged fix PR) or remove the entry. A PR that is
merged in a *different* milestone is reported as an anomaly in
step 11 — do not silently remove it.
6. **Issue type changed.** Did the issue type change (e.g., from Bug to
Task)? If the new type is `Task`, the issue is internal and should
be removed.
7. **Cross-section completeness.** For every closed, non-excluded
milestone issue that is *not* referenced in any version section of
the changelog, add it to the current section (per the categorization
rules in step 5).
After these checks, the changelog should be internally consistent with
the milestone. **Do not defer these fixes to step 11 — they are
workflow errors, not anomalies.** Step 11 only reports milestone
mismatches that require human judgment about the team's release
intent.
### 7. Build the description text
Derive the description from the issue title, not the PR title. Strip leading
emoji prefixes (`:bug:`, `:sparkles:`, `:tada:`) and focus on the
user-facing behavior.
Examples:
| Issue title | Changelog description |
|-------------|----------------------|
| `Plugin API token methods fail with schema validation error on PRO` | `Fix Plugin API token methods failing with schema validation error on PRO` |
| `Comment content is not sanitized before rendering, enabling stored XSS` | `Sanitize comment content on rendering` |
| `Custom uploaded font family names are not sanitized` | `Sanitize font family names on custom uploaded fonts` |
### 8. Insert the section into CHANGES.md
Insert the new version section right after the `# CHANGELOG` header (before
the previous version entry). Use the `edit` tool with enough context to make
a unique match.
### 9. Verify
Read the top of `CHANGES.md` and confirm:
- The version header is correct
- Every entry has a GitHub link
- Entries with a fix PR have the PR sub-line
- The section ordering is correct (newest first)
- Formatting matches the surrounding entries
### 10. Cross-reference milestone PRs against the changelog
Issues can be fixed by PRs that aren't in the milestone, and merged PRs in
the milestone may not close any tracked issue. After writing, run a full
cross-reference to catch gaps:
```bash
# List all merged PRs in the milestone
python3 scripts/gh.py prs --milestone "<MILESTONE>" --state merged > /tmp/milestone-prs.json
# Extract PR numbers from the changelog section
python3 -c "
import json, re
with open('CHANGES.md') as f:
content = f.read()
# Extract the version section (adjust regex to match the actual version)
match = re.search(r'## <MILESTONE> \(Unreleased\)\n(.*?)(?:\n## |\Z)', content, re.DOTALL)
section = match.group(1)
# Collect all PR numbers referenced
changelog_prs = set()
for m in re.findall(r'\[#(\d+)\]\(https://github\.com/penpot/penpot/pull/\d+\)', section):
changelog_prs.add(int(m))
# Collect all milestone PRs (filtered)
with open('/tmp/milestone-prs.json') as f:
milestone_prs = json.load(f)
milestone_merged = {pr['number'] for pr in milestone_prs}
# PRs in milestone but not in changelog
missing = sorted(milestone_merged - changelog_prs)
print(f'Milestone merged PRs: {len(milestone_merged)}')
print(f'Changelog referenced PRs: {len(changelog_prs)}')
print(f'PRs in milestone but NOT in changelog: {len(missing)}')
for num in missing:
pr = next(p for p in milestone_prs if p['number'] == num)
print(f' #{num} {pr[\"title\"][:80]}')
"
```
For each missing PR found, decide whether it should be added to the
changelog or is legitimately excluded (check its labels).
Also verify that no closed-unmerged PRs remain in the changelog:
```bash
python3 scripts/gh.py prs --milestone "<MILESTONE>" --state all | python3 -c "
import json, sys
data = json.load(sys.stdin)
closed = [p for p in data if p['state'] == 'CLOSED']
if closed:
print('WARNING: CLOSED (unmerged) PRs in milestone:')
for p in closed:
print(f' #{p[\"number\"]} {p[\"title\"][:80]}')
"
```
**Post-edit audit checklist:**
- ✅ All referenced PRs are merged (no closed-unmerged artifacts)
- ✅ Every merged milestone PR is either in the changelog or excluded by label
- ✅ PR and issue counts are internally consistent
- ✅ No false-positive PR-to-issue associations
- ✅ Advisory (GHSA) entries are not milestone PRs — their absence from the
cross-reference is intentional (see step 5b)
## Version section template
```markdown
## <VERSION>
### :bug: Bugs fixed
- <fix description> [#<ISSUE>](https://github.com/penpot/penpot/issues/<ISSUE>) (PR: [#<PR>](https://github.com/penpot/penpot/pull/<PR>))
- <fix description> (by @contributor) [#<ISSUE>](https://github.com/penpot/penpot/issues/<ISSUE>) (PR: [#<PR>](https://github.com/penpot/penpot/pull/<PR>))
- <fix description> (https://github.com/penpot/penpot/security/advisories/GHSA-XXXX-XXXX-XXXX)
```
Advisory (GHSA) entries have no issue or PR link — just the advisory URL. See
step 5b.
### 11. Generate anomaly report and save to CHANGES-ISSUES.md
After all edits and cross-referencing are complete, generate a structured
report and save it to `CHANGES-ISSUES.md` (overwriting if exists).
This provides a persistent record of any discrepancies between the milestone
and the changelog.
**Every issue and PR number in the report must be rendered as a full GitHub
Markdown link** using the same URL format as `CHANGES.md`:
- Issue N → `[#N](https://github.com/penpot/penpot/issues/N)`
- PR N → `[#N](https://github.com/penpot/penpot/pull/N)`
The titles and notes should also link to the corresponding issue/PR page
where applicable, so the report is self-contained and clickable from any
Markdown viewer.
## What is an anomaly
**An anomaly is a milestone-mismatch between an issue and its referenced
PR.** It indicates that the changelog claim "this issue is fixed by this PR,
all in milestone M" is inconsistent with the actual milestone assignments.
There are exactly two types:
1. **Issue is in the milestone, but its referenced PR is in a different
milestone (or has no milestone).** The changelog claims a fix in this
release, but the PR is being released elsewhere — the fix may not
actually ship here.
2. **PR is in the milestone, but the issue it closes is in a different
milestone.** The PR is being released here, but the issue it fixes is
being released in a different version — the changelog pairing is
misleading.
**Exception — issue with no milestone is NOT an anomaly.** Milestones
are only required for issues tracked in the "Main" project. A milestone
PR that closes an issue with no milestone references an issue from
another (probably private) project; that is expected and the issue is
not part of this changelog. Do not report it.
**Anything else is not an anomaly.** Other discrepancies (exclusion
labels on in-changelog issues, missing valid issues, unmerged PR
references, duplicates across versions, stale milestone assignments)
are **rule violations** that the LLM must fix directly in `CHANGES.md`
during step 6a (pre-flight checks). They should not appear in this
report — if they do, the LLM has skipped the pre-flight step and
needs to re-run the workflow.
The changelog's primary unit is the **issue**, not the PR, so a missing or
mismatched PR only matters when its issue is part of this milestone.
Run this self-contained script:
```bash
python3 << 'PYEOF'
import json, re, subprocess, sys
from datetime import datetime, timezone
MILESTONE = "<MILESTONE>"
CHANGES_MD = "CHANGES.md"
OUTPUT = "CHANGES-ISSUES.md"
REPO = "penpot/penpot"
# --- URL helpers (match CHANGES.md format exactly) ---
def issue_url(n): return f"https://github.com/{REPO}/issues/{n}"
def pr_url(n): return f"https://github.com/{REPO}/pull/{n}"
def issue_link(n): return f"[#{n}]({issue_url(n)})"
def pr_link(n): return f"[#{n}]({pr_url(n)})"
def issue_link_title(n, title):
url = issue_url(n)
if title:
return f"[#{n}]({url}) — [{title}]({url})"
return f"[#{n}]({url})"
def pr_link_title(n, title):
url = pr_url(n)
if title:
return f"[#{n}]({url}) — [{title}]({url})"
return f"[#{n}]({url})"
def fmt_pr_list(nums):
return ", ".join(pr_link(n) for n in nums)
def fmt_issue_list(nums):
return ", ".join(issue_link(n) for n in nums)
# --- Fetch milestone data ---
result = subprocess.run(
["python3", "scripts/gh.py", "issues", MILESTONE, "--state", "all"],
capture_output=True, text=True)
all_issues = json.loads(result.stdout)
issue_by_num = {i['number']: i for i in all_issues}
result = subprocess.run(
["python3", "scripts/gh.py", "prs", "--milestone", MILESTONE, "--state", "all"],
capture_output=True, text=True)
all_prs = json.loads(result.stdout)
pr_by_num = {p['number']: p for p in all_prs}
# --- Read changelog section ---
with open(CHANGES_MD) as f:
content = f.read()
m = re.search(rf'## {re.escape(MILESTONE)}(?:\s*\([^)]*\))?\n(.*?)(?:\n## |\Z)', content, re.DOTALL)
section = m.group(1) if m else ""
changelog_issues = set()
for num in re.findall(r'\[#(\d+)\]\(https://github\.com/penpot/penpot/issues/\d+\)', section):
changelog_issues.add(int(num))
for num in re.findall(r'\[Github #(\d+)\]', section):
changelog_issues.add(int(num))
changelog_prs = set()
for num in re.findall(r'\[#(\d+)\]\(https://github\.com/penpot/penpot/pull/\d+\)', section):
changelog_prs.add(int(num))
for num in re.findall(r'PR:\[(\d+)\]', section):
changelog_prs.add(int(num))
# --- Milestone lookup caches ---
# PRs and issues returned by milestone queries are KNOWN to be in MILESTONE.
# For everything else, fall back to `gh` per-item lookups.
pr_milestone_cache = {p['number']: MILESTONE for p in all_prs}
issue_milestone_cache = {i['number']: MILESTONE for i in all_issues}
def get_pr_milestone(pr_num):
"""Return the milestone title for a PR, or None if unassigned / unknown."""
if pr_num in pr_milestone_cache:
return pr_milestone_cache[pr_num]
try:
r = subprocess.run(
["gh", "pr", "view", str(pr_num), "--json", "milestone"],
capture_output=True, text=True, check=True)
data = json.loads(r.stdout)
ms = data.get('milestone')
pr_milestone_cache[pr_num] = (ms or {}).get('title')
except (subprocess.CalledProcessError, json.JSONDecodeError):
pr_milestone_cache[pr_num] = None
return pr_milestone_cache[pr_num]
def get_issue_milestone(issue_num):
"""Return the milestone title for an issue, or None if unassigned / unknown."""
if issue_num in issue_milestone_cache:
return issue_milestone_cache[issue_num]
try:
r = subprocess.run(
["gh", "issue", "view", str(issue_num), "--json", "milestone"],
capture_output=True, text=True, check=True)
data = json.loads(r.stdout)
ms = data.get('milestone')
issue_milestone_cache[issue_num] = (ms or {}).get('title')
except (subprocess.CalledProcessError, json.JSONDecodeError):
issue_milestone_cache[issue_num] = None
return issue_milestone_cache[issue_num]
# --- Exclusion rules (shared) ---
EXCLUDED_LABELS = {'release blocker', 'no changelog'}
EXCLUDED_ISSUE_TYPES = {'Task'}
EXCLUDED_PROJECT_STATUS = {'Rejected'}
def issue_excluded(issue):
if not issue: return True
if issue.get('state') != 'CLOSED': return True
if issue.get('issue_type') in EXCLUDED_ISSUE_TYPES: return True
if issue.get('project_status') in EXCLUDED_PROJECT_STATUS: return True
if EXCLUDED_LABELS & set(issue.get('labels', [])): return True
return False
# --- ANOMALIES: milestone mismatches between issues and their referenced PRs ---
# These are the ONLY items that should appear in the report. All other
# discrepancies (exclusion labels, missing valid issues, unmerged PRs,
# duplicates, stale milestone assignments) are workflow errors that the
# LLM must fix in step 6a (pre-flight checks) — they are not anomalies.
# Type A: issue in MILESTONE, referenced PR in different milestone or no milestone
anomalies_a = [] # list of dicts: {issue, issue_title, pr, pr_milestone}
for issue_num in sorted(changelog_issues):
issue = issue_by_num.get(issue_num)
if not issue: continue
if get_issue_milestone(issue_num) != MILESTONE: continue
for pr_num in issue.get('closing_prs', []):
pr_ms = get_pr_milestone(pr_num)
if pr_ms != MILESTONE:
anomalies_a.append({
'issue': issue_num,
'issue_title': issue.get('title', ''),
'pr': pr_num,
'pr_milestone': pr_ms, # may be None
})
# Type B: PR in MILESTONE, the issue it closes is in different milestone or no milestone
anomalies_b = [] # list of dicts: {pr, pr_title, issue, issue_milestone}
for pr_num in sorted(changelog_prs):
pr = pr_by_num.get(pr_num)
if not pr: continue
if get_pr_milestone(pr_num) != MILESTONE: continue
for issue_num in pr.get('closing_issues', []):
issue_ms = get_issue_milestone(issue_num)
# No milestone = issue from another (probably private) project —
# milestones are only required for the "Main" project. Not an
# anomaly, and the issue never belongs in this changelog.
if issue_ms is None: continue
if issue_ms != MILESTONE:
anomalies_b.append({
'pr': pr_num,
'pr_title': pr.get('title', ''),
'issue': issue_num,
'issue_milestone': issue_ms, # may be None
})
# --- Write report ---
def fmt_ms(ms):
return ms if ms else "_none_"
with open(OUTPUT, 'w') as f:
f.write(f'# Changelog Anomaly Report — {MILESTONE}\n\n')
f.write(f'Generated: {datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")}\n\n')
f.write('---\n\n')
n_a = len(anomalies_a)
n_b = len(anomalies_b)
f.write('## Summary\n\n')
f.write(f'- **Issue in {MILESTONE}, referenced PR in different milestone or no milestone:** {n_a}\n')
f.write(f'- **PR in {MILESTONE}, closing issue in a different milestone:** {n_b}\n')
f.write(f'- **Total anomalies:** {n_a + n_b}\n\n')
# --- Anomalies section ---
if n_a or n_b:
f.write('## Anomalies\n\n')
f.write('These are milestone mismatches between an issue in the changelog '
'and its referenced PR (or vice-versa). The changelog claim '
'"this issue is fixed by this PR, all in this milestone" is '
'inconsistent with the actual milestone assignments. '
'Resolve by either updating the milestone on the issue/PR or '
'removing the misleading entry from the changelog.\n\n')
if n_a:
f.write(f'### Issue in {MILESTONE}, PR in different milestone or no milestone\n\n')
by_issue = {}
for a in anomalies_a:
by_issue.setdefault(a['issue'], []).append(a)
for issue_num in sorted(by_issue):
entries = by_issue[issue_num]
title = entries[0]['issue_title']
f.write(f'- {issue_link_title(issue_num, title[:80])}\n')
for e in entries:
ms_label = fmt_ms(e['pr_milestone'])
badge = '🔴' if e['pr_milestone'] is None else '⚠️'
f.write(f' - {badge} Referenced {pr_link(e["pr"])} is in milestone **{ms_label}** (expected: {MILESTONE})\n')
f.write('\n')
if n_b:
f.write(f'\n### PR in {MILESTONE}, closing issue in a different milestone\n\n')
by_pr = {}
for b in anomalies_b:
by_pr.setdefault(b['pr'], []).append(b)
for pr_num in sorted(by_pr):
entries = by_pr[pr_num]
title = entries[0]['pr_title']
f.write(f'- {pr_link_title(pr_num, title[:80])}\n')
for e in entries:
ms_label = fmt_ms(e['issue_milestone'])
badge = '🔴' if e['issue_milestone'] is None else '⚠️'
f.write(f' - {badge} Closing {issue_link(e["issue"])} is in milestone **{ms_label}** (expected: {MILESTONE})\n')
f.write('\n')
else:
f.write('✅ No anomalies found. All (issue, PR) pairs in the changelog have aligned milestone assignments.\n\n')
# --- Context ---
f.write('---\n\n')
f.write('## Context\n\n')
f.write(f'- Milestone: **{MILESTONE}**\n')
f.write(f'- Milestone total issues (all states): {len(all_issues)}\n')
f.write(f'- Closed issues in milestone: {sum(1 for i in all_issues if i.get("state") == "CLOSED")}\n')
f.write(f'- Valid issues after exclusions (after step 5/6a): {len([i for i in all_issues if not issue_excluded(i)])}\n')
f.write(f'- Issues referenced in changelog: {len(changelog_issues)}\n')
f.write(f'- PRs referenced in changelog: {len(changelog_prs)}\n')
print(f"Anomaly report written to {OUTPUT}")
PYEOF
```
This generates `CHANGES-ISSUES.md` containing **only the anomalies**
milestone mismatches between issues and their referenced PRs:
1. **Issue in milestone, referenced PR in different milestone or no milestone**
the changelog claims a fix here, but the PR is released elsewhere.
2. **PR in milestone, closing issue in a different milestone**
the PR is released here, but the issue it fixes belongs to another version.
(An issue with *no* milestone belongs to another, probably private,
project — milestones are only required on the "Main" project — so it is
neither an anomaly nor a changelog candidate.)
**Rule violations are not in the report** — they are workflow errors the
LLM must fix directly in `CHANGES.md` during step 6a (pre-flight checks).
If the report contains a rule violation, the LLM has skipped the pre-flight
step and needs to re-run the workflow before re-generating the report.
The report is overwritten each time it's generated, reflecting the current
state of the milestone and changelog. Every number is rendered as a full
`[#N](https://github.com/penpot/penpot/issues/N)` or
`[#N](https://github.com/penpot/penpot/pull/N)` link so the report is
self-contained and clickable in any Markdown viewer.
## Key Principles
- **Issue = changelog unit.** The primary link always points to the
user-facing issue, not the implementation PR.
- **PR = implementation detail.** Reference the PR inline so readers
can find the code changes.
- **Latest version first.** New sections are inserted at the top of the
changelog, below the `# CHANGELOG` header.
- **Issue Type determines section — exclusively.** Use the `issue_type` field from `gh.py` output (Bug → `:bug:`, Feature/Enhancement → `:sparkles:`). **Do not** use labels (`bug`, `enhancement`) or title emoji prefixes (`:bug:`, `:sparkles:`) — they are frequently wrong or contradictory. The `issue_type` is the single source of truth.
- **User-facing descriptions.** Write from the user's perspective — describe
what broke and what was fixed, not internal implementation details.
- **Community attribution.** When the issue or fix PR has the
`community contribution` label, add `(by @<username>)` on the entry line
between the description and the issue link. Use the **PR author** (not the
issue author) for the attribution.
- **Only closed issues.** An issue must have `state: "closed"` to appear in
the changelog. Open/unresolved issues are omitted.
- **Rejected project status.** Issues marked as "Rejected" in the "Main"
project board are automatically excluded by `gh.py`, even if they are
closed. The project status is distinct from the GitHub issue state.
Use `--include-rejected` to override this behavior.
- **Excluded issues.** Issues with `no changelog` label must be excluded.
Issues with `issue_type: "Task"` must also be excluded — they are internal
chores, not user-facing changes.
- **Multiple PRs per issue.** If multiple PRs fix the same issue, list them
comma-separated inline: `(PR: [#A](url), [#B](url))`.
- **Duplicate removal.** If an entry already exists in a prior version section,
remove it from the current version. Check for text-level duplicates (after
stripping links and attributions) across version sections.
- **Taiga references.** If a changelog entry references a Taiga URL
(`tree.taiga.io`), attempt to find a corresponding GitHub issue via the
Taiga description text or by searching GitHub PRs that reference the Taiga
URL. Replace the Taiga reference with the GitHub issue link and add the PR
reference if applicable.
- **Security advisory (GHSA) entries.** Advisories fixed in the release are
listed under `### :bug: Bugs fixed` with the advisory URL and **no issue or
PR link**, even though they are not in the milestone. The GHSA ID and
description come from the user — do **not** fetch or verify the URL, and do
not drop a draft (unpublished) advisory. Precedent:
`- Fix arbitrary file read security issue on create-font-variant rpc method
(https://github.com/penpot/penpot/security/advisories/GHSA-xp3f-g8rq-9px2)`.
See step 5b.
- **Re-fetch before editing.** Milestones can change — always re-fetch issues
before making edits, don't rely on cached data.
- **Use `scripts/gh.py`.** Prefer the helper script over raw `gh api` calls for
milestone issue listing and PR detail fetching. It handles GraphQL
pagination, batching, and label filtering automatically.
- **Verify PR merge status.** Not all closing PRs are merged — community PRs
can be superseded and closed without merging. Always check that every PR
referenced in the changelog has `state: MERGED`.
- **PR-level exclusions apply.** A PR can carry its own exclusion labels
(`release blocker`, `no issue required`) independent of its linked issue's
labels. Check both.
- **Cross-reference milestone PRs, not just issues.** The `--compare` flag on
the `issues` command only compares issue numbers. Merged PRs not linked to
any milestone issue can be missed. Use `python3 scripts/gh.py prs --milestone`
for a full PR cross-reference.
- **False-positive PR-to-issue associations.** A PR may claim to close an
issue from a different project or context. If the PR title and issue title
are clearly unrelated, or the PR predates the issue by years, treat it as a
data glitch and skip it.
- **Anomaly = milestone mismatch only.** The report contains only milestone
mismatches: (1) the issue is in this milestone but the referenced PR is
in a different milestone (or unassigned), and (2) the PR is in this
milestone but the issue it closes is in a different milestone. An
*unassigned* (milestone-less) issue closed by a milestone PR is **not**
an anomaly: milestones are required only for the "Main" project, so such
issues come from another (probably private) project and are not changelog
candidates. These anomalies are reported because the changelog pairing is
*misleading* — the human needs to decide whether the milestone or the
changelog is wrong. All other discrepancies (exclusion labels, missing
valid issues, unmerged PR references, duplicates, stale milestone
assignments) are **rule violations** that the LLM must fix directly in
`CHANGES.md` during step 6a (pre-flight checks). They never appear in
the report — if they do, the pre-flight step was skipped.
-2
View File
@@ -1,2 +0,0 @@
/cache
/project.local.yml
@@ -1,40 +0,0 @@
# Backend Auth, Permissions, and Product Domain Subtleties
## Auth and sessions
- Main auth RPC commands live in `app.rpc.commands.auth`; LDAP and OIDC provider logic live in `app.auth.ldap` and `app.auth.oidc`, with LDAP-specific RPC checks in `app.rpc.commands.ldap`.
- Public auth endpoints must explicitly set `::rpc/auth false`; RPC auth defaults to enabled. Session cookie creation/deletion is usually attached as an RPC response transform.
- Basic Penpot registration is token staged: prepare/register creates or verifies temporary tokens, then profile creation/session setup is reused by other auth backends. The frontend `/auth/verify-token` flow is a hub for registration confirmation, email change, and invitation tokens.
- OIDC-compatible providers share a generic flow: redirect to provider, validate callback/request token, fetch identity data, then login an existing profile or register a new one. Known providers may have hardcoded endpoints; generic OIDC can use discovery/configured endpoints.
- LDAP login validates credentials against the external directory, fetches identity data, then logs in or registers a matching Penpot profile. LDAP registration is not a separate Penpot signup flow.
- Logout may return an OIDC provider redirect URI when the session claims include provider/session data and the provider has a logout URI.
- Invitation tokens are verified through token issuers and only accepted when the token member id/email matches the authenticated profile; otherwise login proceeds without consuming the invitation.
- HTTP/session parsing details such as cookie/header precedence, JWT session token versions, and SameSite behavior are in `mem:backend/http-storage-filedata-subtleties`.
## Permission model
- `app.rpc.permissions` provides predicate/check factories. Failed permission checks intentionally raise `:not-found` / `:object-not-found`, not an authorization-specific error, to avoid leaking object existence.
- Team role flags are normalized as owner > admin > editor > viewer. Owner/admin imply edit; any membership row implies read.
- File/project/comment checks are implemented in the owning command namespaces, often via helpers imported from `files`, `teams`, or `projects`; do not bypass those helpers with direct DB lookups unless preserving their not-found semantics.
- Comment permission includes both logged-in state and the file/team comment policy. Shared viewer paths may pass `share-id`; preserve that path when changing comment queries.
## Teams, projects, and invitations
- Team/project commands mix DB changes, email, message bus notifications, media/storage cleanup, feature flags, quotas, and audit metadata. Keep mutations transactional when the existing command does so.
- Invitation flows validate muted/bounced emails before sending and use tokenized invitation state. Accepting an invitation is tied to the invited member identity, not just possession of a token.
- Logical deletion is used for many product objects; prefer existing logical-deletion helpers over hard deletes unless the command already performs permanent cleanup.
- Bounced/spam-complaint emails can mute/block a profile for login/registration and email sending. Devenv MailCatcher is the normal local path for registration/email-flow testing.
## Comments, webhooks, and audit
- Comment thread queries join file/project/profile state and exclude deleted files/projects. Unread comment counts depend on `comment_thread_status.modified-at` and profile notification preferences.
- Webhook edits are allowed for team editors/admins or the webhook creator. Webhook validation performs a synchronous HEAD request with a short timeout; validation errors are mapped through `app.loggers.webhooks`.
- Audit events are prepared from RPC metadata, result metadata, params, request context, and selected auth identifiers. Webhook event batching can be controlled through audit/webhook metadata on commands or results.
- Webhook and audit logging are cross-cutting side effects of product commands; when adding a command, check nearby command metadata and result metadata patterns before inventing a new event shape.
## Local testing notes
- Enable LDAP login locally with frontend flag `enable-login-with-ldap`; the devenv includes a configured test LDAP service.
- OIDC testing requires external provider app credentials plus matching backend/frontend config.
- Backend domain tests usually live under `backend/test/backend_tests/rpc/commands/*_test.clj` or nearby backend test namespaces. Use focused `clojure -M:dev:test --focus ...` from `backend/` when possible.
- For auth/session or HTTP behavior, combine backend tests with the HTTP/session notes in `mem:backend/http-storage-filedata-subtleties` because RPC-level tests may not exercise cookie/header transforms.
-107
View File
@@ -1,107 +0,0 @@
# Backend Architecture and Workflow
Backend: JVM Clojure; Integrant; PostgreSQL; Redis/Valkey; RPC; HTTP; storage; mail; audit/logging; workers.
## Focused memories
- RPC, DB helpers, workers, cron: `mem:backend/rpc-db-worker-subtleties`
- Storage abstraction, logical buckets, object lifecycle, deduplication, access, and garbage collection: `mem:backend/storage`.
- HTTP sessions, config, media processing, and file data persistence: `mem:backend/http-storage-filedata-subtleties`.
- Embedded Ladybug graph experiment, projection, incremental sync, console, and risks: `mem:backend/graph-experiment`
- Auth flows, permission model, teams, projects, invitations, comments, webhooks, audit: `mem:backend/auth-permissions-product-domains`
- Services, task-queue/Pub-Sub topology constraints -> `mem:prod-infra/core`.
## Stable namespace map
- `app.rpc.commands.*`: RPC command implementations exposed under `/api/rpc/command/<cmd-name>`.
- `app.rpc.permissions`: permission predicate/check helper factories.
- `app.http.*`: HTTP routes and middleware.
- `app.auth.*`: provider-specific authentication helpers such as LDAP/OIDC.
- `app.loggers.*`: audit, webhook, database, and external log integrations.
- `app.db.*` / `app.db`: next.jdbc wrapper and SQL helpers.
- `app.tasks.*`: background task handlers.
- `app.worker`: task execution/cron plumbing.
- `app.main`: Integrant system map and component wiring.
- `app.config`: `PENPOT_*` env config and feature flags.
- `app.srepl.*`: development REPL helpers for manual backend operations (data inspection, migration helpers, one-off admin tasks).
- `app.nitrate`, `app.rpc.commands.nitrate`, and `app.rpc.management.nitrate`: external Nitrate subscription/organization integration, gated by the `:nitrate` feature flag and shared-key HTTP calls.
## RPC conventions
RPC commands are defined with `app.util.services/defmethod` and schemas. Use `get-` prefixes for read operations. Command metadata usually includes auth, docs version, params schema, and result schema. Return plain maps/vectors or raise structured exceptions from `app.common.exceptions`.
Backend RPC command areas without focused memories include access tokens, binfile, demo, feedback, file snapshots, fonts, management, Nitrate, and webhooks beyond the notes in `mem:backend/auth-permissions-product-domains`; inspect nearby command tests and command metadata before changing them.
## DB conventions
`app.db` helpers accept cfg, pool, or conn in most places and convert kebab-case to snake_case:
- `db/get`, `db/get*`, `db/query`, `db/insert!`, `db/update!`, `db/delete!`.
- Use `db/run!` for multiple operations on one connection.
- Use `db/tx-run!` for transactions.
Database migrations live in `backend/src/app/migrations/`; pure SQL migrations are under `backend/src/app/migrations/sql/`. SQL filenames conventionally start with a sequence and verb/table description, e.g. `0026-mod-profile-table-add-is-active-field`. Applied migrations are tracked in the `migrations` table.
For interactive PostgreSQL access with correct dev defaults, use `scripts/psql`; to dump
the current DDL schema, use `scripts/db-schema` (see `mem:scripts/psql`).
For deeper details on transaction semantics, advisory locks, Transit vs JSON helpers, and dev/test DB URLs: `mem:backend/rpc-db-worker-subtleties`.
## Background tasks
A task handler is an Integrant component with `ig/assert-key`, `ig/expand-key`, and `ig/init-key`, returning the function run by the worker. New tasks also need wiring in `app.main`: handler config, worker registry entry, and cron entry if scheduled.
For worker dispatch, cron, retry semantics, deduplication, and queue internals: `mem:backend/rpc-db-worker-subtleties`.
## REPL
In devenv, backend nREPL is exposed on port 6064.
### Non-interactive eval (preferred for agents)
`./scripts/nrepl-eval.mjs` connects to an already-running nREPL server and evaluates code. Session state (defs, `in-ns`) persists across invocations via a stored session ID in `/tmp/penpot-nrepl-session-<host>-<port>`.
```bash
./scripts/nrepl-eval.mjs '(+ 1 2)' # single expression
./scripts/nrepl-eval.mjs "(require '[my.ns :as ns] :reload)" # reload after edits
./scripts/nrepl-eval.mjs -e # inspect last exception (*e)
./scripts/nrepl-eval.mjs --reset-session '(def x 0)' # discard session, start fresh
./scripts/nrepl-eval.mjs <<'EOF' # multi-expression heredoc
(def x 10)
(+ x 20)
EOF
```
Default port is 6064. Use `-p <PORT>` for a different port. Use `-t <MS>` to override the 120s timeout. Do not start the nREPL server — assume it is already running.
### Interactive REPL
`backend/scripts/nrepl` starts a REPLy client connected to the running nREPL.
For an in-process backend REPL (where you control the JVM lifecycle), stop the running backend first so port 9090 is free, then run `backend/scripts/repl`. Useful top-level helpers include `(start)`, `(stop)`, `(restart)`, `(run-tests)`, and `(repl/refresh-all)`. Many `app.srepl.main` helpers accept the global `system` var, e.g. manual email or maintenance operations.
## Fixtures
Fixtures can populate local data for manual testing/perf work. From the backend REPL, run `(app.cli.fixtures/run {:preset :small})`; fixture users conventionally look like `profileN@example.com` with password `123123`. Standalone fixture aliases may exist, but check current `backend/deps.edn` before relying on old command names.
## Performance
* **Type Hinting:** Use explicit JVM type hints (e.g. `^String`, `^long`) in performance-critical paths to avoid reflection overhead.
* **Batch inserts:** Use `db/insert-many!` for bulk row inserts — generates a single SQL with multiple parameter tuples. Avoid on very large datasets (SQL length / parameter count limits).
* **Server-side cursors:** Use `db/plan` (fetch-size 1000, forward-only, read-only) or `db/cursor` for large result sets. Never fetch large collections into memory at once.
* **Transaction discipline:** Use `tx-run!` for writes (opens a transaction), `run!` for reads (single connection, no transaction). Set `:read-only` on `tx-run!` when applicable to let PostgreSQL optimize.
## Lint and Format
IMPORTANT: all CLI commands must be executed from the `backend/` subdirectory.
* **Linting:** `pnpm run lint:clj`.
* **Formatting:** `pnpm run check-fmt:clj` to check, `pnpm run fmt:clj` to fix. After running `fmt:clj`, `check-fmt:clj` is redundant. Avoid unrelated whitespace diffs.
**Before linting:** if delimiter errors are suspected (after LLM edits), run
`scripts/paren-repair` on the affected files first. Delimiter errors produce
misleading linter/compiler output. See `mem:scripts/paren-repair`.
## Testing
Backend test commands, coverage rules, and conventions: `mem:backend/testing`.
Cross-cutting testing principles, anti-patterns, and verification checklist: `mem:testing`.
@@ -1,632 +0,0 @@
# Graph Experiment
## Scope
- Purpose: project Penpot file data into an embedded Ladybug graph database.
- Purpose: keep the graph current with Penpot file changes.
- Purpose: expose a read-only graph console for backend debugging.
- This is an experiment, not a replacement for PostgreSQL file storage.
- The graph subsystem is off unless `:graph` is in the backend flags.
- The main Penpot frontend has no graph feature code for this subsystem.
- The graph console is a backend-served HTML template with JavaScript.
## Memory Links
- Read `mem:backend/core` for backend architecture, HTTP routes, DB rules, and test commands.
- Read `mem:backend/rpc-db-worker-subtleties` for RPC and message bus behavior.
- Read `mem:backend/http-storage-filedata-subtleties` for file data loading and realization.
- Read `mem:common/changes-architecture` for the change record vocabulary.
- Read `mem:frontend/routing-app-shell-subtleties` for the existing notification WebSocket.
- Read `mem:prod-infra/core` for Redis or Valkey message bus topology.
## Branch Surface
- The graph experiment adds about 6,336 lines and changes about 27 files.
- The graph implementation lives under `backend/src/app/graph/`.
- The graph console lives at `backend/resources/app/templates/graph-console.tmpl`.
- The existing debug page gains graph links in `backend/resources/app/templates/debug.tmpl`.
- The existing debug HTTP routes gain graph handlers in `backend/src/app/http/debug.clj`.
- The backend system passes the message bus to the debug route component in `backend/src/app/main.clj`.
- The backend adds Ladybug and Arrow dependencies in `backend/deps.edn`.
- The backend adds JVM options for Ladybug and Arrow native access.
- The common flag registry adds `:graph` in `common/src/app/common/flags.cljc`.
- The graph experiment adds `graph_sync_parity_test.clj` and `graph_binder_gate_test.clj`.
## System Model
### Storage layers
- PostgreSQL remains the source of truth for Penpot files.
- The graph database stores a projection of one file.
- A persistent graph uses a `.lbug` path under `PENPOT_GRAPH_DIR`.
- The default graph directory is `/tmp/penpot-graph`.
- A debug session uses a Ladybug `:memory:` database.
- A debug session database lives inside the backend JVM process.
- A debug session does not survive a backend restart.
- A debug session does not store file data back to PostgreSQL.
### Two graph update paths
- Cold projection reads the complete file and rebuilds the graph.
- Incremental sync reads file change records and updates the open graph.
- Both paths must produce the same graph for the same file state.
- The parity test treats cold projection as the reference path.
- A reload discards the session graph and uses cold projection again.
## Main Namespaces
### `app.graph.ladybug`
- Opens and closes Ladybug `Database` and `Connection` objects.
- Installs and loads the Ladybug JSON extension.
- Executes Cypher statements.
- Executes prepared statements.
- Binds scalar parameters.
- Formats UUID, string, integer, number, JSON, and timestamp values.
- Formats compound values such as arrays, maps, and structs.
- Converts Ladybug values back to Clojure values.
- Limits normal query results to 200 rows by default.
- Detects result truncation with `:truncated?`.
- Uses query timeout `0` by default.
- Query timeout `0` disables the timeout.
- Provides `validate-on-connection!` for parse, bind, and read-only checks.
- `exec-prepared-on-connection!` prepares every statement before the first execution.
- A prepare failure stops the batch before a mutation runs.
### `app.graph.schema`
- Provides the public schema facade.
- Exposes schema version `penpot-graph-slice-4`.
- Delegates node and relationship definitions to `app.graph.schema.nodes`.
### `app.graph.schema.nodes`
- Holds the single registry for graph node tables.
- Generates node DDL.
- Generates relationship DDL.
- Maps Penpot shape types to graph tables.
- Projects source attributes into graph attributes.
- Formats graph column values.
- Quotes reserved graph labels such as `Group` and `Boolean`.
- Defines container tables and shape tables.
- Defines `IsChildOf`, `IsInstanceOf`, `RefersTo`, and `FillsSwapSlot`.
### `app.graph.schema.contract`
- Records deliberate graph contract decisions.
- Renames graph columns such as `:revn` to `revision`.
- Drops attributes that do not belong in this graph slice.
- Records attributes that the graph does not project.
- Applies per-table dropped attributes.
- Defines type overrides for vectors, transforms, colors, maps, and JSON arrays.
- Maps selected map keys to the frontend JSON naming convention.
- `:background-blur` remains a declared unprojected attribute.
### `app.graph.schema.projection`
- Derives projected schemas from canonical Malli schemas.
- Builds the projected document schema.
- Builds projected shape schemas.
- Selects the schema for each shape type.
### `app.graph.schema.types`
- Maps Malli types to Ladybug types.
- Maps matrices to `DOUBLE[6]`.
- Maps points to `DOUBLE[2]`.
- Maps rectangles to `DOUBLE[4]`.
- Maps colors to `UINT32`.
- Maps collections to Ladybug arrays.
- Maps `:map-of` schemas to `MAP`.
- Maps closed scalar maps to `STRUCT`.
- Maps other complex values to `JSON`.
### `app.graph.schema.values`
- Coerces source values to graph column values.
- Writes fixed vectors with deterministic order.
- Packs colors into the graph color representation.
- Sorts set values when deterministic output is needed.
### `app.graph.arrow`
- Loads projection rows with Apache Arrow.
- Creates temporary staged node and relationship tables.
- Uses `COPY ... FROM (MATCH ...)` for bulk loading.
- Groups relationship loads by source and target table pair.
- Resolves relationship endpoints with joins.
- Does not use `createArrowRelTable` for UUID relationship endpoints.
- Keeps the Arrow `RootAllocator` alive until Ladybug releases staged buffers.
- Closes the allocator after the connection and database close sequence.
### `app.graph.ingest`
- Fetches a complete file with `bfc/get-file` and `:realize? true`.
- Rejects missing files.
- Rejects files without file data.
- Can run file data validation before projection.
- Creates the DDL.
- Loads nodes and edges through Arrow.
- Executes post-load transforms.
- Writes graph metadata last.
- Treats the final metadata write as the complete-build marker.
- Supports a persistent database path and an open connection.
### `app.graph.projection.document`
- Projects `Document`, `Page`, `Component`, and supported shape nodes.
- Skips the page root frame.
- Creates `IsChildOf` edges from shapes to parents.
- Creates page edges to the document.
- Creates component edges to the document.
- Stores page order in `Page.index` and edge `position`.
- Reverses the stored `:shapes` list for Penpot z-order.
- Adds `page-id` to every projected shape.
- Propagates an instance head `component-id` to descendants.
- Stops component inheritance at a non-Frame shape with its own component ID.
- Skips deleted components during cold projection.
- Logs unsupported shape types and missing shape records.
### `app.graph.projection.transforms`
- Runs after the base nodes and edges load.
- `link-component-instances` creates `IsInstanceOf` edges.
- A Frame needs `component-file` to qualify as an instance head.
- `link-shape-refs` creates `RefersTo` edges from `shape-ref`.
- Ladybug limits multi-label relationship `MERGE` statements.
- The transform emits one statement for each shape-table pair.
- `link-swap-slots` creates `FillsSwapSlot` edges.
- Swap slot IDs come from `swap-slot-<uuid>` entries in `touched`.
- The transform removes swap slot entries from `touched` after edge creation.
- The transform order matters because it reads and then changes `touched`.
### `app.graph.meta`
- Stores graph provenance in `GraphMeta`.
- Stores schema version, source revision, producer, and build time.
- The source revision identifies the file revision used for cold projection.
### `app.graph.stats` and `app.graph.report`
- `app.graph.stats` counts graph nodes and relationships from the live catalog.
- `app.graph.report` prints ingest information for REPL use.
## Cold Projection Flow
1. Get the file row and realized file data from PostgreSQL.
2. Read the file revision from the file row.
3. Build the node and edge projection.
4. Create all graph tables from the graph schema.
5. Load node rows with Arrow.
6. Load relationship rows with Arrow.
7. Run `CHECKPOINT;`.
8. Run the registered derived transforms.
9. Write `GraphMeta` as the final build step.
10. Return file ID, file revision, database path, projection stats, and transform stats.
### Projection node groups
- `Document` contains file-level attributes without the file data blob.
- `Document.options` receives file-level options from the data blob.
- `Page` contains page attributes without the page object map.
- `Component` contains component attributes without component object maps.
- Shape tables contain the supported shape attributes.
- The graph stores selected derived attributes such as `page-id`.
### Projection relationship groups
- Structural edges use `IsChildOf`.
- Page and component edges point to `Document`.
- Derived edges come from the post-load transform registry.
## Incremental Sync
### Change source
- `app.rpc.commands.files-update` persists the file update first.
- The same command publishes a `:file-change` message to the file topic.
- The topic key is the file UUID.
- The message contains the file ID, profile ID, session ID, revision, version, and changes.
- Library changes also publish a team-topic message.
- The graph session only consumes the file-topic `:file-change` messages.
### Session subscription
- `app.graph.debug/start-sync-loop!` creates a channel with a dropping buffer of 64.
- The session subscribes the channel to the file UUID topic.
- The loop reads one message at a time.
- The loop ignores message types other than `:file-change`.
- The loop stops when the channel closes.
- `destroy-session!` closes the channel and purges its message bus subscription.
### Session state
- Sessions are stored in a global `defonce` atom.
- The map key is the string form of `profile-id`.
- One profile has one graph session.
- Loading another file first destroys the old session.
- A session stores the Ladybug database and connection.
- A session stores a shared lock for graph access.
- A session stores file metadata.
- A session stores the incremental sync index.
- A session stores the message bus channel.
- A session stores load time and profile ID.
- The session keeps projection statistics but drops full projection rows after index creation.
### Sync index
- `build-index` starts from the complete cold projection.
- The index stores the graph file ID and document ID.
- The index stores the current graph revision.
- The index stores page IDs, names, and positions.
- The index stores component IDs, names, and deleted state.
- The index stores shape table, parent, position, frame, page, and component context.
- The index stores child IDs by parent ID.
- The index supports later change application without another PostgreSQL file read.
### Change application
- `apply-changes!` processes the change list in source order.
- Each supported change returns a new index and a list of Cypher statements.
- Unsupported changes enter the `:skipped` result.
- Supported changes enter the `:applied` result.
- The function collects all statements before it executes them.
- The function appends a document revision statement when at least one change applies.
- The index revision advances only when at least one change applies.
- A larger incoming revision than the index revision creates a warning.
- A revision gap does not trigger catch-up.
### Shape change rules
- `:add-obj` reuses `projection.document/denormalized-shape`.
- `:add-obj` creates the shape node and its parent edge.
- `:mod-obj` applies supported `:set` operations to graph columns.
- `:mod-obj` keeps false and zero values as values.
- `:del-obj` deletes shapes in deep post-order.
- `:mov-objects` detaches shapes from the old parent.
- `:mov-objects` closes the old sibling position gap.
- `:mov-objects` inserts shapes at the new position.
- `:mov-objects` updates `parent_id` and `frame_id`.
- `:mov-objects` rewrites container `shapes` values.
- The parent columns and child lists must match a cold projection.
### Page and component change rules
- Page add creates a projected page node and a document edge.
- Page delete removes the page subtree.
- Page modification updates supported page attributes.
- Component add creates a component node and document edge.
- Component modification updates supported component attributes.
- Component delete uses a soft-delete state.
- Component restore removes the soft-delete state.
- Component purge removes the component node and document edge.
- Component sync paths need more parity coverage than the current tests provide.
## Session Locking
- The sync loop and HTTP handlers share one lock per session.
- The lock protects one Ladybug connection from concurrent access.
- Queries acquire the lock before binder validation and execution.
- Graph data export acquires the lock before catalog reads.
- Session export acquires the lock before `EXPORT DATABASE`.
- A long query blocks sync for the same session.
- A sync batch blocks queries for the same session.
- Ladybug connection thread safety is not assumed.
## Graph Query Rules
- The console accepts Cypher text.
- Blank query text raises a validation error.
- The query first passes Ladybug prepare and bind checks.
- The query must pass the engine read-only analysis.
- A mutating query is rejected.
- The graph console does not provide a write path.
- A session graph is rebuilt from the file by Reload.
- Normal query results have a 200-row limit.
- Query results use string values for the HTML console representation.
- JSON requests receive a Transit JSON response with the query and result.
- HTML requests receive the rendered console with the result.
## Graph Data Export
### G6 data
- `/dbg/actions/graph-data` reads the live Ladybug database.
- It does not read the sync index for nodes and edges.
- It therefore shows database drift if a batch fails after index update.
- Node export covers all registered node tables.
- Relationship export reads the Ladybug relationship catalog.
- Relationship export includes source, target, relationship name, and position.
- Node and relationship export uses a 100,000-row limit.
- The response reports `truncated` when a limit cuts the result.
- The response reports buffer-manager memory usage.
### `.lbug` export
- `source=file` rebuilds the persistent graph from PostgreSQL file data.
- `source=file` runs a synchronous full ingest for each request.
- `source=session` exports the caller profile's live in-memory graph.
- Session export uses Ladybug `EXPORT DATABASE` to Parquet files.
- Session export creates a new `.lbug` database with `IMPORT DATABASE`.
- The temporary Parquet staging directory is deleted after import.
- The final session `.lbug` file remains in the system temporary directory.
- The HTTP response streams the database file to the caller.
## HTTP Routes and Access
- The graph routes live in `backend/src/app/http/debug.clj`.
- The graph route list is added only when `:graph` is enabled.
- `/dbg/graph` serves the graph console page.
- `/dbg/actions/graph-files` returns the profile file tree.
- `/dbg/actions/graph-load` loads a file into the profile session.
- `/dbg/actions/graph-unload` closes the profile session.
- `/dbg/actions/graph-reload` rebuilds the loaded file graph.
- `/dbg/actions/graph-query` runs a read-only Cypher query.
- `/dbg/actions/graph-sync-status` returns the sync state.
- `/dbg/actions/graph-data` returns nodes and edges for G6.
- `/dbg/actions/graph-export` streams a `.lbug` database.
- The `/dbg` session middleware remains active.
- The `/dbg` admin middleware remains active.
- A devenv host with a profile ID passes the debug authorization rule.
- Other hosts need a profile email in the configured admin set.
- `/dbg/actions/graph-files` lists reachable teams, projects, and files.
- The file tree query has a 500-file limit.
- The graph handlers resolve graph namespaces at call time.
- The backend requires `app.graph.debug` and `app.graph.ingest` when the flag is on.
- Ladybug native loading then fails during route initialization instead of first use.
## Console Frontend
### Page type
- `graph-console.tmpl` is a backend resource template.
- It is not a Rumext component.
- It is not part of the main frontend route table.
- The page uses browser `fetch` calls and a browser WebSocket.
- The page loads G6 version `5.1.1` from jsDelivr.
### File tree
- The page fetches `/dbg/actions/graph-files`.
- The response contains team, project, and file groups.
- The page creates the tree with DOM APIs.
- A file click submits the graph load form.
- The page shows a message when no file exists.
### Graph rendering
- The page fetches `/dbg/actions/graph-data`.
- The page converts graph nodes and edges to G6 data.
- The page skips repaint when the node and edge signature does not change.
- The page marks added, removed, and changed graph entities.
- The page supports tree, dagre, circular, force, and combo layouts.
- The page supports collapsed container combos.
- The page has render guards at 4,000 nodes and 8,000 edges.
- The `?safe` query option bypasses the render guard.
- The page shows graph size by node count and relationship count.
- The page shows buffer-manager memory in MiB.
- The page reports a CDN failure when G6 is undefined.
### Query result filtering
- A query can return `filter_*` columns with node IDs.
- The HTML result table hides columns with the `filter_` prefix.
- The JSON result keeps the full result.
- The graph view uses the hidden IDs to select matching nodes.
- The graph view re-runs the query after graph refresh.
- This keeps the query filter aligned with the current graph.
- A user column named `filter_*` follows the same hiding rule.
### Node inspector
- A node click creates a query for that node.
- The inspector calls `/dbg/actions/graph-query` with JSON negotiation.
- The inspector displays the full projected row.
- The inspector uses table and ID values from the graph data.
## WebSocket Data Flow
1. The page opens `/ws/notifications` with a random `session-id` query value.
2. The page sends `:subscribe-file` with a Transit UUID value.
3. The server makes sure that the file exists and that the profile has read permission.
4. The server subscribes the connection to the file topic.
5. `files_update` publishes `:file-change` to the same topic.
6. The graph session consumes the message from its message bus subscription.
7. The WebSocket server sends the message to the browser connection.
8. The browser adds the change to the changelog.
9. The browser fetches sync status after 150 milliseconds.
10. The browser fetches graph data after a 400-millisecond debounce.
11. The browser repaints the G6 graph when the graph data changes.
### WebSocket reconnect behavior
- The page reconnects after three seconds when the socket closes.
- The page resubscribes to the file after the socket opens.
- The page refreshes sync status after reconnect.
- The page refreshes graph data after reconnect.
- Reconnect does not recover dropped message-bus changes.
- The page shows the sync error or skipped-change state when the status reports it.
## Feature Flag and Runtime Dependencies
- `:graph` is defined in `common/src/app/common/flags.cljc`.
- The flag is off by default.
- `com.ladybugdb/lbug` version `0.19.1` is a backend dependency.
- `org.apache.arrow/arrow-memory-netty` version `18.2.0` supports Arrow `RootAllocator`.
- The JVM uses `--enable-native-access=ALL-UNNAMED`.
- The JVM uses `--add-opens=java.base/java.nio=ALL-UNNAMED`.
- The JVM uses `--sun-misc-unsafe-memory-access=allow`.
- The JVM options appear in the development alias and backend launch scripts.
- A Ladybug version change needs new binder and parity tests.
- A JDK version change needs a startup test with the graph flag enabled.
## Tests
### `backend-tests.graph-sync-parity-test`
- Uses two Ladybug `:memory:` databases.
- Does not use PostgreSQL or a live graph session.
- Projects initial file data into database A.
- Applies changes to database A through incremental sync.
- Applies the same changes to file data.
- Projects the changed file data into database B.
- Compares every node row and relationship row.
- Reports differences by table, row key, and column.
- Covers shape add, shape modification, shape deletion, movement, and page changes.
- Contains a test that injects a sync defect and expects a graph difference.
- Does not cover all component change variants.
- Does not cover every movement insertion mode.
### `backend-tests.graph-binder-gate-test`
- Creates the live graph DDL in a Ladybug `:memory:` database.
- Prepares each sync statement template without executing it.
- Detects parse errors and missing tables.
- Detects missing columns and bad label quoting.
- Reports the expected read-only classification.
- Covers reserved node labels across the node registry.
- Reports an error result for an invalid statement.
### Test gaps
- No automated HTTP handler tests cover graph routes.
- No automated session lifecycle tests cover load and unload.
- No automated WebSocket tests cover graph subscription.
- No automated export tests cover persistent and session sources.
- Component add, modify, delete, restore, and purge need parity tests.
- Page delete needs parity coverage.
- Movement with `:after-shape` needs parity coverage.
- Buffer overflow and revision gap behavior need tests.
- Partial batch failure and recovery need tests.
- Query timeout and long-query behavior need tests.
## Known Risks and Limits
### Dropped changes
- The sync channel uses a dropping buffer of 64.
- A burst can discard file-change messages.
- The sync loop logs a revision gap when it sees a larger revision.
- The sync loop does not fetch missing rows from `file_change`.
- Reload is the only built-in recovery path.
### Partial batch state
- `apply-changes!` does not provide Ladybug transaction atomicity.
- A statement failure can leave a partly changed graph.
- The in-memory index can advance before the database state is complete.
- `/dbg/actions/graph-data` reads the database and exposes this drift.
- Reload rebuilds the graph from PostgreSQL file data.
### Query resource use
- The default session query timeout is zero.
- A costly query can hold the session lock for a long time.
- The same lock blocks incremental sync.
- The graph export also holds the same lock during catalog reads.
- The graph schema has a high memory floor.
- The console reports about 115 MiB for the wide slice before file data.
### Session lifecycle
- Sessions have no TTL.
- Sessions remain until unload, replacement, or process shutdown.
- Each session owns native Ladybug memory.
- Many profiles can create many native databases.
- A profile load replaces its previous session.
- Two browser tabs for one profile share one graph session.
### Temporary files
- Session export leaves the final `.lbug` file in the system temporary directory.
- Long-lived servers can accumulate exported session databases.
- The staging directory is deleted after import.
### Browser dependency
- The graph view depends on a runtime CDN request.
- A network restriction can remove the G6 view.
- Queries and session status still use backend endpoints without G6.
### Data exposure
- The graph console can list many files available to the profile.
- The console can load complete projected file data.
- The console can export a graph database.
- The console can inspect all projected node attributes.
- The console is safe only when the `/dbg` access boundary is correct.
- The graph flag must remain off for deployments that do not need this tool.
### Contract drift
- The graph schema is a deliberate slice of the Penpot file model.
- New source attributes do not enter the graph automatically in all cases.
- Dropped and unprojected attributes need an explicit contract decision.
- `applied_tokens` key mapping depends on the JSON naming function.
- `filter_*` is a frontend convention, not a graph schema guarantee.
### Ladybug dialect coupling
- Cypher strings contain Ladybug-specific syntax.
- Label quoting handles reserved labels explicitly.
- Relationship transforms depend on Ladybug relationship limits.
- Arrow loading depends on Ladybug `COPY FROM (MATCH ...)` behavior.
- A dependency upgrade needs schema, binder, Arrow, and parity checks.
## REPL Helpers
- `app.srepl.main` resolves graph functions only when a helper runs.
- `graph-smoke-test!` runs a basic Ladybug operation.
- `graph-query-test!` runs a graph query test.
- `ingest-file-to-graph!` projects a file into a graph database.
- These helpers use `requiring-resolve` to keep the graph dependency lazy.
## Operational Invariants
- PostgreSQL file data remains authoritative.
- Cold projection and incremental sync must produce equal graph state.
- The graph revision must identify the last applied file revision.
- The document revision must update when a sync batch applies.
- A missing or skipped change must remain visible in sync status.
- A graph query from the console must be read-only.
- A graph session must serialize connection access.
- Graph routes must remain behind the `:graph` flag and `/dbg` access control.
- The Arrow allocator must outlive all Ladybug operations that use its buffers.
- `GraphMeta` must be written after the full ingest and transforms finish.
## Key Files
- `backend/src/app/graph/ladybug.clj`: Ladybug API and query gates.
- `backend/src/app/graph/arrow.clj`: Arrow bulk load.
- `backend/src/app/graph/ingest.clj`: Complete file ingest.
- `backend/src/app/graph/debug.clj`: Session lifecycle, sync loop, query, and export.
- `backend/src/app/graph/sync.clj`: Incremental change application.
- `backend/src/app/graph/meta.clj`: Graph provenance.
- `backend/src/app/graph/stats.clj`: Graph counts.
- `backend/src/app/graph/report.clj`: REPL ingest report.
- `backend/src/app/graph/projection/document.clj`: Base document projection.
- `backend/src/app/graph/projection/transforms.clj`: Derived relationship transforms.
- `backend/src/app/graph/schema/nodes.clj`: Node and relationship registry.
- `backend/src/app/graph/schema/contract.clj`: Projection contract decisions.
- `backend/src/app/graph/schema/projection.clj`: Malli projection schemas.
- `backend/src/app/graph/schema/types.clj`: Malli-to-Ladybug type mapping.
- `backend/src/app/graph/schema/values.clj`: Value coercion.
- `backend/src/app/http/debug.clj`: Graph route registration and handlers.
- `backend/src/app/http/websocket.clj`: File WebSocket subscription handlers.
- `backend/src/app/rpc/commands/files_update.clj`: File-change publication.
- `backend/src/app/main.clj`: Integrant message bus wiring.
- `backend/resources/app/templates/graph-console.tmpl`: Graph console browser code.
- `backend/resources/app/templates/debug.tmpl`: Debug page graph links.
- `common/src/app/common/flags.cljc`: `:graph` feature flag.
- `backend/test/backend_tests/graph_sync_parity_test.clj`: Cold versus sync parity.
- `backend/test/backend_tests/graph_binder_gate_test.clj`: Cypher binder gate.
## Development Commands
- Run backend commands from the `backend/` directory.
- Run focused parity tests with `clojure -M:dev:test --focus backend-tests.graph-sync-parity-test`.
- Run focused binder tests with `clojure -M:dev:test --focus backend-tests.graph-binder-gate-test`.
- Run the backend test suite with `clojure -M:dev:test`.
- Examine Clojure formatting with `pnpm run check-fmt:clj`.
- Run backend Clojure lint with `pnpm run lint:clj`.
- Write test output to a file before reading or filtering it.
@@ -1,28 +0,0 @@
# Backend HTTP, Storage, Media, and File Data Subtleties
## Config and HTTP/session middleware
- `app.config/config` and `flags` are dynamic `defonce` vars populated from `PENPOT_*` env vars through the shared schema string transformer. Tests and tooling can bind them.
- `parse-flags` automatically adds `:disable-secure-session-cookies` when `public-uri` is plain HTTP and not localhost. This changes cookie defaults without an explicit env flag.
- The backend sets Clojure `*assert*` globally from the `:backend-asserts` feature flag. Assertion-dependent checks can therefore differ by runtime flags.
- Request body parsing is mostly POST-oriented and supports Transit JSON plus plain JSON. Plain JSON request keys are kebab-decoded before being merged into `:params`.
- Response formatting negotiates with `Accept` or `_fmt=json`. Transit is the default for collection/boolean bodies; JSON encoding has special pointer-map handling.
- Auth prefers the session cookie token before the `Authorization` header. Headers may be `Token` or `Bearer`; JWTs with `kid=1` and `ver=1` are decoded as v1 session tokens, otherwise they are treated as legacy tokens.
- Shared-key auth requires `x-shared-key` as `<key-id> <key>` and stores the lowercased key id on the request. If no shared keys are configured it always rejects.
- Session management uses DB storage unless the DB pool is read-only, then falls back to the in-memory manager. DB sessions support both legacy string ids and v2 UUID session ids.
- Session cookies are renewed when using a legacy string id or when `modified-at` is older than the renewal interval. SameSite is `none` for CORS, otherwise strict/lax based on config.
## Storage and media
- Storage abstraction, backend configuration, logical buckets, object lifecycle, deduplication, access rules, and garbage collection: `mem:backend/storage`.
- SVG validation strips DOCTYPE and uses secure SAX parsing. Basic SVG info falls back to 100x100 dimensions when width/height/viewBox are missing.
- Raster metadata is shell-derived with ImageMagick `identify`, verifies detected MIME against the supplied MIME, and swaps dimensions for EXIF orientations 6/8.
- Remote image download requires 2xx status, `content-length`, a known MIME, and size under the configured maximum before writing the temp file; mismatched byte count is an internal error.
- Font processing shells out to FontForge and WOFF conversion tools and can derive TTF/OTF/WOFF variants from uploaded fonts.
## File data persistence
- File data backends are `legacy-db`, `db`, and `storage`. The storage backend keeps encoded file data in storage bucket `file-data`; the DB row stores metadata with `storage-ref-id` and nil data.
- `fdata/upsert!` touches any storage object referenced by incoming metadata before storing the new row/blob.
- Pointer-map fragments are persisted separately as type `fragment`, and only modified pointer maps are written.
- `fdata/realize` combines pointer realization and object-map realization. Use it before operations that need complete in-memory file data instead of pointer placeholders.
@@ -1,26 +0,0 @@
# Backend RPC/DB/Worker Subtleties
## RPC exposure and wrappers
- RPC commands are discovered from vars created by `app.util.services/defmethod`; adding a command namespace is not enough unless `backend/src/app/rpc.clj` includes it in `resolve-methods`.
- `GET`/`HEAD` RPC calls are only allowed for method names starting with `get-`. Other methods are method-not-allowed even if they are read-only internally.
- RPC auth defaults to enabled. Public endpoints must set `::auth false` metadata explicitly.
- The wrapper stack does auth before params validation, then auditing/rate/concurrency/metrics/retry/condition handling, with DB transaction handling inside that stack. `::db/transaction` metadata controls transaction wrapping.
- Params with `::sm/params` are decoded/conformed through the JSON transformer and successful IObj results get `:encode/json` metadata. Legacy spec conforming only applies when no Malli params schema exists.
- Nil RPC bodies become HTTP 204 unless explicit status metadata is present. Stream bodies default to `application/octet-stream` when no content type is set.
## DB helpers
- Most `app.db` helpers accept a pool, connection, or map containing `::db/pool` / `::db/conn`; preserve that convention in shared code.
- `db/tx-run!` uses `next.jdbc.transaction/*nested-tx* :ignore`: nested transaction calls reuse the outer transaction, not a savepoint. Use explicit savepoints when nested rollback semantics matter.
- `db/run!` opens/reuses one connection but does not create a transaction.
- `db/tjson` is Transit JSON for jsonb storage; `db/json` is plain JSON. Worker task props use Transit and are decoded with `decode-transit-pgobject`.
- Advisory transaction locks accept UUIDs or ints. UUID locks are hashed using a zero-UUID seeded siphash.
## Workers and cron
- Task queues are tenant-prefixed. Submit dedupe only removes not-yet-due `new` tasks with the same name/queue/label; it does not dedupe due, scheduled, retry, running, or completed work.
- The dispatcher selects `new`/`retry` tasks with `FOR UPDATE SKIP LOCKED`, marks them `scheduled`, and publishes Redis payload `[id scheduled-at]`. The runner skips Redis messages whose scheduled timestamp no longer matches DB state.
- Lost `scheduled` tasks are rescheduled after about 5 minutes; `running` tasks older than about 24 hours are marked failed as orphans.
- A task handler that is missing or returns an invalid result currently defaults to completed after warning. Throwing with `ex-data :type ::retry` controls retry behavior; `:strategy ::noop` retries without incrementing retry count.
- Cron jobs lock their `scheduled_task` row with `FOR UPDATE SKIP LOCKED`, disable statement/idle-in-transaction timeouts locally, and reschedule themselves in `finally` unless interrupted. Worker, dispatcher, and cron components do not start when the DB pool is read-only.
-119
View File
@@ -1,119 +0,0 @@
# Backend Storage
## Abstraction
- `app.storage` stores binary objects.
- Each object has a `storage_object` database row.
- The row stores the UUID, size, backend, timestamps, and Transit metadata.
- The backend stores the binary content.
- Supported backends are `:fs` and `:s3`.
- FS uses one root directory and a UUID-derived path.
- S3 uses one configured bucket and an optional prefix.
- A Penpot bucket is metadata. It is not an S3 bucket or a filesystem directory.
- FS and S3 use the same UUID-derived object path. The bucket does not change the path.
- `PENPOT_OBJECTS_STORAGE_*` configures the current object backend.
- Deprecated asset-storage config keys remain supported for migration.
- Database rows keep the backend name. Keep the legacy `:assets-fs` and `:assets-s3` aliases.
## Object Lifecycle
- `put-object!` creates the database row before it writes backend content.
- Backend content is written only when the row is new.
- A failed backend write can leave an unreferenced database row.
- Callers often set `:touched-at` so garbage collection can remove such rows.
- `get-object` excludes rows with `deleted_at`.
- Existing object values can remain readable until physical deletion.
- `:expired-at` blocks reads after the expiration time.
- `del-object!` sets `deleted_at`. It does not remove backend content.
- `storage-gc-deleted` removes the database row and backend content after the deletion delay.
- `storage-gc-touched` finds references before it sets `deleted_at`.
- `objects-gc` removes deleted domain rows and touches their storage object IDs.
- Use `::db/reuse-conn true` with `sto/resolve` inside a database transaction.
## Connection Reuse Details
### `app.storage/resolve` patterns:
**1. Pool mode (default)** - `(sto/resolve cfg)`
- Returns storage abstraction from config
- Uses whatever database pool is available
- **Safe to call outside transaction context**
- Used in: `rpc/commands/media.clj:363`, `rpc/commands/auth.clj:327`, `rpc/commands/profile.clj:362`
**2. Connection reuse mode** - `(sto/resolve cfg ::db/reuse-conn true)`
- Internally calls `db/get-connection cfg` to obtain connectable
- Configures storage with the specific connection from config
- **Must be paired with transaction that owns this connection**
- Used in: `features/fdata.clj:100`, `rpc/commands/media.clj:425`, `rpc/commands/files_thumbnails.clj:307,319`, `binfile/v3.clj:722`
**3. Explicit configuration** - `(sto/configure storage conn)`
- Sets `::db/conn` on storage map directly
- Asserts `db/conn? connection` (storage.clj:349)
- Used inside `db/tx-run!` blocks where `conn` is already available
- Used in: `tasks/file_gc.clj:256`, `rpc/commands/files_thumbnails.clj:347,371`
### Key Warning (from function notes):
The improved note in `import-storage-objects` and `handle-persistence` warns:
**Do not reuse the main database connection for storage operations within a transaction.** The storage upload process can fail mid-operation, leaving orphaned objects on the backend. If the outer transaction aborts, pending storage objects become unreconciliable because the storage subsystem registers its pending state in separate transactions.
### Rule of Thumb for `sto/put-object!`:
Since `put-object!` uses backend-specific operations (`impl/resolve-backend` + `impl/put-object`) and does not directly use `::db/conn` or `::db/pool`, **all usage of `put-object!` will never run inside a common transaction** (if configured at all). The storage backend operations are independent of the database transaction boundary.
## Deduplication
- Deduplication requires `::sto/deduplicate?`, a content hash, and bucket metadata.
- The lookup matches hash, bucket, backend, and `deleted_at IS NULL`.
- The lookup only considers rows with `status='valid'`; pending rows are invisible.
- A hit whose blob is missing is repaired in place: the same row/id is kept,
and `put-object!` rewrites the blob under that id. This heals all existing
references to the object. If the rewrite fails, the row is left live and
valid for a later retry.
- The lookup does not include file ID, profile ID, team ID, or organization ID.
- Objects can therefore share content across users and files within one bucket.
- Deleted objects are not reused.
- `tempfile` objects never use deduplication, even when the caller requests it.
- Use `sto/wrap-with-hash` when the caller already calculated the content hash.
## Bucket Rules
| Bucket | Content and references | Dedup | Direct `/assets/by-id` access | Cleanup |
| --- | --- | --- | --- | --- |
| `file-media-object` | Original file images and generated media thumbnails. References: `file_media_object.media_id` and `thumbnail_id`. | Yes | Public | Reference scan. |
| `team-font-variant` | Font variants in `team_font_variant`. References: `woff1_file_id`, `woff2_file_id`, `otf_file_id`, and `ttf_file_id`. | Yes | Public | Reference scan. |
| `file-object-thumbnail` | Frame and component thumbnails in `file_tagged_object_thumbnail.media_id`. | Yes | Public | Reference scan. |
| `file-thumbnail` | File grid thumbnails in `file_thumbnail.media_id`. | Yes | Authentication required | Reference scan. |
| `profile` | User and team profile photos. References: `profile.photo_id` and `team.photo_id`. | Yes | Authentication required | Reference scan. |
| `organization` | Organization logos uploaded by the Nitrate management API. | Yes | Public | No reference scan. A touched object is deleted. |
| `tempfile` | Export files, chunked-upload chunks, and temporary font downloads. | No | Authentication required | No reference scan. A touched object uses a two-hour deletion delay. |
| `file-data` | Encoded file data when `file-data-backend` is `storage`. Reference metadata has `storage-ref-id`, `file-id`, and the `file_data` row ID. | Yes | Authentication required | Reference scan. |
| `file-data-fragment` | Compatibility value for file-data fragments. The current backend has no dedicated producer for this bucket. | No current write semantics | Public | No touched-object collector case. |
| `file-change` | Compatibility value for file changes. Current snapshots store data in `file_data`, not this bucket. | No current write semantics | Authentication required | No touched-object collector case. |
- The valid bucket set lives in `app.storage/valid-buckets`.
- `file-media-object` is the default bucket for old rows without bucket metadata.
- Do not assign a new bucket without adding its access and cleanup behavior.
- The touched-object collector raises an internal error for an unknown bucket.
- It supports `file-media-object`, `team-font-variant`, `file-object-thumbnail`, `file-thumbnail`, `profile`, `file-data`, `tempfile`, and `organization`.
- It does not support `file-data-fragment` or `file-change`.
## Access Rules
- `app.http.assets` decides direct object authentication from the bucket.
- Public buckets are `file-media-object`, `file-object-thumbnail`, `team-font-variant`, `file-data-fragment`, and `organization`.
- Other valid buckets require a session or access-token profile ID.
- File-media routes also require file read permission.
- Non-public direct responses set `content-disposition: attachment`.
- FS responses use `x-accel-redirect` for the configured asset path.
- S3 responses use a presigned URL and an HTTP redirect.
## File Data
- `file-data-backend` accepts `legacy-db`, `db`, or `storage`.
- `legacy-db` stores main data in `file.data` and snapshots in `file_change.data`.
- `db` stores encoded data in `file_data.data`.
- `storage` stores encoded data in storage subsystem with `file-data` bucket and keeps `data` nil in `file_data` table.
- The `file_data.metadata.storage-ref-id` value points to the storage object.
- `fdata/upsert!` touches a storage object from incoming metadata before it stores the new row.
- File snapshots use `file_data` for snapshot data and `file_change` for snapshot metadata.
-11
View File
@@ -1,11 +0,0 @@
# Backend Testing
JVM `clojure.test` (kaocha runner) under `backend/test/backend_tests/`.
- READ `mem:testing` FIRST — it defines the execution discipline (no piping, tee to file, preferred commands) that applies to all JVM test runs.
- All CLI commands must be executed from the `backend/` subdirectory.
- Tests are invoked directly via `clojure -M:dev:test` (kaocha) — there is no pnpm wrapper. Kaocha auto-discovers test namespaces, so no runner registration is needed.
- Coverage: if code is added or modified in `src/`, corresponding tests in `test/backend_tests/` must be added or updated.
- Isolated run: `clojure -M:dev:test --focus backend-tests.my-ns-test` for a specific test namespace, or `clojure -M:dev:test --focus backend-tests.my-ns-test/my-test-var` for a specific test var.
- Regression run: `clojure -M:dev:test` to ensure no regressions in related functional areas.
- If you need to filter output, tee to a temp file first: `clojure -M:dev:test 2>&1 | tee /tmp/penpot-test-output.txt`.
@@ -1,39 +0,0 @@
# File Mutations: Changes and Undo Architecture
Penpot mutates file data through change records. A change set is both the persistence payload and the basis for undo/redo, so UI actions, tests, backend file updates, and library/file tooling should drive the production change pipeline instead of ad hoc object-map mutation.
## Change shape
Each change is a map such as `{:type ... :id ... :page-id ...}`. Common families:
- `:add-obj`, `:mod-obj`, `:del-obj`: shape lifecycle. `:mod-obj` contains `:operations`, commonly `{:type :set :attr ... :val ... :ignore-geometry ... :ignore-touched ...}` or `{:type :set-touched ...}`.
- `:add-component`, `:mod-component`, `:del-component`: component/library metadata.
- `:add-children`, `:remove-children`, `:reg-objects`: tree and object-map edits.
- `:set-option`, `:add-page`, `:mov-page`, and related file/page metadata changes.
Each transaction carries `:redo-changes` and inverse `:undo-changes`. The undo stack stores transactions and can move its index backward/forward.
## changes-builder API
`common/src/app/common/files/changes_builder.cljc` (usually alias `pcb`) is the fluent builder. Start from `(pcb/empty-changes <it> <page-id>)` or `(pcb/empty-changes nil <page-id>)` for tests.
High-value builder operations:
- `pcb/with-page-id`, `pcb/with-objects`, `pcb/with-library-data`: set context for following operations.
- `pcb/update-shapes ids update-fn`: emits `:mod-obj` with diff-derived `:set` ops. Options include `{:with-objects? true}`, `{:ignore-touched true}`, and `{:attrs #{...}}`.
- `pcb/add-objects`, `pcb/change-parent`, `pcb/remove-objects`, `pcb/resize-parents`: shape/tree edits.
- `pcb/add-component`, `pcb/update-component`, `pcb/mod-component`: component/library edits.
- `pcb/set-translation? true`: marks the whole change set as translation-only, which lets component sync skip expensive work.
## Applying changes in tests
`thf/apply-changes` in `app.common.test-helpers.files` is the test analog of the production applier. It validates by default; pass `:validate? false` only for intentionally-invalid intermediate states.
The applier uses the same `process-operation` multimethod as production (`common/src/app/common/files/changes.cljc`), so tests that use it exercise production behavior.
## :touched and geometry
For component touched semantics and sync groups, read `mem:common/component-data-model`. For the exact `set-shape-attr` / second-pass behavior during change application, read `mem:common/file-change-validation-migration-subtleties`. For transform-specific ignore-geometry behavior, read `mem:frontend/workspace-transform-subtleties`.
## Inspection
To inspect what a UI action emitted, use `mem:frontend/cljs-repl` with the snippets in `mem:common/component-debugging-recipes` rather than adding temporary source instrumentation.
@@ -1,47 +0,0 @@
# Component and Variant Data Model
## Shape roles relative to components
A shape can occupy multiple roles at once:
1. Master/main instance: defines a component and has `:main-instance true` plus `:component-id`.
2. Copy/non-main instance: produced by instantiating a component and carries `:shape-ref` pointing at the master shape. `(ctk/in-component-copy? shape)` is essentially `(some? (:shape-ref shape))`.
3. Component root: topmost shape of an instance, marked `:component-root true` and carrying surface attrs such as `:component-id` and `:component-file`.
Variant masters are main instances and component roots. Their descendants may themselves be component copies, so master/copy logic must handle nested instances rather than assuming those roles are exclusive.
## :shape-ref chains
`:shape-ref` walks up the inheritance hierarchy and can cross files for remote libraries. `find-ref-shape` and `get-ref-chain-until-target-ref` in `app.common.types.file` follow this chain.
`find-shape-ref-child-of` in `app.common.logic.variants` walks the chain looking for the first ref-shape whose ancestors include a specific parent. Variant switch uses this to locate the equivalent master child in the target variant.
## :touched flags
`:touched` is a set of override-group keywords such as `:geometry-group`, `:fill-group`, and `:text-content-group`. It means a copy diverged from its master for attrs in that sync group.
`sync-attrs` in `app.common.types.component` maps attrs to groups. `set-touched-group` is the legitimate setter; the central `set-shape-attr` path calls it only for copies and only when ignore flags allow it.
Masters are not normally touched through `set-shape-attr`, but touched flags can appear on master shapes through cloning/duplication paths. `add-touched-from-ref-chain` in `app.common.logic.variants` unions touched flags from ancestors into the copy being processed, so upstream/master touched state can affect downstream switch behavior.
## Swap slots and positional matching
- A swap slot (stored via `ctk/set-swap-slot`, a `:touched` group `swap-slot-<uuid>`) marks a copy sub-head that was SWAPPED to another component; `compare-children` then pairs it to the main child by slot instead of by `shape-ref`.
- Copy sub-heads without a slot are paired to main children by `shape-ref` (seek, not index). `find-near-match` (positional) is only a validator/repair heuristic; validity requires membership of the ref among the near-main parent's children, not index equality (`mem:common/file-change-validation-migration-subtleties`).
- Copy child ORDER converges to the main's via the async sync (`moved` branch of `compare-children`); local code must never reorder copy children directly (guards in `:mov-objects`/`:reorder-children`).
## Cloning paths
`make-component-instance` in `app.common.types.container` produces a clean component copy through `update-new-shape`, dissociating attrs such as `:touched`, `:variant-id`, and `:variant-name` on cloned shapes.
`duplicate-component` in `app.common.logic.libraries` creates a new component master by cloning existing component shapes, setting component metadata, and applying a position delta. It does not have the same clean-copy semantics as `make-component-instance`, so inherited attrs on the source can matter.
When a bug depends on touched state, identify which cloning path produced the shape before changing sync logic.
## Variant containers
A variant container is a frame with `:is-variant-container true`. Its children are variant masters with `:variant-id` pointing at the container and `:variant-name` naming the variant value. Component records in the library carry `:variant-properties`.
Predicates are broad: `ctk/is-variant?` checks `:variant-id` and applies to both variant master shapes and component rows; `ctk/is-variant-container?` checks the container shape flag.
Moving/dropping a shape into a variant container through the move-to-frame path can auto-convert it into a variant via `generate-make-shapes-variant`, which may duplicate the underlying component. Treat drag/drop into variant containers as a component/variant operation, not a plain reparent.
@@ -1,58 +0,0 @@
# Common Component and Change Debugging Recipes
Keep source changes out of these recipes unless the task requires a durable fix.
## Inspect recent workspace changes
From `cljs_repl` after triggering an action:
```clojure
(let [items (get-in @app.main.store/state [:workspace-undo :items])
n (count items)]
(->> items
(drop (max 0 (- n 5)))
(map-indexed (fn [i it]
{:idx (+ i (max 0 (- n 5)))
:tags (:tags it)
:n (count (:redo-changes it))
:types (frequencies (map :type (:redo-changes it)))
:ids (mapv :id (:redo-changes it))}))))
```
To inspect operations within the latest `:mod-obj`:
```clojure
(let [items (get-in @app.main.store/state [:workspace-undo :items])
mod-obj (->> (:redo-changes (last items))
(filter #(= :mod-obj (:type %)))
first)]
(:operations mod-obj))
```
## Trace variant switch attribute copying
To capture what `update-attrs-on-switch` saw during a real UI swap, patch it temporarily in `cljs_repl`:
```clojure
(def orig (deref #'app.common.logic.libraries/update-attrs-on-switch))
(def trace-buf (atom []))
(set! app.common.logic.libraries/update-attrs-on-switch
(fn [& args]
(swap! trace-buf conj
(let [[_ curr prev _ _ origin _] args]
{:curr (select-keys curr [:name :x :y :selrect :points :touched])
:prev (select-keys prev [:name :x :y :selrect :points :touched])
:origin-ref (select-keys origin [:id :name :x :y :width :height :selrect])}))
(apply orig args)))
;; trigger UI action, then inspect @trace-buf
(set! app.common.logic.libraries/update-attrs-on-switch orig)
```
Runtime patching is faster than adding temporary source instrumentation and avoids recompilation cleanup. Restore the var or reload the frontend when finished.
## Test-side helpers
- Use `thf/dump-file file :keys [...]` to print a shape tree with selected keys during common tests.
- Prefer production-path helpers such as `cls/generate-update-shapes` plus `thf/apply-changes` for shape mutations.
- For component swaps with keep-touched behavior, use `tho/swap-component-in-shape` with `{:keep-touched? true}`.
- Temporary `prn` calls in production code are acceptable while investigating but should be removed before committing.
@@ -1,42 +0,0 @@
# Component Swap and Variant Switch Pipeline
## Entry points
Frontend entry points under `frontend/src/app/main/data/workspace/`:
- `variants.cljs`: `variants-switch` and `variant-switch` events feed property-toggle UI and Plugin API `switchVariant` behavior into `dwl/component-swap`.
- `libraries.cljs`: `component-swap` is the single-swap workhorse; `component-multi-swap` batches swaps and calls `component-swap` with `keep-touched? = false`.
`keep-touched? = true` is the discriminator for preserving user overrides during variant switch. Batch/multi-swap paths intentionally bypass that logic.
## Common-side pipeline
For a single swap with `keep-touched? = true`:
1. `cll/generate-component-swap` in `common/src/app/common/logic/libraries.cljc` builds the base changes: remove old shape and instantiate the target component in its place through `generate-new-shape-for-swap`, `generate-instantiate-component`, and `make-component-instance`.
2. `clv/generate-keep-touched` in `common/src/app/common/logic/variants.cljc` walks pre-swap children, augments each with chain-derived touched flags through `add-touched-from-ref-chain`, finds the equivalent target shape through `find-shape-ref-child-of`, then calls `update-attrs-on-switch`.
3. `update-attrs-on-switch` in `app.common.logic.libraries` decides which touched attrs from the previous shape should be copied onto the freshly instantiated target shape.
## update-attrs-on-switch hazards
The routine compares `current-shape` (fresh target copy), `previous-shape` (pre-swap shape with chain-derived touched), and `origin-ref-shape` (source variant master's equivalent shape). It loops over sync attrs except `swap-keep-attrs` and copies only attrs that pass several guards:
- skip equal previous/current values;
- skip equal composite geometry for selected attrs;
- require the corresponding touched group;
- for most attrs, require source and target masters to agree;
- for fixed-size selrect/points/width/height, use dedicated fixed-layout geometry handling;
- text and path shapes have specialized value conversion paths.
The generic fallback branch copies from `previous-shape`. It represents the intended "carry user override through switch" behavior, but bugs usually appear when guards fail to reject incompatible geometry or master differences before reaching that branch.
## Known sharp edges
- Composite `:selrect` and `:points` bypass the simple different-master skip; width/height checks catch some but not all positional differences.
- `previous-shape` may be repositioned by destination-root minus origin-root before copying. For normal variant switch this is often zero, but do not assume it for all swap entry points.
- Touched flags can be inherited through ref chains, so a shape that looks untouched locally may still behave as touched after `add-touched-from-ref-chain`.
## Test harness
`common/src/app/common/test_helpers/compositions.cljc` has `swap-component-in-shape`, which drives `generate-component-swap` plus `generate-keep-touched` with the production `keep-touched?` flag. Use it for focused common tests of variant-switch behavior.
`common/test/common_tests/logic/variants_switch_test.cljc` is the canonical reference suite for swap+touched scenarios. Read nearby tests before adding another case.
-56
View File
@@ -1,56 +0,0 @@
# Common Architecture and Workflow
`common/` intro: shared CLJC for frontend, backend, exporter, library/file tooling, tests. Small semantic changes can affect multiple runtimes.
## Stable namespace map
- `app.common.data` and `app.common.data.macros`: generic data helpers and performance macros that do not depend on Penpot domain entities.
- `app.common.types.*`: shared shape/file/page/component/token data types, schemas, predicates, and entity-local operations. `app.common.types.organization` contains organization schemas, `apply-organization`, and fail-closed organization/team permission rules (`allowed?`, `can-send-invitations?`).
- `app.common.files.*`: file-level operations, shape tree helpers, change application, migrations, validation, and undo/redo-related logic.
- `app.common.logic.*`: higher-level workflows/algorithms over files, shapes, components, variants, libraries, tokens, etc.
- `app.common.geom.*`: geometry helpers and transformations.
- `app.common.schema` / `app.common.schema.*`: Malli abstraction layer.
- `app.common.math`, `app.common.time`, `app.common.uuid`, `app.common.json`, etc.: cross-runtime utilities.
- `app.common.test_helpers.*`: test builders and production-path helpers.
## Layering and cross-runtime rules
Use reader conditionals for platform-specific code. Because CLJC runs on JVM and CLJS targets, avoid assuming browser-only or JVM-only behavior unless the reader conditional isolates it.
Respect the intended abstraction direction in new/refactored code:
- generic data utilities should not know Penpot domain concepts;
- `types.*` should preserve invariants for a single domain entity or ADT;
- `files.*` can coordinate several entities inside a file and preserve referential integrity;
- `changes*` should adapt serializable change records to lower-level operations and avoid embedding broad business algorithms;
- `logic.*` and frontend/backend event layers own higher workflow/business behavior.
Some legacy code violates this layering; do not copy those violations into new code when a focused refactor is practical.
## Focused memory routing
Model, schema, and persistence shape:
- File/page/shape/component attr changes, import/export surfaces, inspector/codegen, and cross-module checklist: `mem:common/data-model-change-checklist`.
- Token data structures, token import/export, active theme/set semantics, and schema/coercion behavior: `mem:common/tokens-schema-subtleties`.
Geometry and layout:
- Shape geometry invariants, redundant geometry fields, and geometry-sensitive tests: `mem:common/geometry-invariants`.
- Coordinate drift and approximate float comparisons: `mem:common/decimals-and-coordinates`.
- Layout/grid assignment, deassignment, metadata cleanup, and auto-positioning: `mem:common/layout-grid-subtleties`.
Change pipeline, validation, and migrations:
- Change records, undo/redo architecture, changes-builder API, and production-path mutation guidance: `mem:common/changes-architecture`.
- Change application, shape-tree edits, validation/repair, migrations, and second-pass touched behavior: `mem:common/file-change-validation-migration-subtleties`.
Components, variants, and debugging:
- Component/variant data model, ref chains, touched override semantics, and cloning paths: `mem:common/component-data-model`.
- Component swap, variant switch, and keep-touched pipeline: `mem:common/component-swap-pipeline`.
- Live inspection snippets, temporary runtime patching, and test-side debugging helpers for common change/component behavior: `mem:common/component-debugging-recipes`.
Text and tests:
- Shared text data conversion, DraftJS compatibility, modern text content, and derived position data: `mem:common/text-subtleties`.
- Common test commands, helper conventions, production-path test mutations, and runtime coverage choices: `mem:common/testing`.
- Cross-cutting testing principles, anti-patterns, and verification checklist: `mem:testing`.
## Areas without focused memories
Common areas with little or no dedicated memory include colors, media/SVG helpers, path operations, thumbnail helpers, generic pools, weak refs, and some utility namespaces. Treat work there as source/test-led unless a focused memory exists.
@@ -1,25 +0,0 @@
# Common Data Model Change Checklist
## Attribute conventions
- Prefer optional page/shape attrs with default behavior when absent. Reverting to default should usually remove the attr instead of storing nil.
- Do not treat nil as a distinct persisted state from absence. Import/export and cleanup paths may filter nil attrs away.
- Avoid Clojure-special naming in exported object attrs, especially boolean names ending in `?`; exported/imported data must survive JSON/SVG/Transit and external tooling.
- Any new shape attr that participates in component sync must be listed in `app.common.types.component/sync-attrs` with the correct touched group. Attrs absent from `sync-attrs` are ignored by component synchronization.
## Cross-module update checklist
When changing the file data model, check the relevant paths:
- Schema/type definitions under `common/src/app/common/types*` and helpers under `common/src/app/common/files*` / `logic*`.
- File migrations in `common/src/app/common/files/migrations.cljc` when old files cannot safely use absence/default behavior.
- Frontend edit forms under `frontend/src/app/main/ui/workspace/sidebar/options/`; multi-selection behavior is usually in `multiple.cljs` and must handle `:multiple` values.
- SVG/file render and export metadata under `frontend/src/app/main/ui/shapes/*`, especially `export.cljs` when an attr is not a native SVG property.
- SVG import/parser paths under `frontend/src/app/worker/import/parser.cljs`; attrs not exported and imported will be lost on reimport.
- Viewer inspect and code generation under `frontend/src/app/main/ui/viewer/inspect/*` and `frontend/src/app/util/code_gen.cljs` / markup/style helpers when handoff output should expose the attr.
- Exporter/library consumers when the change affects file construction, rendering, or packaged `.penpot` archives.
## Migrations
Existing files should keep working unchanged when possible. If absence cannot preserve old behavior, add a migration and preserve append/order semantics described in `mem:common/file-change-validation-migration-subtleties`.
Model changes can also require file feature flags or migration metadata updates; check nearby migrations and `common/src/app/common/features.cljc` before inventing a new pattern.
@@ -1,54 +0,0 @@
# Decimals and Coordinates in Penpot
Penpot stores all geometry as JS numbers (doubles in CLJS, doubles in
CLJ for the JVM-side common code). Several Penpot-specific facts
about how this plays out are not obvious from reading the code.
## Sub-pixel drift is routine
Coordinate values that "should" be integers are routinely off by ~1e-5
in production data. A `:width` of 107 will frequently appear as
`107.00001275539398` after the value has passed through:
- the modifier propagation pipeline (`apply-wasm-modifiers` and the
Rust WASM transform engine)
- any rotation/scale composition
- repeated translations
This drift is invisible in the UI (the renderer rounds at draw time)
but defeats exact equality comparisons in business logic. It does NOT
appear in JVM-only test setups because the WASM pipeline isn't
involved — tests that build shapes via `setup-shape` and `add-sample-shape`
get clean integer values. Bugs that depend on drift will pass tests
but fire in production unless tests explicitly inject drift.
## Use the close? helpers, not `=`
For comparing coordinate-like floats, the established convention is:
- `app.common.math/close?` — scalar tolerance comparison.
Default precision 0.001 (sub-pixel; tight enough to keep distinct
shapes distinct, loose enough to absorb arithmetic noise).
Two-arity uses default precision; three-arity takes a custom one.
- `app.common.geom.point/close?` — element-wise close on `gpt/Point`
records. Compares :x and :y via `mth/close?`.
- `app.common.geom.matrix/close?` — close on transform matrices.
- `app.common.geom.shapes/close-attrs?` — used inside `set-shape-attr`
to decide whether a re-assigned `:width`/`:height` should be treated
as a no-op (suppresses spurious touched marking from drift).
Treat `=` on `:x`, `:y`, `:width`, `:height`, `:selrect`, or `:points`
fields as a code smell when the inputs may have flowed through any
transform. The `set-shape-attr`-style precedent (already using
`close-attrs?`) is the right model.
## The redundancy multiplies failure modes
A shape's position lives in `:x/:y`, `:selrect`, AND `:points` (see
`mem:common/geometry-invariants` memory). Each is a separate set of float
values. After any operation that touches geometry, all three should
agree, but each is computed by a different path and accumulates
its own drift. Comparing `:selrect.width` from shape A to
`:selrect.width` from shape B is comparing two values that
"semantically" should be equal but were computed via different
operation chains — exact equality will often be false.
@@ -1,31 +0,0 @@
# Common File Change, Validation, and Migration Subtleties
## Change application
- `process-changes` validates the whole change vector once by default, reduces changes, then performs a second pass for collected touched changes. Callers that already validated can pass `verify? false`.
- `process-operation :set` delegates to `ctn/set-shape-attr`; `:assign` first decodes attrs with the shape-attrs JSON transformer and then emits per-attr set operations.
- `set-shape-attr` treats `:position-data` as derived and never touched. Geometry/content-path changes use approximate equality; geometry differences under about 1px can be ignored for touched purposes.
- Width/height are excluded from the `is-geometry?` branch in `set-shape-attr`; do not assume all geometry-group attrs follow identical ignore-geometry behavior.
- `process-touched-change` marks the owning component modified when a touched shape belongs to a main instance; component-data changes can come from shape ops through this second pass.
- Copy structure is guarded at change application: `:mov-objects` (`is-valid-move?`) and `:reorder-children` both refuse to alter children of shapes inside component copies unless the change carries `allow-altering-copies` (sync/swap flows set it). New structural change types must follow the same rule.
- `cls/generate-delete-shapes` propagates deletions from INSIDE a component main to the copy shapes referencing them (transitively, all pages of the file) so no dangling `shape-ref`s remain; skipped when the main root itself is deleted (copies then resolve into the deleted component) and for `allow-altering-copies` flows (swap replaces the shape; sync reconciles).
## Shape tree edits
- `shape-tree/add-shape` falls back invalid/missing parent or frame ids to root (`uuid/zero`), ensures parent `:shapes` is a vector, avoids duplicate child ids, and clears `:remote-synced` on copy parents unless `ignore-touched` is true.
- `shape-tree/delete-shape` removes the shape and all descendants from the objects map and removes the id from its parent. This is different from render-wasm deletion, which may keep deleted children for undo/redo internals.
- Page object maps can carry metadata indexes such as cached frame lists. `start-page-index` / `update-page-index` rebuild those metadata indexes; `frontend` commit application calls `ctst/update-object-indices` after page changes.
## Validation and repair
- Full referential/semantic validation currently runs only when file features contain `"components/v2"`.
- Validation starts at root plus orphan shapes, then validates component records. `validate-file!` raises `:validation :referential-integrity` with collected details.
- `repair-file` does not mutate data directly; it reduces validation errors into redo changes using `changes-builder`. Callers must apply or persist those changes.
- `:missing-slot` fires only for a REAL swap: a copy sub-head whose `shape-ref` is no longer a child of the near main parent. A pure positional mismatch (ref still a sibling elsewhere) is a reorder — valid, realigned by the async component sync; do not "repair" it by assigning swap slots (a slot freezes the child out of normal sync). `fix-missing-swap-slots` (migration 0019) follows the same membership rule.
## Migrations
- Prefer optional attrs/default behavior so old files continue working without migration. If absence cannot preserve old behavior, add a migration.
- Migrations are an ordered set mixing legacy version-derived ids and newer named ids. Keep append order stable; `migrate` applies the set difference between available migrations and file migrations.
- `migrate-file` synthesizes legacy migration ids from old numeric versions when `:migrations` is absent, migrates legacy features, and records feature flags created through `cfeat/*new*`.
- When a file had no previous `:migrations`, `migrate-file` marks all migrations as migrated in metadata so callers persist the complete migration set, not only transformations that changed data.
@@ -1,48 +0,0 @@
# Geometry Invariants in Penpot Shapes
Core invariant: shape position is stored redundantly, and all geometry fields must stay coherent.
## Redundant fields
For a shape at `(x, y)` with width `w` and height `h`:
- `:x`, `:y`, `:width`, `:height`: top-left and dimensions.
- `:selrect`: `{:x :y :width :height :x1 :y1 :x2 :y2}`, where `x2 = x + w` and `y2 = y + h`.
- `:points`: four corners for an axis-aligned rect, clockwise from top-left.
- `:transform` and `:transform-inverse`: identity for axis-aligned shapes; populated for transformed shapes.
After a geometric mutation, equivalent fields such as `:y`, `(:y :selrect)`, and the first point's `:y` should agree. The renderer and hit-testing read `:selrect` / `:points`, so a shape can render or select incorrectly even when `:x` / `:y` look right.
## Helpers that preserve the invariant
- `gsh/move`: translates by delta and updates geometry consistently.
- `gsh/absolute-move`: moves to an absolute position by computing a delta from the current selrect.
- `gsh/transform-shape`: applies a full transform.
- `cts/setup-shape`: initializes geometry for new shapes; variant test helpers such as `thv/add-variant-with-child` use it.
## Edits that break the invariant
- `(assoc shape :x ...)` or `(assoc shape :y ...)`: updates only one field and leaves `:selrect` / `:points` stale.
- `ths/update-shape file label :y val`: goes through `set-shape-attr`, but does not repair all position fields for `:y` alone.
- Direct `update-in` edits to `:selrect`, `:points`, or dimensions.
## Test setup warning
When positioning test shapes, use `gsh/absolute-move`, `gsh/move`, or production change helpers. Do not set only `:x` / `:y`.
```clojure
(cls/generate-update-shapes
(pcb/empty-changes nil page-id)
#{(:id child)}
#(gsh/absolute-move % (gpt/point (:x %) 101))
(:objects page)
{})
```
Using `(ths/update-shape file label :y 101)` leaves `:selrect.y` stale. Downstream code that reads `:selrect` can then fail in ways that look like product bugs but are only invalid test setup.
## :touched and geometry mutation
When a copy shape changes geometry through the proper pipeline (`set-shape-attr` via `process-operation :set`), `:touched` gains `:geometry-group` unless ignored. Tests can either drive the production update with `cls/generate-update-shapes`, or inject `(assoc shape :touched #{:geometry-group})` when only touched state matters.
If a test needs both a new position and touched state, move the shape first with geometry-preserving helpers, then inject or assert touched state.
@@ -1,16 +0,0 @@
# Common Layout and Grid Subtleties
## Layout metadata
- Layout container data and child layout-item data are removed by different helpers. Do not assume clearing a layout frame also clears all child layout metadata.
- Layout data can affect both container attrs and immediate child attrs; validate behavior for both sides when changing cleanup or propagation.
## Grid assignment
- Grid `assign-cells` ensures at least one column and row, skips absolute-position children, creates non-tracked rows/cols when children exceed tracked cells, and asserts that assigned cells do not overlap.
- `position-absolute?` counts HIDDEN shapes as absolute: hiding a grid child frees its cell on the next `assign-cells`.
- `reorder-grid-children` rewrites the parent's `:shapes` to the REVERSE of the sorted cell order, but children with no cell (hidden/absolute) keep their original index — do not "fix" this into moving them to an end; that broke copy/main positional slot alignment (referential-integrity crash).
- The `:reorder-children` change it emits is refused on parents inside component copies unless `allow-altering-copies` (same rule as `:mov-objects`); `pcb/reorder-grid-children` also skips copy grids producer-side. Copy child order is owned by the component sync engine.
- Grid deassignment removes cells for shapes that are no longer direct children or have become absolute-positioned.
- Auto-positioning is not just sorting: some auto cells are converted to manual when empty/manual/span state would break the auto sequence, then auto single-span items can be compacted.
- `fix-overlaps` is marked dev-only and removes one overlapping cell, preferring empty cells first. Avoid depending on it as normal production repair.
-53
View File
@@ -1,53 +0,0 @@
# Common Testing and Verification
`common/` is CLJC shared code. Tests should cover the relevant runtime(s): JVM for backend/common logic and JS for frontend/exporter behavior. For geometry, component, and file-model changes, JVM tests are common and fast, but JS/browser behavior can differ when WASM modifier math or CLJS-specific state is involved.
## Unit tests
READ `mem:testing` FIRST — it defines the execution discipline (no piping, tee to file, preferred commands) that applies to all CLJS/JS and JVM test runs.
Common tests live under `common/test/common_tests/` and use `clojure.test`.
They are CLJC and run on both JVM and JS.
From `common/`:
- Full JVM test run: `clojure -M:dev:test`
- Full JS test run (always builds, suppressed output): `pnpm run test:quiet`
- Full JS test run (always builds, build output visible): `pnpm run test`
- Focus a JVM test namespace: `clojure -M:dev:test --focus common-tests.logic.variants-switch-test`
- Focus a JVM test var: `clojure -M:dev:test --focus common-tests.logic.variants-switch-test/test-basic-switch`
- Focus a JS test namespace: `pnpm run test:quiet -- --focus common-tests.logic.comp-sync-test`
- Focus a JS test var: `pnpm run test:quiet -- --focus common-tests.logic.comp-sync-test/test-sync-when-changing-attribute`
- Quiet logging during a JS run: append `--log-level warn` (or `trace|debug|info|warn|error`)
- Build JS test target only (no run): `pnpm run build:test`
- After `build:test` has been run, run the compiled runner directly: `node target/tests/test.js [--focus ...] [--log-level ...]`.
New common JS test namespaces must be required/listed in `common_tests/runner.cljc`;
new vars in existing namespaces need no runner change. Multiple JVM `--focus` flags
compose as a union.
## Test helpers
Helpers live under `common/src/app/common/test_helpers/` and are usually aliased with short `th*` prefixes. Test namespaces using label->uuid helpers should start with `(t/use-fixtures :each thi/test-fixture)` so labels reset between tests.
Useful builders:
- `thf/sample-file` creates a base file.
- `tho/add-simple-component` creates a simple component.
- `thc/instantiate-component` instantiates a component copy.
- `thv/add-variant-with-child` creates a variant container with two child variants.
- `thv/add-variant-with-copy` creates variants whose children are component instances.
`add-variant-with-copy` does not accept position params for children; use `gsh/absolute-move` after creation if positions matter.
## Driving production paths
For shape mutations, prefer production-path helpers such as `cls/generate-update-shapes` plus `thf/apply-changes`. For component swaps with keep-touched behavior, use `tho/swap-component-in-shape` with `{:keep-touched? true}`.
`thf/apply-changes` validates by default and usually gives the most useful invariant failure. Pass `:validate? false` only for intentionally malformed intermediate state.
## Geometry setup caution
For geometry-sensitive tests, read `mem:common/geometry-invariants` before positioning shapes. Use geometry-preserving helpers or production change helpers rather than direct single-field edits.
## Debugging
Use `mem:common/component-debugging-recipes` for shape-tree dumps, undo/change inspection, and temporary live instrumentation recipes.
@@ -1,14 +0,0 @@
# Common Text Subtleties
## DraftJS compatibility
- `app.common.text` is legacy DraftJS conversion support. New text work should prefer the newer text type namespaces unless specifically touching DraftJS conversion.
- DraftJS style values are encoded as Transit strings under `PENPOT$$$<key>$$$<encoded>` style names. `PENPOT_SELECTION` is a special marker.
- Text conversion uses Unicode code points on both CLJ and CLJS paths, not UTF-16 code units. This matters for offsets around emoji and astral characters.
- Draft conversion fixes gradient type strings back to keywords.
## Modern text content
- Modern text content schema is narrow: root -> paragraph-set -> paragraph -> text nodes.
- `position-data` is derived layout/geometry/font fragment data and should be treated as generated state, not source-of-truth file data.
- Token propagation and some non-current-page text updates drop `:position-data` so it can be regenerated in the right runtime context.
@@ -1,17 +0,0 @@
# Common Tokens and Schema Subtleties
## Tokens
- `TokensLib` always ensures an internal hidden theme exists and defaults active themes to that hidden theme path. That hidden theme represents the UI state where active sets are controlled without modifying a user-created theme.
- `get-tokens-in-active-sets` merges tokens from sets selected by active themes in set order, so later active sets with the same token name override earlier ones.
- Activating a real theme in `common.logic.tokens` removes the hidden theme from the active-theme set unless it is the only active theme. Toggling active sets directly copies current active sets into the hidden theme.
- DTCG import/export deliberately hides the internal hidden theme: exports omit it from `$themes` and activeThemes, while `activeSets` records the hidden/current active sets.
- Single-set DTCG/legacy imports throw if no supported tokens are found. Multi-set import normalizes set names, keeps `tokenSetOrder`, rejects conflicting token path names, discards unsupported token types, and validates theme sets against existing sets.
- Token values stored on shapes are token names in `:applied-tokens`, not token ids. Renames and group renames must update those name paths.
- Token serialization has both Transit handlers for frontend/backend transport and Fressian handlers for internal file-data storage, with migrations for older token-lib internal versions.
## Schema
- `app.common.schema/json-transformer` has custom map-of key decoding/encoding, so map keys can be transformed based on the key schema instead of only the value schema.
- `check-fn` throws `ex-info` with default `:type :assertion`, `:code :data-validation`, and `::explain`. Prefer reusable `check-fn`/lazy validators in hot or repeated paths; `sm/check` creates a checker every call.
- `coercer` decodes with the JSON transformer and then checks. This is the common pattern for accepting external JSON-shaped data into internal types.
-91
View File
@@ -1,91 +0,0 @@
You are working on the GitHub project `penpot/penpot`, a monorepo.
# Memory system
- Memories are the primary project guidance (not docs or other readme files).
- A section's top-level memory is `<section>/core`. When a section is relevant, read the core memory
before focused memories.
- Edits/stale refs/duplication cleanup: `mem:memory-maintenance`.
- Cross-cutting testing principles, TDD workflow, and anti-patterns: `mem:testing`.
# Development workflow
- Commit/PR/issue creation is **on explicit request only**. Before any of these actions, read the relevant memory — don't infer format from prior examples:
- Before `git commit``mem:workflow/creating-commits` (subject format, body, `AI-assisted-by: model-name` trailer)
- Before `gh issue create``mem:workflow/creating-issues` (title derivation, body template, labels, Issue Type)
- Before `gh pr create` / `gh pr edit``mem:workflow/creating-prs` (title format, body structure, "Note:" line)
- **Never `git push`, force-push, or modify `git origin`** (or any other remote). The user pushes from their own shell; if a push is required, say so and wait. Never amend a commit that the user has already pushed unless explicitly asked.
- You have access to the GitHub CLI `gh` or corresponding MCP tools.
- Issues are also managed on Taiga. Read issues using the `read_taiga_issue` tool.
- Before writing code, analyze the task in depth and describe your plan. If the task is complex, break it down into atomic steps.
*After making changes, run the applicable lint and format checks for the affected module before considering the work done (per example `mem:backend/core` or `mem:frontend/core`).
- Align `let` binding values: when a `let` form has multiple bindings spanning
several lines, align the value forms to the same column with spaces.
- If you introduce delimiter errors (mismatched parens/brackets) in Clojure/CLJS files,
fix them with `scripts/paren-repair` BEFORE running lint/format checks.
See `mem:scripts/paren-repair` for usage.
- Never run anything that destroys data without explicit permission, including `drop-devenv`, `docker compose down -v`, `docker volume rm ...`. The user's real work lives in the volumes of the shared infra.
# Project modules
This is a monorepo. Principles that apply to one module do *not* generally apply to others. Do not make assumptions.
- `frontend/`: ClojureScript + SCSS SPA/design editor; core conventions: `mem:frontend/core`.
- `backend/`: JVM Clojure HTTP/RPC server with PostgreSQL, Redis, storage, mail, and workers; core conventions: `mem:backend/core`. Runtime services and the task-queue vs Pub/Sub topology that constrains horizontal scaling: `mem:prod-infra/core`.
- `common/`: shared CLJC data types, geometry, schemas, file/change logic, and utilities; core conventions: `mem:common/core`.
- `render-wasm/`: Rust -> WebAssembly Skia renderer consumed by frontend; core conventions: `mem:render-wasm/core`.
- `exporter/`: ClojureScript/Node headless Playwright SVG/PDF export; core conventions: `mem:exporter/core`.
- `mcp/`: TypeScript Model Context Protocol integration; core conventions: `mem:mcp/core`.
- `plugins/`: TypeScript plugin runtime/examples and Plugin API types; core conventions: `mem:plugins/core`.
- `library/`: design library workflows; core conventions: `mem:library/core`.
- `docs/`: documentation site; core workflow and conventions: `mem:docs/core`.
- `media-processor/`: TypeScript/Node.js HTTP service for image (sharp) and font (FontForge) processing; core conventions: `mem:media-processor/core`.
The memory is structured in a way that you can get the critical information about the
module. You can read it from `mem:<MODULE>/core`
# Low-centrality project paths
- `docker/` contains devenv related code, not needed unless specifically instructed.
When working on devenv startup, compose layout, instance config (`defaults.env`),
tmux session lifecycle, MinIO provisioning, or anything in `manage.sh`'s
`*-devenv` commands, read `mem:devenv/core`.
- `experiments/` contains standalone experimental HTML/JS/scripts; treat it as non-core unless the user explicitly asks about it.
- `sample_media/` contains sample image/icon media and config used as fixtures/demo material; do not infer app behavior from it.
# Dev Scripts (scripts/)
- `scripts/nrepl-eval.mjs` — Evaluate Clojure/ClojureScript code via nREPL.
Supports `--backend` (port 6064) and `--frontend` (port 3447) aliases.
See `mem:scripts/nrepl-eval`.
- `scripts/paren-repair` — Fix mismatched delimiters in Clojure/CLJS files
and reformat with cljfmt. Run before lint checks when LLM edits break parens.
See `mem:scripts/paren-repair`.
- `scripts/psql` — PostgreSQL client wrapper with devenv defaults.
Companion: `scripts/db-schema` for DDL dumps. See `mem:scripts/psql`.
- `scripts/taiga.py` — Fetch public issues, user stories, and tasks from the
Penpot Taiga project without authentication. See `mem:scripts/taiga`.
- `scripts/gh.py` — GitHub operations helper: list milestone issues, fetch PR
details, compare against CHANGES.md. Requires `gh` CLI. See `mem:scripts/gh`.
- `scripts/error-reports.mjs` — Query error reports via RPC API with token
authentication. Supports list/get operations with filtering and pagination.
See `mem:scripts/error-reports`.
# Dependency graph
`frontend -> common`, `backend -> common`, `exporter -> common`, and `frontend -> render-wasm`. Changes in `common` can
affect frontend, backend, exporter, file migrations, and design-library behavior; validate across consumers when
semantics change.
# Working with Penpot designs
- Before automating or inspecting Penpot designs through the Plugin API, call the Penpot MCP `high_level_overview` tool.
- connection between the JavaScript plugin API and the ClojureScript code: `mem:frontend/plugin-api-to-cljs-binding`.
- executing ClojureScript code in the frontend: `mem:frontend/cljs-repl`.
- handling Clojure compiler errors, runtime patching and debug helpers: `mem:frontend/handling-errors-and-debugging`.
## Detecting Crashes
The Penpot frontend can crash silently from the JS API's perspective: `execute_code` calls return successfully, but 1-2s later the workspace becomes unusable (Internal Error page).
The `execute_code` tool then stops working, but `cljs_repl` still works. Use it to detect a crash via `(some? (:exception @app.main.store/state))`.
For details on handling crashes, read memory `mem:frontend/handling-crashes`.
-76
View File
@@ -1,76 +0,0 @@
# Devenv startup and configuration
Compose-based dev environment under `docker/devenv/`, driven by `manage.sh`. Parallel instances share infra + Postgres + MinIO; each instance has its own `main` container, Valkey, source checkout, tmux session.
## Compose project layout
- `penpotdev-infra`: shared `postgres`, `minio`, `minio-setup`, `mailer`, `ldap`. File: `docker-compose.infra.yml`.
- `penpotdev-wsN` (N=0,1,…): per-instance `main` + `redis` (Valkey). File: `docker-compose.main.yml`. ws0 (a.k.a. `main`) binds `$PWD`; ws1+ bind clones at `${PENPOT_WORKSPACES_DIR}/wsN/` (default `~/.penpot/penpot_workspaces/`), maintained by the developer.
- Optional overlay `docker-compose.opencode.yml`: added by `instance-compose` as an extra `-f` only when `PENPOT_OPENCODE_CONFIG_DIR` is set (i.e. `run-devenv --opencode-config-dir DIR` ran in this process). Bind-mounts the host dir at `/home/penpot/.config/opencode` (`:z`). Flag-only, per-call; not read from ambient env. Parser `parse-opencode-config-dir` absolutizes (`~`, realpath) because compose resolves relative bind sources against the compose file's dir. Only instances brought up with the flag get the mount.
- All projects join external network `penpot_shared`. Created idempotently by `ensure-devenv-network`, never removed by lifecycle commands.
## Source-of-truth files
- `docker/devenv/defaults.env`: ws0 baseline — container/volume names, runtime env, published host ports, tmux defaults. `manage.sh` aborts if unreadable.
- For ws1+, `instance-env-overrides` computes the per-instance overrides (container/volume names, host ports offset `10000·N`, `PENPOT_PUBLIC_URI`, `PENPOT_REDIS_URI`, `PENPOT_BACKEND_WORKER=false`) and `instance-compose` injects them as env vars at compose time — never written to disk, recomputed each call so they can't drift. ws0 uses `defaults.env` as-is.
- `backend/scripts/_env`: backend-internal only — secret keys, `PENPOT_FLAGS` (with `enable-backend-worker` gated on `PENPOT_BACKEND_WORKER`), `JAVA_OPTS`, `setup_minio()`. Never duplicates `defaults.env`.
- Compose files use pure `${VAR}` substitution; missing var = compose fails.
## Invariants
- `infra-compose` / `instance-compose` wrap `docker compose` with `env -i`, then re-inject what compose needs. Stripping is required because `defaults.env` is sourced into manage.sh's shell at startup (stale values would leak); the ws1+ overrides are deliberately re-injected as shell env vars precisely because Compose gives shell precedence over `--env-file`, so they override the `defaults.env` baseline.
- Volume names pinned via `name:` (PENPOT_*_VOLUME), decoupled from the compose project name. ws1+ inject distinct per-instance volume names; ws0 keeps the historical `penpotdev_*` physical names so project renames never require data migration.
- Network aliases (`- main`, `- redis`) are not declared in main.yml. Compose's auto-service-alias still registers `redis` on the shared network, so DNS for `redis` is non-deterministic with multiple instances. Backend uses `PENPOT_REDIS_URI=redis://penpot-devenv-wsN-valkey/0` (container_name) instead.
- No cross-project `depends_on`. `manage.sh ensure-infra-up` `docker wait`s on the `minio-setup` one-shot.
- `JAVA_OPTS` in `manage.sh` is shadowed inside the container by `_env`. The `-e JAVA_OPTS=...` flag only matters for processes that don't source `_env`.
## Worker policy
Backend workers run only on ws0. `_env` gates `enable-backend-worker` on `PENPOT_BACKEND_WORKER`; ws1+ inject it as false. Workers are pure fire-and-forget: `wrk/submit!` inserts a row into the shared Postgres `task` table and returns; RPC handlers never wait on completion and workers never publish to msgbus. The reason for "ws0 only" is avoiding multi-instance worker races (cron dedup is best-effort across instances, `wrk/submit!` `dedupe` is racy across submitters); details in `mem:prod-infra/core`.
Each workspace is independent and can be started/stopped in any order. Shared infra (postgres, minio, etc.) is shut down only when no instances remain running.
## Port layout
Container-internal ports fixed; host side offset `10000·N`.
| ws0 | ws1 | wsN | container | role |
|---|---|---|---|---|
| 3449 | 13449 | 3449+10000·N | 3449 | public HTTPS (Caddy; `/mcp/ws` same-origin) |
| 3449/udp | 13449/udp | … | 3449/udp | HTTP/3 |
| 4401 | 14401 | … | 4401 | MCP HTTP stream |
| 4403 | 14403 | … | 4403 | MCP REPL |
| 14181 | 24181 | … | 14281 | Serena MCP |
| 14182 | 24182 | … | 24282 | Serena dashboard |
Everything else (frontend dev, backend API, exporter, storybook, REPLs, plugin dev, MCP inspector/WebSocket) is in-process or same-origin via Caddy/nginx. Infra publishes: mailer 1080, ldap 10389/10636 (singletons, not offset).
## Tmux + MCP routing
`docker/devenv/files/start-tmux.sh` is session-level idempotent. Reads `PENPOT_TMUX_ATTACH`. If the session exists it attaches or exits; otherwise creates 4 base windows (frontend watch / storybook / exporter / backend) plus `mcp` (when `enable-mcp` in `PENPOT_FLAGS`) and `serena` (when `SERENA_ENABLED=true`). `run-devenv --agentic` always sets both env vars. The legacy `run-devenv` alias doesn't, hence its 4-window-only session. To switch from a legacy session to agentic, `stop-devenv` then `run-devenv --agentic` — the conditional windows are only added at session create time.
MCP plugin routing is same-origin: frontend uses `<public-uri>/mcp/ws`, per-instance nginx proxies to MCP port 4401 in-container. For the plugin↔MCP server wiring (how the browser plugin discovers the URL, the in-memory connection registry, why DB-mediated routing isn't needed), see `mem:mcp/core`.
## Workspace orchestration (ws1+)
Workspace directories are user-maintained at `${PENPOT_WORKSPACES_DIR}/wsN`. `run-devenv --agentic --ws i` syncs only when `--sync` is passed, with one exception: if the workspace directory is missing on first use, sync runs implicitly to seed it.
`sync-workspace wsN`:
1. `assert-clean-git-state` — refuses on `.git/{rebase-apply,rebase-merge,MERGE_HEAD,CHERRY_PICK_HEAD,index.lock}`. No `--sync-force` escape.
2. `rsync -a --delete $PWD/.git/ $workspace/.git/`.
3. `git ls-files -z --cached --others --exclude-standard``rsync --files-from` (Git is the authority on tracked files; rsync's gitignore filter would drop committed files under gitignored parents like `.clj-kondo/config.edn`).
4. Initial-only copy of `frontend/resources/public/js/config.js` (gitignored, but agentic mode needs it). After the first sync the workspace's copy belongs to the developer — subsequent syncs leave it alone.
5. `git switch -C "wsN/<current-branch>"` inside the workspace.
No `--delete` on the working-tree pass: gitignored caches in the workspace survive. Workspace dir + named volumes survive `compose down`.
## CLI surface
- `run-devenv --agentic [--ws main|0|wsN|N] [--sync] [--serena-context CTX] [--opencode-config-dir DIR]`: bring one instance up. Agentic only — MCP and Serena windows are always created. Default target main. Errors out if the target is already running. `--sync` is rejected on main; on ws1+ it's optional (forced only when the workspace dir does not exist yet). `--opencode-config-dir DIR` bind-mounts DIR at `~/.config/opencode` in-container via the optional overlay above; mount applies at container creation, so changing it requires stop + re-run.
- `stop-devenv [--ws main|0|wsN|N] [--all]`: stop instances. Flags mutually exclusive. `--ws N` stops just that workspace. `--ws 0` or no flag stops ws0; shared infra shuts down only if no other instances remain. `--all` stops every ws highest-first then ws0, then infra.
- `run-devenv`: legacy alias, ws0 non-agentic attached.
- `attach-devenv [--ws main|0|wsN|N]`: pure attach. Fails fast if instance/session missing.
- `run-devenv-shell [--instance 0|wsN|N] [cmd...]`: bash in target instance. (`--instance` flag not yet renamed to `--ws`.)
- `start-devenv` / `log-devenv` / `drop-devenv`: legacy paths around ws0 + shared infra. `drop-devenv` never removes volumes.
`exporter/scripts/run` and `wait-and-start.sh` source `backend/scripts/_env` then `_env.local` if present.
-19
View File
@@ -1,19 +0,0 @@
# Docs
`docs/`: Penpot documentation site; Eleventy.
## Layout and Tooling
- `docs/package.json`: Eleventy commands and documentation-site dependencies.
- `docs/README.md`: local setup and tooling notes.
- Diagrams may use PlantUML, svgbob, mermaid, and C4/arc42 conventions depending on the existing page.
## Commands
From `docs/`:
- Install deps if needed: `pnpm install`.
- Local server: `pnpm start` or `pnpm run serve` (serves on `http://localhost:8080`).
- Build: `pnpm run build`.
- Watch: `pnpm run watch`.
Documentation changes should follow the existing page structure and rendered Help Center conventions rather than inventing a new style locally.
-35
View File
@@ -1,35 +0,0 @@
# Exporter Architecture and Workflow
`exporter/`: CLJS/Node headless export service. Depends on `common/`; uses Playwright plus export JS/CLJS deps for SVG/PDF/assets.
## Layout and commands
- Source: `exporter/src/`; config: `deps.edn`, `shadow-cljs.edn`, `package.json`; runtime helpers/assets: `vendor/`, `scripts/`.
- From `exporter/`: setup `./scripts/setup`; watch `pnpm run watch` or `pnpm run watch:app`; production build `pnpm run build`; test bundle `pnpm run build:test`; tests `pnpm run test` or `pnpm run test:quiet`; lint `pnpm run lint:clj`; format check/fix `pnpm run check-fmt:clj` / `pnpm run fmt:clj`.
- Because exporter consumes `common/`, shared file/shape/model changes may need exporter verification even when the immediate change is not under `exporter/`.
- Cross-cutting testing principles and anti-patterns: `mem:testing`.
- Exporter test conventions and CI: `mem:exporter/testing`.
## HTTP and browser pool
- POST body limit is about 60 MB. Exporter supports `application/transit+json`; request params merge query params and body params.
- Map response bodies are Transit JSON and force HTTP 200; nil 200 bodies become 204.
- Auth token comes from cookie `auth-token`, then uploads use Bearer auth plus the management shared key.
- Each export job gets a fresh Playwright browser context. On success, the context closes and the browser returns to the pool; on error, the browser is destroyed instead of reused.
- Borrow validates browser connection. Pool acquire timeout is about 10s; font loading timeout logs a warning and continues after about 15s.
## Export batching and async behavior
- `prepare-exports` groups entries by `[scale type]` and partitions groups into chunks of 50. Each partition uses file/page/share/name from its first item, so be careful if entries might cross those boundaries.
- Single-export response is used only when multiple export is not forced and there is exactly one prepared export containing exactly one object.
- Multi-object export can run async: when `wait` is false it returns a resource immediately and publishes progress/end/error to Redis by profile topic; when `wait` is true it waits for upload and returns the uploaded resource.
- Frame export returns a resource immediately and publishes Redis updates; it does not follow the same `wait` option path.
- ZIP entry names are sanitized and duplicates receive numeric suffixes.
## Render details
- Bitmap export differs for WASM vs non-WASM render paths: WASM forces Playwright `deviceScaleFactor` to 1 and passes scale through the render URL; non-WASM uses `deviceScaleFactor = scale`.
- WebP is produced by taking a PNG screenshot and converting it with ImageMagick.
- SVG export rasterizes text foreignObjects to PNG, converts through PPM/color masks/potrace, and reassembles SVG paths. It also replaces non-breaking spaces for SVG compatibility and drops empty defs/paths.
- PDF export injects `@page` sizing through raw browser `evaluate` JavaScript; that code cannot rely on CLJS runtime helpers.
- Temporary resources schedule local deletion, then uploads POST to `/api/management/methods/upload-tempfile` with `X-Shared-Key: exporter <management-key>` and Bearer auth.
-16
View File
@@ -1,16 +0,0 @@
# Exporter Testing
- READ `mem:testing` first.
- Tests use `cljs.test` and live under `exporter/test/exporter_tests/`.
- Register every test namespace in `exporter-tests.runner`.
- From `exporter/`: `pnpm run build:test` builds the Node test bundle without running tests.
- From `exporter/`: `pnpm run test` builds and runs tests with full output.
- From `exporter/`: `pnpm run test:quiet` builds and runs tests with reduced build output.
- After `build:test`, reuse the compiled bundle with `node target/tests/test.js`.
- For iterative focused runs, build once and reuse the compiled bundle.
- Focus a test namespace with `node target/tests/test.js --focus exporter-tests.renderer-svg-test`.
- Focus a test var with `node target/tests/test.js --focus exporter-tests.renderer-svg-test/creates-the-correct-gradient-element`.
- Set app log level by appending `--log-level warn` (or `trace|debug|info|warn|error`).
- `test:quiet` accepts forwarded options but rebuilds the bundle; prefer the direct runner after `build:test` for focused runs.
- From `exporter/`: `pnpm run check-fmt:clj` checks ClojureScript formatting.
- From `exporter/`: `pnpm run lint:clj` runs ClojureScript linting.
-94
View File
@@ -1,94 +0,0 @@
# ClojureScript REPL and Frontend Debugging
Execute code in the live frontend via the Penpot MCP `cljs_repl` tool. For browser-console debugging, the frontend also exports a `debug` JS namespace in development builds.
## Accessing app state
The main store is `app.main.store/state`. It contains workspace metadata, selection, UI state, profile, route, etc. Page objects are not under a `:workspace-data` key; use derived refs.
```clojure
;; Current selection
(mapv str (get-in @app.main.store/state [:workspace-local :selected]))
;; Current page objects
(let [objects @app.main.refs/workspace-page-objects
shape (get objects (parse-uuid "some-uuid-here"))]
(select-keys shape [:name :type :x :y :width :height :fills :strokes :rotation :opacity :frame-id :parent-id]))
```
Shape keys use kebab-case keywords. Internal `:rect` corresponds to "rectangle" in the JS Plugin API, and `:frame` corresponds to "board".
Component instance shapes carry `:component-id` and `:component-file` directly; `:component-root` flags the root of an instance. Use `app.common.types.container/get-head-shape` for nearest head and `get-instance-root` for outermost root; they differ for nested instances.
## Navigation recipe
To programmatically open a workspace file, all three ids are required:
```clojure
(do (require '[app.main.data.common :as dcm])
(app.main.store/emit! (dcm/go-to-workspace
:team-id (parse-uuid "<team-id>")
:file-id (parse-uuid "<file-id>")
:page-id (parse-uuid "<page-id>"))))
```
Get `team-id` from `(:current-team-id @app.main.store/state)`. Get file ids from `(vals (:files @app.main.store/state))`. Get page ids by fetching file data, e.g. through `rp/cmd! :get-file` with current features.
## Reload the live runtime
`(.reload js/location)` (alias `app.util.dom/reload-current-window`) from `cljs_repl` reloads the browser page: clears `set!` runtime patches, re-fetches file state, and is the simplest crash recovery while the repl is live (`mem:frontend/handling-crashes`). To re-fetch only the current file's data without a full page reload, emit `(app.main.store/emit! (potok.v2.core/event :app.main.data.workspace/reload-current-file))`.
## Useful lookup helpers
`app.plugins.utils` contains state lookup helpers that are useful from any CLJS, despite living under `plugins/`:
- `locate-shape`, `locate-objects`, `locate-file`.
- `locate-component` resolves through the outermost instance root.
- `locate-head-component` resolves through the nearest component head.
- `locate-library-component` does direct file-id/component-id lookup.
## Runtime patching with `set!`
Some frontend vars are deliberately mutable escape hatches for runtime instrumentation or circular-dependency patching. From `cljs_repl`, use `set!` for temporary debugging of CLJS vars such as `app.main.store/on-event`, `app.main.errors/reload-file`, `app.main.errors/is-plugin-error?`, `app.main.errors/last-report`, or `app.main.errors/last-exception`. These patches affect only the live browser runtime and disappear on reload or recompilation.
```clojure
;; Log non-noisy Potok events temporarily.
(set! app.main.store/on-event
(fn [event]
(when (potok.v2.core/event? event)
(.log js/console (potok.v2.core/repr-event event)))))
```
Restore mutable hooks after debugging, or reload the frontend. Use JVM `alter-var-root` only for JVM Clojure; it is not the normal way to patch live CLJS browser vars.
## Browser-console debug namespace
In development, the JS console exposes `debug` helpers from `frontend/src/debug.cljs`:
```javascript
debug.set_logging("namespace", "debug");
debug.dump_state();
debug.dump_buffer();
debug.get_state(":workspace-local :selected");
debug.dump_objects();
debug.dump_object("Rect-1");
debug.dump_selected();
debug.dump_tree(true, true);
```
Visual workspace debug overlays can be toggled with `debug.toggle_debug("bounding-boxes")`, `"group"`, `"events"`, or `"rotation-handler"`; `debug.debug_all()` and `debug.debug_none()` toggle all visual aids.
For temporary source traces, prefer existing logging (`app.common.logging` / `app.util.logging`) or short-lived `prn`, `app.common.pprint/pprint`, `js/console.log`, or `js-debugger` calls. Remove temporary source instrumentation before committing.
## Runtime targeting
`cljs_repl` may connect to the wrong runtime when several are attached, such as workspace plus rasterizer. Verify with `(.-title js/document)`; it should show the workspace file name, not "Penpot - Rasterizer".
To list or target shadow-cljs runtimes, run from `/home/penpot/penpot/frontend`:
```bash
printf '(shadow.cljs.devtools.api/repl-runtimes :main)\n' | timeout 10 npx shadow-cljs clj-eval --stdin
printf '(shadow.cljs.devtools.api/cljs-eval :main "<cljs-code>" {:client-id 5})\n' | timeout 10 npx shadow-cljs clj-eval --stdin
```
Use command timeouts so a disconnected browser does not hang the session.
@@ -1,22 +0,0 @@
# Frontend Compile Diagnostics
Separate from runtime crash recovery.
## First check the shadow-cljs build
Use the Penpot MCP `cljs_compiler_output` tool to inspect the latest shadow-cljs `:main` build status. This is the fastest way to distinguish a bad build from a runtime error in the browser.
Recommended order after CLJ/CLJC/CLJS source edits:
1. Run `cljs_compiler_output`.
2. If the compiler reports a Clojure syntax problem, especially unmatched delimiters or a confusing location, run `clj_check_parentheses` on the absolute path of the suspect `.clj`, `.cljc`, or `.cljs` file.
3. After the build is healthy, use `mem:frontend/cljs-repl`, browser tools, or runtime crash checks for behavior.
## Parentheses checker
`clj_check_parentheses` analyzes one Clojure/ClojureScript source file and reports the area likely responsible for unclosed parentheses/brackets/braces. Use it when compiler output points near EOF, points at a misleading later form, or says delimiter-related syntax errors.
## Hot reload notes
When the frontend shadow-cljs watch process is running, edits to CLJC files in `common/` are normally recompiled into the browser automatically. Do not restart the frontend before checking `cljs_compiler_output`; stale behavior is often a failed build.
For production/minified stack traces, build the production bundle from `frontend/` with `pnpm run build:app`. Output and source maps are generated under `frontend/resources/public/js`; inspect source maps or shadow-cljs reports using build ids from `shadow-cljs.edn`.
@@ -1,152 +0,0 @@
# Composable component tests
A framework concept for systematically testing Penpot's component subsystem
(synchronisation/propagation, swaps, variant switches, nesting, overrides), implemented in TWO test
suites that share the principles below:
1. **ClojureScript suite** — in the frontend test tree (`frontend/test/frontend_tests/
composable_tests/`), driving a minimally-assembled real app headlessly. The original.
2. **TypeScript suite** — a Penpot plugin (`plugins/apps/composable-test-suite/`), driving the FULL
production app end-to-end through the Plugin API, with a slightly more elaborate set of
abstractions. Runs interactively (panel), remotely (Playwright), and headlessly in CI. Its
README is the authoritative operational reference.
## Shared core idea
A test is a **composition of operations** over a starting configuration, plus assertions. You
describe a test as data (a setup + a sequence of operations) rather than writing bespoke imperative
code, and coverage grows by COMPOSITION: a new variation is one combinator wrapped around existing
pieces, not a copied test. Choice points (one-of alternatives, optional steps) EXPAND the
composition into a full sweep of variants — one written case stands for a whole matrix of concrete
tests.
## Shared principles
- **Every producing object is the accessor interface to what it produces downstream.** An
operation — and related objects such as content-creation strategies — is not merely an action:
the SAME object instance the case holds is the typed interface through which everything it
created or changed is later retrieved, checked, and asserted, parameterized by the situation. A
foundation operation exposes accessors for the participants it built; an edit operation exposes
its dual check (`assertHasChangedProperty` / `has-property-of`); a choice is recovered by asking
the one-of object (`getChoice`/`get-choice`); "did this step run" is asked of the step
(`wasApplied`/`applied?`). NEVER reach into a situation (or the document) for something an
upstream object produced — ask the producer. This is what keeps sweeps sound (object identity
ties the question to the exact node that ran) and what keeps retrieval logic in exactly one
place. Particularly explicit in the TS OOP implementation, where these accessors are methods on
the operation/strategy classes; repeatedly violating it (reading the document directly,
duplicating retrieval) was the most common review correction while building the suites.
- **Operations are data with identity.** Each operation node has a unique id at construction and
records what it did under that id; interrogation is by identity. Bind an operation to a value
ONCE and reuse it in the composition and in every query about it.
- **Drive the real production pipeline.** Operations route through genuine Penpot logic — real
change functions / real workspace events / the real Plugin API, never raw field writes — so the
production watcher's AUTOMATIC propagation is what's under test.
- **Roles, not internals.** A starting configuration names its participants (roles). Role→id
capture happens when the configuration is built; operation TARGETS resolve at apply-time and may
be re-bound, so an operation targeting a role follows it as state-building ops re-point it —
which lets a single operation be swept across depth.
- **Enumeration is authored, not exhaustive.** Compose only VALID cases, so outcomes are just
pass / fail / error — no not-applicable cells.
- **Naming discipline.** Penpot domain nouns ("component", "variant") must not name framework
abstractions; an operation may name the domain ACTION it performs.
- **Operator algebra** (same in both suites): sequence (cartesian product of the steps' variants),
one-of (union, choice recorded), optional(X) = one-of([X, skip]), inline assertion ops, trailing
asserters.
- **Case authoring:** a case carries a CamelCase identifier and a plain-terms description in three
parts — situation setup, actions/variations, asserted requirement.
---
# ClojureScript suite (frontend test tree)
Test-only `.cljs` code in the frontend test tree (nothing "common" about it). A **situation** =
the in-memory file value + named roles + `:vars` + an ordered applied-log. Operations are records
implementing `IOperation`/`apply-to` (`apply` collides with core). Assertions = inline `Test` ops
and/or a trailing asserter; the runner makes no judgment. Failures carry `describe-applied` (the
transcript), which is what makes a failing variant in a sweep identifiable.
Layout: `core.cljs` (the domain-agnostic engine: situation, identity/transcript, roles/targets,
operators, runners), `comp/setups.cljs` (setups + role accessors), `comp/nodes.cljs` (the component
operations and their check duals), `interpreter.cljs` (runs cases against the real frontend),
`comp/sync_test.cljs` (the cases; registered in `frontend_tests/runner.cljs`). Case letters B..N;
the sweeps (K: depth × edit-precedence; L: swaps; M: variant switches; N: rotated-instance
geometry, on the #10109 fix branch until merged) are the flagship pattern — read them before
writing a new sweep.
**Scenario lineage model** (behind the sweeps): scenario ops track named component lineages as
objects under `:vars`, each holding the FIXED deepest origin (`:remote-*`), the ADVANCING outer
main (`:main-*`), and per-nesting-level data whose `:nested-head` (the deepest instance at that
level, found by descending the `:shape-ref` chain — matching chain MEMBERSHIP, not terminus) is
the swap/switch target, anchored by its swap-stable parent. Nesting seeks the FIXED origin, not
the advancing main — that is what makes each level's `:nested-head` land on the deepest instance.
A variant nesting re-points the lineage's remote to the chosen member. Construction lesson:
cross-level propagation requires progressively NESTED levels (one variant + plain wraps); sibling
nestings do not propagate between each other.
**Interpreter:** installs the situation's files into the global `st/state` (aux files tagged
`:library-of`), starts the real `watch-component-changes` (+ harness `watch-undo-stack`), maps
event-ops to REAL workspace events (`dwsh/update-shapes`, `dwl/component-swap`,
`dwv/variants-switch`, `dwt/increase-rotation` — which runs the `check-delta` placement
classification — `dwt/update-dimensions`, `dwu/undo`, `dwl/sync-file`, …) and runs sync-ops'
`apply-to` against the live store file; awaits settlement (idle-gap heuristic + per-op grace) and
re-reads `:file` each step so the shared accessors keep working.
STORE-SWAP IMMUNITY: other test namespaces `set!` `st/state`/`st/stream` and never restore, while
the `app.main.refs` lenses stay bound to the ORIGINAL atoms — propagation then dies silently. The
interpreter captures the atoms at namespace-load time and re-`set!`s them per variant.
Running: `cd frontend && pnpm run build:test`, then
`node target/tests/test.js --focus frontend-tests.composable-tests.comp.sync-test`
(var-level focus for one case).
**Fidelity warning:** the harness drives a MINIMALLY-ASSEMBLED app — only some
`initialize-workspace` subscriptions are wired. Risk = SILENT UNDER-WIRING (e.g. undo needs the
harness `watch-undo-stack`). When a case needs app behaviour beyond a raw edit, check for an
unwired subscription and verify by PROBING store state, not by trusting a green assertion.
**Caveats:** inline `Test` exceptions are UNCAUGHT on the frontend (crash the runner — assert in
the trailing asserter). `(optional (in-sequence …))` is not flattened for the interpreter — use
independent optionals. The Serena/clj-kondo cache for `nodes.cljs` goes stale (phantom symbols) —
trust the build. Cross-namespace global-state leaks land in this suite first; suspect them before
the framework on inexplicable full-run-only failures. Case H's `sync-file` schedules a delayed RPC
that fails headless (benign; absorbed by per-op grace).
---
# TypeScript suite (the plugin) — full e2e
`plugins/apps/composable-test-suite/` — same principles against the FULL production app through the
Plugin API (real frontend, real propagation). Continuation of the CLJS suite per issue #10584.
Operational details (build/run, connect URL, remote control, reading logs, auto-reload, CI): the
plugin README.
Distinguishing abstractions (the OOP articulation of the shared principles):
- `TestCase {identifier, description, operation}` with the three-part description mandated in the
constructor docstring.
- The accessor-interface principle is class-level: foundation operations (e.g.
`OpCreateSimpleComponentWithCopy`) expose the roles they build; **content-creation strategies**
(pluggable: what content a foundation builds around) expose accessors for the content they
created; edit operations expose their checks (`OpChangeProperty.assertHasChangedProperty`);
`OpOneOf`/`OpOptional` are queried for what ran. Tests never grope the document for something a
producer can be asked for.
- `ShapeProp` model: property duals with numeric tolerance; rotation is a writable attr, height
goes via resize (readonly in the Plugin API).
- `TestSuite` enumerates cases into a `TestTree` with stable per-test ids;
`run(ids, TestRunObserver)` is the ONLY output channel — the framework is UI-free by
construction. `plugin.ts` (panel adapter), `main.ts` (panel UI) and `src/ci/headless.ts`
(CI adapter) are three thin consumers.
- Cases live in `src/composable-tests/cases/` as `case<Identifier>.ts` (e.g. `MainEditSyncs` — the
sweep that found #10109).
- Panel checkboxes carry stable DOM ids (case identifier / `Identifier-N` composites) for remote
control via Playwright; recipe in the README.
## CI
Headless per-PR gate: `.github/workflows/tests-composable-suite.yml` runs
`pnpm --filter composable-test-suite run test:ci` — mocked backend (frontend e2e static server +
Playwright RPC fixtures, no backend/login), the in-sandbox bundle injected via `ɵloadPlugin`,
results streamed via console markers, `TEST_FILTER` by identifier substring. The mocked backend is
NOT a limitation for this suite (everything asserted is frontend store logic; empirically
confirmed against the interactive runs). Architecture mirrors `plugin-api-test-suite`'s CI driver;
the mock harness exists in THREE places that must stay in sync (provenance note in `ci/run-ci.ts`).
Details: README, "Running in CI".
## Substrate
`mem:common/test-setup`, `mem:common/component-data-model`, `mem:common/component-swap-pipeline`,
`mem:frontend/testing`.
-60
View File
@@ -1,60 +0,0 @@
# Frontend Architecture and Workflow
Frontend: CLJS SPA; React/Rumext; Potok; RxJS; okulary refs; SCSS modules; shared `common/`; JS/TS workspace packages.
## Stable namespace map
- `app.main.ui.*`: Rumext/React UI components for workspace, dashboard, viewer, settings, auth, nitrate, etc.
- `app.main.data.*`: Potok event handlers and side effects.
- `app.main.refs`: reactive refs/lenses over store and derived workspace data.
- `app.main.store`: Potok store and `emit!`.
- `app.plugins.*` and `app.plugins`: CLJS implementation of Plugin JS API proxies.
- `app.render_wasm.*`: frontend bridge to Rust/WASM renderer.
- `app.util.*`: DOM, HTTP, i18n, keyboard, codegen, and general frontend utilities.
- `frontend/packages/*` and `frontend/text-editor`: JS/TS workspace packages consumed by the app.
- Nitrate subscription/organization UI and flows live under `app.main.data.nitrate` and `app.main.ui.nitrate*`; backend/API behavior is covered by backend memories, and shared permission rules are in `common/src/app/common/types/nitrate_permissions.cljc`.
## Lint and Format
From `frontend/`:
- CLJ/CLJS lint: `pnpm run lint:clj`.
- JS lint currently no-ops via `pnpm run lint:js`.
- SCSS lint: `pnpm run lint:scss`.
- Format checks: `pnpm run check-fmt:clj`, `pnpm run check-fmt:js`, `pnpm run check-fmt:scss`.
- Format fix: `pnpm run fmt`, or targeted `fmt:clj` / `fmt:js` / `fmt:scss`. After running `fmt:*`, `check-fmt:*` is redundant.
- Translation formatting after i18n edits: `pnpm run translations`.
**Before linting:** if delimiter errors are suspected (after LLM edits, or
lint/compiler reports syntax errors), run `scripts/paren-repair` on the
affected files first. Delimiter errors produce misleading linter output.
See `mem:scripts/paren-repair`.
## Focused memory routing
UI and packages:
- App UI components, SCSS modules, style-system boundaries, accessibility, i18n, and render performance: `mem:frontend/ui-conventions-and-style-system`.
- JS/TS packages, shared UI package, text editor, Storybook, and package builds: `mem:frontend/ui-packages-text-editor-workflow`.
Workspace behavior:
- Workspace state, commits, persistence, undo, repo calls, and refs: `mem:frontend/workspace-state-persistence-subtleties`.
- Workspace transforms, modifier previews, WASM modifier integration, and transform commits: `mem:frontend/workspace-transform-subtleties`.
- Workspace token application/propagation: `mem:frontend/workspace-token-subtleties`; shared token data/schema: `mem:common/tokens-schema-subtleties`.
App shell and product flows:
- Routing, root app shell, websocket, and global errors: `mem:frontend/routing-app-shell-subtleties`.
- Dashboard and viewer flows: `mem:frontend/dashboard-viewer-subtleties`.
- Plugin JS API runtime inside the frontend app: `mem:frontend/plugin-api-to-cljs-binding`.
Diagnostics and validation:
- Runtime inspection and navigation: `mem:frontend/cljs-repl`.
- Source-edit compile/hot-reload diagnostics: `mem:frontend/compile-diagnostics`.
- Runtime crash recovery: `mem:frontend/handling-crashes`.
- Tests and live verification: `mem:frontend/testing`.
- Cross-cutting testing principles and anti-patterns: `mem:testing`.
- Real pointer/keyboard gesture reproduction: `mem:frontend/playwright-gestures`.
## Areas without focused memories
These frontend areas currently have no dedicated Serena memory beyond this architecture entry and nearby source/tests: clipboard, drawing tools, boolean/path operations, interactions/prototyping, color/style asset management, grid-layout editing UI, comments UI, fonts UI, and many dashboard/settings subflows. Treat work there as less memory-covered and inspect source/tests more carefully.
@@ -1,16 +0,0 @@
# Frontend Dashboard and Viewer Subtleties
## Dashboard
- Dashboard initialization fetches projects and fonts for the team, then listens to websocket messages only for global topic `uuid/zero` or the current profile id.
- Project fetch replaces each project map completely instead of merging, so fields such as `deleted-at` can disappear cleanly.
- Dashboard file/project mutations are often optimistic local updates with fire-and-forget RPC watchers. Bulk permanent delete/restore paths use SSE progress and progress notifications.
- File creation/duplication strips file `:data` before putting file summaries into dashboard state.
## Viewer
- Viewer initialization sets `:current-file-id`, `:current-share-id`, and `:viewer-local`, then fetches the view-only bundle. Comment threads are fetched only for logged-in users.
- Viewer bundle fetch sends the full supported feature set because anonymous shared viewers may not know team-enabled features.
- View-only bundles can contain pointer values in `:pages-index` and file data. Viewer resolves those fragments with `:get-file-fragment` before storing the bundle.
- `bundle-fetched` indexes pages and precomputes viewer frames/all-frames, stores libraries/users/thumbnails/permissions under `:viewer`, then navigates to frame id, query index, or auto-selected frame.
- Viewer zoom and interaction mode changes update both `:viewer-local` and the `:viewer` route query params.
@@ -1,38 +0,0 @@
# Frontend Runtime Crash Handling
## Detect a runtime workspace crash
Runtime crashes usually show the Internal Error page with title text "Something bad happened" and class `main_ui_static__download-link`. A common pattern is: changes go through via JS API / `execute_code`, then 1-2s later an `update-file` request reaches the backend and is rejected.
After a crash, `execute_code` can become unusable because no plugin instances are connected and any data in its `storage` is lost, but `cljs_repl` usually still works.
Check crash state:
```clojure
(some? (:exception @app.main.store/state))
```
It returns `true` when the Internal Error page is showing and `false` on a healthy workspace or after a successful reload.
## Read the runtime cause
The exception is stored at `(:exception @app.main.store/state)`. Useful keys:
- `:type`, `:code`, `:status`: error class, e.g. `:validation`, `:referential-integrity`, `400`.
- `:hint`, `:details`: human-readable explanation; `:details` often contains validation problems with `:shape-id`, `:page-id`, `:args`, etc.
- `:uri`: API endpoint that returned the error, e.g. `update-file`.
- `:app.main.errors/instance`: underlying JS Error object.
- `:app.main.errors/trace`: JS stack trace string, usually response-handling path rather than the dispatch site that produced the bad change.
```clojure
(let [ex (:exception @app.main.store/state)]
(select-keys ex [:type :code :status :hint :details :uri]))
```
For backend validation errors (`:type :validation`), `:details` is usually the most informative field; it identifies the shape and invariant that failed.
## Recover and continue testing
Simplest path when `cljs_repl` is still live (usually true after a crash): reload via repl with `(.reload js/location)` — see `mem:frontend/cljs-repl`. Alternatively via Playwright: find the workspace tab (URL contains `/#/workspace`, title ends `- Penpot`), select it if not current, then `playwright:browser_navigate` to that same URL. Either way, confirm recovery with `(some? (:exception @app.main.store/state))` returning `false`.
For backend-rejected changes, such as validation errors on `update-file`, changes are not persisted. Reload restores the pre-crash state, so it is safe to retry after fixing the cause.
Loaded 100 of 2693 files, more files were not shown because too many files have changed in this diff. Show more