diff --git a/.agents/README.md b/.agents/README.md index 1f4e2364c5..657ab6eac3 100644 --- a/.agents/README.md +++ b/.agents/README.md @@ -508,6 +508,7 @@ JSON, REPL access, and so on. | [`ste`](skills/ste/SKILL.md) | Rewrites prose in Simplified Technical English. Loads only when you name it. | | [`refine-prompt`](skills/refine-prompt/SKILL.md) | Rewrites a rough prompt into a clearer one. Never runs the prompt. | | [`update-changelog`](skills/update-changelog/SKILL.md) | Regenerates `CHANGES.md` from a GitHub milestone. | +| [`find-project-anomalies`](skills/find-project-anomalies/SKILL.md) | Checks a milestone against the Main project board and reports anomalies to `tmp/-ANOMALIES.md`. | ### A typical round diff --git a/.agents/skills/find-project-anomalies/SKILL.md b/.agents/skills/find-project-anomalies/SKILL.md new file mode 100644 index 0000000000..208db481a9 --- /dev/null +++ b/.agents/skills/find-project-anomalies/SKILL.md @@ -0,0 +1,116 @@ +--- +name: find-project-anomalies +description: Check a GitHub milestone against the Main project board, report the five anomaly types to tmp/-ANOMALIES.md, and fix missing milestone assignments on request. +--- + +# Skill: find-project-anomalies + +Check every issue and PR in a given GitHub milestone against the Main +project board. Report the five anomaly types below, then fix the ones a +human confirms. + +## Getting the version ($ARGUMENTS) + +The milestone version comes from the skill arguments (`$ARGUMENTS`), the +same way other skills receive theirs. Accept `2.17.0`, `v2.17.0`, +`milestone 2.17.0`, or any text clearly containing one `X.Y.Z` version. + +If no version is given and none can be deduced from the arguments or the +conversation context, **stop and ask the user** which milestone to check. +Do not guess, do not default to the latest release. + +## When to Use + +- Before a release, to catch board/milestone drift while there is still time + to fix it +- After triage sessions, to verify every milestone item is tracked, owned, + and correctly assigned +- Whenever someone asks "is milestone X.Y.Z clean on Main?" + +## Prerequisites + +- `gh` CLI authenticated (`gh auth status`) +- Python 3.8+ +- `scripts/gh.py` for GitHub queries, `scripts/project-anomalies.py` for + detection and reporting (`python3 scripts/project-anomalies.py check --help`) + +## Workflow + +### 1. Resolve the version (see above), then run the check + +```bash +python3 scripts/project-anomalies.py check "2.17.0" +# explicit path: +python3 scripts/project-anomalies.py check "2.17.0" --output tmp/2.17.0-ANOMALIES.md +``` + +This fetches milestone issues and PRs in bulk (one GraphQL call per 50 +items), resolves outsiders the same way, and overwrites +`tmp/-ANOMALIES.md`. Every number renders as a full +`[#N](https://github.com/penpot/penpot/issues/N)` or +`[#N](https://github.com/penpot/penpot/pull/N)` link. + +### 2. Read the report and judge each anomaly + +The five types, in report order: + +1. **OPEN issue with a MERGED PR** — the fix landed but the issue never + closed (or it reopened). Fix: close the issue, or move it out of the + milestone if the fix did not actually land here. +2. **Milestone issue, PR elsewhere or issue off Main** — a *merged* closing + PR in another milestone (or none), or the issue itself not on the Main + board. An unmerged PR's milestone means nothing (it landed nowhere), so + it never triggers this type. Fix: align the milestones (see step 3), or + add the issue to Main. +3. **Milestone PR, issue elsewhere or off Main** — the mirror view over + *merged* PRs only: a PR released here closes an issue tracked elsewhere + (or nowhere on Main). Unmerged PRs never trigger this type — their + milestone means nothing yet. Fix: align the milestones (see step 3), or + add the issue to Main. +4. **MERGED PR on a `needs triage` issue** — landed without triage. Only a + human can triage; never auto-remove the label. +5. **MERGED PR on an unassigned, non-community issue** — no owner and no + `community contribution` label (checked on both issue and PR). Either + assign an owner or confirm it is community work. Never invent an owner. + +Scope notes (deliberate, not bugs): +- Project membership is checked **on the issue side only**. PRs are not the + unit tracked on Main, so an unprojected PR alone is never an anomaly. +- An issue with *no* milestone closed by a milestone PR **is** reported + here (type 3): unlike the changelog flow, this pairing always needs a + human look. + +### 3. Fix missing milestone assignments (only these, only on confirmation) + +The only mechanical fix in this skill is assigning a missing milestone. +The report already prints the exact command next to each such case: + +```bash +gh pr edit --milestone "" +gh issue edit --milestone "" +``` + +Rules: +- Apply **only** when one side lacks a milestone. When both sides have + *different* milestones, a human decides which one moves — never pick a + side yourself. +- Confirm **each fix (or each small batch)** with the user before running + it. Read back what the command will change. +- Everything else (closing issues, triaging, assigning owners, adding + items to the Main board) is human work: point at it, do not do it. + +### 4. Re-run until clean + +After fixes land, re-run step 1 and confirm the report shows zero +anomalies. The `tmp/` report is scratch output (gitignored): quote or +paste entries into chat when reporting back, do not commit it. + +## Key Principles + +- **No version, no run.** Stop and ask when the milestone is unknown. +- **Report first, touch nothing.** No board or milestone changes without + explicit per-item confirmation. +- **Milestone gaps auto-fix; judgment calls do not.** Only a missing + milestone is mechanical. triage, ownership, and close/reopen decisions + belong to humans. +- **Every number clickable.** The report is useless without full GitHub links. diff --git a/.agents/skills/update-changelog/SKILL.md b/.agents/skills/update-changelog/SKILL.md index 5fa2883d6f..bd279754bc 100644 --- a/.agents/skills/update-changelog/SKILL.md +++ b/.agents/skills/update-changelog/SKILL.md @@ -20,7 +20,8 @@ primary link, with the fix PR inline on the same line. - `gh` CLI authenticated (`gh auth status`) - Python 3.8+ -- `scripts/gh.py` helper script available +- `scripts/gh.py` for GitHub queries, `scripts/changelog.py` for changelog + checks (run `python3 scripts/changelog.py --help` for usage) ## Workflow @@ -31,9 +32,6 @@ if not specified. ### 2. Fetch all issues in the milestone -Use the helper script. It uses GraphQL for efficient single-pass fetching -(closing PRs are included in the same query — no N+1): - ```bash # All closed issues (default) python3 scripts/gh.py issues "2.16.0" @@ -46,214 +44,135 @@ python3 scripts/gh.py issues "2.16.0" --exclude "release blocker,no changelog" ``` **Exclusion rules (issue-level):** -- `no changelog` label — Chore/refactor work that doesn't need a changelog entry -- `release blocker` label — Blocked issues not yet ready for changelog -- `Task` issue type — Internal chores are not user-facing; automatically excluded by `gh.py`. Use `--include-tasks` to override. -- **Rejected project status** — Issues with a "Rejected" status in the "Main" project board are automatically excluded by `gh.py`. This project-level status (independent of the GitHub issue `state`) indicates the issue was rejected from the release. Use `--include-rejected` to override. +- `no changelog` label — chore/refactor work, no entry needed +- `release blocker` label — blocked issues not yet ready for changelog +- `Task` issue type — internal chores, not user-facing; excluded by `gh.py` + (use `--include-tasks` to override) +- **Rejected project status** — issues with "Rejected" status on the "Main" + project board are excluded by `gh.py` (use `--include-rejected` to override). + This status is independent of the GitHub issue `state`. -**Exclusion rules (PR-level):** -In addition to issue-level exclusions, PRs with these labels should be -excluded regardless of their linked issue's labels: -- `release blocker` — PR is part of a pending release blocker batch -- `no issue required` — Trivial fix not tracked as an issue +**Exclusion rules (PR-level):** PRs with these labels stay out regardless of +their linked issue's labels: `release blocker`, `no issue required`. -The script outputs JSON with each entry containing `number`, `title`, `state`, -`issue_type`, `labels`, `closing_prs` (the PRs that fix each issue), and -`project_status` (the "Main" project board status, e.g. "Done", "Rejected", -or `null` if not tracked in a project). +Each entry carries `number`, `title`, `state`, `issue_type`, `labels`, +`closing_prs`, and `project_status`. ### 3. Identify missing entries (optional) -If updating from an existing `CHANGES.md`, find issues in the milestone that -are NOT yet referenced in the changelog: - ```bash python3 scripts/gh.py issues "2.16.0" --exclude "release blocker,no changelog" --compare CHANGES.md ``` -This returns a filtered JSON array with only the missing issues. - -> **Note:** The `--compare` flag checks **issues** only (via issue number -> references in the changelog). To find merged **PRs** not yet referenced, -> use the milestone PR cross-reference described in step 10 below. +Returns only milestone issues not yet referenced in the changelog. Note: it +compares **issues** only. For unreferenced merged **PRs**, use the +cross-reference in step 8. ### 4. Fetch additional PR details when needed -When you need more context for specific PRs (e.g. to find the PR author for -community contribution attribution, or to read the PR body for -"Fixes/Closes #NNN" patterns): - ```bash -# One or more PR numbers +# One or more PR numbers (also: --file prs.txt, or --stdin) python3 scripts/gh.py prs 9179 9204 9311 -# From a file -python3 scripts/gh.py prs --file prs.txt - -# From stdin -cat prs.txt | python3 scripts/gh.py prs --stdin -``` - -The `prs` command also supports listing all PRs in a milestone in one call: - -```bash -# All merged PRs in a milestone (default) +# All merged PRs in a milestone (default); --state all/open/closed for others python3 scripts/gh.py prs --milestone "2.16.0" - -# All states (merged, open, closed) -python3 scripts/gh.py prs --milestone "2.16.0" --state all ``` -The `prs` command returns JSON with `number`, `title`, `body`, `state`, -`merged_at`, `author`, `labels`, and `closing_issues`. PRs are fetched in -batches of 50 via GraphQL to stay within API limits (milestone mode uses -paginated GraphQL on the milestone's `pullRequests` connection). - -You can also list all PRs in a milestone in a single call: - -```bash -# All merged PRs in a milestone (default) -python3 scripts/gh.py prs --milestone "2.16.0" - -# All states (merged, open, closed) -python3 scripts/gh.py prs --milestone "2.16.0" --state all - -# Open PRs only -python3 scripts/gh.py prs --milestone "2.16.0" --state open -``` - -The milestone path uses paginated GraphQL on the milestone's `pullRequests` -connection (100 per page), avoiding one-by-one fetches. +Returns `number`, `title`, `body`, `state`, `merged_at`, `author`, `labels`, +and `closing_issues`. Milestone mode uses paginated GraphQL (100 per page). ### 5. Categorize entries — strictly by issue type, never by labels or emoji -Use the **Issue Type** field (GitHub's native issue type, exposed as -`issue_type` in the `gh.py` JSON output) to determine which section an entry -belongs to. +Use the **Issue Type** field (`issue_type` in the `gh.py` output). No separate +query is needed. > **⚠️ CRITICAL: Never use labels or title emoji prefixes for categorization.** -> Labels like `bug` and `enhancement`, as well as title prefixes like `:bug:` -> and `:sparkles:`, are frequently inaccurate, missing, or contradictory to the -> actual issue type. The `issue_type` field from `gh.py` is the single source -> of truth. +> Labels like `bug`/`enhancement` and prefixes like `:bug:`/`:sparkles:` are +> often wrong or missing. `issue_type` is the single source of truth. | `issue_type` value | Changelog section | |--------------------|-------------------| | `Bug` | `### :bug: Bugs fixed` | | `Feature` or `Enhancement` | `### :sparkles: New features & Enhancements` | -| `Task` | **Exclude** — internal chores are not user-facing | -| `null` (not set) | Check labels as a fallback: `bug` label → bugs, otherwise enhancements | +| `Task` | **Exclude** — internal chores, not user-facing | +| `null` (not set) | Fallback to labels: `bug` label → bugs, otherwise enhancements | -The `gh.py` issues command already includes `issue_type` in every entry's -output. **No separate GraphQL query is needed.** +**Breaking changes override everything:** an issue with the `breaking change` +label goes under `### :boom: Breaking changes & Deprecations`, no matter its +issue type. This is the only label-based rule — it is an explicit exception +to the "never by labels" principle above. The `:boom:` subsection goes first +in the version, before `:rocket:` (matching the existing precedent). -**Preserve highlighted entries:** If an entry is already featured in -`### :rocket: Epics and highlights`, keep it in that section when refreshing a -changelog version. Do not remove a highlighted entry just because issue type -categorization would otherwise place it under `### :sparkles: New features & -Enhancements`. +**Preserve highlighted entries:** if an entry already sits under +`### :rocket: Epics and highlights`, keep it there when refreshing. Do not +move it down just because its type would place it under `:sparkles:`. -**Community contribution attribution:** If the issue or its fix PR has the -`community contribution` label, add an attribution `(by @)` -on the changelog entry line, **before** the GitHub issue/PR references. +**Community attribution:** if the issue or its fix PR has the +`community contribution` label, add `(by @)` on the entry +line, **before** the issue/PR references. Use the **PR author** (the `author` +field from step 4), not the issue author: -The attribution should reference the **PR author**, not the issue author. -The `prs` subcommand includes the `author` field — use that: - -```bash -python3 scripts/gh.py prs | python3 -c "import sys,json; print(json.load(sys.stdin)[0]['author'])" -``` - -Placement in the entry line: ```markdown - Fix description of the bug (by @username) [#](...) (PR: [#](...)) ``` -**Only closed issues are included.** An issue must have `state: "closed"` to -appear in the changelog. Open/unresolved issues are omitted, even if they are -tracked in the milestone. +**Only closed issues are included**, even if open ones are tracked in the +milestone. **Pairing rules:** | Pattern | Changelog format | |---------|-----------------| -| Closed issue + one or more PRs fix it | Primary link = issue, PR inline comma-separated | -| PR exists with no linked issue | If a corresponding closed issue exists in the same milestone, link the issue. Otherwise, skip the entry (the issue must be the changelog unit). | -| Closed issue with no fix PR in milestone | Link the issue directly, without a PR reference. | +| Closed issue + one or more fix PRs | Primary link = issue, PRs inline comma-separated | +| PR with no linked issue | Link the issue if a matching closed one exists in the milestone; otherwise skip (the issue is the changelog unit) | +| Closed issue with no fix PR in milestone | Link the issue directly, no PR reference | -> **False-positive associations:** A PR may incorrectly claim to close an issue -> from a different context (e.g., a very old PR referencing a modern issue, or a -> cross-project reference). If the PR title and issue title are clearly unrelated, -> or the PR was created years before the issue, treat it as a data glitch and -> skip it. PR [#3](https://github.com/penpot/penpot/pull/3) (ancient License PR -> claiming to close a plugin API issue) is a known example. +> **False-positive associations:** a PR may wrongly claim to close an issue +> from another context (ancient PR, cross-project reference). If titles are +> clearly unrelated or the PR predates the issue by years, treat it as a data +> glitch and skip it. -### 5a. ⚠️ Verify PR merge status before writing +### 5a. Verify PR merge status before writing A closed issue may list closing PRs that were **closed without merging** -(e.g., a community PR that was superseded by another). The changelog must -only reference **merged** PRs. Verify before writing: +(e.g. a superseded community PR). Only **merged** PRs go in the changelog: ```bash -# Collect all PR numbers from the candidate entries and check them -python3 scripts/gh.py prs | python3 -c " -import json, sys -for pr in json.load(sys.stdin): - if pr['state'] != 'MERGED': - print(f'WARNING: #{pr[\"number\"]} is {pr[\"state\"]} (not merged)') -" +python3 scripts/changelog.py check-merged +# also accepts: --file prs.txt, or numbers via --stdin ``` -If a closing PR is closed-unmerged, find the actual merged PR that -superseded it: -1. Check the issue's closing PRs list for other PRs (there may be multiple) -2. Look for other PRs with similar titles or descriptions referencing the same issue -3. Inspect the closed PR's conversation timeline for a pointer to the replacement - -Replace the reference in the changelog entry with the correct merged PR number. +If a closing PR is closed-unmerged, find the merged PR that superseded it +(other PRs in the issue's closing list, similar titles, or pointers in the +closed PR's timeline) and reference that one instead. ### 5b. Security advisory (GHSA) entries -Security advisories fixed in a release are documented in the changelog even -though they are **neither milestone issues nor PRs**. The GHSA ID and its -description are supplied by the user or the release notes — they never come -from the milestone fetch in step 2. - -**Format** (matches the existing precedent in `CHANGES.md`, e.g. the -`create-font-variant` arbitrary file read advisory): +Advisories fixed in a release go in the changelog even though they are +**neither milestone issues nor PRs**. The GHSA ID and description come from +the user or the release notes — never from the milestone fetch. ```markdown - Fix (https://github.com/penpot/penpot/security/advisories/GHSA-XXXX-XXXX-XXXX) ``` -Rules: -- Place the entry under `### :bug: Bugs fixed`, with **no issue or PR link** — - only the advisory URL. -- The advisory may be **draft/unpublished** at changelog time (the URL 404s - publicly). Do **not** web-fetch or verify the URL, and do **not** drop the - entry because of that. Rely on the GHSA ID provided by the user. -- Derive the description from the supplied advisory title, imperative mood and - user-facing (e.g. `Fix command injection in SVG exporter via legacy fill-color`). -- These entries are **invisible to the automation**: they are not returned by - `gh.py issues`, not matched by `--compare` (step 3), not part of the PR - cross-reference (step 10), and not scanned by the anomaly-report regexes - (step 11, which only match `issues/` and `pull/` links). Add them manually. -- During pre-flight checks (step 6a) apply only the **backport/duplicate** - check: if the same GHSA already appears in an earlier version section, remove - it from the current section. Their absence from milestone cross-references - is expected, not an anomaly. +Rules: place under `### :bug: Bugs fixed` with **no issue or PR link**; do +**not** fetch or verify the URL (it may be draft/unpublished and 404); write +the description in imperative mood from the advisory title. These entries are +invisible to the automation — add them by hand, and in step 9 apply only the +backport/duplicate check to them. -### 6. Read the current CHANGES.md +### 6. Read the current CHANGES.md and run pre-flight checks -Read the top of `CHANGES.md` to understand the existing format and find the -insertion point (newest version goes at the top, after the `# CHANGELOG` -header). - -Key format rules from the existing file: +Newest version goes at the top, right after the `# CHANGELOG` header: ```markdown ## +### :boom: Breaking changes & Deprecations + +- [#](https://github.com/penpot/penpot/issues/) (PR: [#](https://github.com/penpot/penpot/pull/)) + ### :bug: Bugs fixed - Fix description of the bug [#](https://github.com/penpot/penpot/issues/) (PR: [#](https://github.com/penpot/penpot/pull/)) @@ -264,86 +183,38 @@ Key format rules from the existing file: - Add new feature description [#](https://github.com/penpot/penpot/issues/) (PR: [#](https://github.com/penpot/penpot/pull/)) ``` -Format details: -- Entries start with `- ` followed by a short description in imperative mood -- Primary link is **always the issue** (user-facing artifact) -- PR references are inline on the same line: `(PR: [#]())` - If an issue has multiple fix PRs, they are comma-separated: - `(PR: [#](), [#]())` -- The description should describe the fix/feature from the user's perspective -- Community contributions get `(by @)` **before** the issue link -- Sections are separated by a blank line between the last entry and the next - section title -- Only include a section if there are entries for it -- When an entry already exists in an earlier version section, it must be removed - from the current version to avoid duplicates +Format details: entries start with `- ` plus a short imperative description; +PR refs stay inline (`(PR: [#]())`, comma-separated for several); +`(by @)` goes before the issue link; only include non-empty +sections; blank line between a section's last entry and the next title; never +duplicate an entry from an earlier version section (the earlier version wins +for backports). -### 6a. Pre-flight checks — fix rule violations in the changelog +**Pre-flight checks — fix violations directly in `CHANGES.md` before writing +the new section.** Reconcile every existing entry (any version section) and +every milestone candidate against the current milestone state (re-fetch, do +not trust cached data): -**The LLM must apply these checks during the workflow and fix any -violations directly in `CHANGES.md`. They are not anomalies — they are -process errors that should be corrected before writing the new section.** - -The changelog is a *snapshot* of the milestone at a point in time, but -milestones and changelog entries can drift. The LLM must reconcile the -existing changelog against the current state of the milestone and the -existing changelog entries. - -For each entry that already exists in `CHANGES.md` (in any version -section) or in the candidate set for the current milestone, check: - -1. **Duplicate across versions.** Is the same issue already documented - in another (older) version section? If yes, this is a *backport*: - - The user-facing fix was already released. Remove the duplicate - from the current section. The earlier version is the canonical - reference. - -2. **Stale milestone assignment.** Has the issue been moved out of the - current milestone since the changelog was last updated (e.g., a fix - arrived late and the issue was reassigned to a future milestone)? - - Verify the issue is still in the current milestone via - `python3 scripts/gh.py issues --state all`. If it's no - longer there, remove the entry from the current section. (If the - target section doesn't exist yet, the entry is simply dropped.) - -3. **Exclusion labels newly applied.** Did the issue acquire a - `no changelog` or `release blocker` label since the changelog was - last updated? If yes, remove the entry from the current section. - -4. **Issue state changed.** Is the issue still closed? Has it been - reopened, deleted, or moved to a `Rejected` project status? If yes, +1. **Duplicate across versions** → remove from the current section. +2. **Stale milestone assignment** (issue moved out of this milestone) → + remove the entry (or drop it if the section does not exist yet). +3. **Newly applied exclusion labels** (`no changelog`, `release blocker`) → remove the entry. - -5. **Unmerged or removed PR references.** For every PR referenced in - the entry, is the PR still merged? Was the PR closed without - merging (superseded)? Was the PR moved to a different milestone? - If the only referenced PR is no longer merged, fix the reference - (find the actual merged fix PR) or remove the entry. A PR that is - merged in a *different* milestone is reported as an anomaly in - step 11 — do not silently remove it. - -6. **Issue type changed.** Did the issue type change (e.g., from Bug to - Task)? If the new type is `Task`, the issue is internal and should - be removed. - -7. **Cross-section completeness.** For every closed, non-excluded - milestone issue that is *not* referenced in any version section of - the changelog, add it to the current section (per the categorization - rules in step 5). - -After these checks, the changelog should be internally consistent with -the milestone. **Do not defer these fixes to step 11 — they are -workflow errors, not anomalies.** Step 11 only reports milestone -mismatches that require human judgment about the team's release -intent. +4. **Issue no longer closed/deleted/Rejected** → remove the entry. +5. **Unmerged or moved PR reference** → fix the reference or remove the + entry (a PR merged in a *different* milestone is a step-9 anomaly, do not + silently remove it). +6. **Issue type changed to `Task`** → remove the entry. +7. **Breaking change misplaced** (issue has the `breaking change` label but + sits in another section) → move the entry to `:boom:`. +8. **Missing valid issues** (closed, non-excluded, unreferenced anywhere) → + add them to the current section per step 5. ### 7. Build the description text -Derive the description from the issue title, not the PR title. Strip leading -emoji prefixes (`:bug:`, `:sparkles:`, `:tada:`) and focus on the -user-facing behavior. - -Examples: +Derive it from the **issue title**, not the PR title. Strip leading emoji +prefixes (`:bug:`, `:sparkles:`, `:tada:`) and describe the user-facing +behavior: | Issue title | Changelog description | |-------------|----------------------| @@ -351,589 +222,72 @@ Examples: | `Comment content is not sanitized before rendering, enabling stored XSS` | `Sanitize comment content on rendering` | | `Custom uploaded font family names are not sanitized` | `Sanitize font family names on custom uploaded fonts` | -### 8. Insert the section into CHANGES.md +Insert the new version section right after the `# CHANGELOG` header with the +`edit` tool and enough context for a unique match. -Insert the new version section right after the `# CHANGELOG` header (before -the previous version entry). Use the `edit` tool with enough context to make -a unique match. +### 7b. Propose and populate `:rocket: Epics and highlights` -### 8b. Propose and populate the `:rocket: Epics and highlights` subsection +Create the subsection if missing (place it before `### :sparkles:`) and pick +2–5 of the most impactful/user-visible `:sparkles:` entries: new visible +features, big capabilities, items that make self-hosted users want to update. +`frontend/src/app/main/ui/releases/v2_.cljs` slide titles are optional +hints (they may not exist for every version). Every `:rocket:` entry MUST +carry issue AND PR references; never remove entries from a prior run. -After inserting the version section, proactively create or populate the -`### :rocket: Epics and highlights` subsection. This section surfaces the -most impactful changes for self-hosted users checking for updates. - -**When to create:** If the version section does not already have a -`### :rocket: Epics and highlights` subsection, create one. Place it before -`### :sparkles:` (matching existing order in CHANGES.md). - -**How to identify highlights:** Review the `:sparkles:` entries for the -version and select 2–5 of the most impactful/user-visible ones. Criteria: -- New user-visible features (not internal refactors) -- Significant capability additions -- Items that create "FOMO" for self-hosted users on older versions - -**Use release notes as hints:** Check -`frontend/src/app/main/ui/releases/v2_.cljs` for the corresponding -version. The slide titles and feature descriptions there are curated -marketing content indicating what the team considers highlight-worthy. Match -those themes to changelog entries. Treat these files as optional hints — they -may not exist for every version. - -**Format requirement:** Every `:rocket:` entry MUST follow the standard -changelog format with issue/PR references: -``` -- [#](https://github.com/penpot/penpot/issues/) (PR: [#](https://github.com/penpot/penpot/pull/)) -``` -An entry without issue AND PR references is a highlight gap (warning, not an anomaly). - -**Preserve existing entries:** If the `:rocket:` section already exists from -a prior run, preserve its entries. Do not remove or rewrite them. - -### 9. Verify - -Read the top of `CHANGES.md` and confirm: -- The version header is correct -- Every entry has a GitHub link -- Entries with a fix PR have the PR sub-line -- The section ordering is correct (newest first) -- Formatting matches the surrounding entries - -### 10. Cross-reference milestone PRs against the changelog - -Issues can be fixed by PRs that aren't in the milestone, and merged PRs in -the milestone may not close any tracked issue. After writing, run a full -cross-reference to catch gaps: +### 8. Cross-reference milestone PRs against the changelog ```bash -# List all merged PRs in the milestone -python3 scripts/gh.py prs --milestone "" --state merged > /tmp/milestone-prs.json - -# Extract PR numbers from the changelog section -python3 -c " -import json, re - -with open('CHANGES.md') as f: - content = f.read() - -# Extract the version section (adjust regex to match the actual version) -match = re.search(r'## \(Unreleased\)\n(.*?)(?:\n## |\Z)', content, re.DOTALL) -section = match.group(1) - -# Collect all PR numbers referenced -changelog_prs = set() -for m in re.findall(r'\[#(\d+)\]\(https://github\.com/penpot/penpot/pull/\d+\)', section): - changelog_prs.add(int(m)) - -# Collect all milestone PRs (filtered) -with open('/tmp/milestone-prs.json') as f: - milestone_prs = json.load(f) - -milestone_merged = {pr['number'] for pr in milestone_prs} - -# PRs in milestone but not in changelog -missing = sorted(milestone_merged - changelog_prs) -print(f'Milestone merged PRs: {len(milestone_merged)}') -print(f'Changelog referenced PRs: {len(changelog_prs)}') -print(f'PRs in milestone but NOT in changelog: {len(missing)}') -for num in missing: - pr = next(p for p in milestone_prs if p['number'] == num) - print(f' #{num} {pr[\"title\"][:80]}') -" +python3 scripts/changelog.py cross-ref "" [--changes CHANGES.md] ``` -For each missing PR found, decide whether it should be added to the -changelog or is legitimately excluded (check its labels). +Lists merged milestone PRs missing from the changelog section (decide per PR: +add it or confirm its exclusion labels) and warns about CLOSED (unmerged) PRs +in the milestone. GHSA entries never appear here — that absence is expected. -Also verify that no closed-unmerged PRs remain in the changelog: +### 9. Generate the anomaly report ```bash -python3 scripts/gh.py prs --milestone "" --state all | python3 -c " -import json, sys -data = json.load(sys.stdin) -closed = [p for p in data if p['state'] == 'CLOSED'] -if closed: - print('WARNING: CLOSED (unmerged) PRs in milestone:') - for p in closed: - print(f' #{p[\"number\"]} {p[\"title\"][:80]}') -" +python3 scripts/changelog.py report "" [--changes CHANGES.md --output CHANGES-ISSUES.md] ``` -**Post-edit audit checklist:** -- ✅ All referenced PRs are merged (no closed-unmerged artifacts) -- ✅ Every merged milestone PR is either in the changelog or excluded by label -- ✅ PR and issue counts are internally consistent -- ✅ No false-positive PR-to-issue associations -- ✅ Advisory (GHSA) entries are not milestone PRs — their absence from the - cross-reference is intentional (see step 5b) +Overwrites `CHANGES-ISSUES.md` with the current state. Every number renders as +a full `[#N](https://github.com/penpot/penpot/issues/N)` or +`[#N](https://github.com/penpot/penpot/pull/N)` link. -## Version section template +**An anomaly is a milestone mismatch or a `:boom:` mislabel** (the changelog +pairing is misleading and a human must judge intent): -```markdown -## +1. Issue in this milestone, referenced PR in another milestone (or none). +2. PR in this milestone, closed issue in another milestone — except an issue + with *no* milestone, which belongs to another (probably private) project + and is neither anomaly nor changelog candidate. +3. `:boom:` entry whose issue lacks the `breaking change` label. These stay + listed in the report **and** in the changelog: either label the issue or + move the entry to its regular section. -### :bug: Bugs fixed +**Highlight gaps are warnings, not anomalies:** missing `:rocket:` on a +released X.Y.0 (patches never carry highlights); `:rocket:` entry without +issue AND PR references. Gaps never count toward the anomaly total. -- [#](https://github.com/penpot/penpot/issues/) (PR: [#](https://github.com/penpot/penpot/pull/)) -- (by @contributor) [#](https://github.com/penpot/penpot/issues/) (PR: [#](https://github.com/penpot/penpot/pull/)) -- (https://github.com/penpot/penpot/security/advisories/GHSA-XXXX-XXXX-XXXX) -``` - -Advisory (GHSA) entries have no issue or PR link — just the advisory URL. See -step 5b. - -### 11. Generate anomaly report and save to CHANGES-ISSUES.md - -After all edits and cross-referencing are complete, generate a structured -report and save it to `CHANGES-ISSUES.md` (overwriting if exists). -This provides a persistent record of any discrepancies between the milestone -and the changelog. - -**Every issue and PR number in the report must be rendered as a full GitHub -Markdown link** using the same URL format as `CHANGES.md`: -- Issue N → `[#N](https://github.com/penpot/penpot/issues/N)` -- PR N → `[#N](https://github.com/penpot/penpot/pull/N)` - -The titles and notes should also link to the corresponding issue/PR page -where applicable, so the report is self-contained and clickable from any -Markdown viewer. - -## What is an anomaly - -**An anomaly is a milestone-mismatch between an issue and its referenced -PR.** There are two anomaly types, plus two highlight gaps (warnings that -do not count toward the anomaly total): - -1. **Issue is in the milestone, but its referenced PR is in a different - milestone (or has no milestone).** The changelog claims a fix in this - release, but the PR is being released elsewhere — the fix may not - actually ship here. -2. **PR is in the milestone, but the issue it closes is in a different - milestone.** The PR is being released here, but the issue it fixes is - being released in a different version — the changelog pairing is - misleading. - - **Exception — issue with no milestone is NOT an anomaly.** Milestones - are only required for issues tracked in the "Main" project. A milestone - PR that closes an issue with no milestone references an issue from - another (probably private) project; that is expected and the issue is - not part of this changelog. Do not report it. -3. **missing-highlights (gap):** A released X.Y.0 version section has no - `### :rocket: Epics and highlights` subsection. Patches (X.Y.Z) never - carry highlights, so only minors/majors are checked. -4. **missing-highlight-reference (gap):** A `:rocket:` entry lacks the - required issue AND PR references. Every highlight entry must follow the - standard changelog format with `[#ISSUE]` and `(PR: [#PR])` links - (multi-PR `(PR: [#A](...), [#B](...))` accepted). - -**Anything else is not an anomaly.** Other discrepancies (exclusion -labels on in-changelog issues, missing valid issues, unmerged PR -references, duplicates across versions, stale milestone assignments) -are **rule violations** that the LLM must fix directly in `CHANGES.md` -during step 6a (pre-flight checks). They should not appear in this -report — if they do, the LLM has skipped the pre-flight step and -needs to re-run the workflow. - -The changelog's primary unit is the **issue**, not the PR, so a missing or -mismatched PR only matters when its issue is part of this milestone. - -Run this self-contained script: - -```bash -python3 << 'PYEOF' -import json, re, subprocess, sys -from datetime import datetime, timezone - -MILESTONE = "" -CHANGES_MD = "CHANGES.md" -OUTPUT = "CHANGES-ISSUES.md" -REPO = "penpot/penpot" - -# --- URL helpers (match CHANGES.md format exactly) --- -def issue_url(n): return f"https://github.com/{REPO}/issues/{n}" -def pr_url(n): return f"https://github.com/{REPO}/pull/{n}" -def issue_link(n): return f"[#{n}]({issue_url(n)})" -def pr_link(n): return f"[#{n}]({pr_url(n)})" -def issue_link_title(n, title): - url = issue_url(n) - if title: - return f"[#{n}]({url}) — [{title}]({url})" - return f"[#{n}]({url})" -def pr_link_title(n, title): - url = pr_url(n) - if title: - return f"[#{n}]({url}) — [{title}]({url})" - return f"[#{n}]({url})" -def fmt_pr_list(nums): - return ", ".join(pr_link(n) for n in nums) -def fmt_issue_list(nums): - return ", ".join(issue_link(n) for n in nums) - -# --- Fetch milestone data --- -result = subprocess.run( - ["python3", "scripts/gh.py", "issues", MILESTONE, "--state", "all"], - capture_output=True, text=True) -all_issues = json.loads(result.stdout) -issue_by_num = {i['number']: i for i in all_issues} - -result = subprocess.run( - ["python3", "scripts/gh.py", "prs", "--milestone", MILESTONE, "--state", "all"], - capture_output=True, text=True) -all_prs = json.loads(result.stdout) -pr_by_num = {p['number']: p for p in all_prs} - -# --- Read changelog section --- -with open(CHANGES_MD) as f: - content = f.read() - -m = re.search(rf'## {re.escape(MILESTONE)}(?:\s*\([^)]*\))?\n(.*?)(?:\n## |\Z)', content, re.DOTALL) -section = m.group(1) if m else "" - -changelog_issues = set() -for num in re.findall(r'\[#(\d+)\]\(https://github\.com/penpot/penpot/issues/\d+\)', section): - changelog_issues.add(int(num)) -for num in re.findall(r'\[Github #(\d+)\]', section): - changelog_issues.add(int(num)) - -changelog_prs = set() -for num in re.findall(r'\[#(\d+)\]\(https://github\.com/penpot/penpot/pull/\d+\)', section): - changelog_prs.add(int(num)) -for num in re.findall(r'PR:\[(\d+)\]', section): - changelog_prs.add(int(num)) - -# --- Milestone lookup caches --- -# PRs and issues returned by milestone queries are KNOWN to be in MILESTONE. -# For everything else, fall back to `gh` per-item lookups. -pr_milestone_cache = {p['number']: MILESTONE for p in all_prs} -issue_milestone_cache = {i['number']: MILESTONE for i in all_issues} - -def get_pr_milestone(pr_num): - """Return the milestone title for a PR, or None if unassigned / unknown.""" - if pr_num in pr_milestone_cache: - return pr_milestone_cache[pr_num] - try: - r = subprocess.run( - ["gh", "pr", "view", str(pr_num), "--json", "milestone"], - capture_output=True, text=True, check=True) - data = json.loads(r.stdout) - ms = data.get('milestone') - pr_milestone_cache[pr_num] = (ms or {}).get('title') - except (subprocess.CalledProcessError, json.JSONDecodeError): - pr_milestone_cache[pr_num] = None - return pr_milestone_cache[pr_num] - -def get_issue_milestone(issue_num): - """Return the milestone title for an issue, or None if unassigned / unknown.""" - if issue_num in issue_milestone_cache: - return issue_milestone_cache[issue_num] - try: - r = subprocess.run( - ["gh", "issue", "view", str(issue_num), "--json", "milestone"], - capture_output=True, text=True, check=True) - data = json.loads(r.stdout) - ms = data.get('milestone') - issue_milestone_cache[issue_num] = (ms or {}).get('title') - except (subprocess.CalledProcessError, json.JSONDecodeError): - issue_milestone_cache[issue_num] = None - return issue_milestone_cache[issue_num] - -# --- Exclusion rules (shared) --- -EXCLUDED_LABELS = {'release blocker', 'no changelog'} -EXCLUDED_ISSUE_TYPES = {'Task'} -EXCLUDED_PROJECT_STATUS = {'Rejected'} - -def issue_excluded(issue): - if not issue: return True - if issue.get('state') != 'CLOSED': return True - if issue.get('issue_type') in EXCLUDED_ISSUE_TYPES: return True - if issue.get('project_status') in EXCLUDED_PROJECT_STATUS: return True - if EXCLUDED_LABELS & set(issue.get('labels', [])): return True - return False - -# --- ANOMALIES: milestone mismatches between issues and their referenced PRs --- -# These are the ONLY items that should appear in the report. All other -# discrepancies (exclusion labels, missing valid issues, unmerged PRs, -# duplicates, stale milestone assignments) are workflow errors that the -# LLM must fix in step 6a (pre-flight checks) — they are not anomalies. - -# Type A: issue in MILESTONE, referenced PR in different milestone or no milestone -anomalies_a = [] # list of dicts: {issue, issue_title, pr, pr_milestone} -for issue_num in sorted(changelog_issues): - issue = issue_by_num.get(issue_num) - if not issue: continue - if get_issue_milestone(issue_num) != MILESTONE: continue - for pr_num in issue.get('closing_prs', []): - pr_ms = get_pr_milestone(pr_num) - if pr_ms != MILESTONE: - anomalies_a.append({ - 'issue': issue_num, - 'issue_title': issue.get('title', ''), - 'pr': pr_num, - 'pr_milestone': pr_ms, # may be None - }) - -# Type B: PR in MILESTONE, the issue it closes is in different milestone or no milestone -anomalies_b = [] # list of dicts: {pr, pr_title, issue, issue_milestone} -for pr_num in sorted(changelog_prs): - pr = pr_by_num.get(pr_num) - if not pr: continue - if get_pr_milestone(pr_num) != MILESTONE: continue - for issue_num in pr.get('closing_issues', []): - issue_ms = get_issue_milestone(issue_num) - # No milestone = issue from another (probably private) project — - # milestones are only required for the "Main" project. Not an - # anomaly, and the issue never belongs in this changelog. - if issue_ms is None: continue - if issue_ms != MILESTONE: - anomalies_b.append({ - 'pr': pr_num, - 'pr_title': pr.get('title', ''), - 'issue': issue_num, - 'issue_milestone': issue_ms, # may be None - }) - -# --- Type C: released X.Y.0 version sections without :rocket: subsection --- -# Patches (X.Y.Z with Z != 0) never carry :rocket: by design — only minors/majors (X.Y.0). -anomalies_c = [] # list of version strings -rocket_heading_re = re.compile(r'^### :rocket:', re.MULTILINE) -version_sections = re.split(r'(?=^## \d+\.\d+\.\d+)', content, flags=re.MULTILINE) -for vs in version_sections: - m = re.match(r'^## (\d+\.\d+\.\d+)(.*)', vs) - if not m: continue - ver, suffix = m.group(1), m.group(2) - if 'unreleased' in suffix.lower(): continue - if ver.split('.')[2] != '0': continue - if not rocket_heading_re.search(vs): - anomalies_c.append(ver) - -# --- Type D: :rocket: entries without issue AND PR references --- -# Both are required: `[#ISSUE](.../issues/N)` and `(PR: [#PR](.../pull/M))`. -# Multi-PR entries `(PR: [#A](...), [#B](...))` are accepted. -anomalies_d = [] # list of dicts: {version, line} -issue_ref_re = re.compile(r'\[#\d+\]\(https://github\.com/penpot/penpot/issues/\d+\)') -pr_ref_re = re.compile(r'\(PR:\s*\[#\d+\]\(https://github\.com/penpot/penpot/pull/\d+\)(\s*,\s*\[#\d+\]\(https://github\.com/penpot/penpot/pull/\d+\))*\)') -for vs in version_sections: - m = re.match(r'^## (\d+\.\d+\.\d+)(.*)', vs) - if not m: continue - ver = m.group(1) - rocket_match = rocket_heading_re.search(vs) - if not rocket_match: continue - # Extract the :rocket: subsection body (up to next ### or ##) - rocket_body = vs[rocket_match.end():] - rocket_body = re.split(r'(?m)^#{2,3}\s', rocket_body)[0] - for line in rocket_body.splitlines(): - line = line.strip() - if line.startswith('- ') and not (issue_ref_re.search(line) and pr_ref_re.search(line)): - anomalies_d.append({'version': ver, 'line': line[:100]}) - -# --- Write report --- -def fmt_ms(ms): - return ms if ms else "_none_" - -with open(OUTPUT, 'w') as f: - f.write(f'# Changelog Anomaly Report — {MILESTONE}\n\n') - f.write(f'Generated: {datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")}\n\n') - f.write('---\n\n') - - n_a = len(anomalies_a) - n_b = len(anomalies_b) - n_c = len(anomalies_c) - n_d = len(anomalies_d) - - f.write('## Summary\n\n') - f.write(f'- **Issue in {MILESTONE}, referenced PR in different milestone or no milestone:** {n_a}\n') - f.write(f'- **PR in {MILESTONE}, closing issue in a different milestone:** {n_b}\n') - f.write(f'- **Total anomalies:** {n_a + n_b}\n') - f.write(f'- **Released X.Y.0 version missing :rocket: section (gap):** {n_c}\n') - f.write(f'- **:rocket: entry without issue AND PR references (gap):** {n_d}\n\n') - - # --- Anomalies section (milestone mismatches only) --- - if n_a or n_b: - f.write('## Anomalies\n\n') - f.write('These are milestone mismatches between an issue in the changelog ' - 'and its referenced PR (or vice-versa). The changelog claim ' - '"this issue is fixed by this PR, all in this milestone" is ' - 'inconsistent with the actual milestone assignments. ' - 'Resolve by either updating the milestone on the issue/PR or ' - 'removing the misleading entry from the changelog.\n\n') - - if n_a: - f.write(f'### Issue in {MILESTONE}, PR in different milestone or no milestone\n\n') - by_issue = {} - for a in anomalies_a: - by_issue.setdefault(a['issue'], []).append(a) - for issue_num in sorted(by_issue): - entries = by_issue[issue_num] - title = entries[0]['issue_title'] - f.write(f'- {issue_link_title(issue_num, title[:80])}\n') - for e in entries: - ms_label = fmt_ms(e['pr_milestone']) - badge = '🔴' if e['pr_milestone'] is None else '⚠️' - f.write(f' - {badge} Referenced {pr_link(e["pr"])} is in milestone **{ms_label}** (expected: {MILESTONE})\n') - f.write('\n') - - if n_b: - f.write(f'\n### PR in {MILESTONE}, closing issue in a different milestone\n\n') - by_pr = {} - for b in anomalies_b: - by_pr.setdefault(b['pr'], []).append(b) - for pr_num in sorted(by_pr): - entries = by_pr[pr_num] - title = entries[0]['pr_title'] - f.write(f'- {pr_link_title(pr_num, title[:80])}\n') - for e in entries: - ms_label = fmt_ms(e['issue_milestone']) - badge = '🔴' if e['issue_milestone'] is None else '⚠️' - f.write(f' - {badge} Closing {issue_link(e["issue"])} is in milestone **{ms_label}** (expected: {MILESTONE})\n') - f.write('\n') - - else: - f.write('✅ No anomalies found. All (issue, PR) pairs in the changelog have aligned milestone assignments.\n\n') - - # --- Highlight gaps (warnings, not anomalies) --- - if n_c or n_d: - f.write('## Highlight gaps\n\n') - f.write('These are warnings, not anomalies: they do not affect the ' - 'milestone-mismatch total above. They track `:rocket:` coverage ' - 'across all released X.Y.0 versions. Historical entries (e.g. ' - 'Taiga links) predate the current reference convention and are ' - 'expected to appear here.\n\n') - - if n_c: - f.write(f'### Released X.Y.0 version missing :rocket: section\n\n') - f.write('These released minors/majors have no `### :rocket: Epics and highlights` subsection. ' - 'Add highlights to help self-hosted users understand what they are missing.\n\n') - for ver in anomalies_c: - f.write(f'- Version **{ver}**\n') - f.write('\n') - - if n_d: - f.write(f'### :rocket: entry without issue AND PR references\n\n') - f.write('These highlight entries lack the required issue AND PR references. ' - 'Add `[#ISSUE](...)` and `(PR: [#PR](...))` links.\n\n') - for d in anomalies_d: - f.write(f'- **{d["version"]}**: `{d["line"]}`\n') - f.write('\n') - elif not (n_a or n_b): - f.write('✅ No highlight gaps found. All released X.Y.0 versions have properly referenced :rocket: entries.\n\n') - - # --- Context --- - f.write('---\n\n') - f.write('## Context\n\n') - f.write(f'- Milestone: **{MILESTONE}**\n') - f.write(f'- Milestone total issues (all states): {len(all_issues)}\n') - f.write(f'- Closed issues in milestone: {sum(1 for i in all_issues if i.get("state") == "CLOSED")}\n') - f.write(f'- Valid issues after exclusions (after step 5/6a): {len([i for i in all_issues if not issue_excluded(i)])}\n') - f.write(f'- Issues referenced in changelog: {len(changelog_issues)}\n') - f.write(f'- PRs referenced in changelog: {len(changelog_prs)}\n') - -print(f"Anomaly report written to {OUTPUT}") -PYEOF -``` - -This generates `CHANGES-ISSUES.md` containing anomalies and highlight gaps: - -1. **Issue in milestone, referenced PR in different milestone or no milestone** — - the changelog claims a fix here, but the PR is released elsewhere. -2. **PR in milestone, closing issue in a different milestone** — - the PR is released here, but the issue it fixes belongs to another version. - (An issue with *no* milestone belongs to another, probably private, - project — milestones are only required on the "Main" project — so it is - neither an anomaly nor a changelog candidate.) -3. **missing-highlights (gap, warning)** — a released X.Y.0 version section - has no `### :rocket: Epics and highlights` subsection. Patches (X.Y.Z) - never carry highlights. -4. **missing-highlight-reference (gap, warning)** — a `:rocket:` entry lacks - the required issue AND PR references. - -Gaps do not count toward the anomaly total. - -**Rule violations are not in the report** — they are workflow errors the -LLM must fix directly in `CHANGES.md` during step 6a (pre-flight checks). -If the report contains a rule violation, the LLM has skipped the pre-flight -step and needs to re-run the workflow before re-generating the report. - -The report is overwritten each time it's generated, reflecting the current -state of the milestone and changelog. Every number is rendered as a full -`[#N](https://github.com/penpot/penpot/issues/N)` or -`[#N](https://github.com/penpot/penpot/pull/N)` link so the report is -self-contained and clickable in any Markdown viewer. +**Anything else is a rule violation**, not a report item: fix it in step 6 +pre-flight. If one shows up in the report, re-run the workflow. ## Key Principles -- **Issue = changelog unit.** The primary link always points to the - user-facing issue, not the implementation PR. -- **PR = implementation detail.** Reference the PR inline so readers - can find the code changes. -- **Latest version first.** New sections are inserted at the top of the - changelog, below the `# CHANGELOG` header. -- **Issue Type determines section — exclusively.** Use the `issue_type` field from `gh.py` output (Bug → `:bug:`, Feature/Enhancement → `:sparkles:`). **Do not** use labels (`bug`, `enhancement`) or title emoji prefixes (`:bug:`, `:sparkles:`) — they are frequently wrong or contradictory. The `issue_type` is the single source of truth. -- **User-facing descriptions.** Write from the user's perspective — describe - what broke and what was fixed, not internal implementation details. -- **Community attribution.** When the issue or fix PR has the - `community contribution` label, add `(by @)` on the entry line - between the description and the issue link. Use the **PR author** (not the - issue author) for the attribution. -- **Only closed issues.** An issue must have `state: "closed"` to appear in - the changelog. Open/unresolved issues are omitted. -- **Rejected project status.** Issues marked as "Rejected" in the "Main" - project board are automatically excluded by `gh.py`, even if they are - closed. The project status is distinct from the GitHub issue state. - Use `--include-rejected` to override this behavior. -- **Excluded issues.** Issues with `no changelog` label must be excluded. - Issues with `issue_type: "Task"` must also be excluded — they are internal - chores, not user-facing changes. -- **Multiple PRs per issue.** If multiple PRs fix the same issue, list them - comma-separated inline: `(PR: [#A](url), [#B](url))`. -- **Duplicate removal.** If an entry already exists in a prior version section, - remove it from the current version. Check for text-level duplicates (after - stripping links and attributions) across version sections. -- **Taiga references.** If a changelog entry references a Taiga URL - (`tree.taiga.io`), attempt to find a corresponding GitHub issue via the - Taiga description text or by searching GitHub PRs that reference the Taiga - URL. Replace the Taiga reference with the GitHub issue link and add the PR - reference if applicable. -- **Security advisory (GHSA) entries.** Advisories fixed in the release are - listed under `### :bug: Bugs fixed` with the advisory URL and **no issue or - PR link**, even though they are not in the milestone. The GHSA ID and - description come from the user — do **not** fetch or verify the URL, and do - not drop a draft (unpublished) advisory. Precedent: - `- Fix arbitrary file read security issue on create-font-variant rpc method - (https://github.com/penpot/penpot/security/advisories/GHSA-xp3f-g8rq-9px2)`. - See step 5b. -- **Re-fetch before editing.** Milestones can change — always re-fetch issues - before making edits, don't rely on cached data. -- **Use `scripts/gh.py`.** Prefer the helper script over raw `gh api` calls for - milestone issue listing and PR detail fetching. It handles GraphQL - pagination, batching, and label filtering automatically. -- **Verify PR merge status.** Not all closing PRs are merged — community PRs - can be superseded and closed without merging. Always check that every PR - referenced in the changelog has `state: MERGED`. -- **PR-level exclusions apply.** A PR can carry its own exclusion labels - (`release blocker`, `no issue required`) independent of its linked issue's - labels. Check both. -- **Cross-reference milestone PRs, not just issues.** The `--compare` flag on - the `issues` command only compares issue numbers. Merged PRs not linked to - any milestone issue can be missed. Use `python3 scripts/gh.py prs --milestone` - for a full PR cross-reference. -- **False-positive PR-to-issue associations.** A PR may claim to close an - issue from a different project or context. If the PR title and issue title - are clearly unrelated, or the PR predates the issue by years, treat it as a - data glitch and skip it. -- **Anomaly = milestone mismatch only; gaps are warnings.** The report's - anomaly total counts only milestone mismatches: (1) the issue is in this - milestone but the referenced PR is in a different milestone (or unassigned), - and (2) the PR is in this milestone but the issue it closes is in a - different milestone. `:rocket:` highlight gaps (missing section on a - released X.Y.0, entry without issue AND PR references) are reported in a - separate `Highlight gaps` section and never count toward the anomaly total. An - *unassigned* (milestone-less) issue closed by a milestone PR is **not** - an anomaly: milestones are required only for the "Main" project, so such - issues come from another (probably private) project and are not changelog - candidates. These anomalies are reported because the changelog pairing is - *misleading* — the human needs to decide whether the milestone or the - changelog is wrong. All other discrepancies (exclusion labels, missing - valid issues, unmerged PR references, duplicates, stale milestone - assignments) are **rule violations** that the LLM must fix directly in - `CHANGES.md` during step 6a (pre-flight checks). They never appear in - the report — if they do, the pre-flight step was skipped. +- **Issue = changelog unit**, PR = implementation detail inline. +- **Latest version first**, below the `# CHANGELOG` header. +- **Issue Type decides the section — exclusively**, except `breaking change` + label → `:boom:` first. +- **User-facing descriptions** in imperative mood. +- **Community attribution** uses the PR author, placed before the issue link. +- **Only closed issues**; **Rejected** project status excludes. +- **`no changelog` and `Task`** stay out. +- **Multiple fix PRs** go comma-separated inline. +- **Duplicates**: the earlier version section wins. +- **Taiga references**: resolve to the GitHub issue via description text or + PRs mentioning the Taiga URL, then link issue + PR. +- **GHSA entries** live under `:bug:` with only the advisory URL (see 5b). +- **Re-fetch before editing**; prefer `scripts/gh.py` over raw `gh api`. +- **Verify PR merge status** (`check-merged`); PR-level exclusions apply. +- **Cross-reference PRs, not just issues** (`cross-ref`); watch for + false-positive PR-to-issue links. diff --git a/.gitignore b/.gitignore index 5d3cbd93ef..7cd1fdc50c 100644 --- a/.gitignore +++ b/.gitignore @@ -110,6 +110,7 @@ opencode.json /.opencode/reports /.opencode/prompts /.ci-logs +/tmp/ /.codex/ /tools/__pycache__ /scripts/__pycache__ diff --git a/.serena/memories/critical-info.md b/.serena/memories/critical-info.md index f0cec128dc..5e84c9834e 100644 --- a/.serena/memories/critical-info.md +++ b/.serena/memories/critical-info.md @@ -55,6 +55,8 @@ The memory is structured in a way that you can get the critical information abou - `scripts/psql` — PostgreSQL client wrapper with devenv defaults. Companion: `scripts/db-schema` for DDL dumps. See `mem:scripts/psql`. - `scripts/taiga.py` — Fetch public issues, user stories, and tasks from the Penpot Taiga project without authentication. See `mem:scripts/taiga`. - `scripts/gh.py` — GitHub operations helper: list milestone issues, fetch PR details, compare against CHANGES.md. Requires `gh` CLI. See `mem:scripts/gh`. +- `scripts/project-anomalies.py` — Main-board anomaly check for a milestone (`check` writes `tmp/-ANOMALIES.md`). Bulk resolution via `scripts/gh.py`. See `mem:scripts/project-anomalies`. +- `scripts/changelog.py` — Changelog checks for the `update-changelog` skill: `check-merged` (PR merge status), `cross-ref` (milestone PRs vs changelog section), `report` (anomaly report to CHANGES-ISSUES.md). Calls `scripts/gh.py` via subprocess. See `mem:scripts/changelog`. - `scripts/error-reports.mjs` — Query error reports via RPC API with token authentication. Supports list/get operations with filtering and pagination. See `mem:scripts/error-reports`. - `scripts/clean-node-modules` — Remove stale `node_modules` from all pnpm workspaces (root, modules, member packages). Keeps the shared pnpm store at `/.pnpm-store` unless `--store`; ignores `external/` and `.opencode/`. Usage and reinstall steps: `mem:workflow/updating-pnpm`. - `scripts/ci` — CI orchestration script: runs lint, tests, and format checks per module (`frontend backend common render-wasm exporter mcp plugins library`). Logs go to `.ci-logs/`; read the log file on failure. See `mem:scripts/ci`. diff --git a/.serena/memories/scripts/changelog.md b/.serena/memories/scripts/changelog.md new file mode 100644 index 0000000000..6dc0dd5c94 --- /dev/null +++ b/.serena/memories/scripts/changelog.md @@ -0,0 +1,18 @@ +# Changelog helper + +`scripts/changelog.py` holds the checks used by the `update-changelog` skill (extracted from the skill file so the skill stays workflow-only). Calls `scripts/gh.py` via subprocess; never call the GitHub API directly from this script. + +## Subcommands + +- `check-merged ` (`--file`, `--stdin` accepted) — warn on any non-merged PR, exit 1 when found; run before writing changelog entries. +- `cross-ref [--changes CHANGES.md]` — fetch milestone PRs with `--state all` once, report merged PRs missing from the version section plus CLOSED (unmerged) warnings; GHSA entries never appear here by design. +- `report [--changes CHANGES.md --output CHANGES-ISSUES.md]` — overwrite the anomaly report (milestone mismatches type A/B plus `:rocket:` gaps C/D); rule violations never belong in the report, they are fixed in CHANGES.md first. + +## Conventions + +- Milestone resolution is batched, never per-item: unknown PRs go through `gh.py prs ` and unknown issues through `gh.py issue ` (one GraphQL call per 50 items); a lookup miss resolves to null, never to an extra call. + +- Version-section lookup accepts an optional `(suffix)` after the version header (e.g. `(Unreleased)`); a missing section is an error for `cross-ref` and an empty section for `report`. +- PR/issue reference parsing covers standard `[#N](.../pull/N)` / `[#N](.../issues/N)` links plus legacy `PR:[N]` / `[Github #N]` forms. +- Shared parsing helpers (`extract_version_section`, `extract_subsection`, `collect_changelog_prs/issues`) live at module top so future subcommands reuse them instead of duplicating regexes. +- Report anomaly types: A/B milestone mismatches, E `:boom:` entry whose issue lacks the `breaking change` label (counted, entries preserved — human labels the issue or moves the entry); C/D are `:rocket:` gaps (warnings, never counted). diff --git a/.serena/memories/scripts/gh.md b/.serena/memories/scripts/gh.md index 89ca288bb7..f40bddadf4 100644 --- a/.serena/memories/scripts/gh.md +++ b/.serena/memories/scripts/gh.md @@ -49,11 +49,13 @@ python3 scripts/gh.py issues "2.16.0" --compare CHANGES.md - Issues with type "Task" are excluded (`--include-tasks` to keep them). - Issues with "Rejected" project status are excluded (`--include-rejected` to keep them). +Every issue entry carries `assignees` (logins) and `projects` (board titles); every PR entry carries `assignees`. + **Output**: JSON array to stdout; progress to stderr. ### `prs` -Fetch PR details by number or by milestone. +Fetch PR details by number or by milestone. Every entry includes a `milestone` field (title or null). ```bash # Fetch specific PRs @@ -74,6 +76,12 @@ python3 scripts/gh.py prs --milestone "2.16.0" --state all **Output**: JSON array to stdout; progress to stderr. +### `issue` + +Fetch issue details by number in batches of 50 (same input styles as `prs`: numbers, `--file`, `--stdin`). No filters. A dead number fails its whole batch (unlike PRs, issues error instead of resolving to null), so the batch falls back to one-by-one lookups and reports misses as `{"number", "error": "not_found"}`. Any other failure aborts. + +**Output**: JSON array to stdout; progress to stderr. + ### `link-issue` Explicitly assign a GitHub issue to a pull request: @@ -111,7 +119,10 @@ python3 scripts/gh.py advisories GHSA-xvj6-fh9w-gjw7 ## Key principles +- Changelog-specific checks (merge-status verification, milestone-vs-changelog cross-reference, anomaly report) live in `scripts/changelog.py`, which calls this script via subprocess — see `mem:scripts/changelog`. + - All output is JSON — pipe into `jq` or other tools for further processing. +- Transient HTTP 504 responses from `gh` are retried automatically (3 attempts, 5s/15s backoff); any other error fails fast with no retry. - Milestone lookup is by exact title match. - `issues` subcommand auto-paginates (100 items per page). - `prs` subcommand batches PR number lookups (50 per GraphQL query). diff --git a/.serena/memories/scripts/project-anomalies.md b/.serena/memories/scripts/project-anomalies.md new file mode 100644 index 0000000000..9d0bfe6ecc --- /dev/null +++ b/.serena/memories/scripts/project-anomalies.md @@ -0,0 +1,17 @@ +# Project anomalies helper + +`scripts/project-anomalies.py` implements the `find-project-anomalies` skill: `check ` fetches milestone issues/PRs with `--state all` via `scripts/gh.py`, resolves outsiders in bulk (one GraphQL call per 50 items), and writes `tmp/-ANOMALIES.md` (gitignored scratch output, every number a full GitHub link). + +## Anomaly types (all scoped to the milestone) + +- Open-with-merged-PR: OPEN issue with a MERGED closing PR (close it or move it out). +- Issue-PR mismatch: milestone issue whose *merged* closing PR is elsewhere/nowhere (an unmerged PR's milestone is irrelevant), or issue off the Main board. +- PR-issue mismatch: *merged* milestone PR closing an issue elsewhere/nowhere (milestone-less issues ARE reported here, unlike the changelog flow), or off the Main board. Unmerged PRs never count anywhere: their milestone is meaningless. +- Needs-triage: MERGED PR on a `needs triage` issue (human triage only, never auto-unlabel). +- Unassigned: MERGED PR on an issue with no assignees and no `community contribution` label on either side (valid states: assigned, or unassigned+community). + +## Conventions + +- Project membership is checked on the issue side only (`projects` list from `gh.py`); PRs are not the Main-tracked unit, so an unprojected PR alone is never an anomaly. +- Detection (`find_anomalies`) and rendering (`render_report`) are pure functions over pre-resolved dicts; outsiders must be merged into the lookup maps first (missing numbers are skipped, never flagged). +- The only mechanical fix is assigning a missing milestone (`gh pr/issue edit --milestone`); everything else is human judgment with per-item confirmation. diff --git a/CHANGES.md b/CHANGES.md index 2abeb0612b..2e4f118bef 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -308,6 +308,7 @@ - Use hard reload for render engine switching in the workspace menu [#10441](https://github.com/penpot/penpot/issues/10441) (PR: [#10444](https://github.com/penpot/penpot/pull/10444)) - Rotate size badge when shape is rotated [#10386](https://github.com/penpot/penpot/issues/10386) (PR: [#10393](https://github.com/penpot/penpot/pull/10393)) - Add separate internal URI for exporter to handle Docker deployments where internal and public URIs differ [#10627](https://github.com/penpot/penpot/issues/10627) (PR: [#10630](https://github.com/penpot/penpot/pull/10630)) +- Make throwValidationErrors default to true for v2 manifest plugins [#10401](https://github.com/penpot/penpot/issues/10401) (PR: [#10433](https://github.com/penpot/penpot/pull/10433)) ### :bug: Bugs fixed @@ -415,6 +416,14 @@ - Fix text shape position-data to include required fills in WASM and DOM calculation paths [#10646](https://github.com/penpot/penpot/issues/10646) (PR: [#10650](https://github.com/penpot/penpot/pull/10650)) - Log expired OIDC tokens as auth failures instead of server errors [#10635](https://github.com/penpot/penpot/issues/10635) (PR: [#10636](https://github.com/penpot/penpot/pull/10636)) - Return 400 instead of 500 when ImageMagick rejects invalid uploaded images [#10642](https://github.com/penpot/penpot/issues/10642) (PR: [#10643](https://github.com/penpot/penpot/pull/10643)) +- Fix Plugin API board.guides setting and clearing throwing malli invalid-schema error (by @filipsajdak) [#9773](https://github.com/penpot/penpot/issues/9773) (PR: [#10503](https://github.com/penpot/penpot/pull/10503)) +- Fix Plugin API rejecting open-overlay and toggle-overlay interactions when position is not set (by @mvanhorn) [#10201](https://github.com/penpot/penpot/issues/10201) (PR: [#10503](https://github.com/penpot/penpot/pull/10503)) +- Fix Plugin API close-overlay interaction requiring animation field (by @mvanhorn) [#10202](https://github.com/penpot/penpot/issues/10202) (PR: [#10503](https://github.com/penpot/penpot/pull/10503)) +- Fix boards hidden from View Mode reappearing when adding prototype interactions (by @jeffrey701) [#10289](https://github.com/penpot/penpot/issues/10289) (PR: [#9695](https://github.com/penpot/penpot/pull/9695)) +- Fix Plugin API PenpotUtils.createVariantContainer assigning variant properties to wrong components [#10506](https://github.com/penpot/penpot/issues/10506) (PR: [#10562](https://github.com/penpot/penpot/pull/10562)) +- Fix mask inside flex layout shifting position or size when child image visibility changes [#10537](https://github.com/penpot/penpot/issues/10537) (PR: [#10697](https://github.com/penpot/penpot/pull/10697)) +- Fix path ends selector missing arrow icons and incorrect divider colors [#10593](https://github.com/penpot/penpot/issues/10593) (PR: [#10631](https://github.com/penpot/penpot/pull/10631)) +- Fix webhook form prefilling hardcoded metadata URI default [#10722](https://github.com/penpot/penpot/issues/10722) (PR: [#10723](https://github.com/penpot/penpot/pull/10723)) ## 2.16.2 diff --git a/scripts/changelog.py b/scripts/changelog.py new file mode 100644 index 0000000000..b9106f201f --- /dev/null +++ b/scripts/changelog.py @@ -0,0 +1,616 @@ +#!/usr/bin/env python3 +""" +changelog.py — Helper commands for the ``update-changelog`` skill. + +Uses ``scripts/gh.py`` (via subprocess) and the local ``CHANGES.md`` so the +skill file itself stays free of inline programs. + +Subcommands: + check-merged Verify that every given PR is merged (warns otherwise) + cross-ref Compare merged milestone PRs against the changelog section + report Generate the anomaly report (CHANGES-ISSUES.md) + +Usage: + python3 scripts/changelog.py check-merged 9179 9204 9311 + cat prs.txt | python3 scripts/changelog.py check-merged --stdin + python3 scripts/changelog.py cross-ref "2.16.0" + python3 scripts/changelog.py report "2.16.0" + python3 scripts/changelog.py report "2.16.0" --changes CHANGES.md --output CHANGES-ISSUES.md + +Prerequisites: + - gh CLI authenticated (gh auth status) + - Python 3.8+ +""" + +import argparse +import json +import re +import subprocess +import sys +from datetime import datetime, timezone +from pathlib import Path + + +REPO = "penpot/penpot" +GH_PY = Path(__file__).resolve().parent / "gh.py" + + +# ───────────────────────────────────────────── +# Shared helpers +# ───────────────────────────────────────────── + + +def run_gh_py(*args: str, input_text: str | None = None) -> str: + """Run ``scripts/gh.py`` with the given arguments, return stdout.""" + cmd = [sys.executable, str(GH_PY), *args] + result = subprocess.run(cmd, input=input_text, capture_output=True, text=True) + if result.returncode != 0: + print(f"gh.py error: {result.stderr}", file=sys.stderr) + sys.exit(1) + return result.stdout + + +def read_pr_numbers(args: argparse.Namespace) -> list[int]: + """Collect PR numbers from positional args, --file, and/or --stdin.""" + numbers: list[int] = [] + if args.numbers: + numbers.extend(args.numbers) + if args.file: + with open(args.file) as f: + for line in f: + line = line.strip() + if line: + numbers.append(int(line)) + if args.stdin: + for line in sys.stdin: + line = line.strip() + if line: + numbers.append(int(line)) + seen: set[int] = set() + return [n for n in numbers if not (n in seen or seen.add(n))] + + +def fetch_prs_by_numbers(pr_numbers: list[int]) -> list[dict]: + """Fetch PR details via ``gh.py prs`` for explicit PR numbers.""" + stdout = run_gh_py("prs", *[str(n) for n in pr_numbers]) + return json.loads(stdout) + + +def fetch_milestone_prs(milestone: str, state: str) -> list[dict]: + """Fetch all PRs in a milestone via ``gh.py prs --milestone``.""" + stdout = run_gh_py("prs", "--milestone", milestone, "--state", state) + return json.loads(stdout) + + +def extract_version_section(content: str, milestone: str) -> str: + """Return the changelog body of a ``## `` section (or "").""" + match = re.search( + rf"^## {re.escape(milestone)}(?:\s*\([^)]*\))?\n(.*?)(?=^## |\Z)", + content, + re.DOTALL | re.MULTILINE, + ) + return match.group(1) if match else "" + + +def extract_subsection(section: str, heading_re: str) -> str: + """Return the body of a ``### `` subsection (or "").""" + match = re.search(rf"^{heading_re}", section, re.MULTILINE) + if not match: + return "" + body = section[match.end():] + return re.split(r"(?m)^#{2,3}\s", body)[0] + + +def collect_changelog_prs(section: str) -> set[int]: + """Collect all PR numbers referenced in a changelog section.""" + numbers = set() + for num in re.findall( + r"\[#(\d+)\]\(https://github\.com/penpot/penpot/pull/\d+\)", section + ): + numbers.add(int(num)) + for num in re.findall(r"PR:\[(\d+)\]", section): + numbers.add(int(num)) + return numbers + + +def collect_changelog_issues(section: str) -> set[int]: + """Collect all issue numbers referenced in a changelog section.""" + numbers = set() + for num in re.findall( + r"\[#(\d+)\]\(https://github\.com/penpot/penpot/issues/\d+\)", section + ): + numbers.add(int(num)) + for num in re.findall(r"\[Github #(\d+)\]", section): + numbers.add(int(num)) + return numbers + + +# ───────────────────────────────────────────── +# Subcommand: check-merged +# ───────────────────────────────────────────── + + +def cmd_check_merged(args: argparse.Namespace) -> None: + """Warn about any given PR that is not merged. Exits 1 when found.""" + pr_numbers = read_pr_numbers(args) + if not pr_numbers: + print( + "ERROR: no PR numbers provided (pass numbers, --file, or --stdin)", + file=sys.stderr, + ) + sys.exit(1) + + prs = fetch_prs_by_numbers(pr_numbers) + bad = 0 + for pr in prs: + number = pr.get("number") + state = pr.get("state") + if state != "MERGED": + bad += 1 + print(f"WARNING: #{number} is {state} (not merged)") + + if bad: + print(f"{bad} of {len(prs)} PRs are NOT merged", file=sys.stderr) + sys.exit(1) + print(f"OK: all {len(prs)} PRs are merged", file=sys.stderr) + + +# ───────────────────────────────────────────── +# Subcommand: cross-ref +# ───────────────────────────────────────────── + + +def cmd_cross_ref(args: argparse.Namespace) -> None: + """Compare merged milestone PRs against the changelog section.""" + with open(args.changes) as f: + content = f.read() + section = extract_version_section(content, args.milestone) + if not section: + print( + f'ERROR: no "## {args.milestone}" section found in {args.changes}', + file=sys.stderr, + ) + sys.exit(1) + + changelog_refs = collect_changelog_prs(section) + milestone_prs = fetch_milestone_prs(args.milestone, "all") + + merged = {pr["number"] for pr in milestone_prs if pr.get("state") == "MERGED"} + closed = [pr for pr in milestone_prs if pr.get("state") == "CLOSED"] + + missing = sorted(merged - changelog_refs) + print(f"Milestone merged PRs: {len(merged)}") + print(f"Changelog referenced PRs: {len(changelog_refs)}") + print(f"PRs in milestone but NOT in changelog: {len(missing)}") + for num in missing: + pr = next(p for p in milestone_prs if p["number"] == num) + print(f" #{num} {pr['title'][:80]}") + + if closed: + print("WARNING: CLOSED (unmerged) PRs in milestone:") + for pr in closed: + print(f" #{pr['number']} {pr['title'][:80]}") + + +# ───────────────────────────────────────────── +# Subcommand: report +# ───────────────────────────────────────────── + + +def issue_url(n: int) -> str: + return f"https://github.com/{REPO}/issues/{n}" + + +def pr_url(n: int) -> str: + return f"https://github.com/{REPO}/pull/{n}" + + +def issue_link(n: int) -> str: + return f"[#{n}]({issue_url(n)})" + + +def pr_link(n: int) -> str: + return f"[#{n}]({pr_url(n)})" + + +def issue_link_title(n: int, title: str) -> str: + url = issue_url(n) + if title: + return f"[#{n}]({url}) — [{title}]({url})" + return f"[#{n}]({url})" + + +def pr_link_title(n: int, title: str) -> str: + url = pr_url(n) + if title: + return f"[#{n}]({url}) — [{title}]({url})" + return f"[#{n}]({url})" + + +def cmd_report(args: argparse.Namespace) -> None: + """Generate the changelog anomaly report and write it to a file.""" + milestone = args.milestone + + all_issues = json.loads(run_gh_py("issues", milestone, "--state", "all")) + issue_by_num = {i["number"]: i for i in all_issues} + + all_prs = fetch_milestone_prs(milestone, "all") + pr_by_num = {p["number"]: p for p in all_prs} + + with open(args.changes) as f: + content = f.read() + section = extract_version_section(content, milestone) + changelog_issues = collect_changelog_issues(section) + changelog_prs = collect_changelog_prs(section) + + # PRs and issues returned by milestone queries are KNOWN to be in the + # milestone. Everything else is resolved in bulk via batched gh.py + # lookups (one GraphQL call per 50 items, never one call per item). + pr_milestone_cache = {p["number"]: milestone for p in all_prs} + issue_milestone_cache = {i["number"]: milestone for i in all_issues} + + unknown_prs = set() + for issue in issue_by_num.values(): + unknown_prs.update(issue.get("closing_prs", [])) + unknown_prs -= set(pr_milestone_cache) + if unknown_prs: + print( + f"Resolving milestones for {len(unknown_prs)} PRs outside {milestone}...", + file=sys.stderr, + ) + for pr in json.loads(run_gh_py("prs", *[str(n) for n in sorted(unknown_prs)])): + pr_milestone_cache[pr["number"]] = pr.get("milestone") + + unknown_issues = set() + for pr in pr_by_num.values(): + unknown_issues.update(pr.get("closing_issues", [])) + unknown_issues |= collect_changelog_issues(extract_subsection(section, r"### :boom:")) + unknown_issues -= set(issue_milestone_cache) + external_issues: dict[int, dict] = {} + if unknown_issues: + print( + f"Resolving milestones for {len(unknown_issues)} issues outside {milestone}...", + file=sys.stderr, + ) + for issue in json.loads( + run_gh_py("issue", *[str(n) for n in sorted(unknown_issues)]) + ): + issue_milestone_cache[issue["number"]] = issue.get("milestone") + if "error" not in issue: + external_issues[issue["number"]] = issue + + def get_issue_labels(issue_num: int) -> list[str] | None: + """Return the labels of an issue, or None when they cannot be known.""" + issue = issue_by_num.get(issue_num) + if issue is not None: + return issue.get("labels", []) + issue = external_issues.get(issue_num) + if issue is not None: + return issue.get("labels", []) + return None + + def get_pr_milestone(pr_num: int) -> str | None: + """Return the milestone title for a PR, or None if unassigned.""" + return pr_milestone_cache.get(pr_num) + + def get_issue_milestone(issue_num: int) -> str | None: + """Return the milestone title for an issue, or None if unassigned.""" + return issue_milestone_cache.get(issue_num) + + excluded_labels = {"release blocker", "no changelog"} + + def issue_excluded(issue: dict | None) -> bool: + if not issue: + return True + if issue.get("state") != "CLOSED": + return True + if issue.get("issue_type") in {"Task"}: + return True + if issue.get("project_status") in {"Rejected"}: + return True + if excluded_labels & set(issue.get("labels", [])): + return True + return False + + # Type A: issue in milestone, referenced PR in different milestone or none + anomalies_a = [] + for issue_num in sorted(changelog_issues): + issue = issue_by_num.get(issue_num) + if not issue: + continue + if get_issue_milestone(issue_num) != milestone: + continue + for pr_num in issue.get("closing_prs", []): + pr_ms = get_pr_milestone(pr_num) + if pr_ms != milestone: + anomalies_a.append( + { + "issue": issue_num, + "issue_title": issue.get("title", ""), + "pr": pr_num, + "pr_milestone": pr_ms, + } + ) + + # Type B: PR in milestone, the issue it closes is in a different milestone. + # An issue with NO milestone belongs to another (probably private) project + # and is NOT an anomaly. + anomalies_b = [] + for pr_num in sorted(changelog_prs): + pr = pr_by_num.get(pr_num) + if not pr: + continue + if get_pr_milestone(pr_num) != milestone: + continue + for issue_num in pr.get("closing_issues", []): + issue_ms = get_issue_milestone(issue_num) + if issue_ms is None: + continue + if issue_ms != milestone: + anomalies_b.append( + { + "pr": pr_num, + "pr_title": pr.get("title", ""), + "issue": issue_num, + "issue_milestone": issue_ms, + } + ) + + # Type E: :boom: entry whose issue lacks the `breaking change` label. + # These stay in the changelog (they are preserved, not removed) — the + # human decides whether to label the issue or move the entry elsewhere. + anomalies_e = [] + boom_body = extract_subsection(section, r"### :boom:") + if boom_body: + for issue_num in sorted(collect_changelog_issues(boom_body)): + labels = get_issue_labels(issue_num) + if labels is None: + continue + if "breaking change" not in labels: + issue = issue_by_num.get(issue_num) or external_issues.get(issue_num) or {} + anomalies_e.append( + { + "issue": issue_num, + "issue_title": issue.get("title", ""), + } + ) + + # Type C: released X.Y.0 sections without a :rocket: subsection. + # Patches (X.Y.Z with Z != 0) never carry :rocket: — only minors/majors. + anomalies_c = [] + rocket_heading_re = re.compile(r"^### :rocket:", re.MULTILINE) + version_sections = re.split( + r"(?=^## \d+\.\d+\.\d+)", content, flags=re.MULTILINE + ) + for vs in version_sections: + m = re.match(r"^## (\d+\.\d+\.\d+)(.*)", vs) + if not m: + continue + ver, suffix = m.group(1), m.group(2) + if "unreleased" in suffix.lower(): + continue + if ver.split(".")[2] != "0": + continue + if not rocket_heading_re.search(vs): + anomalies_c.append(ver) + + # Type D: :rocket: entries without issue AND PR references. + anomalies_d = [] + issue_ref_re = re.compile( + r"\[#\d+\]\(https://github\.com/penpot/penpot/issues/\d+\)" + ) + pr_ref_re = re.compile( + r"\(PR:\s*\[#\d+\]\(https://github\.com/penpot/penpot/pull/\d+\)" + r"(\s*,\s*\[#\d+\]\(https://github\.com/penpot/penpot/pull/\d+\))*\)" + ) + for vs in version_sections: + m = re.match(r"^## (\d+\.\d+\.\d+)(.*)", vs) + if not m: + continue + ver = m.group(1) + rocket_body = extract_subsection(vs, r"### :rocket:") + if not rocket_body: + continue + for line in rocket_body.splitlines(): + line = line.strip() + if line.startswith("- ") and not ( + issue_ref_re.search(line) and pr_ref_re.search(line) + ): + anomalies_d.append({"version": ver, "line": line[:100]}) + + def fmt_ms(ms: str | None) -> str: + return ms if ms else "_none_" + + with open(args.output, "w") as f: + f.write(f"# Changelog Anomaly Report — {milestone}\n\n") + f.write( + f"Generated: {datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M UTC')}\n\n" + ) + f.write("---\n\n") + + n_a, n_b, n_c, n_d, n_e = ( + len(anomalies_a), + len(anomalies_b), + len(anomalies_c), + len(anomalies_d), + len(anomalies_e), + ) + + f.write("## Summary\n\n") + f.write( + f"- **Issue in {milestone}, referenced PR in different milestone or no milestone:** {n_a}\n" + ) + f.write( + f"- **PR in {milestone}, closing issue in a different milestone:** {n_b}\n" + ) + f.write(f"- **:boom: entry without breaking change label:** {n_e}\n") + f.write(f"- **Total anomalies:** {n_a + n_b + n_e}\n") + f.write( + f"- **Released X.Y.0 version missing :rocket: section (gap):** {n_c}\n" + ) + f.write(f"- **:rocket: entry without issue AND PR references (gap):** {n_d}\n\n") + + if n_a or n_b or n_e: + f.write("## Anomalies\n\n") + f.write( + "Types A and B are milestone mismatches between an issue in the changelog " + "and its referenced PR (or vice-versa). The changelog claim " + '"this issue is fixed by this PR, all in this milestone" is ' + "inconsistent with the actual milestone assignments. " + "Resolve by either updating the milestone on the issue/PR or " + "removing the misleading entry from the changelog.\n\n" + "Type E entries stay in the changelog: either label the issue " + "as `breaking change` or move the entry out of `:boom:`.\n\n" + ) + + if n_a: + f.write( + f"### Issue in {milestone}, PR in different milestone or no milestone\n\n" + ) + by_issue: dict[int, list[dict]] = {} + for a in anomalies_a: + by_issue.setdefault(a["issue"], []).append(a) + for issue_num in sorted(by_issue): + entries = by_issue[issue_num] + title = entries[0]["issue_title"] + f.write(f"- {issue_link_title(issue_num, title[:80])}\n") + for e in entries: + ms_label = fmt_ms(e["pr_milestone"]) + badge = "🔴" if e["pr_milestone"] is None else "⚠️" + f.write( + f" - {badge} Referenced {pr_link(e['pr'])} is in milestone **{ms_label}** (expected: {milestone})\n" + ) + f.write("\n") + + if n_b: + f.write( + f"\n### PR in {milestone}, closing issue in a different milestone\n\n" + ) + by_pr: dict[int, list[dict]] = {} + for b in anomalies_b: + by_pr.setdefault(b["pr"], []).append(b) + for pr_num in sorted(by_pr): + entries = by_pr[pr_num] + title = entries[0]["pr_title"] + f.write(f"- {pr_link_title(pr_num, title[:80])}\n") + for e in entries: + ms_label = fmt_ms(e["issue_milestone"]) + badge = "🔴" if e["issue_milestone"] is None else "⚠️" + f.write( + f" - {badge} Closing {issue_link(e['issue'])} is in milestone **{ms_label}** (expected: {milestone})\n" + ) + f.write("\n") + + if n_e: + f.write("\n### :boom: entry without breaking change label\n\n") + f.write( + "These entries sit under `### :boom: Breaking changes & Deprecations` " + "but their issue has no `breaking change` label. They are kept " + "in the changelog — add the label on the issue or move the " + "entry to its regular section.\n\n" + ) + for e in anomalies_e: + f.write(f"- ⚠️ {issue_link_title(e['issue'], e['issue_title'][:80])}\n") + f.write("\n") + else: + f.write( + "✅ No anomalies found. All (issue, PR) pairs in the changelog have aligned milestone assignments.\n\n" + ) + + if n_c or n_d: + f.write("## Highlight gaps\n\n") + f.write( + "These are warnings, not anomalies: they do not affect the " + "milestone-mismatch total above. They track `:rocket:` coverage " + "across all released X.Y.0 versions. Historical entries (e.g. " + "Taiga links) predate the current reference convention and are " + "expected to appear here.\n\n" + ) + + if n_c: + f.write("### Released X.Y.0 version missing :rocket: section\n\n") + f.write( + "These released minors/majors have no `### :rocket: Epics and highlights` subsection. " + "Add highlights to help self-hosted users understand what they are missing.\n\n" + ) + for ver in anomalies_c: + f.write(f"- Version **{ver}**\n") + f.write("\n") + + if n_d: + f.write("### :rocket: entry without issue AND PR references\n\n") + f.write( + "These highlight entries lack the required issue AND PR references. " + "Add `[#ISSUE](...)` and `(PR: [#PR](...))` links.\n\n" + ) + for d in anomalies_d: + f.write(f"- **{d['version']}**: `{d['line']}`\n") + f.write("\n") + elif not (n_a or n_b or n_e): + f.write( + "✅ No highlight gaps found. All released X.Y.0 versions have properly referenced :rocket: entries.\n\n" + ) + + f.write("---\n\n") + f.write("## Context\n\n") + f.write(f"- Milestone: **{milestone}**\n") + f.write(f"- Milestone total issues (all states): {len(all_issues)}\n") + f.write( + f"- Closed issues in milestone: {sum(1 for i in all_issues if i.get('state') == 'CLOSED')}\n" + ) + f.write( + f"- Valid issues after exclusions: {len([i for i in all_issues if not issue_excluded(i)])}\n" + ) + f.write(f"- Issues referenced in changelog: {len(changelog_issues)}\n") + f.write(f"- PRs referenced in changelog: {len(changelog_prs)}\n") + + print(f"Anomaly report written to {args.output}") + + +# ───────────────────────────────────────────── +# CLI entrypoint +# ───────────────────────────────────────────── + + +def main() -> None: + parser = argparse.ArgumentParser( + description="Helper commands for the update-changelog skill" + ) + sub = parser.add_subparsers(dest="command", required=True, title="subcommands") + + p_check = sub.add_parser( + "check-merged", help="Verify that every given PR is merged" + ) + p_check.add_argument("numbers", type=int, nargs="*", + help="PR numbers to check (space-separated)") + p_check.add_argument("--file", type=str, + help="File with one PR number per line") + p_check.add_argument("--stdin", action="store_true", + help="Read PR numbers from stdin (one per line)") + p_check.set_defaults(func=cmd_check_merged) + + p_cross = sub.add_parser( + "cross-ref", + help="Compare merged milestone PRs against the changelog section", + ) + p_cross.add_argument("milestone", help="Milestone title (e.g. '2.16.0')") + p_cross.add_argument("--changes", default="CHANGES.md", + help="Path to the changelog file (default: CHANGES.md)") + p_cross.set_defaults(func=cmd_cross_ref) + + p_report = sub.add_parser( + "report", help="Generate the anomaly report (CHANGES-ISSUES.md)" + ) + p_report.add_argument("milestone", help="Milestone title (e.g. '2.16.0')") + p_report.add_argument("--changes", default="CHANGES.md", + help="Path to the changelog file (default: CHANGES.md)") + p_report.add_argument("--output", default="CHANGES-ISSUES.md", + help="Report output path (default: CHANGES-ISSUES.md)") + p_report.set_defaults(func=cmd_report) + + args = parser.parse_args() + args.func(args) + + +if __name__ == "__main__": + main() diff --git a/scripts/gh.py b/scripts/gh.py index 9e2454e556..c430fbe843 100755 --- a/scripts/gh.py +++ b/scripts/gh.py @@ -25,6 +25,7 @@ Usage: cat prs.txt | python3 scripts/gh.py prs --stdin python3 scripts/gh.py prs --milestone "2.16.0" (default: state=merged) python3 scripts/gh.py prs --milestone "2.16.0" --state all + python3 scripts/gh.py issue 11235 11236 python3 scripts/gh.py advisories (list all advisories) python3 scripts/gh.py advisories --severity critical (filter by severity) python3 scripts/gh.py advisories GHSA-xvj6-fh9w-gjw7 (single advisory detail) @@ -40,6 +41,7 @@ import json import re import subprocess import sys +import time from typing import Any @@ -47,21 +49,70 @@ REPO = "penpot/penpot" OWNER = "penpot" REPO_NAME = "penpot" +# Transient gateway timeouts from the GitHub API are retried, nothing else. +GH_RETRIES = 3 +GH_RETRY_DELAYS = (5, 15) # seconds waited before retry N (last delay repeats) + + +def run_gh_command(cmd: list[str], input_text: str | None = None) -> str: + """Run a ``gh`` command, retrying transient HTTP 504s. + + Raises `GhCommandFailed` on failure so callers can choose between + aborting (normal commands) and degrading (batched lookups). + """ + last_stderr = "" + for attempt in range(GH_RETRIES): + if attempt: + delay = GH_RETRY_DELAYS[min(attempt - 1, len(GH_RETRY_DELAYS) - 1)] + print( + f"gh: HTTP 504, retrying in {delay}s" + f" (attempt {attempt + 1}/{GH_RETRIES})...", + file=sys.stderr, + ) + time.sleep(delay) + result = subprocess.run( + cmd, input=input_text, capture_output=True, text=True + ) + if result.returncode == 0: + return result.stdout + last_stderr = result.stderr + if "504" not in result.stderr: + break + raise GhCommandFailed(last_stderr) + + +class GhCommandFailed(Exception): + """A ``gh`` invocation failed (stderr kept for the caller to judge).""" + + def __init__(self, stderr: str) -> None: + super().__init__(stderr) + self.stderr = stderr + + +def fail_gh(stderr: str) -> None: + """Report a ``gh`` failure and exit (standard behavior for CLI commands).""" + print(f"gh error: {stderr}", file=sys.stderr) + sys.exit(1) + # ───────────────────────────────────────────── # Shared helpers # ───────────────────────────────────────────── +def post_graphql(query: str, variables: dict) -> Any: + """POST a GraphQL query via ``gh`` and return the raw response body.""" + payload = json.dumps({"query": query, "variables": variables}) + stdout = run_gh_command(["gh", "api", "graphql", "--input", "-"], payload) + return json.loads(stdout) + + def run_gh_graphql(query: str, variables: dict) -> Any: """Run a GraphQL query via ``gh api graphql --input -``.""" - payload = json.dumps({"query": query, "variables": variables}) - cmd = ["gh", "api", "graphql", "--input", "-"] - result = subprocess.run(cmd, input=payload, capture_output=True, text=True) - if result.returncode != 0: - print(f"gh error: {result.stderr}", file=sys.stderr) - sys.exit(1) - body = json.loads(result.stdout) + try: + body = post_graphql(query, variables) + except GhCommandFailed as err: + fail_gh(err.stderr) if "errors" in body: for err in body["errors"]: print(f"GraphQL error: {err.get('message')}", file=sys.stderr) @@ -71,12 +122,10 @@ def run_gh_graphql(query: str, variables: dict) -> Any: def run_gh_rest(path: str) -> Any: """Run a REST API call via ``gh api``.""" - cmd = ["gh", "api", path] - result = subprocess.run(cmd, capture_output=True, text=True) - if result.returncode != 0: - print(f"gh error: {result.stderr}", file=sys.stderr) - sys.exit(1) - return json.loads(result.stdout) + try: + return json.loads(run_gh_command(["gh", "api", path])) + except GhCommandFailed as err: + fail_gh(err.stderr) # ───────────────────────────────────────────── @@ -219,6 +268,7 @@ query($owner: String!, $repo: String!, $milestone: Int!, $cursor: String) { state issueType { name } labels(first: 20) { nodes { name } } + assignees(first: 10) { nodes { login } } closedByPullRequestsReferences(first: 5) { nodes { number } } projectItems(first: 10) { nodes { @@ -257,6 +307,7 @@ query($query: String!, $cursor: String) { milestone { title } issueType { name } labels(first: 20) { nodes { name } } + assignees(first: 10) { nodes { login } } closedByPullRequestsReferences(first: 5) { nodes { number } } projectItems(first: 10) { nodes { @@ -275,6 +326,32 @@ query($query: String!, $cursor: String) { """ +def node_assignees(node: dict) -> list[str]: + """Extract assignee logins from a GraphQL issue/PR node.""" + return [ + a["login"] for a in (node.get("assignees") or {}).get("nodes") or [] + ] + + +def node_projects(node: dict) -> list[str]: + """Extract project board titles from a GraphQL issue node.""" + return [ + (pi.get("project") or {}).get("title") + for pi in (node.get("projectItems") or {}).get("nodes") or [] + if (pi.get("project") or {}).get("title") + ] + + +def node_main_status(node: dict) -> str | None: + """Extract the "Main" project board status from a GraphQL issue node.""" + for pi in (node.get("projectItems") or {}).get("nodes") or []: + project = pi.get("project") or {} + if project.get("title") == "Main": + status_field = pi.get("fieldValueByName") or {} + return status_field.get("name") + return None + + def fetch_no_milestone_issues(states: str, labels: str | None = None) -> list[dict]: """ Fetch all issues that belong to NO milestone via paginated GraphQL search. @@ -284,7 +361,7 @@ def fetch_no_milestone_issues(states: str, labels: str | None = None) -> list[di labels: optional comma-separated labels to include (built into the search query) Returns: - List of {number, title, state, milestone, issue_type, labels, closing_prs, project_status} + List of {number, title, state, milestone, issue_type, labels, assignees, closing_prs, project_status, projects} """ all_nodes: list[dict] = [] cursor: str | None = None @@ -315,13 +392,6 @@ def fetch_no_milestone_issues(states: str, labels: str | None = None) -> list[di continue issue_type = node.get("issueType") ms = node.get("milestone") - project_status = None - for pi in (node.get("projectItems") or {}).get("nodes") or []: - project = pi.get("project") or {} - if project.get("title") == "Main": - status_field = pi.get("fieldValueByName") or {} - project_status = status_field.get("name") - break all_nodes.append({ "number": node["number"], "title": node["title"], @@ -329,8 +399,10 @@ def fetch_no_milestone_issues(states: str, labels: str | None = None) -> list[di "milestone": ms["title"] if ms else None, "issue_type": issue_type["name"] if issue_type else None, "labels": [lbl["name"] for lbl in node["labels"]["nodes"]], + "assignees": node_assignees(node), "closing_prs": [pr["number"] for pr in node["closedByPullRequestsReferences"]["nodes"]], - "project_status": project_status, + "project_status": node_main_status(node), + "projects": node_projects(node), }) total = len(all_nodes) @@ -352,7 +424,7 @@ def fetch_milestone_issues(milestone_num: int, states: str) -> list[dict]: states: GraphQL states enum array literal, e.g. ``"[CLOSED]"`` or ``"[OPEN CLOSED]"`` Returns: - List of {number, title, state, issue_type: str|None, labels: [str], closing_prs: [int]} + List of {number, title, state, issue_type: str|None, labels: [str], assignees: [str], closing_prs: [int], project_status, projects} """ query = GQL_ISSUES_QUERY.replace("__STATES__", states) all_nodes: list[dict] = [] @@ -373,22 +445,16 @@ def fetch_milestone_issues(milestone_num: int, states: str) -> list[dict]: if node is None: continue issue_type = node.get("issueType") - # Extract project status from the "Main" project board (if present) - project_status = None - for pi in (node.get("projectItems") or {}).get("nodes") or []: - project = pi.get("project") or {} - if project.get("title") == "Main": - status_field = pi.get("fieldValueByName") or {} - project_status = status_field.get("name") - break all_nodes.append({ "number": node["number"], "title": node["title"], "state": node["state"], "issue_type": issue_type["name"] if issue_type else None, "labels": [lbl["name"] for lbl in node["labels"]["nodes"]], + "assignees": node_assignees(node), "closing_prs": [pr["number"] for pr in node["closedByPullRequestsReferences"]["nodes"]], - "project_status": project_status, + "project_status": node_main_status(node), + "projects": node_projects(node), }) total = len(all_nodes) @@ -490,7 +556,9 @@ GQL_PRS_QUERY_ITEM = """\ state mergedAt createdAt + milestone {{ title }} author {{ login }} + assignees(first: 10) {{ nodes {{ login }} }} labels(first: 20) {{ nodes {{ name }} }} closingIssuesReferences(first: 5) {{ nodes {{ number }} }} }} @@ -537,7 +605,9 @@ def fetch_prs_batch(pr_numbers: list[int]) -> list[dict]: "state": pr["state"], "merged_at": pr.get("mergedAt"), "created_at": pr.get("createdAt"), + "milestone": (pr.get("milestone") or {}).get("title"), "author": pr["author"]["login"] if pr["author"] else None, + "assignees": node_assignees(pr), "labels": [lbl["name"] for lbl in pr["labels"]["nodes"]], "closing_issues": [iss["number"] for iss in pr["closingIssuesReferences"]["nodes"]], }) @@ -561,6 +631,7 @@ query($owner: String!, $repo: String!, $milestone: Int!, $cursor: String) { createdAt headRefName author { login } + assignees(first: 10) { nodes { login } } labels(first: 20) { nodes { name } } files(first: 100) { nodes { path } } closingIssuesReferences(first: 5) { nodes { number } } @@ -583,7 +654,7 @@ def fetch_milestone_prs(milestone_num: int, states: str) -> list[dict]: Returns: List of {number, title, body, state, merged_at, created_at, - head_ref_name, author, labels: [str], files: [str], + head_ref_name, author, assignees, labels: [str], files: [str], closing_issues: [int]} """ query = GQL_MILESTONE_PRS_QUERY.replace("__STATES__", states) @@ -613,6 +684,7 @@ def fetch_milestone_prs(milestone_num: int, states: str) -> list[dict]: "created_at": node.get("createdAt"), "head_ref_name": node.get("headRefName"), "author": node["author"]["login"] if node["author"] else None, + "assignees": node_assignees(node), "labels": [lbl["name"] for lbl in node["labels"]["nodes"]], "files": [file["path"] for file in node["files"]["nodes"]], "closing_issues": [iss["number"] for iss in node["closingIssuesReferences"]["nodes"]], @@ -641,38 +713,19 @@ def cmd_prs(args: argparse.Namespace) -> None: print(f"Fetching {args.state} PRs via GraphQL...", file=sys.stderr) prs = fetch_milestone_prs(ms["number"], gql_states) + for pr in prs: + pr["milestone"] = ms["title"] print(f"Fetched {len(prs)} PRs total", file=sys.stderr) print(json.dumps(prs, indent=2)) return # ── Number-based path ─────────────────────────────────────────── - pr_numbers: list[int] = [] - - if args.numbers: - pr_numbers.extend(args.numbers) - - if args.file: - with open(args.file) as f: - for line in f: - line = line.strip() - if line: - pr_numbers.append(int(line)) - - if args.stdin: - for line in sys.stdin: - line = line.strip() - if line: - pr_numbers.append(int(line)) - + pr_numbers = read_numbers_from_args(args) if not pr_numbers: print("ERROR: no PR numbers provided (pass numbers, --file, --stdin, or --milestone)", file=sys.stderr) sys.exit(1) - # Deduplicate while preserving order - seen: set[int] = set() - pr_numbers = [n for n in pr_numbers if not (n in seen or seen.add(n))] - print(f"Fetching {len(pr_numbers)} PRs in batches of {PRS_BATCH_SIZE}...", file=sys.stderr) @@ -686,6 +739,155 @@ def cmd_prs(args: argparse.Namespace) -> None: print(json.dumps(all_results, indent=2)) +# ───────────────────────────────────────────── +# Subcommand: issue (fetch issues by number, batched) +# ───────────────────────────────────────────── + +GQL_ISSUE_BY_NUMBER_QUERY_ITEM = """\ + issue_{num}: issue(number: {num}) {{ + number + title + state + milestone {{ title }} + issueType {{ name }} + labels(first: 20) {{ nodes {{ name }} }} + assignees(first: 10) {{ nodes {{ login }} }} + closedByPullRequestsReferences(first: 5) {{ nodes {{ number }} }} + projectItems(first: 10) {{ + nodes {{ + project {{ title }} + fieldValueByName(name: "Status") {{ + ... on ProjectV2ItemFieldSingleSelectValue {{ + name + }} + }} + }} + }} + }} +""" + + +def issue_node_to_dict(node: dict) -> dict: + """Convert a GraphQL issue node to the shared issue dict shape.""" + issue_type = node.get("issueType") + ms = node.get("milestone") + return { + "number": node["number"], + "title": node["title"], + "state": node["state"], + "milestone": ms["title"] if ms else None, + "issue_type": issue_type["name"] if issue_type else None, + "labels": [lbl["name"] for lbl in node["labels"]["nodes"]], + "assignees": node_assignees(node), + "closing_prs": [pr["number"] for pr in node["closedByPullRequestsReferences"]["nodes"]], + "project_status": node_main_status(node), + "projects": node_projects(node), + } + + +class IssueBatchFailed(Exception): + """One aliased issue lookup failed; the batch cannot be trusted as a whole.""" + + +def _fetch_issues_batch(issue_numbers: list[int]) -> list[dict]: + """Single batched issue lookup; raises `IssueBatchFailed` on any failure.""" + items = "\n".join( + GQL_ISSUE_BY_NUMBER_QUERY_ITEM.format(num=n) for n in issue_numbers + ) + query = GQL_PRS_QUERY_WRAPPER.format(items=items) + variables = {"owner": OWNER, "repo": REPO_NAME} + + try: + body = post_graphql(query, variables) + except GhCommandFailed as err: + raise IssueBatchFailed(err.stderr) from err + if "errors" in body: + raise IssueBatchFailed("; ".join(e.get("message", "") for e in body["errors"])) + repo = body["data"]["repository"] + + results: list[dict] = [] + for num in issue_numbers: + node = repo.get(f"issue_{num}") + if node is None: + results.append({ + "number": num, + "error": "not_found", + }) + continue + results.append(issue_node_to_dict(node)) + return results + + +def fetch_issues_batch(issue_numbers: list[int]) -> list[dict]: + """ + Fetch details for a list of issue numbers in a single GraphQL query. + + Uses numbered aliases (issue_1234, …) so each issue is looked up by + number in one round-trip. Returns entries in the same order as the input. + + Unlike PRs (which resolve to null), a dead issue number fails the whole + batch, so when the failure names an unresolvable issue each number is + retried on its own and reported as ``not_found`` instead of aborting. + Any other failure (auth, outage, exhausted 504 retries) still aborts. + """ + try: + return _fetch_issues_batch(issue_numbers) + except IssueBatchFailed as err: + if "Could not resolve" not in str(err): + print(f"GraphQL error: {err}", file=sys.stderr) + sys.exit(1) + print(" batch lookup failed, retrying issues one by one...", + file=sys.stderr) + results: list[dict] = [] + for num in issue_numbers: + try: + results.extend(_fetch_issues_batch([num])) + except IssueBatchFailed: + results.append({"number": num, "error": "not_found"}) + return results + + +def read_numbers_from_args(args: argparse.Namespace) -> list[int]: + """Collect numbers from positional args, --file, and/or --stdin (deduplicated).""" + numbers: list[int] = [] + if args.numbers: + numbers.extend(args.numbers) + if args.file: + with open(args.file) as f: + for line in f: + line = line.strip() + if line: + numbers.append(int(line)) + if args.stdin: + for line in sys.stdin: + line = line.strip() + if line: + numbers.append(int(line)) + seen: set[int] = set() + return [n for n in numbers if not (n in seen or seen.add(n))] + + +def cmd_issue(args: argparse.Namespace) -> None: + """Handle the ``issue`` subcommand (batched by-number lookup, no filters).""" + issue_numbers = read_numbers_from_args(args) + if not issue_numbers: + print("ERROR: no issue numbers provided (pass numbers, --file, or --stdin)", + file=sys.stderr) + sys.exit(1) + + print(f"Fetching {len(issue_numbers)} issues in batches of {PRS_BATCH_SIZE}...", + file=sys.stderr) + + all_results: list[dict] = [] + for i in range(0, len(issue_numbers), PRS_BATCH_SIZE): + batch = issue_numbers[i : i + PRS_BATCH_SIZE] + print(f" batch {i // PRS_BATCH_SIZE + 1}: issues {batch[0]}..{batch[-1]}", + file=sys.stderr) + all_results.extend(fetch_issues_batch(batch)) + + print(json.dumps(all_results, indent=2)) + + # ───────────────────────────────────────────── # Subcommand: advisories # ───────────────────────────────────────────── @@ -860,6 +1062,24 @@ def main() -> None: p_link.add_argument("pr_number", type=int, help="Pull request number") p_link.set_defaults(func=cmd_link_issue) + # --- issue (by-number lookup) --- + p_issue = sub.add_parser( + "issue", help="Fetch details for one or more issues by number (batched)" + ) + p_issue.add_argument( + "numbers", type=int, nargs="*", + help="Issue numbers to fetch (space-separated)" + ) + p_issue.add_argument( + "--file", type=str, + help="File with one issue number per line" + ) + p_issue.add_argument( + "--stdin", action="store_true", + help="Read issue numbers from stdin (one per line)" + ) + p_issue.set_defaults(func=cmd_issue) + # --- advisories --- p_adv = sub.add_parser("advisories", help="List or inspect GitHub security advisories") p_adv.add_argument( diff --git a/scripts/project-anomalies.py b/scripts/project-anomalies.py new file mode 100644 index 0000000000..120681234a --- /dev/null +++ b/scripts/project-anomalies.py @@ -0,0 +1,468 @@ +#!/usr/bin/env python3 +""" +project-anomalies.py — Find anomalies on the Main project board for a milestone. + +Used by the ``find-project-anomalies`` skill. Fetches milestone issues and +PRs in bulk via ``scripts/gh.py`` (one GraphQL call per 50 items, never one +call per item) and writes a clickable report to +``tmp/-ANOMALIES.md``. + +Anomaly types (all scoped to the given milestone): + 1. open-with-merged-pr — OPEN issue with a MERGED closing PR. + 2. issue-pr-mismatch — milestone issue whose closing PR is in a different + milestone (or none), or issue not associated to the Main project. + 3. pr-issue-mismatch — milestone PR whose closed issue is in a different + milestone (or none), or not associated to the Main project. + 4. needs-triage — MERGED PR whose closed issue has the `needs triage` label. + 5. unassigned — MERGED PR whose closed issue has no assignees and no + `community contribution` label (on the issue or the PR). + +Project membership is only checked on the issue side: PRs are not the unit +tracked on the Main board, so an unprojected PR alone is never an anomaly. + +Usage: + python3 scripts/project-anomalies.py check "2.17.0" + python3 scripts/project-anomalies.py check "2.17.0" --output tmp/2.17.0-ANOMALIES.md + +Prerequisites: + - gh CLI authenticated (gh auth status) + - Python 3.8+ +""" + +import argparse +import json +import subprocess +import sys +from datetime import datetime, timezone +from pathlib import Path + + +REPO = "penpot/penpot" +GH_PY = Path(__file__).resolve().parent / "gh.py" +MAIN_PROJECT = "Main" +COMMUNITY_LABEL = "community contribution" +TRIAGE_LABEL = "needs triage" + + +# ───────────────────────────────────────────── +# Shared helpers +# ───────────────────────────────────────────── + + +def run_gh_py(*args: str) -> str: + """Run ``scripts/gh.py`` with the given arguments, return stdout.""" + cmd = [sys.executable, str(GH_PY), *args] + result = subprocess.run(cmd, capture_output=True, text=True) + if result.returncode != 0: + print(f"gh.py error: {result.stderr}", file=sys.stderr) + sys.exit(1) + return result.stdout + + +def issue_url(n: int) -> str: + return f"https://github.com/{REPO}/issues/{n}" + + +def pr_url(n: int) -> str: + return f"https://github.com/{REPO}/pull/{n}" + + +def issue_link(n: int) -> str: + return f"[#{n}]({issue_url(n)})" + + +def pr_link(n: int) -> str: + return f"[#{n}]({pr_url(n)})" + + +def issue_link_title(n: int, title: str) -> str: + url = issue_url(n) + if title: + return f"[#{n}]({url}) — [{title}]({url})" + return f"[#{n}]({url})" + + +def pr_link_title(n: int, title: str) -> str: + url = pr_url(n) + if title: + return f"[#{n}]({url}) — [{title}]({url})" + return f"[#{n}]({url})" + + +def in_main_project(item: dict) -> bool: + """Whether an issue/PR dict is associated to the Main project board.""" + return MAIN_PROJECT in (item.get("projects") or []) + + +# ───────────────────────────────────────────── +# Anomaly detection (pure: no network inside) +# ───────────────────────────────────────────── + + +def find_anomalies( + milestone: str, + issues: list[dict], + prs: list[dict], + pr_by_num: dict[int, dict], + issue_by_num: dict[int, dict], +) -> dict[str, list[dict]]: + """Detect the five anomaly types for a milestone. + + Callers pre-resolve outsiders: any closing PR/issue not in the milestone + must already sit in ``pr_by_num``/``issue_by_num`` (with at least + ``milestone``, ``state``, ``labels``, ``assignees`` and ``projects``). + Unknown numbers are skipped, never flagged. + """ + t1_open_merged: list[dict] = [] + t2_issue_pr: list[dict] = [] + t3_pr_issue: list[dict] = [] + t4_needs_triage: list[dict] = [] + t5_unassigned: list[dict] = [] + + for issue in issues: + if issue.get("state") != "OPEN": + continue + merged = [ + n for n in issue.get("closing_prs", []) + if (pr_by_num.get(n) or {}).get("state") == "MERGED" + ] + if merged: + t1_open_merged.append({ + "issue": issue["number"], + "issue_title": issue.get("title", ""), + "prs": sorted(merged), + }) + + for issue in issues: + for pr_num in issue.get("closing_prs", []): + pr = pr_by_num.get(pr_num) + if pr is None: + continue + # An unmerged PR has landed nowhere: its milestone says nothing + # about any release, so only merged PRs count here. + if pr.get("state") != "MERGED": + continue + if pr.get("milestone") != milestone: + t2_issue_pr.append({ + "issue": issue["number"], + "issue_title": issue.get("title", ""), + "pr": pr_num, + "pr_milestone": pr.get("milestone"), + }) + if not in_main_project(issue): + t2_issue_pr.append({ + "issue": issue["number"], + "issue_title": issue.get("title", ""), + "pr": None, + "pr_milestone": None, + "projects": issue.get("projects") or [], + }) + + for pr in prs: + # Like type 2: only a merged PR has landed somewhere, so only + # merged PRs take part in milestone pairing. (Whether the issue is + # on the Main board is checked regardless.) + if pr.get("state") != "MERGED": + continue + for issue_num in pr.get("closing_issues", []): + issue = issue_by_num.get(issue_num) + if issue is None: + continue + if issue.get("milestone", milestone) != milestone or not in_main_project(issue): + t3_pr_issue.append({ + "pr": pr["number"], + "pr_title": pr.get("title", ""), + "issue": issue_num, + "issue_title": issue.get("title", ""), + "issue_milestone": issue.get("milestone", milestone), + "projects": issue.get("projects") or [], + }) + + for pr in prs: + if pr.get("state") != "MERGED": + continue + for issue_num in pr.get("closing_issues", []): + issue = issue_by_num.get(issue_num) + if issue is None: + continue + if TRIAGE_LABEL in (issue.get("labels") or []): + t4_needs_triage.append({ + "pr": pr["number"], + "pr_title": pr.get("title", ""), + "issue": issue_num, + "issue_title": issue.get("title", ""), + }) + if not issue.get("assignees") and COMMUNITY_LABEL not in ( + (issue.get("labels") or []) + (pr.get("labels") or []) + ): + t5_unassigned.append({ + "pr": pr["number"], + "pr_title": pr.get("title", ""), + "issue": issue_num, + "issue_title": issue.get("title", ""), + }) + + return { + "open_merged": t1_open_merged, + "issue_pr": t2_issue_pr, + "pr_issue": t3_pr_issue, + "needs_triage": t4_needs_triage, + "unassigned": t5_unassigned, + } + + +# ───────────────────────────────────────────── +# Report rendering (pure) +# ───────────────────────────────────────────── + + +def milestone_fix(kind: str, number: int, milestone: str) -> str: + """Render the one-line `gh` command that assigns a missing milestone.""" + return f"Fix: `gh {kind} edit {number} --milestone \"{milestone}\"`" + + +def render_report( + milestone: str, + anomalies: dict[str, list[dict]], + total_issues: int, + closed_issues: int, + total_prs: int, + merged_prs: int, +) -> str: + """Render the full anomalies report as Markdown.""" + t1 = anomalies["open_merged"] + t2 = anomalies["issue_pr"] + t3 = anomalies["pr_issue"] + t4 = anomalies["needs_triage"] + t5 = anomalies["unassigned"] + total = len(t1) + len(t2) + len(t3) + len(t4) + len(t5) + + lines = [ + f"# Project Anomalies — {milestone}", + "", + f"Generated: {datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M UTC')}", + "", + "---", + "", + "## Summary", + "", + f"- **OPEN issue with a MERGED PR:** {len(t1)}", + f"- **Milestone issue, PR elsewhere/unprojected:** {len(t2)}", + f"- **Milestone PR, issue elsewhere/unprojected:** {len(t3)}", + f"- **MERGED PR on a `needs triage` issue:** {len(t4)}", + f"- **MERGED PR on an unassigned, non-community issue:** {len(t5)}", + f"- **Total anomalies:** {total}", + "", + ] + + if total == 0: + lines += [ + "✅ No anomalies found. The milestone and the Main project board agree.", + "", + ] + else: + lines += ["## Anomalies", ""] + if t1: + lines += [ + "### OPEN issue with a MERGED PR", + "", + "The fix already landed but the issue never closed (or it was " + "reopened). Close the issue or move it out of the milestone.", + "", + ] + for e in t1: + lines.append(f"- {issue_link_title(e['issue'], e['issue_title'][:80])}") + for n in e["prs"]: + lines.append(f" - 🔴 Merged {pr_link(n)}") + lines.append("") + + if t2: + lines += [ + "### Milestone issue, PR elsewhere or issue off the Main board", + "", + ] + by_issue: dict[int, list[dict]] = {} + for e in t2: + by_issue.setdefault(e["issue"], []).append(e) + for issue_num in sorted(by_issue): + entries = by_issue[issue_num] + lines.append( + f"- {issue_link_title(issue_num, entries[0]['issue_title'][:80])}" + ) + for e in entries: + if e["pr"] is None: + projects = ", ".join(e["projects"]) or "_none_" + lines.append( + " - 🔴 Issue is not on the Main project board " + f"(projects: {projects})" + ) + else: + ms = e["pr_milestone"] or "_none_" + badge = "🔴" if e["pr_milestone"] is None else "⚠️" + lines.append( + f" - {badge} Closing {pr_link(e['pr'])} is in milestone " + f"**{ms}** (expected: {milestone})" + ) + if e["pr_milestone"] is None: + lines.append( + f" {milestone_fix('pr', e['pr'], milestone)}" + ) + lines.append("") + + if t3: + lines += [ + "### Milestone PR, issue elsewhere or off the Main board", + "", + ] + by_pr: dict[int, list[dict]] = {} + for e in t3: + by_pr.setdefault(e["pr"], []).append(e) + for pr_num in sorted(by_pr): + entries = by_pr[pr_num] + lines.append( + f"- {pr_link_title(pr_num, entries[0]['pr_title'][:80])}" + ) + for e in entries: + ms = e["issue_milestone"] or "_none_" + if e["issue_milestone"] != milestone: + lines.append( + f" - ⚠️ Closing {issue_link(e['issue'])} is in milestone " + f"**{ms}** (expected: {milestone})" + ) + if e["issue_milestone"] is None: + lines.append( + f" {milestone_fix('issue', e['issue'], milestone)}" + ) + else: + projects = ", ".join(e["projects"]) or "_none_" + lines.append( + f" - 🔴 Closing {issue_link(e['issue'])} is not on the " + f"Main project board (projects: {projects})" + ) + lines.append("") + + if t4: + lines += [ + "### MERGED PR on a `needs triage` issue", + "", + "The fix landed but the issue was never triaged. Human triage needed.", + "", + ] + for e in t4: + lines.append( + f"- ⚠️ {pr_link_title(e['pr'], e['pr_title'][:80])}" + f" closes {issue_link(e['issue'])}" + ) + lines.append("") + + if t5: + lines += [ + "### MERGED PR on an unassigned, non-community issue", + "", + "No owner and no community label. Assign an owner or confirm it " + "is a community contribution.", + "", + ] + for e in t5: + lines.append( + f"- ⚠️ {pr_link_title(e['pr'], e['pr_title'][:80])}" + f" closes {issue_link(e['issue'])}" + ) + lines.append("") + + lines += [ + "---", + "", + "## Context", + "", + f"- Milestone: **{milestone}**", + f"- Milestone issues (all states): {total_issues}", + f"- Open issues in milestone: {total_issues - closed_issues}", + f"- Milestone PRs (all states): {total_prs}", + f"- Merged PRs in milestone: {merged_prs}", + "", + ] + return "\n".join(lines) + + +# ───────────────────────────────────────────── +# CLI entrypoint +# ───────────────────────────────────────────── + + +def cmd_check(args: argparse.Namespace) -> None: + """Fetch milestone data in bulk, detect anomalies, write the report.""" + milestone = args.milestone + + all_issues = json.loads(run_gh_py("issues", milestone, "--state", "all")) + issue_by_num = {i["number"]: i for i in all_issues} + + all_prs = json.loads(run_gh_py("prs", "--milestone", milestone, "--state", "all")) + pr_by_num = {p["number"]: p for p in all_prs} + + # Resolve outsiders in bulk (one GraphQL call per 50 items). + unknown_prs = set() + for issue in all_issues: + unknown_prs.update(issue.get("closing_prs", [])) + unknown_prs -= set(pr_by_num) + if unknown_prs: + print( + f"Resolving {len(unknown_prs)} PRs outside {milestone}...", + file=sys.stderr, + ) + for pr in json.loads(run_gh_py("prs", *[str(n) for n in sorted(unknown_prs)])): + pr_by_num[pr["number"]] = pr + + unknown_issues = set() + for pr in all_prs: + unknown_issues.update(pr.get("closing_issues", [])) + unknown_issues -= set(issue_by_num) + if unknown_issues: + print( + f"Resolving {len(unknown_issues)} issues outside {milestone}...", + file=sys.stderr, + ) + for issue in json.loads( + run_gh_py("issue", *[str(n) for n in sorted(unknown_issues)]) + ): + if "error" not in issue: + issue_by_num[issue["number"]] = issue + + anomalies = find_anomalies(milestone, all_issues, all_prs, pr_by_num, issue_by_num) + report = render_report( + milestone, + anomalies, + total_issues=len(all_issues), + closed_issues=sum(1 for i in all_issues if i.get("state") == "CLOSED"), + total_prs=len(all_prs), + merged_prs=sum(1 for p in all_prs if p.get("state") == "MERGED"), + ) + + output = Path(args.output or f"tmp/{milestone}-ANOMALIES.md") + output.parent.mkdir(parents=True, exist_ok=True) + output.write_text(report) + total = sum(len(v) for v in anomalies.values()) + print(f"{total} anomalies written to {output}") + + +def main() -> None: + parser = argparse.ArgumentParser( + description="Find Main-project anomalies for a milestone" + ) + sub = parser.add_subparsers(dest="command", required=True, title="subcommands") + + p_check = sub.add_parser("check", help="Detect anomalies and write the report") + p_check.add_argument("milestone", help="Milestone title (e.g. '2.17.0')") + p_check.add_argument( + "--output", + default=None, + help="Report path (default: tmp/-ANOMALIES.md)", + ) + p_check.set_defaults(func=cmd_check) + + args = parser.parse_args() + args.func(args) + + +if __name__ == "__main__": + main() diff --git a/scripts/test_project_anomalies.py b/scripts/test_project_anomalies.py new file mode 100644 index 0000000000..ca781c58d2 --- /dev/null +++ b/scripts/test_project_anomalies.py @@ -0,0 +1,278 @@ +#!/usr/bin/env python3 +"""Tests for scripts/project-anomalies.py (pure logic, no network). + +Run with: + + python3 scripts/test_project_anomalies.py +""" + +import importlib.machinery +import importlib.util +import pathlib +import sys +import unittest + + +# Loading scripts/project-anomalies.py should not emit scripts/__pycache__/. +sys.dont_write_bytecode = True + +SCRIPT_PATH = pathlib.Path(__file__).resolve().parent / "project-anomalies.py" + + +def load_pa(): + """Load scripts/project-anomalies.py as a module without running its CLI.""" + loader = importlib.machinery.SourceFileLoader("project_anomalies", str(SCRIPT_PATH)) + spec = importlib.util.spec_from_loader("project_anomalies", loader) + module = importlib.util.module_from_spec(spec) + loader.exec_module(module) + return module + + +pa = load_pa() + +M = "2.17.0" + + +def issue(n, **kw): + base = { + "number": n, + "title": f"issue {n}", + "state": "CLOSED", + "milestone": M, + "labels": [], + "assignees": ["someone"], + "closing_prs": [], + "project_status": "Done", + "projects": ["Main"], + } + base.update(kw) + return base + + +def pr(n, **kw): + base = { + "number": n, + "title": f"pr {n}", + "state": "MERGED", + "milestone": M, + "labels": [], + "assignees": ["someone"], + "closing_issues": [], + } + base.update(kw) + return base + + +def run(issues, prs): + return pa.find_anomalies( + M, issues, prs, + {p["number"]: p for p in prs}, + {i["number"]: i for i in issues}, + ) + + +class OpenMergedTests(unittest.TestCase): + def test_open_issue_with_merged_pr_is_flagged(self): + out = run( + [issue(1, state="OPEN", closing_prs=[11])], + [pr(11, closing_issues=[1])], + ) + self.assertEqual(len(out["open_merged"]), 1) + self.assertEqual(out["open_merged"][0]["prs"], [11]) + + def test_open_issue_with_unmerged_pr_is_clean(self): + out = run( + [issue(1, state="OPEN", closing_prs=[11])], + [pr(11, state="OPEN", closing_issues=[1])], + ) + self.assertEqual(out["open_merged"], []) + + def test_closed_issue_with_merged_pr_is_clean(self): + out = run( + [issue(1, closing_prs=[11])], + [pr(11, closing_issues=[1])], + ) + self.assertTrue(all(v == [] for v in out.values())) + + +class IssuePrMismatchTests(unittest.TestCase): + def test_pr_in_other_milestone_is_flagged(self): + out = run( + [issue(1, closing_prs=[11])], + [pr(11, milestone="2.18.0")], + ) + self.assertEqual(len(out["issue_pr"]), 1) + self.assertEqual(out["issue_pr"][0]["pr_milestone"], "2.18.0") + + def test_pr_without_milestone_is_flagged(self): + out = run( + [issue(1, closing_prs=[11])], + [pr(11, milestone=None)], + ) + self.assertEqual(len(out["issue_pr"]), 1) + + def test_unmerged_pr_milestone_is_ignored(self): + # An OPEN (unmerged) PR has landed nowhere: its milestone is + # irrelevant, even when it differs. + out = run( + [issue(1, closing_prs=[11])], + [pr(11, state="OPEN", milestone="2.18.0")], + ) + self.assertEqual(out["issue_pr"], []) + + def test_off_board_issue_flagged_despite_unmerged_pr(self): + # The project check does not depend on PR merge state. + out = run( + [issue(1, projects=["Other"], closing_prs=[11])], + [pr(11, state="OPEN", milestone="2.18.0")], + ) + self.assertEqual(len(out["issue_pr"]), 1) + self.assertIsNone(out["issue_pr"][0]["pr"]) + + def test_issue_off_main_board_is_flagged_without_pr(self): + out = run([issue(1, projects=["Other"], project_status=None)], []) + self.assertEqual(len(out["issue_pr"]), 1) + self.assertIsNone(out["issue_pr"][0]["pr"]) + + def test_aligned_pair_is_clean(self): + out = run( + [issue(1, closing_prs=[11])], + [pr(11, closing_issues=[1])], + ) + self.assertEqual(out["issue_pr"], []) + self.assertEqual(out["pr_issue"], []) + + +class PrIssueMismatchTests(unittest.TestCase): + def test_issue_in_other_milestone_is_flagged(self): + other = issue(2, milestone="2.16.0") + out = pa.find_anomalies( + M, [issue(1)], [pr(11, closing_issues=[2])], + {11: pr(11, closing_issues=[2])}, {1: issue(1), 2: other}, + ) + self.assertEqual(len(out["pr_issue"]), 1) + + def test_issue_without_milestone_is_flagged(self): + # Unlike the changelog flow, here a milestone-less issue closed by a + # milestone PR is reported: the pairing needs human judgment. + other = issue(2, milestone=None, projects=[], project_status=None) + out = pa.find_anomalies( + M, [issue(1)], [pr(11, closing_issues=[2])], + {11: pr(11, closing_issues=[2])}, {1: issue(1), 2: other}, + ) + self.assertEqual(len(out["pr_issue"]), 1) + + def test_issue_off_main_board_is_flagged(self): + other = issue(2, projects=["Other"], project_status=None) + out = pa.find_anomalies( + M, [issue(1)], [pr(11, closing_issues=[2])], + {11: pr(11, closing_issues=[2])}, {1: issue(1), 2: other}, + ) + self.assertEqual(len(out["pr_issue"]), 1) + + def test_milestone_issue_keeps_milestone_in_record(self): + # Milestone issues carry no "milestone" key (known by construction); + # an off-board one must still render its own milestone, never _none_. + mine = issue(1, projects=["Other"], project_status=None) + del mine["milestone"] + out = pa.find_anomalies( + M, [mine], [pr(11, closing_issues=[1])], + {11: pr(11, closing_issues=[1])}, {1: mine}, + ) + self.assertEqual(len(out["pr_issue"]), 1) + self.assertEqual(out["pr_issue"][0]["issue_milestone"], M) + report = pa.render_report(M, out, 1, 0, 1, 1) + self.assertNotIn("gh issue edit 1", report) + + def test_unmerged_pr_is_ignored(self): + # Closed-unmerged (or open) milestone PRs take no part in milestone + # pairing: only merged PRs landed somewhere. + other = issue(2, milestone="2.16.0") + for state in ("OPEN", "CLOSED"): + out = pa.find_anomalies( + M, [issue(1)], [pr(11, state=state, closing_issues=[2])], + {11: pr(11, state=state, closing_issues=[2])}, + {1: issue(1), 2: other}, + ) + self.assertEqual(out["pr_issue"], [], f"state={state}") + + +class NeedsTriageTests(unittest.TestCase): + def test_merged_pr_on_triaged_issue_is_clean(self): + out = run( + [issue(1, labels=["bug"])], + [pr(11, closing_issues=[1])], + ) + self.assertEqual(out["needs_triage"], []) + + def test_merged_pr_on_needs_triage_issue_is_flagged(self): + out = run( + [issue(1, labels=["needs triage"])], + [pr(11, closing_issues=[1])], + ) + self.assertEqual(len(out["needs_triage"]), 1) + + def test_unmerged_pr_on_needs_triage_issue_is_clean(self): + out = run( + [issue(1, labels=["needs triage"])], + [pr(11, state="OPEN", closing_issues=[1])], + ) + self.assertEqual(out["needs_triage"], []) + + +class UnassignedTests(unittest.TestCase): + def test_assigned_issue_is_clean(self): + out = run( + [issue(1, assignees=["owner"])], + [pr(11, closing_issues=[1])], + ) + self.assertEqual(out["unassigned"], []) + + def test_unassigned_community_issue_is_clean(self): + out = run( + [issue(1, assignees=[], labels=["community contribution"])], + [pr(11, closing_issues=[1])], + ) + self.assertEqual(out["unassigned"], []) + + def test_unassigned_community_pr_is_clean(self): + out = run( + [issue(1, assignees=[])], + [pr(11, closing_issues=[1], labels=["community contribution"])], + ) + self.assertEqual(out["unassigned"], []) + + def test_unassigned_non_community_issue_is_flagged(self): + out = run( + [issue(1, assignees=[])], + [pr(11, closing_issues=[1])], + ) + self.assertEqual(len(out["unassigned"]), 1) + + def test_unmerged_pr_is_clean(self): + out = run( + [issue(1, assignees=[])], + [pr(11, state="CLOSED", closing_issues=[1])], + ) + self.assertEqual(out["unassigned"], []) + + +class RenderTests(unittest.TestCase): + def test_clean_report_says_so(self): + out = run([issue(1, closing_prs=[11])], [pr(11, closing_issues=[1])]) + report = pa.render_report(M, out, 1, 0, 1, 1) + self.assertIn("Total anomalies:** 0", report) + self.assertIn("✅ No anomalies", report) + + def test_every_number_is_a_clickable_link(self): + out = run( + [issue(1, state="OPEN", closing_prs=[11])], + [pr(11, closing_issues=[1])], + ) + report = pa.render_report(M, out, 1, 0, 1, 1) + self.assertIn("[#1](https://github.com/penpot/penpot/issues/1)", report) + self.assertIn("[#11](https://github.com/penpot/penpot/pull/11)", report) + + +if __name__ == "__main__": + unittest.main(verbosity=2)