Félix Malfait 0bcce772d2 Add QA Scout: post-merge browser QA agent on main (#24856)
## What

A post-merge browser QA agent, riding the environment `ci-e2e-main`
already boots on every push to main.

After the deterministic Playwright suite, the QA Scout:

1. Resolves the merged PR for the pushed commit and collects its
metadata, file list, and diff. Skips i18n translation PRs and titles
carrying `[no-qa]` / `[noqa]`, mirroring `mustBeQa` in the eng app.
2. Runs Claude Code (`claude-code-action`, same pin as `claude.yml`)
with the Playwright MCP against the already-running app at
`localhost:3000`, following the new skill at
`.claude/skills/qa-scout/SKILL.md`: derive 2 to 5 user-visible scenarios
from the diff, execute them in the browser, and diff its own window of
the server and worker logs to catch swallowed backend exceptions behind
a green-looking UI (the 2.35 timelineActivity failure mode).
3. Writes `verdict.json` (`PASS | INVESTIGATE | FAIL`) and `report.md`,
uploaded as the `qa-scout-report` artifact and rendered in the job
summary.
4. On `FAIL` / `INVESTIGATE`, a separate job posts (or updates) a single
marked comment on the merged PR, opening with a `[!CAUTION]` admonition.
`PASS` stays quiet.

The skill also works locally: engineers can run `/qa-scout` in Claude
Code against a dev stack.

## Why

Post-merge QA of merged PRs is today a fully manual Build Companion
queue, and the deterministic e2e suite covers 7 flows. The Scout
pre-chews that QA: it picks the risky flows for each specific diff, and
it reads the logs while it clicks, which is exactly where
silently-swallowed regressions hide. `hasQaBeenDoneOnMain` stays
human-owned; the Scout's report lands where the human QA starts.

## Safety and rollout

- **Shadow-safe**: every Scout step is `continue-on-error`, so it can
never redden `ci-e2e-main` or affect `ci-e2e-main-status-check`. The
suite's own behavior is unchanged (server/worker logs are now teed to
files; console output is identical).
- **Privilege separation**: the agent step runs under the workflow's
read-only token and only gets `Read/Grep/Glob/Write`, a short Bash
allowlist, and the Playwright MCP (headless Chrome, isolated profile,
origin-restricted to localhost). `pull-requests: write` exists only in
the `qa-scout-comment` job, which reads the structured artifact and
never the agent's raw context, so untrusted page/log content the agent
reads can never reach a privileged credential.
- **Never on fork or PR events**: the Scout only runs on push to main
and manual `workflow_dispatch`; merge-queue PR runs of this workflow are
untouched.
- **Kill switch**: set repo variable `QA_SCOUT_DISABLE_COMMENTS=true` to
keep verdicts in artifacts/summaries only (shadow mode).
- **Re-run**: Actions, CI E2E Main, Run workflow with `pr_number`
re-runs the Scout for any merged PR.

## Validation

- Workflow YAML parses; both embedded scripts pass `bash -n`; the
heredoc MCP config is valid JSON.
- The `mustBeQa` mirror was table-tested against real title shapes
(`[no-qa]`, `[NOQA]`, `i18n - translations`, mid-title i18n mentions).
- `@playwright/mcp@0.0.79` (current latest) flags used (`--headless`,
`--browser chrome`, `--isolated`, `--output-dir`, `--allowed-origins`)
verified against its CLI help.
- `.gitignore` gains `!.claude/skills/` so the skill can live next to
the committed `.claude/settings.json`.

## Follow-ups (separate PRs)

- Wire the verdict into the eng app: a webhook route (like
`/s/main-ci-failing`) writing the verdict onto the `pullRequest` record
so Build Companion shows it in the QA queue.
- Point the same skill at staging after deploy/upgrade, where aged data
lives.
- Have the nightly/deep variant draft Playwright specs for stable
scenarios so the deterministic suite grows.

---
_Generated by [Claude
Code](https://claude.ai/code/session_01B1ML7TTsrqPbPnCtox9rBX)_

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/24856?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
2026-08-27 11:42:11 +02:00

Twenty logo

The #1 Open-Source CRM

Website · Documentation · Roadmap · Discord · Figma

Twenty banner


Why Twenty

Twenty gives technical teams the building blocks for a custom CRM that meets complex business needs and quickly adapts as the business evolves. Twenty is the CRM you build, ship, and version like the rest of your stack.

Learn more about why we built Twenty


Installation

Cloud

The fastest way to get started. Sign up at twenty.com and spin up a workspace in under a minute, with no infrastructure to manage and always up to date.

Build an app

Scaffold a new app with the Twenty CLI:

npx create-twenty-app my-app

Define objects, fields, and views as code:

import { defineObject, FieldType } from 'twenty-sdk/define';

export default defineObject({
  nameSingular: 'deal',
  namePlural: 'deals',
  labelSingular: 'Deal',
  labelPlural: 'Deals',
  fields: [
    { name: 'name', label: 'Name', type: FieldType.TEXT },
    { name: 'amount', label: 'Amount', type: FieldType.CURRENCY },
    { name: 'closeDate', label: 'Close Date', type: FieldType.DATE_TIME },
  ],
});

Then ship it to your workspace:

npx twenty app:publish --private

See the app development guide for objects, views, agents, and logic functions.

Self-hosting

Run Twenty on your own infrastructure with Docker Compose, or contribute locally via the local setup guide.



Everything you need

Twenty gives you the building blocks of a modern CRM (objects, views, workflows, and agents) and lets you extend them as code. Here's a tour of what's in the box.

Want to go deeper? Read the User Guide for product walkthroughs, or the Documentation for developer reference.

Create your apps

Learn more about apps in doc

Stay on top with version control

Learn more about version control in doc

All the tools you need to build anything

Learn more about primitives in doc

Customize your layouts

Learn more about layouts in doc

AI agents and chats

Learn more about AI in doc

Plus all the tools of a good CRM

Learn more about CRM features in doc


Stack

Thanks

Greptile      Sentry      Crowdin

Thanks to these amazing services that we use and recommend for code review (Greptile), catching bugs (Sentry) and translating (Crowdin).

Join the Community

Star the repo · Discord · Feature requests · Releases · X · LinkedIn · Crowdin · Contribute

S
Description
No description provided
Readme AGPL-3.0
2.5 GiB
0 Stars 1 Watchers 0 Forks
Languages
TypeScript 81.6%
MDX 15.7%
JavaScript 2.3%
Python 0.2%