Commit Graph

27 Commits

Author SHA1 Message Date
martmull
056e3a4cd8 Add check for breaking api changes (#20848)
- update ci-breaking-changes.yaml so it check for api contrat breaks
- check fails properly when removing fix
https://github.com/twentyhq/twenty/pull/20825
- check it turns green again when adding fix back
2026-05-23 08:50:05 +00:00
neo773
565995e715 security: harden CI against supply-chain attacks (#20476)
- Pin all third-party actions to SHA
- Gate claude.yml triggers to internal authors with Harden-Runner egress
audit
- Ignore fork-PR lifecycle scripts
- Narrow cross-repo dispatch payloads
- Add 7d npm release-age gate
- Add CODEOWNERS on .github/** and .yarnrc.yml

---------

Co-authored-by: prastoin <paul@twenty.com>
2026-05-12 12:20:29 +00:00
Félix Malfait
8d54ff6ca0 fix(ci): probe real schema in breaking-changes server readiness check (#20465)
## Summary

The `GraphQL and OpenAPI Breaking Changes Detection` workflow has been
posting graphql-inspector stack traces as PR comments — see [#20445
comment](https://github.com/twentyhq/twenty/pull/20445#issuecomment-4421142635)
for an example.

### Root cause

- The wait step probed readiness with `curl -s URL > /dev/null 2>&1`,
which exits 0 for **any** HTTP response — including 5xx and GraphQL
error JSON. NestJS opens the HTTP listener before the workspace schema
cache is fully populated, so the wait often completed while the server
still served auth/metadata error JSON.
- The introspection download therefore wrote a small (~154-byte) error
payload instead of the real schema. `jq empty` in the validation step
only checks JSON *syntax*, so `{"errors":[...]}` passed validation.
- `graphql-inspector diff` then failed with `Unable to read JSON file:
... Not valid JSON content`, the workflow swallowed the error into the
diff markdown, and the bot posted that stack trace verbatim on the PR.

In the failing run, the main-branch files were 154 B (GraphQL) and 112 B
(REST 500); the current-branch files in the same run were 600 KB–2.8 MB.

### Fix

- Wait steps now POST an authenticated introspection (`{ __schema {
queryType { name } } }`) and require `.data.__schema` plus a 2xx
response from `/rest/open-api/core` (`curl -f`) before declaring the
server ready.
- Validation step now checks for the expected shape (`.data.__schema`
for GraphQL, `.openapi`/`.swagger` for OpenAPI) and includes the first
200 bytes of any bad payload in the warning, so when something genuinely
goes wrong the next debugger has a real lead instead of a generic stack
trace.

## Test plan

- [ ] CI runs against this branch — the workflow's own readiness probes
are now exercised against the real server, so a green run validates the
new check.
- [ ] If the readiness probe still passes but downloads regress, the
strengthened validation step will surface the payload in the workflow
logs instead of posting a graphql-inspector stack trace on the PR.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-05-12 06:43:35 +00:00
Félix Malfait
9b8cb610c3 Flush Redis between server runs in breaking changes CI (#19491)
## Summary
Added a Redis cache flush step in the breaking changes CI workflow to
prevent stale data contamination between consecutive server runs.

## Key Changes
- Added a new workflow step that executes `redis-cli FLUSHALL` between
the current branch server run and the main branch server run
- Includes error handling to log a warning if the Redis flush fails,
without blocking the workflow
- Added explanatory comments documenting why this step is necessary

## Implementation Details
The Redis flush is necessary because both the current branch and main
branch servers share the same Redis instance during CI testing. The
`CoreEntityCacheService` and `WorkspaceCacheService` persist cached
entities across process restarts, which can cause stale data from the
current branch server to contaminate the main branch server comparison.
This step ensures a clean cache state before switching branches.

https://claude.ai/code/session_01BVMacfAXDMNx5WAFtP7GgW

Co-authored-by: Claude <noreply@anthropic.com>
2026-04-09 12:56:24 +02:00
Charles Bochet
ac8e0d4217 Replace twentycrm/twenty-postgres-spilo with official postgres:16 in CI (#19182)
## Summary
- Replaces `twentycrm/twenty-postgres-spilo` with the official
`postgres:16` image across all 7 CI workflow files
- Removes Docker Hub `credentials` blocks from all service containers
(postgres, redis, clickhouse)
- Removes the `Login to Docker Hub` step from the breaking changes
workflow

## Context
Fork PRs cannot access repository secrets/variables, causing `${{
vars.DOCKERHUB_USERNAME }}` and `${{ secrets.DOCKERHUB_PASSWORD }}` to
resolve to empty strings. GitHub Actions rejects empty credential values
at template validation time, failing the job before any step runs.

The custom spilo image was the original reason credentials were needed
(to avoid Docker Hub rate limits on non-official images). The only
Postgres extensions required in CI (`uuid-ossp`, `unaccent`) are built
into the official `postgres:16` image. Official Docker Hub images have
significantly higher pull rate limits and don't require authentication.
2026-03-31 21:41:42 +02:00
Félix Malfait
8fa3962e1c feat: add resumable stream support for agent chat (#19107)
## Overview
Add resumable stream support for agent chat to allow clients to
reconnect and resume streaming responses if the connection is
interrupted (e.g., during page refresh).

## Changes

### Backend (Twenty Server)
- Add `activeStreamId` column to `AgentChatThreadEntity` to track
ongoing streams
- Create `AgentChatResumableStreamService` to manage Redis-backed
resumable streams using the `resumable-stream` library with ioredis
- Extend `AgentChatController` with:
  - `GET /:threadId/stream` endpoint to resume an existing stream
  - `DELETE /:threadId/stream` endpoint to stop an active stream
- Update `AgentChatStreamingService` to store streams in Redis and track
active stream IDs
- Add `resumable-stream@^2.2.12` dependency to package.json

### Frontend (Twenty Front)
- Update `useAgentChat` hook to:
- Use a persistent transport with `prepareReconnectToStreamRequest` for
resumable streams
  - Export `resumeStream` function from useChat
  - Add `handleStop` callback to clear active stream on DELETE endpoint
- Use thread ID as stable message ID instead of including message count
- Add stream resumption logic in `AgentChatAiSdkStreamEffect` component
to automatically call `resumeStream()` when switching threads

## Database Migration
New migration `1774003611071-add-active-stream-id-to-agent-chat-thread`
adds the `activeStreamId` column to store the current resumable stream
identifier.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 18:19:25 +02:00
Charles Bochet
191a277ddf fix: invalidate rolesPermissions cache + add Docker Hub auth to CI (#19044)
## Summary

### Cache invalidation fix
- After migrating object/field permissions to syncable entities (#18609,
#18751, #18567), changes to `flatObjectPermissionMaps`,
`flatFieldPermissionMaps`, or `flatPermissionFlagMaps` no longer
triggered `rolesPermissions` cache invalidation
- This caused stale permission data to be served, leading to flaky
`permissions-on-relations` integration tests and potentially incorrect
permission enforcement in production after object permission upserts
- Adds the three permission-related flat map keys to the condition that
triggers `rolesPermissions` cache recomputation in
`WorkspaceMigrationRunnerService.getLegacyCacheInvalidationPromises`
- Clears memoizer after recomputation to prevent concurrent
`getOrRecompute` calls from caching stale data

### Docker Hub rate limit fix
- CI service containers (postgres, redis, clickhouse) and `docker
run`/`docker build` steps were pulling from Docker Hub
**unauthenticated**, hitting the 100-pull-per-6-hour rate limit on
shared GitHub-hosted runner IPs
- Adds `credentials` blocks to all service container definitions and
`docker/login-action` steps before `docker run`/`docker compose`
commands
- Uses `vars.DOCKERHUB_USERNAME` + `secrets.DOCKERHUB_PASSWORD`
(matching the existing twenty-infra convention)
- Affected workflows: ci-server, ci-merge-queue, ci-breaking-changes,
ci-zapier, ci-sdk, ci-create-app-e2e, ci-website,
ci-test-docker-compose, preview-env-keepalive, spawn-twenty-docker-image
action
2026-03-27 17:32:53 +01:00
Paul Rastoin
281bb6d783 Guard yarn database:migrate:prod (#19008)
## Motivations
A lot of self hosters hands up using the `yarn database:migrated:prod`
either manually or through AI assisted debug while they try to upgrade
an instance while their workspace is still blocked in a previous one
Leading to their whole database permanent corruption

## What happened
Replaced the direct call the the typeorm cli to a command calling it
programmatically, adding a layer of security in case a workspace seems
to be blocked in a previous version than the one just before the one
being installed ( e.g 1.0 when you try to upgrade from 1.1 to 1.2 )

For our cloud we still need a way to bypass this security explaining the
-f flag

## Remark
Centralized this logic and refactored creating new services
`WorkspaceVersionService` and `CoreEngineVersionService` that will
become useful for the upcoming upgrade refactor

Related to https://github.com/twentyhq/twenty-infra/pull/529
2026-03-27 14:39:18 +00:00
Charles Bochet
9d57bc39e5 Migrate from ESLint to OxLint (#18443)
## Summary

Fully replaces ESLint with OxLint across the entire monorepo:

- **Replaced all ESLint configs** (`eslint.config.mjs`) with OxLint
configs (`.oxlintrc.json`) for every package: `twenty-front`,
`twenty-server`, `twenty-emails`, `twenty-ui`, `twenty-shared`,
`twenty-sdk`, `twenty-zapier`, `twenty-docs`, `twenty-website`,
`twenty-apps/*`, `create-twenty-app`
- **Migrated custom lint rules** from ESLint plugin format to OxLint JS
plugin system (`@oxlint/plugins`), including
`styled-components-prefixed-with-styled`, `no-hardcoded-colors`,
`sort-css-properties-alphabetically`,
`graphql-resolvers-should-be-guarded`,
`rest-api-methods-should-be-guarded`, `max-consts-per-file`, and
Jotai-related rules
- **Migrated custom rule tests** from ESLint `RuleTester` + Jest to
`oxlint/plugins-dev` `RuleTester` + Vitest
- **Removed all ESLint dependencies** from `package.json` files and
regenerated lockfiles
- **Updated Nx targets** (`lint`, `lint:diff-with-main`, `fmt`) in
`nx.json` and per-project `project.json` to use `oxlint` commands with
proper `dependsOn` for plugin builds
- **Updated CI workflows** (`.github/workflows/ci-*.yaml`) — no more
ESLint executor
- **Updated IDE setup**: replaced `dbaeumer.vscode-eslint` with
`oxc.oxc-vscode` extension, configured `source.fixAll.oxc` and
format-on-save with Prettier
- **Replaced all `eslint-disable` comments** with `oxlint-disable`
equivalents across the codebase
- **Updated docs** (`twenty-docs`) to reference OxLint instead of ESLint
- **Renamed** `twenty-eslint-rules` package to `twenty-oxlint-rules`

### Temporarily disabled rules (tracked in `OXLINT_MIGRATION_TODO.md`)

| Rule | Package | Violations | Auto-fixable |
|------|---------|-----------|-------------|
| `twenty/sort-css-properties-alphabetically` | twenty-front | 578 | Yes
|
| `typescript/consistent-type-imports` | twenty-server | 3814 | Yes |
| `twenty/max-consts-per-file` | twenty-server | 94 | No |

### Dropped plugins (no OxLint equivalent)

`eslint-plugin-project-structure`, `lingui/*`, `@stylistic/*`,
`import/order`, `prefer-arrow/prefer-arrow-functions`,
`eslint-plugin-mdx`, `@next/eslint-plugin-next`,
`eslint-plugin-storybook`, `eslint-plugin-react-refresh`. Partial
coverage for `jsx-a11y` and `unused-imports`.

### Additional fixes (pre-existing issues exposed by merge)

- Fixed `EmailThreadPreview.tsx` broken import from main rename
(`useOpenEmailThreadInSidePanel`)
- Restored truthiness guard in `getActivityTargetObjectRecords.ts`
- Fixed `AgentTurnResolver` return types to match entity (virtual
`fileMediaType`/`fileUrl` are resolved via `@ResolveField()`)

## Test plan

- [x] `npx nx lint twenty-front` passes
- [x] `npx nx lint twenty-server` passes
- [x] `npx nx lint twenty-docs` passes
- [x] Custom oxlint rules validated with Vitest: `npx nx test
twenty-oxlint-rules`
- [x] `npx nx typecheck twenty-front` passes
- [x] `npx nx typecheck twenty-server` passes
- [x] CI workflows trigger correctly with `dependsOn:
["twenty-oxlint-rules:build"]`
- [x] IDE linting works with `oxc.oxc-vscode` extension
2026-03-06 01:03:50 +01:00
Charles Bochet
d48c58640c Migrate CI runners from Depot back to GitHub-hosted runners (#18347)
## Summary
- Replaces all `depot-ubuntu-24.04` runners with `ubuntu-latest`
- Replaces all `depot-ubuntu-24.04-8` runners with
`ubuntu-latest-8-cores`
- Updates storybook build cache keys in ci-front.yaml to reflect the
runner name change

Reverts the temporary Depot migration introduced in #18163 / #18179
across all 23 workflow files.
2026-03-03 14:14:27 +01:00
Félix Malfait
0223975bbd Harden GitHub Actions: fix injections, isolate privileged operations to ci-privileged repo (#18318)
## Summary

- Fix expression injection vulnerabilities in composite actions
(`restore-cache`, `nx-affected`) and workflow files (`claude.yml`)
- Reduce overly broad permissions in `ci-utils.yaml` (Danger.js) and
`ci-breaking-changes.yaml`
- Restructure `preview-env-dispatch.yaml`: auto-trigger for members,
opt-in for contributor PRs via `preview-app` label (safe because
keepalive has no write tokens)
- Isolate all write-access operations (PR comments, cross-repo posting)
to a new dedicated
[`twentyhq/ci-privileged`](https://github.com/twentyhq/ci-privileged)
repo via `repository_dispatch`, so that workflows in twenty that execute
contributor code never have write tokens
- Create `post-ci-comments.yaml` (`workflow_run` bridge) to dispatch
breaking changes results to ci-privileged, solving the [fork PR comment
issue](https://github.com/twentyhq/twenty/pull/13713#issuecomment-3168999083)
- Delete 5 unused secrets and broken `i18n-qa-report` workflow
- Remove `TWENTY_DISPATCH_TOKEN` from twenty (moved to ci-privileged as
`CORE_TEAM_ISSUES_COMMENT_TOKEN`)
- Use `toJSON()` for all `client-payload` values to prevent JSON
injection

## Security model after this PR

| Workflow | Executes fork code? | Write tokens available? |
|----------|---------------------|------------------------|
| preview-env-keepalive | Yes | None (contents: read only) |
| preview-env-dispatch | No (base branch) | CI_PRIVILEGED_DISPATCH_TOKEN
only |
| ci-breaking-changes | Yes | None (contents: read only) |
| post-ci-comments (workflow_run) | No (default branch) |
CI_PRIVILEGED_DISPATCH_TOKEN only |
| claude.yml | No (base branch) | CI_PRIVILEGED_DISPATCH_TOKEN,
CLAUDE_CODE_OAUTH_TOKEN |
| ci-utils (Danger.js) | No (base branch) | GITHUB_TOKEN (scoped) |

All actual write tokens (`TWENTY_PR_COMMENT_TOKEN`,
`CORE_TEAM_ISSUES_COMMENT_TOKEN`) live in `twentyhq/ci-privileged` with
strict CODEOWNERS review and branch protection.

## Test plan

- [ ] Verify preview environment comments still appear on member PRs
- [ ] Verify adding `preview-app` label triggers preview for contributor
PRs
- [ ] Verify breaking changes reports still post on PRs (including fork
PRs)
- [ ] Verify Claude cross-repo responses still post on core-team-issues
- [ ] Confirm ci-privileged branch protection is enforced
2026-03-02 10:57:14 +01:00
Charles Bochet
1fa55bfd02 Fix bugs tied to jotai migration (#18227)
Fixing a few bugs:
- CommandMenu not reactive
- Filtering on index view infinite loop
2026-02-25 17:11:12 +01:00
Charles Bochet
129d1ede86 Change runners temp (#18163)
Temporarily moving all ubuntu-latest 1 core to depot except ci-website
2026-02-23 10:53:31 +01:00
Félix Malfait
e6491d6a80 feat(i18n): fix translation QA issues and add automation (#16756)
## Summary

This PR fixes translation QA issues and adds automation to prevent
future issues.

### Translation Fixes
- Fixed **escaped Unicode sequences** in translations (e.g.,
`\u62db\u5f85` → `招待`)
- Removed **corrupted control characters** from .po files (null bytes,
invalid characters)
- Fixed **missing/incorrect placeholders** in various languages
- Deleted **35 problematic translations** via Crowdin API that had
variable mismatches

### New Scripts (in `packages/twenty-utils/`)
- `fix-crowdin-translations.ts` - Auto-fixes encoding issues and syncs
to Crowdin
- `fix-qa-issues.ts` - Fixes specific QA issues via Crowdin API
- `translation-qa-report.ts` - Generates weekly QA report from Crowdin
API

### New Workflow
- `i18n-qa-report.yaml` - Weekly workflow that creates a PR with
translation QA issues for review

### Other Changes
- Moved GitHub Actions from `.github/workflows/actions/` to
`.github/actions/`
- Fixed `date-utils.ts` to avoid nested `t` macros in plural expressions
(root cause of confusing placeholders)

### QA Status After Fixes
| Category | Count | Status |
|----------|-------|--------|
| variables | 0  | Fixed |
| tags | 1 | Minor |
| empty | 0  | Fixed |
| spaces | 127 | Low priority |
| numbers | 246 | Locale-specific |
| special_symbols | 268 | Locale-specific |
2025-12-22 17:30:46 +01:00
Félix Malfait
3d7f332f80 Change runner for breaking change CI (#15205)
As per title
2025-10-20 13:55:12 +02:00
Félix Malfait
5375a478db Fix: Make CI .env manipulation robust against missing trailing newlines (#15189)
## Problem

CI workflow started timing out on October 14, 2025 after commit
`d750df7fff` removed the trailing newline from `.env.example`.

## Root Cause

When `.env.example` lacks a trailing newline:
```bash
# Last line without newline
# CLICKHOUSE_URL=...twenty
```

And CI runs:
```bash
echo "NODE_PORT=3002" >> .env
```

Result:
```bash
# CLICKHOUSE_URL=...twentyNODE_PORT=3002  ← Commented out!
```

Server starts on default port 3000 instead of 3002, health check fails.

## Fix

1. **Restore trailing newline** to `.env.example`
2. **Make all CI `.env` operations robust** by adding `echo "" >> .env`
before appending
3. **Simplified `set_env_var`** function to always add newline first

Now works regardless of whether template files have trailing newlines.

## Files Changed

- 6 CI workflow files
- 1 .env.example file
2025-10-18 13:46:56 +02:00
Félix Malfait
d76abefdee Fix CI concurrency: prevent test cancellation on main branch (#15188)
## Problem
The concurrency rules in CI workflows were cancelling in-progress test
runs even on the main branch. This caused inconsistent check counts when
multiple commits were pushed in quick succession.

## Solution
Updated `cancel-in-progress` in all CI workflows to be conditional:
- **On main branch**: Tests run to completion (no cancellation)
- **On feature branches**: Tests are cancelled when new commits are
pushed (saves CI resources)

## Changes
Modified 11 workflow files to use:
```yaml
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
```

This ensures every commit to main gets fully tested while maintaining
efficiency on feature branches.
2025-10-18 10:50:54 +02:00
Etienne
c693c4d9cf Common API - create common find one query (#14720)
closes https://github.com/twentyhq/core-team-issues/issues/1418

Tested : 
- findOne on Rest and Gql

### Vision
#### Common
- Common is kind of renamed Gql base resolver
- Common handles args (filter, values, ...) validation
- Common accepts depth or raw gql selected fields to compute
selectedFields
- Common is directly called by each CommonQueries (findOne, ...)
service, which extend CommonBaseQuery service
- Common sequence : 
| - Parse & Validate args (args-handlers, to create)
| - Build query (query-parsers : currently in gql-query-parsers, to
move)
| - Execute query
| - Fetch relation + format
#### Rest
- Simple parsing (without metadata validation)
- Calling Common API
- Simple rest response formatting
#### Gql
- Calling Common API
2025-10-08 10:18:42 +00:00
Abdul Rahman
216d72b5d7 AI SDK v5 migration (#14549)
Co-authored-by: Félix Malfait <felix@twenty.com>
2025-09-22 22:13:43 +02:00
Charles Bochet
a21daa2670 Optimize CI runner cost (#14628) 2025-09-20 12:14:29 +02:00
Félix Malfait
30a2164980 First Application POC (#14382)
Quick proof of concept for twenty-apps + twenty-cli, with local
development / hot reload

Let's discuss it!



https://github.com/user-attachments/assets/c6789936-cd5f-4110-a265-863a6ac1af2d
2025-09-10 15:12:38 +02:00
Charles Bochet
953f1e7207 Fix CI cache for storybook (#14038)
In this PR:
- Migrate twenty-ui to ES2020 (removing undesired type errors during
build)
- Make CI cache work again between `sb-build` and `sb-test` steps

(Note: also fixing a minor bug on Notes)

Total CI time: ~8min
<img width="1088" height="531" alt="image"
src="https://github.com/user-attachments/assets/0bd0a99a-c69e-491d-91b2-9ddf6622464a"
/>

Both node_modules and sb-build caches are being hit now
<img width="1154" height="722" alt="image"
src="https://github.com/user-attachments/assets/a5addcee-51d2-4f51-9ff6-d02cd25da49a"
/>
2025-08-22 09:34:35 +02:00
nitin
a3322745b5 revert ci changes on ci-breaking-changes (#13682)
<img width="990" height="513" alt="Screenshot 2025-08-06 at 16 27 36"
src="https://github.com/user-attachments/assets/23a99dab-09cf-4978-8c10-5d8a2f7a84c9"
/>
2025-08-06 16:30:00 +05:30
nitin
60257a9701 fix api mismatch on rest metadata (#13680)
## Context

This PR fixes a REST API metadata schema mismatch that occurred after
the webhook and apiKey migration from workspace entities to metadata
level.

## The Issue

After PR #13576 deleted the webhook and apiKey workspace entities, the
metadata OpenAPI specification ended up in a broken state:
- The `/metadata` endpoints still had paths for `/webhooks` and
`/apiKeys`
- But the schema definitions were missing (they were previously pulled
from workspace entities)
- This created dangling references in the OpenAPI document

## The Fix

Added proper schema definitions for webhook and apiKey directly in
`computeMetadataSchemaComponents`:
- `Webhook`, `WebhookForUpdate`, `WebhookForResponse`
- `ApiKey`, `ApiKeyForUpdate`, `ApiKeyForResponse`

## Why the CI is Failing

The CI breaking changes detection is comparing:
- **Main branch**: Has a broken OpenAPI document with endpoints that
reference non-existent schemas
- **This branch**: Has the fixed OpenAPI document with proper schemas

The OpenAPI diff tool can't even parse the main branch's document due to
the missing schema references, hence the error -- tested locally
2025-08-06 16:15:05 +05:30
Félix Malfait
d59604c92d Fix small CI escape issue (#13331)
Fixes #13298
2025-07-22 12:13:53 +02:00
Félix Malfait
d744ef129e Rebase main before applying breaking change CI (#12992)
Avoid false positives when a breaking change was recently merged and the
PR branch isn't up to date
2025-07-02 07:39:53 +02:00
Félix Malfait
21f9db1f41 Improve breaking change CI (#12816)
It used to post a comment if the API schema changed, even if there's no
breaking change. I thought this could be OK as an FYI. But it is not
since now we generate the examples dynamically with Faker the OpenAPI
schema is always different
2025-06-24 14:53:14 +02:00