mirror of
https://github.com/twentyhq/twenty.git
synced 2026-06-12 01:46:39 -04:00
d75685b8dc
## TL;DR
The metadata API silently drops relation rows whenever a batched
relation query exceeds the requested page size. A dev-seeded workspace
already has **558 fieldMetadata rows across 31 objects**, so
`objects(paging:{first:50}) { fields(paging:{first:500}) }` executes:
```sql
SELECT DISTINCT ... FROM core."fieldMetadata" fields
WHERE workspaceId = $1 AND objectMetadataId IN (...31 ids...)
LIMIT 501 OFFSET 0 -- no ORDER BY
```
…and returns exactly **501 of 558** fields — ~57 rows dropped, and
*which object loses which field is scan-order-dependent*. This is what
made `example-app-postcard` CI flap with "PostCard object missing field
X" (different X per run).
## Root cause
`@ptc-org/nestjs-query-typeorm`'s `batchQueryRelations` (the DataLoader
batch path behind every `@CursorConnection`) applies the **per-parent**
page size as a **single global LIMIT** on the batched query, then groups
rows per parent in memory. Any batch whose combined relation rows exceed
`first + 1` truncates arbitrary parents. This affects production
metadata reads, not just CI — any workspace with enough fields/objects
loses rows in `objects.fields`-style connections.
## Fix
Yarn patch on `@ptc-org/nestjs-query-typeorm@9.4.0` (same vehicle as the
existing `nestjs-query-graphql` patch):
- `RelationQueryBuilder.batchSelect`: only apply LIMIT/OFFSET when the
batch has a **single parent**; multi-parent batches stay **bounded**
with `parents × (offset + limit)` — the upper bound a correct per-parent
pager can ever need, so it cannot wrongly truncate while still guarding
against unbounded fetches on high-cardinality relations;
- `batchQueryRelations`: enforce paging **per parent** by slicing after
`mapRelations` (preserves the `first + 1` hasNextPage probe semantics).
## Verification
- On a frozen repro DB (postcard installed, 558 fields): unpatched
returns 501 fields with `postCard` missing `deliveredAt`; patched
returns **558/558** with the full `postCard` field set. Reproduced
identically on typeorm 0.3.20 and 0.3.26 — pre-existing bug, **not** a
typeorm regression (this unblocks the typeorm upgrade that was reverted
from #21448).
- Postcard install/uninstall stress loop: unpatched fails within 1–2
iterations; patched **12/12 green**.
- `npx nx typecheck twenty-server` clean, full `twenty-server` unit
suite green (5651 passed).
## Related
#21435 chases the **same CI symptom** (postcard randomly missing a
freshly synced field) at a different layer — a workspace-cache write
racing invalidation. The two are complementary: the repro behind this PR
survives a **cold server restart + `redis-cli FLUSHALL`** with all rows
intact in Postgres, which no cache race can explain — the truncation
happens on the DB read itself (`LIMIT 501` over 558 matching rows,
captured via `log_statement=all`). Both fixes are likely needed for the
postcard job to be fully reliable.
## Notes
Worth upstreaming to `@ptc-org/nestjs-query` eventually; the proper
upstream fix is per-parent windowed pagination (`ROW_NUMBER() OVER
(PARTITION BY parentId)`), but the in-memory per-parent slice is correct
and proportionate at metadata-API scale.