Testing
Vitest for the API and packages; Playwright E2E for web. No frontend unit suite.
- API — Vitest +
fastify.inject(), blackbox HTTP, in-memory PGlite. Group entry files are*.spec.ts; route tests are imported*.test.ts(see Fastify testing rules). - Packages — Vitest unit tests for packages with a
testscript (core,react,error). CI:packages-test.ymlwhenpackages/ortools/change. - Web — Playwright E2E only. Commands, fixtures, and specs: E2E Testing.
- Product Ready — Fork-and-run checklist for adopters (Product Ready). That page is the release bar for R0. CI green is Workflow, not Quality.
PGLite does not support concurrent writers. API Vitest runs with fileParallelism: false, maxWorkers: 1, and sequence.concurrent: false. Between group entry files, cleanupGroupDatabase() truncates all 15 schema tables and clears the JWT session pool.
Assertion rule
Each inject() test should assert one HTTP status. Assert catalog code only on JSON error responses. Schema validation failures return 400 with BAD_REQUEST. Do not use expect([A, B]).toContain(...) for codes — remove unreachable handler branches instead.
Remote AI tests may call ctx.skip() when the provider is unreachable or out of credits — never return early without skip (soft-pass-as-green is forbidden).
Error contract
HTTP errors use { code, message } from the app catalog (getError / sendCatalogError in apps/api/src/lib/catalogs/mapper.ts). INTERNAL_SERVER_ERROR and TOO_MANY_REQUESTS are normalized to SERVER_ERROR and RATE_LIMIT_EXCEEDED. Global rate-limit 429 bodies include { code, message, retryAfter }.
Group entry map
| Group entry | Routes covered |
|---|---|
account.spec.ts | Account linking, API keys, email change, profile |
auth/session/session.spec.ts | Logout, refresh, validate-tokens, user |
auth/magiclink/magiclink.spec.ts | Magic link request/verify |
auth/oauth/oauth.spec.ts | OAuth authorize/exchange and link-authorize-url (unconfigured 503 contract) |
auth/passkey/passkey.spec.ts | Passkey start/verify/exchange/resolve-user |
auth/web3/web3.spec.ts | EIP-155 and Solana verify + lib/web3/domain unit tests |
health.spec.ts | /health + security headers |
reference/reference.spec.ts | /reference HTML + OpenAPI JSON (including jwtFromServer on magic-link callback) |
ai.spec.ts | AI chat + generate (lib/ai/messages.spec.ts inlines download + message unit tests) |
Vitest include is **/*.spec.ts only. Every *.test.ts must be imported by a group entry *.spec.ts or it never runs. pnpm --filter @repo/api test:unit runs scripts/check-test-imports.mjs first and fails on orphans.
Coverage
pnpm --filter @repo/api test:cov runs the same Vitest suite with v8 coverage (apps/api/coverage/). CI uploads the report from the api-e2e unit job. Playwright is not included. Known gaps (OAuth IdP exchange, skipped AI remote paths) stay uncovered. No coverage floors in CI yet. The unit job does not set ANTHROPIC_API_KEY; remote AI tests are skipped when the key is missing or placeholder.
AI test split
- Contract (local, hard) —
POST /ai/chatand/ai/generatevalidation: 401 unauthenticated, 400 bad payloads, SSRF file-URL matrix (data:only). No provider calls; no soft skip. - Remote (provider-dependent) — Live Anthropic/Open Router/Ollama paths. The remote
describeblocks skip entirely whenANTHROPIC_API_KEYis missing or placeholder (sk-ant-xxx). With a real key,test/utils/ai-remote.tsskips on 402 credits; 502/503/504 fail the suite.
Web chat E2E (chat-assistant.spec.ts) uses Playwright test.skip for provider outages — not SSRF coverage.
Auth helpers
@test/utils/auth-helper.js: getOrCreateSession, getWeb3Session, createAuthenticatedUser, insertTestPasskey, getApiKeyToken, clearSessionPool.
ALLOW_TEST
When ALLOW_TEST=true (non-production), magic-link and change-email flows for *@test.ai store plain tokens in verification.token_plain. E2E reads them via GET /test/verification/last?type=&email= or GET /test/magic-link/last?email= (both require a @test.ai email). Do not enable on production deployments.
Vitest pins ALLOWED_ORIGINS to http://localhost:3000,http://127.0.0.1:3000,https://example.com unconditionally so passkey origin tests are not no-ops under * or inherited CI values.
Known gaps
- OAuth IdP exchange — Unit tests assert unconfigured 503 and link-authorize-url 401/503 only. No live GitHub/Google/Facebook/Twitter calls in CI.
- AI remote — Skips (not passes) on 402 or upstream errors when
ANTHROPIC_API_KEYis missing or placeholder (sk-ant-xxx). Default model:claude-haiku-4-5(aliassonnet→claude-sonnet-4-6). See AI Architecture.