Basilic
Testing

Testing

Vitest for the API and packages; Playwright E2E for web. No frontend unit suite.

  • API — Vitest + fastify.inject(), blackbox HTTP, in-memory PGlite. Group entry files are *.spec.ts; route tests are imported *.test.ts (see Fastify testing rules).
  • Packages — Vitest unit tests for packages with a test script (core, react, error). CI: packages-test.yml when packages/ or tools/ change.
  • Web — Playwright E2E only. Commands, fixtures, and specs: E2E Testing.
  • Product Ready — Fork-and-run checklist for adopters (Product Ready). That page is the release bar for R0. CI green is Workflow, not Quality.

PGLite does not support concurrent writers. API Vitest runs with fileParallelism: false, maxWorkers: 1, and sequence.concurrent: false. Between group entry files, cleanupGroupDatabase() truncates all 15 schema tables and clears the JWT session pool.

Assertion rule

Each inject() test should assert one HTTP status. Assert catalog code only on JSON error responses. Schema validation failures return 400 with BAD_REQUEST. Do not use expect([A, B]).toContain(...) for codes — remove unreachable handler branches instead.

Remote AI tests may call ctx.skip() when the provider is unreachable or out of credits — never return early without skip (soft-pass-as-green is forbidden).

Error contract

HTTP errors use { code, message } from the app catalog (getError / sendCatalogError in apps/api/src/lib/catalogs/mapper.ts). INTERNAL_SERVER_ERROR and TOO_MANY_REQUESTS are normalized to SERVER_ERROR and RATE_LIMIT_EXCEEDED. Global rate-limit 429 bodies include { code, message, retryAfter }.

Group entry map

Group entryRoutes covered
account.spec.tsAccount linking, API keys, email change, profile
auth/session/session.spec.tsLogout, refresh, validate-tokens, user
auth/magiclink/magiclink.spec.tsMagic link request/verify
auth/oauth/oauth.spec.tsOAuth authorize/exchange and link-authorize-url (unconfigured 503 contract)
auth/passkey/passkey.spec.tsPasskey start/verify/exchange/resolve-user
auth/web3/web3.spec.tsEIP-155 and Solana verify + lib/web3/domain unit tests
health.spec.ts/health + security headers
reference/reference.spec.ts/reference HTML + OpenAPI JSON (including jwtFromServer on magic-link callback)
ai.spec.tsAI chat + generate (lib/ai/messages.spec.ts inlines download + message unit tests)

Vitest include is **/*.spec.ts only. Every *.test.ts must be imported by a group entry *.spec.ts or it never runs. pnpm --filter @repo/api test:unit runs scripts/check-test-imports.mjs first and fails on orphans.

Coverage

pnpm --filter @repo/api test:cov runs the same Vitest suite with v8 coverage (apps/api/coverage/). CI uploads the report from the api-e2e unit job. Playwright is not included. Known gaps (OAuth IdP exchange, skipped AI remote paths) stay uncovered. No coverage floors in CI yet. The unit job does not set ANTHROPIC_API_KEY; remote AI tests are skipped when the key is missing or placeholder.

AI test split

  • Contract (local, hard)POST /ai/chat and /ai/generate validation: 401 unauthenticated, 400 bad payloads, SSRF file-URL matrix (data: only). No provider calls; no soft skip.
  • Remote (provider-dependent) — Live Anthropic/Open Router/Ollama paths. The remote describe blocks skip entirely when ANTHROPIC_API_KEY is missing or placeholder (sk-ant-xxx). With a real key, test/utils/ai-remote.ts skips on 402 credits; 502/503/504 fail the suite.

Web chat E2E (chat-assistant.spec.ts) uses Playwright test.skip for provider outages — not SSRF coverage.

Auth helpers

@test/utils/auth-helper.js: getOrCreateSession, getWeb3Session, createAuthenticatedUser, insertTestPasskey, getApiKeyToken, clearSessionPool.

ALLOW_TEST

When ALLOW_TEST=true (non-production), magic-link and change-email flows for *@test.ai store plain tokens in verification.token_plain. E2E reads them via GET /test/verification/last?type=&email= or GET /test/magic-link/last?email= (both require a @test.ai email). Do not enable on production deployments.

Vitest pins ALLOWED_ORIGINS to http://localhost:3000,http://127.0.0.1:3000,https://example.com unconditionally so passkey origin tests are not no-ops under * or inherited CI values.

Known gaps

  • OAuth IdP exchange — Unit tests assert unconfigured 503 and link-authorize-url 401/503 only. No live GitHub/Google/Facebook/Twitter calls in CI.
  • AI remote — Skips (not passes) on 402 or upstream errors when ANTHROPIC_API_KEY is missing or placeholder (sk-ant-xxx). Default model: claude-haiku-4-5 (alias sonnetclaude-sonnet-4-6). See AI Architecture.

On this page