E2E Testing

End-to-end tests run full user flows using Playwright. Two suites: Next E2E (apps/next) tests the web app + auth; Fastify E2E (apps/fastify) tests Scalar UI login. Both use test@test.ai for magic-link when ALLOW_TEST=true.

Root QA: pnpm qa runs the full pipeline (install → checktypes → lint → build → test) and ends with pnpm test:e2e, which runs Fastify E2E then Next E2E via scripts/run-e2e.mjs (both use test:e2e:local to spawn servers). The E2E script kills any processes on ports 3000 and 3001 before starting (unless SKIP_KILL_PORTS=1).

Commands

URL Configuration

test:e2e accepts URL params. Next: --app and --api. Fastify: --api only. Params override environment variables.

# Next (both app and API)
pnpm test:e2e --app=https://my-app.vercel.app --api=https://my-api.vercel.app
pnpm test:e2e -- --app https://localhost:3000 --api https://localhost:3001

# Fastify (API only)
pnpm test:e2e --api=https://my-api.vercel.app
pnpm test:e2e -- --api https://localhost:3001

# Environment (defaults when no params)
PLAYWRIGHT_APP_URL=https://... PLAYWRIGHT_API_URL=https://... pnpm test:e2e

Defaults: Next NEXT_PUBLIC_APP_URL or http://localhost:3000; API NEXT_PUBLIC_API_URL or http://localhost:3001.

Local Development

1. Full suite (root): pnpm test:e2e — runs Fastify E2E then Next E2E (both spawn servers, poll, run, cleanup).
2. Next only: pnpm --filter @repo/next test:e2e:local — builds, spawns Fastify + Next, runs Next E2E, cleanup.
3. Fastify only: pnpm --filter @repo/fastify test:e2e:local — spawns API, runs Scalar login E2E, cleanup.
4. Manual (two terminals): Build with pnpm --filter @repo/next build:e2e, run pnpm --filter @repo/next start:e2e:servers in one terminal and pnpm --filter @repo/next test:e2e in another.
5. External URLs: Use --app and --api (Next) or --api (Fastify) to target already-running servers or Vercel previews.

Test Email (test@test.ai)

Magic-link E2E tests use test@test.ai when ALLOW_TEST=true. The token is stored in the database (verification.token_plain) and read from /test/magic-link/last. No fake outbox; works with Vercel serverless (shared DB across instances).

• Local: test:e2e:local and start:ci set ALLOW_TEST=true.
• Vercel previews: Set ALLOW_TEST=true for the Preview environment in project env vars.

CI

Test workflows run on pull requests (path-filtered):

• API E2E (api-e2e.yml) — Runs when apps/fastify or its deps change. Spawns Fastify locally, runs Playwright.
• Next E2E (next-e2e.yml) — Runs when apps/next or its deps change. Spawns Fastify + Next locally, runs Playwright.
• Packages (packages-test.yml) — Runs when packages or tools change. Unit tests only; excludes app E2E.

Lint and security run on every PR. See GitHub Actions.

Vercel Deployment Protection (Manual Testing)

To run E2E manually against Vercel preview deployments with Deployment Protection enabled:

When Deployment Protection is enabled:

1. Protection Bypass for Automation: In each project (Next, Fastify): Settings → Deployment Protection → Protection Bypass for Automation → enable → generate secret.
2. Use the same bypass secret for both projects.
3. Add VERCEL_AUTOMATION_BYPASS_SECRET to GitHub secrets.
4. Playwright adds x-vercel-protection-bypass and x-vercel-set-bypass-cookie headers automatically when the env var is set.
5. OPTIONS Allowlist (for CORS): In Fastify project, Deployment Protection → OPTIONS Allowlist → add / or /auth so CORS preflight succeeds.

See Vercel Protection Bypass for Automation.

Environment Variables

• Chat E2E (chat-assistant.spec.ts): Requires OPEN_ROUTER_API_KEY. Local: add to apps/fastify/.env.test. CI: set in GitHub secrets; test:e2e:local passes process.env and .env.test to the Fastify server. Currently skipped: getAuthToken returns null for the chat transport in E2E. loginAsTestUserWithTokenInjection and /api/auth/test-set-session (ALLOW_TEST only) exist for session injection when cookie propagation from magic-link verify fails.
• E2E CI: Uses local servers only; no Vercel URLs. For manual testing against previews, pass --app/--api URLs.

ALLOW_TEST

ALLOW_TEST=true enables:

• Fake email for @test.ai (no Resend)
• DB-backed token storage for /test/magic-link/last
• Production guard: server exits if ALLOW_TEST is true in production

Next E2E Specs

Project order

Disabled / skipped specs

chat-assistant skipped (test.describe.skip) until getAuthToken fixed for chat transport.

Troubleshooting

• Port in use: scripts/run-e2e.mjs kills processes on 3000/3001 before E2E. If ports remain in use, run bash scripts/kill-test-servers.sh manually.
• Tests fail with unreachable URLs: Ensure both servers are running, or pass correct --app / --api URLs.
• Token extraction fails: Verify ALLOW_TEST=true and email is test@test.ai.
• Vercel CORS errors: Add / or /auth to OPTIONS Allowlist in Fastify Deployment Protection.
• Protection bypass not working: Ensure VERCEL_AUTOMATION_BYPASS_SECRET matches the secret in both Vercel projects.

Related

• Frontend Testing - E2E-only frontend testing strategy
• GitHub Actions - CI workflow
• Vercel Deployment - Preview deployment and env vars