Testing
Five suites, one per runtime boundary. Each app owns its own runner and config: there is no top-level test command.
| Suite | Location | Runner | Touches a real DB? |
|---|---|---|---|
| Data engine | data-engine/tests/ | pytest | No: scripted fake DB |
| API unit | apps/api/tests/unit/ | bun test | No |
| API integration | apps/api/tests/integration/ | bun test | Yes: local Postgres via docker-compose.test.yml (never Neon) |
| Web unit | apps/web/tests/unit/ | Vitest | No: MSW mocks the API |
| Web e2e | apps/e2e/tests/smoke/ | Playwright | No: fixture API server |
Test files are never co-located with source. They live under the app’s tests/ directory (or
apps/e2e/tests/), mirroring the path of the file under test: e.g. tests/unit/common/mappers.test.ts
covers src/common/mappers.ts.
Running everything
# data-engine
cd data-engine && pytest -v
# api (integration suite needs the local test DB: docker-compose.test.yml)
cd apps/api && bun run test:db:up # start Postgres + Neon HTTP proxy, apply migrations
cd apps/api && bun test # unit + integration
cd apps/api && bun run test:unit # unit only
cd apps/api && bun run test:integration
# web
cd apps/web && bun run test # vitest run
# e2e (spins up its own servers; E2E_WEB_PORT=4331 if your own `astro dev` holds :4321)
cd apps/e2e && bun run test
Data engine (data-engine/tests/)
pytest, config in pyproject.toml (pythonpath = ["."], testpaths = ["tests"]). Dev deps:
requirements-dev.txt (= requirements.txt + pytest).
- Pure functions are tested directly:
math_utils.py(normalize_minmax,bayesian_win_rate,clamp),finishing_order.py, the model weight invariants (test_weights.py: the committedsrc/models/gp_v4.jsonscores exactly the computed GP features and its importances matchdocs/feature-weights.json), weighted-v4’s pieces (test_conditional_logit.py: the fit recovers known weights;test_gp_features_v4.py, over the in-memory history intests/support/race_history.py;test_gp_model_v4.py: save/load, importances, training set;test_identity.py, againstdocs/identity-cases.json), the backtest scoring (test_backtest.py: metrics, calibration,GridWinRates),rank_by_probability,race_weekend_window,weekend_pipeline(test_weekend_pipeline.py:plan, the chains,due_statuses,repair_plan,prediction_steps),enrichment_coverage(test_enrichment_coverage.py:missing, compound normalisation from either source; the coverage query through the fake DB), and the FastF1 adapter (test_fastf1_session.py: stint, pit-stop and race-control rows over plain records, telemetry input reshaping). - The live ingest adapter runs against recorded responses:
test_openf1_session.pyfeedsopenf1_session(OpenF1 + Jolpica → ingest rows) the Bahrain 2026 responses intests/fixtures/openf1/andtests/fixtures/jolpica/, checked against the official results and what FastF1 ingested. - DB-touching functions are tested through a fake double:
tests/support/fake_db.pyprovidesFakeCursor/FakeConnection, shaped like psycopg2’sRealDictCursor.FakeConnectiontakes a list of result sets and hands one out perwith conn.cursor()block, in call order. This coverscompute_luck_score,same_person_rows,compute_team_circuit_perf,season_drivers, andingest_runner.py’s two shared runners (run_ingest_job,run_qualifying_ingest_job): the latter via monkeypatched per-job callables andopenf1_sessionreaders, no real session data needed. The same double coversdata_quality_audit._audit_race(per-status gate/threshold branching),data_quality_repair.run(issue grouping, repair plan per race, resolve bookkeeping),main.auto_detect_round, (with FastF1’s schedule /load_sessionmocked)sync_scheduleandsync_season,compute_title_odds.run(incl. replaying an earlier round without seeing later results or predictions),auto_runner._retry_missing_race_events(only the missing coverage’s job reruns, a cleanly finished job is settled in the memo, budget-limited telemetry stays pending), and the CLI’s session-job dispatch (test_main_cli.py). - Anything that runs chain steps (
auto_runner,data_quality_repair) is tested by swapping entries of the one job table,weekend_pipeline.JOBS(monkeypatch.setitem), never by patching job modules. conftest.pysets a placeholderDATABASE_URLso importing a job module (which reads it at import time) works with or without a local.env. Tests never open a real connection.upsert.pyand the prediction jobs write viapsycopg2.extras.execute_batch, which renders SQL through a C extension that needs a real connection even just to quote identifiers: so their tests (test_upsert.py,test_compute_sprint_features.py) monkeypatchexecute_batchitself and assert on the params it receives, the same patterntest_ingest_runner.pyalready uses for its ownexecute_batchcall, rather than extendingFakeCursorto fake SQL rendering.test_compute_sprint_features.pyruns the sprint-v3 job against an in-memoryRaceHistorywith a sprint grid: the committed GP weights scoresprint_features, rows are stored the GP’s way with the predicted winner in one transaction, sprints inside the committed fit are skipped and an unknown race fails.test_compute_features.pycovers the weighted-v4 GP job: each feature’s value and contribution land in thefeaturesJSONB, a plain softmax ofraw_weighted_scoreis the model’s win probability, andrunloads three seasons of history before the race and writes one row per driver.
API unit (apps/api/tests/unit/)
bun test, no DB, no network. Mirrors src/ by domain: the shared src/common/ layer (mappers,
standings, cache, identity, feature-manifest) and each module’s
pure *.helpers.ts (circuits/ track outline, track profile, circuit era/stats; drivers/
classification, qualifying gap; history/ helpers; predictions/ accuracy, prediction
history, intel standings; race-events/openf1.mappers; quality). feature-manifest.test.ts checks the
manifest’s labels against docs/feature-weights.json, the same file the Python and web drift tests read. Fixtures are plain objects typed as
typeof <table>.$inferSelect; see mappers.test.ts for the pattern. identity.test.ts reads its
cases from docs/identity-cases.json, the same file data-engine/tests/test_f1db.py checks
normalize_person_name against, so the two languages can’t drift on the F1DB join key. The GP/sprint reads in
common/session-reads.ts are DB-bound, so they are covered by integration tests instead
(integration/session-kind/: the same assertions run once per kind).
Anything pure that a service delegates to (row→DTO mapping, aggregation, normalisation) belongs
here. If a service method is hard to unit-test, that’s usually a sign the pure part should be
extracted into common/ or a *.helpers.ts (this is what the max-lines budget nudges toward).
API integration (apps/api/tests/integration/)
bun test driving the real Hono app in-process via app.request() against a local Postgres
(docker-compose.test.yml: postgres:16-alpine on :5433 behind local-neon-http-proxy on :4445).
Neon Free’s 100 CU-hours/month are shared by every branch, and TDD runs this suite constantly, so the
suite never touches Neon.
apps/api/.env.test(committed, docker-only credentials):bun testloads it over.env. It setsTEST_DATABASE_URLandNEON_FETCH_ENDPOINT, whichsrc/config/localNeonProxy.tsuses to route the Neon HTTP driver to the proxy.bun run test:db:up/test:db:downstart/stop it (tmpfs: data is thrown away).- Guards in
tests/support/db/test-db.ts: refuses a*.neon.techhost (overrideTEST_DB_ALLOW_NEON=1), and refuses to truncate ifTEST_DATABASE_URLequalsDATABASE_URL(overrideTEST_DB_ALLOW_RESET=1). tests/support/app/request.ts:apiRequest(path, init?)callsapp.request()with aBindingsenv pointing at the test DB.tests/support/factories/: one insert helper per table, each takingPartial<$inferInsert>overrides and returning the inserted row. FK chain:season → team/circuit → driver → race → results/predictions.- Lifecycle: each test file calls
truncateAll(db)inbeforeAllandafterAll, seeds its own fixtures (typically under a sentinel year like2097/2099), and asserts on the JSON response.truncateAllis a singleTRUNCATE … RESTART IDENTITY CASCADEover every table. - Covered:
races(incl. practice long runs, circuit outline + circuit telemetry),race-events,race-story,strategy,telemetry,drivers(incl. career, racecraft, teammate battle),teams(incl. pit stops),seasons,search,predictions(incl. title odds),sprint,history, the home hub’s on-this-day query,/api/health(never wakes the DB) and/api/health/db, theedgeCachemiddleware (common/edge-cache.ts, againsttests/support/cache/fake-edge-cache.ts), the per-kind GP/sprint reads (session-kind/), and the route factories (route/route.test.ts: 400/404 answers and Cache-Control derived from the data’s race status or season, never from?year=). Not yet:quality(dev-only).
Web unit (apps/web/tests/unit/)
Vitest, config in vitest.config.ts (include tests/unit/**/*.test.{ts,tsx}, @ alias, setup file).
- Default environment is
node: pure logic runs there:lib/f1/predictionMath.ts(all exports),lib/f1/predictionTerms.ts(served contributions as terms, edge over the field, What-If Lab),lib/f1/teamColors/lib/f1/teamLogos/features/history/f1dbCountries,features/compare/compareStats.ts, the compare stat rows (features/compare/{driver,team}CompareConfig.ts: Championship Points reads the API’schampionshipPoints, never a client-side GP + sprint sum),features/history/historyView.ts,features/races/{raceView,calendar,loadRaceEvents}(race page and calendar view logic),components/data/{chartScale,labelLayout}, the hub/circuits/drivers/teams feature helpers,features/predictions/{buildPredictionPageData,predictionCopy}(API mocked withvi.mock, not MSW). - Page-data loaders are tested through MSW:
features/drivers/loadDriverPageData,features/teams/loadTeamPageData,features/races/loadRacePageData(GP and sprint) andfeatures/history/loadHistoryPageData. Each test stubs the API routes the page hits withapiOk/apiError(tests/support/msw/routes.ts), calls the loader with a URL and a pinnednow, and asserts on the returned view model;recordApiRequests()captures which endpoints were requested, so “only the open tab is fetched” is asserted directly. Career-year resolution is covered for all three fallbacks (requested → current → latest season). - jsdom is opt-in per file via a
// @vitest-environment jsdomdocblock on line 1: used for hook/component tests (useCompareController,useGlobalSearch,GlobalSearch) with Testing Library. tests/support/setup.ts: jest-dom matchers, aResizeObserverstub (cmdk needs it), and the MSW server lifecycle (onUnhandledRequest: 'error', reset between tests).tests/support/msw/:handlers.ts+fixtures.tsmock thesrc/lib/api.tsendpoints athttp://localhost:8787. Client-side fetches in the components under test hit these.routes.tsadds per-test overrides (apiOk,apiError) andrecordApiRequests();setup.tsresets handlers and request listeners after each test.- Not covered:
.astrocomponent rendering.
Web e2e (apps/e2e/)
Playwright, its own package.json and playwright.config.ts (see
docs/adr/0001-apps-layout-scoped-to-js-bun.md for why it isn’t nested under apps/web/).
webServerstarts two processes: the fixture API server (fixtures/server.ts,Bun.serveon:4310, serves{ data, error: null }+ CORS headers for the routes under test) andastro devforapps/webon:4321withPUBLIC_API_URLpointed at the fixture server.astro previewisn’t used (the@astrojs/cloudflareadapter doesn’t support it) so the SSR frontmatter runs under Vite’s dev server instead. Playwright can’t intercept the server-side fetches Astro makes in frontmatter, which is why a real fixture HTTP server is needed rather than request mocking.- The fixture server sends CORS headers because
GlobalSearch/DriverCompareTool/TeamCompareToolfetch client-side: a genuine cross-origin browser request. Every other route is only ever hit server-side. - Every fixture in
fixtures/data.tsis typed withapps/web/src/types, which re-exports the API’scommon/types:bun run typecheck(tsc --noEmit, also a CI step) fails when a fixture drifts from the real response shape, e.g. a field the API added. - Smoke specs (
tests/smoke/) cover the home hub,/prediction,/prediction/sprint/[id],/races/[id](+ sprint, race story, telemetry),/circuits+/circuits/[key],/drivers+/drivers/[id],/teams+/teams/[id], both compare pages,/history(all four tabs, old-URL redirects), the sidebar, branding, and the global search palette: each asserts key content renders against the fixtures. accessibility.spec.tsruns axe (@axe-core/playwright) on ten key pages and fails on anyseriousorcriticalviolation, colour contrast included.mobile.spec.tsloads the main pages at 375 px and fails if any of them scrolls sideways. It also checks that wide charts keep their labels legible in their own scroller and that the scorecard’s season column stays put.reuseExistingServeris on outside CI, so a developer’s ownastro devon:4321would be silently reused: against real data. SetE2E_WEB_PORT/E2E_API_PORT(e.g.4329/4319) to run against fresh fixture-backed servers while your dev servers are up.
CI (.github/workflows/test.yml)
Four jobs, on every push to master and every PR:
| Job | Steps |
|---|---|
data-engine | pip install -r requirements-dev.txt → pytest -v |
api | bun install → test:db:up (docker compose + migrations) → bun run test → bun run typecheck → bun run lint |
web | bun install → bun run build → bun run test → bun run typecheck → bun run lint |
e2e | bun install (web + e2e) → bun run typecheck (fixtures against the API’s types) → playwright install chromium → bun run test |
The api job runs the integration suite against the same docker-compose.test.yml stack: CI never
spends Neon CU-hours, and no TEST_DATABASE_URL secret is needed any more.
Locally, a husky pre-commit hook runs lint-staged (lint-staged.config.mjs): staged files under
apps/api/ or apps/web/ trigger that app’s bun run typecheck and bun run lint (including the
max-lines budget). Tests are not part of the hook.
Adding a test
- Pure helper (either app) →
tests/unit/, mirror the source path. Prefer extracting the pure part over testing a DB-bound method. - New API endpoint / query → an integration file under
tests/integration/<domain>/; add a factory if a table has none;truncateAllinbeforeAll/afterAll. - New page logic (tabs, fallbacks, shaping) → put it in a page-data loader under
features/<area>/and test the loader with MSW (nodeenvironment), mirroringtests/unit/features/<area>/. - New interactive web island → a
jsdomVitest file; add MSW handlers for any endpoint it calls; consider an e2e smoke spec + a fixture route if it’s a whole page. - New ETL scoring function → pytest; pure if possible, otherwise script
FakeConnectionwith one result set per query the function issues.