This page walks through the whole automated test system: which kinds of tests exist, which tools run them, when they run, and how contract and characterization tests differ from ordinary ones. It is written for someone who has never done test automation before.
The higher a test sits, the faster it runs and the more precisely it points at the broken spot. The lower it sits, the more it behaves like a real user, but the slower it is. The six layers complement each other; none replaces another.
Call a function or the server directly, with no browser, so they finish in a fraction of a second. Examples: the server answers 200/301/304/400/404/405/500 correctly, and the notification script builds the right PASS/FAIL message.
Define "the shape the data must have" once (a schema), then check every place where data crosses from one part to another. If one side changes shape without the other knowing, a test fails immediately. Details in section 2.
They don't ask what the app should do. They record what it actually does right now, odd behaviour included, so you notice the moment someone changes it by accident. Details in section 3.
Open a real browser, click, type, and check the result. Split by area of the app: sign-up/login 27, tasks 27, profile 10, and full multi-page journeys 6. Test data uses made-up users (demo, alice).
axe-core scans every page and every state (normal, with an error, mid-edit, dark mode) against WCAG 2.1 levels A and AA. One special test injects a known violation to prove the scanner really catches it, so a pass is never silent.
Not a new kind of test but a selection: 5 E2E tests for the most important paths run first (tag @smoke). They run on every push and pull request.
| Part of the project | What it is for |
|---|---|
app/index.html, app/server.mjs | The Task Board app and a static server: the target of all the tests |
tests/support/pages.ts | Page Objects: "press this button, fill that field" written once, so tests read well and only one place changes when the UI does |
tests/support/storage.ts | Helpers that read and write localStorage directly: seedStorage (inject data, then reload), readJson, readRaw, dumpStorage |
tests/fixtures/reports/ | Three fake reports (pass, fail-nested, empty) so unit tests can exercise the notify script without a real Playwright run |
contracts/schemas.mjs | The single place that defines the data shape for all 3 boundaries, imported by both Playwright and node:test |
Think of a form two departments agree to fill in the same way. If one changes the form without telling the other, the second can't read it. A contract test is the person who checks the form every time it is handed over.
| Name | What the agreement says | Checked by |
|---|---|---|
| C1 localStorage | The app may only write the listed keys, and the data must have the right shape. For example every task must be { id: integer, title: string, priority: low|medium|high, done: boolean }, ids must be unique, the session must name a user that exists, and tasks must belong to an existing user. |
tests/contracts/storage.spec.ts drives the real UI (register, add/edit/delete tasks, change profile, delete account, switch users) and checks storage after each step. |
| C2 Playwright report | results.json must contain stats (expected, unexpected, flaky, skipped as non-negative integers) and suites with titles, because notify.mjs depends on only this part. |
unit/contracts.test.mjs checks the 3 fixtures and the report from the latest real run (if the file exists), plus cases that must be rejected. |
| C3 webhook body | Must have content (Discord) and text (Slack), both non-empty and identical, with no extra fields. |
unit/contracts.test.mjs starts a fake server, runs the real notify.mjs against it, and checks the body it receives against the schema. |
The key point is that the schema lives in one place (contracts/schemas.mjs, written with zod). Browser tests and node tests import the same file, so the two sides can never disagree about "the right shape". The state being checked comes from driving the real UI, not from hand-built data.
It is like photographing a room before moving furniture. The photo doesn't say the room should look this way, but if something goes missing later you know at once. It suits code with no written spec, or old code you are about to change without knowing what depends on it.
| File | Records that |
|---|---|
validation.spec.ts | The limits of every form: username length (3 to 20 characters; letters, digits and underscore only), password minimum of 8, task title up to 80, the order of checks (username before password; current password, then length, then confirmation), usernames ignore case but passwords do not, and every failed login shows the same message. |
resilience.spec.ts | How the app boots from odd storage (broken JSON, wrong types, good and bad tasks mixed, a strange theme) and the redirect rule of every route, including each page's title. |
unit/contracts.test.mjs | The exact text notify.mjs builds from each kind of report, and that a run with zero tests counts as PASS. |
__proto__ and constructor can be registered.tb.users is stored as an array, the app does not repair it and nobody can log in.Taken from package.json, playwright.config.ts and the workflow files actually in use.
Launches a real Chromium, drives it to click and type, and provides assertions, retries and parallel runs. It starts the test site automatically before running.
playwright.config.ts · 2 retries and 2 workers on CI · screenshot, video and trace kept only for failuresThe test runner that ships with Node, nothing to install. Used for tests that need no browser, so results come back fast. Has a built-in coverage mode.
npm run test:unit · npm run test:unit:coverageA library for writing a data schema and checking that real data matches it. It is the common language of all three contracts.
contracts/schemas.mjsScans a page for WCAG 2.1 A/AA violations, such as images without alt text, text with too little contrast, or form fields without labels.
tests/a11y.spec.tsA hand-written static server with nothing to install. Playwright starts it automatically (port 3000, changeable with PORT), or you can point at another site with BASE_URL.
unit/server.test.mjs)list prints results in the terminal. html builds a report you open with npm run report. json writes test-results/results.json, which the notify script reads next.
--reporter yourself there is no results.json and the notification wrongly says FAILCollects raw results into allure-results and builds a report grouped by behaviour and suite, handy for looking back. It needs Java installed when generating the report.
npm run allure:generate · allure:open · allure:serveRuns on ubuntu with Node 22. Two workflows: Smoke on every push to main and every PR, and Weekend E2E on the weekend schedule, which then uploads reports and keeps them for 14 days.
.github/workflows/smoke.yml · weekend-e2e.ymlThe script reads results.json, writes a ✅/❌ summary and POSTs it to a webhook kept secret in GitHub Secrets (WEBHOOK_URL). It runs every time, even when tests fail.
scripts/notify.mjsThis is the full weekend workflow. Smoke uses only steps 1 to 4, with just the 5 smoke tests.
npm ci and npx playwright install --with-deps chromium.playwright-report and test-results as artifacts.notify.mjs reads results.json and posts a summary to Discord or Slack with a link to the run.| Situation | Command |
|---|---|
| A tiny change, want a quick check | npm run test:smoke |
| Changed the notify script or the server | npm run test:unit |
| Changed the shape of stored data | npx playwright test --grep @contract |
| Changed validation rules or error messages | npx playwright test --grep @characterization |
| Changed look, layout or colours | npx playwright test --grep @a11y |
| Before a release | npm run test:unit then npm test |
| See what a command would run, without running it | npx playwright test --list --grep @contract |
Kind tags in use: @smoke @validation @security @persistence @resilience @destructive @a11y @responsive @contract @characterization, plus area tags @auth @tasks @profile @journey. A test can carry several tags.
smoke.yml and weekend-e2e.yml run only Playwright. The 42 tests in unit/ (including contracts C2 and C3) must be run by hand with npm run test:unit. To have them caught automatically, add that step to a workflow.