03 · Test

QA clicks it. CI runs it.

A request tests an API operation. A flow tests what users actually do, like booking an order.

Publish your flows for QA to run with one click, then run the same checks in CI on every change.

Assert what matters

By default, a request passes on any 2xx response. Add assert checks when you need more, and the response is judged entirely by them.

Use the familiar uvu/assert helpers. Every check runs, and every result tells you what was expected and what actually came back.

export default FindPetsByStatus.request({
  name: 'Find pets by status',
  tags: ['smoke', 'monitor'],
  query: { status: '{{status}}' },
  init: { status: 'available' },
  // keep the pets the response lists: a flow can pick one of them
  capture: { pets: (res) => (res.status === 200 ? res.data : []) },
  assert: (res, { assert }) => {
    assert.is(res.status, 200, 'ok')
    if (res.status === 200) assert.ok(res.data.length > 0, 'returns at least one pet')
  },
})

Let QA change the inputs

A flow has inputs, like a GitHub workflow's inputs or Storybook's controls. QA changes them in a form before pressing Run, and CI takes the defaults or passes --input name=value.

An input can start from generated data. QA gets a realistic name without typing one, can press Generate for another, or press Run with defaults to skip every question.

export default sequence({
  id: 'new-pet',
  title: 'Register a pet',
  description: 'Name a pet, add it to the store and fetch it back.',
  tags: ['petstore', 'smoke'],
  inputs: {
    // a different name every run; QA can edit it or press Generate for another
    petName: c.text({
      default: () => faker.person.firstName(),
      description: 'What the new pet is called.',
    }),
  },
})
  .step('Add the pet', addPet) // keeps the id the store gave it
  .step('Fetch it back', (ctx) =>
    getPet.with({
      vars: { petId: ctx.captured.addedPetId },
      assert: (res, { assert, vars }) => {
        assert.is(res.status, 200, 'found')
        if (res.status === 200) assert.is(res.data.id, Number(vars.petId), 'the pet that was added')
      },
    }),
  )

Give QA a portal

specbook build --minimal publishes only the operations your requests and flows use. QA opens the portal, picks a request or a flow, chooses an environment and presses Run.

Open the QA portal ↗

See exactly what failed

When the API answers with something your checks did not expect, the run shows which check failed, what was expected, what came back, the line of your file and the request that led to it. Here an order has changed under a check that expected it to be placed.

specbook test --env demo

$ specbook test --env demo

A flow that fails once and passes on a retry is not hidden. With --retries 1 it passes, is marked flaky in the summary and exits 0.

specbook test --env demo --retries 1

$ specbook test --env demo --retries 1

Run it in CI too

Add one workflow and CI runs your checks on every change.

# .github/workflows/test.yml
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 24 }
      - run: npm ci
      - run: npx specbook check
      - run: npx specbook test --env staging --reporter list --reporter junit:reports/junit.xml
        env:
          PETSTORE_API_KEY: ${{ secrets.PETSTORE_API_KEY }}
      - uses: actions/upload-artifact@v4
        if: always()
        with: { name: reports, path: reports }

Secrets are environment variables named in the config, so your CI's secret store is all it needs. On GitHub Actions, failures are annotated on the change and summarised on the run. The exit code is 0 when everything passed, 1 when a flow failed or errored, 2 when the run could not start and 3 for a usage error.