Storybook on TestingBot

Screenshot testing and real device testing for Storybook components, running on TestingBot.

What this shows

Storybook builds UI components in isolation, and every state of a component is a story. The useful part for testing is that every story is a plain URL:

https://your-storybook.example.com/iframe.html?id=components-button--primary&viewMode=story

Storybook also publishes an index of everything it knows about at /index.json. Put those two facts together and a component library turns into a list of URLs you can open on any browser or device in the TestingBot cloud. This repo does exactly that, in three tiers:

1. Playwright Test takes a screenshot of every story on two Chrome versions and two Safari versions in the cloud, and compares it against a golden image.
2. The Storybook Test Runner executes the stories, including their play functions, on a remote browser.
3. A WebDriver script opens every story on a real iPhone or Android phone and saves screenshots.

The components in src/components/ are deliberately small: a Button, a Badge, a Card and a SignupForm with play functions.

Quick start

git clone https://github.com/testingbot/storybook-testingbot-example.git
cd storybook-testingbot-example
npm ci

Get a key and secret from your TestingBot account, and put them in .env or export them:

cp .env.example .env
# or
export TB_KEY=your-key
export TB_SECRET=your-secret

Now the important part. The browsers that run your tests live in the TestingBot cloud, so they need a route to your Storybook. http://localhost:6006 means nothing to them: it resolves inside the cloud VM, where nothing is listening, and every test fails to load the story. Pick one of these two:

Option A, a published Storybook. Nothing else to set up:

export STORYBOOK_URL=https://your-storybook.example.com
npm run test:grid

Option B, a local Storybook behind a tunnel. Download TestingBot Tunnel, which needs Java 11 or newer, then use three terminals:

# terminal 1
npm run build-storybook
npm run serve-storybook

# terminal 2
java -jar testingbot-tunnel.jar $TB_KEY $TB_SECRET -i myStorybookTunnel

# terminal 3
export STORYBOOK_URL=http://localhost:6006
export TB_TUNNEL_ID=myStorybookTunnel
npm run test:grid

TB_TUNNEL_ID is the part people forget. testingbot.config.mjs reads it and adds it to the capabilities as tunnelIdentifier. Start the tunnel but leave TB_TUNNEL_ID unset and the sessions still launch, they just have no route back to your machine, and you get ERR_CONNECTION_REFUSED on every page.goto.

Why port 6006 needs one extra capability

A running tunnel is not quite enough, and this catches people out. The tunnel proxies these ports into the cloud VM without being asked:

80, 443, 3000, 3001, 3030, 3400, 8080

Storybook's default port, 6006, is not one of them. Without help you get a healthy tunnel, a session that starts fine, and net::ERR_TIMED_OUT on every story, which looks exactly like a broken tunnel.

The fix is the localHttpPorts capability, and testingbot.config.mjs handles it for you: it reads the port out of STORYBOOK_URL and, when that port is not proxied by default, adds localHttpPorts: [6006] inside tb:options. Serve Storybook on a different port and it follows along. Serve it on 3000 and the capability is omitted, because it is not needed.

This applies to a tunnel only. A published Storybook is on 80 or 443, so there is nothing to configure.

If you would rather not set any of this up locally, the GitHub Actions workflow does the whole sequence for you.

There is no npx playwright install step anywhere in this repo. The browsers run in the TestingBot cloud, so playwright-core is all your machine or your CI image needs. That is a meaningful saving in CI, where downloading browser binaries is usually the slowest part of the job.

How it works

tests/storybook.spec.mjs is the whole screenshot suite:

1. Fetch ${STORYBOOK_URL}/index.json at collection time.
2. Filter the entries down to type === 'story'.
3. Generate one Playwright test per story, which opens the story URL, waits for #storybook-root and for document.fonts.ready, then calls toHaveScreenshot.

Adding a story to your component library adds a test. There is no list to keep in sync.

testingbot.config.mjs builds the connect endpoint. Every capability object you hand it is merged with your credentials and URL encoded into a wss://cloud.testingbot.com/playwright endpoint. One endpoint is one session in the cloud, visible in your TestingBot dashboard with video, logs and screenshots.

A few settings in playwright.config.mjs are worth knowing about:

• maxDiffPixelRatio: 0.02. Remote browsers render text slightly differently than your laptop does. A small tolerance keeps the suite stable without hiding real changes.
• snapshotPathTemplate stores baselines per project. Chrome on Windows and Safari on macOS never produce identical pixels, so they need separate golden images.
• workers: 2 matches a trial plan. Raise it to the parallel session count on your plan.

Baselines

Generate golden images on the grid, not on your laptop, otherwise the first real run fails on rendering differences that have nothing to do with your components:

npm run test:grid:update

Commit the results in tests/__screenshots__/. From then on, npm run test:grid compares against them and fails when a component's rendering changes.

You can also generate them from CI without setting up a tunnel locally. See regenerating the baselines below.

Choosing browsers and versions

Each project in playwright.config.mjs is one browser, version and platform. The default grid is deliberately narrow: Chrome latest and latest-1 on Windows 11, Safari latest and latest-1 on macOS Sequoia. Two versions of each engine catches the regressions that actually happen; a third engine mostly buys you a third set of baselines to maintain. Add to the grid array to widen it.

The full list of what you can ask for is in the browsers and versions documentation.

{
  name: 'firefox@latest:LINUX',
  use: {
    connectOptions: {
      wsEndpoint: getConnectWsEndpoint({
        browserName: 'firefox',
        browserVersion: 'latest',
        platform: 'LINUX',
      }),
    },
  },
}

Pin exact versions when you want to catch regressions that only show up on a version your users are still running.

Storybook Test Runner variant

If you already use @storybook/test-runner to execute play functions, you can point it at TestingBot as well:

npm run test:test-runner

The configuration lives in test-runner-jest.config.js, generated with npx test-storybook --eject and then given a connectOptions.wsEndpoint. The SignupForm stories carry play functions, and those run inside the remote browser, so this tier tests interaction rather than only rendering.

Two caveats:

• @storybook/test-runner is built on jest-playwright-preset, which is deprecated. Its repository was archived in September 2025, the preset is now vendored inside the test runner, and the project is moving towards @playwright/test. This tier works today, and it is pinned to the version it was verified against, but the Playwright Test tier above is the better long-term choice.
• The remote connect settings are internals of that vendored preset. See test-runner/README.md for what changed and what to re-check after an upgrade.

Storybook's Vitest addon is a separate thing and is not used here. It is a good local workflow, but its orchestrator serves tests from localhost, so it does not drive a remote grid. Use the Vitest addon locally, and use this repo for cross browser and real device coverage.

Real device screenshots

Emulated viewports show you a small screen. They do not show you how Mobile Safari or Chrome on Android actually renders your component.

STORYBOOK_URL=https://your-storybook.example.com npm run test:real-ios
STORYBOOK_URL=https://your-storybook.example.com npm run test:real-android

Screenshots land in ./screenshots/ios/ and ./screenshots/android/.

Playwright cannot drive browsers on real iOS devices, so real-devices/storybook-real-device.mjs uses a WebDriver session through Appium with WebdriverIO. Playwright can drive Chrome on real Android devices, and playwright.config.mjs has an opt in project for that:

REAL_DEVICE=1 npm run test:grid

Device names and OS versions come from the device list. Pin both, since the same model is offered on several OS versions.

STORYBOOK_URL has to be reachable from the device. Publishing a static build is much the easier option for this tier, because real devices cannot resolve the literal hostname localhost. To use a tunnel you need a real hostname: add a line like 127.0.0.1 storybook.local to /etc/hosts on the machine running the tunnel, then set STORYBOOK_URL=http://storybook.local:6006. The tunnel FAQ covers this.

Local Storybook and TestingBot Tunnel

Covered in option B of the quick start. The same TB_TUNNEL_ID variable works for the real device script, which builds its capabilities the same way.

The tunnel is a good default even when you could publish, because it keeps unreleased components off the public internet.

GitHub Actions

.github/workflows/testingbot.yml runs the whole thing on every push to main, and on demand. No local setup, no tunnel to babysit:

1. Build Storybook.
2. Serve storybook-static on the runner at localhost:6006.
3. Start the tunnel with the TestingBot Tunnel action. It runs the tunnel container with --network=host, so localhost:6006 on the runner is exactly what the cloud browsers resolve, and the action blocks until the tunnel reports ready.
4. Run the screenshot suite against the grid.
5. Upload the Playwright report as an artifact.

The one thing to set up is TB_KEY and TB_SECRET under Settings, Secrets and variables, Actions. Use a dedicated account rather than a personal key.

The job skips every grid step when those secrets are empty, so forks do not fail red. A concurrency group cancels an in progress run when you push again, so two runs of the same branch never compete for the parallel sessions on your plan.

Regenerating the baselines

Golden images have to come from the grid. If you have no tunnel set up locally, let CI produce them:

1. Actions, "Storybook screenshots on TestingBot", "Run workflow".
2. Tick "Regenerate the golden screenshots on the grid".
3. When it finishes, download the screenshot-baselines artifact.
4. Unpack it into tests/__screenshots__/ and commit.

That run is green: with --update-snapshots, Playwright writes the missing images and passes. It is the ordinary push runs that fail while tests/__screenshots__/ is empty, because a missing baseline is a failed test. So expect the first push after cloning this repo to be red until you have committed baselines.

Do the same whenever you add a story or change a component on purpose.

Links

• Storybook testing with Playwright on TestingBot
• Playwright on TestingBot
• Browsers and versions
• Real device list
• TestingBot Tunnel

License

MIT. See LICENSE.