Release Testing & Pre-Release QA

The problem

Before every release, someone runs through the critical flows by hand: sign up, log in, the main create/edit/delete loop, checkout, settings. It's slow, it's boring, it's error-prone, and it's the step that gets compressed when the release is late — which is exactly when regressions slip through.

Why the usual approach falls short

• Manual pre-release QA is a person-shaped bottleneck. It doesn't fit a same-day hotfix or a team shipping daily.
• A scripted smoke suite helps but decays. It covers a fixed list, and that list stops matching the product.
• Coverage silently narrows under pressure. "Check the important bits" becomes "check the bits I have time for."

How BlackBat AI fits

Point BlackBat AI at the release candidate on staging and it runs the pre-release sweep for you — exploring every reachable flow and checking the plain-English journeys you've defined as must-pass. You get back a clear result: which explicit checks passed, and what broken behaviour the exploration turned up, each with a screenshot and reproduction steps you can hand straight to a developer.

The plain-English checks are where you encode "this must work to ship": "a new user can sign up, verify their email, and complete onboarding." How to write those is its own short guide.

What it looks like in practice

1. Deploy the release candidate to staging.
2. Run BlackBat AI — the autonomous sweep plus your must-pass checks.
3. Review findings; fix or accept; ship.

A release checklist that runs itself

A pre-release run replaces the manual click-through with something repeatable:

1. Deploy the release candidate to staging.
2. Run a full crawl for the broad sweep, since a release is the time to be thorough.
3. Check the must-pass flows. Plain-English suites give you pass/fail per step for sign-up, the core loop, checkout and anything else the release must not break.
4. Review findings in the changed areas first, then the rest.
5. Ship, or fix and re-run. A diff run after a fix is cheaper than a second full crawl.

Making it a required step

If you want the result to gate a release instead of relying on someone to look, trigger the run from your release pipeline with an API key (CI/CD triggering is available on paid plans) and wait on the status: PASSED continues and FAILED, TIMEOUT or CANCELLED stops. The release process guide and the CI/CD walkthrough cover where it fits. Run notifications can post the outcome to Slack or a webhook so the whole team sees it.

Where it fits

This is the layer between your CI test run and production. Unit and integration tests run earlier and faster; the pre-release BlackBat AI run is the "does the whole thing still work for a user" check that used to land on a person. See the end-to-end testing guide for how it sits in the wider strategy.