Write good checks

If a scenario is the promise, its checks are the proof. A check is one thing that must be true for the promise to be believed. Checks are the floor: what the AI agent verifies first, what a human tester covers before exploring, and exactly what gets automated into a script, no more and no less.

That makes checks the point where quality becomes a decision. Vague checks produce arguments about whether something passed. Precise checks produce results you can act on, and automation you can trust.

Why it matters

• Clear pass or fail. Two people should reach the same result from the same check.
• Automation scope. What you write is what a script covers. A vague check becomes a vague script.
• Cost. How each check is verified decides how much of your testing needs a person. See “Choose how it is verified” below.
• Risk coverage. The mix of check types shows whether you test failure as well as success.

What good looks like

• One outcome, as one short clause that can be seen to be true or false.
• No steps, clicks or page elements. Say what must be the case, not how to get there. Steps break when the design changes. The outcome does not.
• Specific enough to settle. “Checkout works” cannot be settled. “Shows an order confirmation after a valid card is accepted” can.
• Independent. It passes or fails by itself.

Cover the failure cases

For each scenario, write the stated case and at least one way it could fail: its opposite, a boundary, a role that must be denied, or a security angle. A scenario that only has happy-path checks has not been tested against failure. If a scenario honestly has one non-negotiable case, one check is fine. Do not pad.

Example: one promise, six checks

Scenario: Shoppers should always be able to complete checkout with a valid card.

Check	Type	Verified via
Accepts a valid Visa card and shows an order confirmation.	happy_path	Browser
Rejects an expired card with a clear message.	negative	Browser
Never charges the card twice when the pay button is double-clicked.	edge_case	API
Never shows the full card number after payment.	security	Browser
Lets every payment field be reached and filled using only the keyboard.	accessibility	Browser
Shows the order confirmation within five seconds of paying.	performance	Browser

Common mistakes

Instead of this	Write this	The problem
Click #pay-now and check the toast.	Shows an order confirmation after paying.	A step and a page element. It breaks on a redesign.
Checkout works.	Accepts a valid Visa card and shows an order confirmation.	Too vague to settle.
Accepts Visa and Mastercard and rejects expired cards.	Three checks.	Three outcomes in one. A single failure hides the others.
Test the payment form.	Rejects a card number with a missing digit with a clear message.	An instruction, not an outcome.

Choose the type

Each check has one tag. It says what kind of risk it covers.

Tag	Covers	Example
happy_path	The normal, successful flow.	Accepts a valid Visa card.
negative	Bad input and expected refusals.	Rejects an expired card.
edge_case	Unusual but possible situations.	A double-click does not charge twice.
security	Access, privacy and abuse.	A shopper cannot open another’s order.
accessibility	Keyboard, screen reader and contrast.	Every field shows a visible focus.
performance	Speed and responsiveness.	Search results appear within two seconds.

Use the mix as a quick read of balance: filter a scenario’s checks by tag and see whether failure, security and accessibility are represented.

Choose how it is verified

Verified via decides who confirms the check, and therefore what it costs to run again.

Choice	Use it when	Typical cost to repeat
Browser	The outcome is seen by using the product.	Low. The agent and scripts can take it.
API	A direct call to the environment settles it, with no page.	Lowest. Fast and automatable.
Database	A database assertion settles it.	Medium. Qarmiq does not run it. A connector or a person does.
Human verification	Only a person can judge it, such as look and feel.	High. Never offered to the agent or a script.
Connector	A connected tool decides, such as an AI evaluation suite.	Low once connected. See Connectors.

Prefer the most automatable choice that is honest. Reserve Human verification for what truly needs judgement. The share of checks that depend on a person is visible on the automation view of Insights, and it is a good number to watch.

Add context where it helps

Check context holds what the check cannot say: what makes it pass, the data to use, and why a boundary sits where it does. Example, for “Rejects an expired card with a clear message”:

Expired means the expiry month is earlier than the current month. The message should name the expiry date. Use the test card with last month’s date.

Keep it to facts and data. No steps. No credentials.

Keep checks honest over time

• When behaviour changes on purpose, change the check’s wording first. A run keeps the wording it tested, so history stays truthful.
• When a defect slips through, add the check that would have caught it. This is the cheapest way to make testing compound.
• When the AI agent finds something beyond your checks that is worth testing every time, promote it to a check. See Autonomous test results.
• Remove checks that no longer matter. A long list nobody trusts is worse than a short one everybody does.

Questions to ask your team

• What share of our checks are verified by a person, and why?
• For our critical scenarios, do we test how they fail, or only that they work?
• When the last serious defect escaped, did we add a check for it?
• If we redesigned the product tomorrow, how many checks would need rewriting? (The answer should be close to none.)