QA teams spend more sprint time documenting than most people want to admit. AI agents can shift that balance — but only when they're built the right way.
01 — THE PROBLEM
Test Case Documentation Takes More Sprint Time Than It Should
It happens every sprint. Before a single test runs, before a single bug gets logged, there's a documentation phase — somewhere between thirty minutes and an hour per user story, just writing test cases by hand. Multiply that across fifteen stories in a two-week sprint and you're looking at fifteen to thirty hours of structured writing. That's real time that could go toward actual testing.
The output of all that work is also inconsistent in ways that compound over time. Every engineer writes test cases a little differently. Negative scenarios get deprioritised under deadline pressure. Edge cases — the ones most likely to catch real bugs — are the first to go when time runs short. And when a defect turns up in production, tracing it back to the requirement it was supposed to cover is usually guesswork, because that link was never made.
This isn't a people problem. It's a tooling gap. The cognitive overhead of manual test case writing is real, and it comes at a direct cost to the work QA engineers are actually good at: exploratory testing, edge case reasoning, usability observation, and defect investigation.
"The question isn't whether AI can write test cases. It's whether it can write them well enough to be trusted — and what needs to be in place for that trust to develop."
02 — THE APPROACH
What an AI Agent for Test Creation Actually Needs to Do
The common assumption when teams first consider AI for test case generation is that you point a language model at a user story and it produces useful output. Sometimes it does. More often, the output is generic, misses negative scenarios, skips edge cases, and formats steps inconsistently. The reason is straightforward: a general-purpose model has no knowledge of your team’s conventions, your platform’s requirements, or your organisation’s definition of adequate test coverage.
A useful QA agent is more than a language model call. It needs three things working together: a reliable source of the actual requirement, a way to inject your team's rules into the generation, and a workflow that keeps the engineer in control of what ends up in the platform.
Source of truth: the user story
The agent needs to pull the story directly from the platform — Azure DevOps, Jira, wherever — not from a copy-paste. That's the only way test cases reliably reflect the current state of the requirement. Acceptance criteria come in all shapes: Gherkin, bullet points, plain paragraphs. The agent should handle all three and infer testable conditions even when the story is thin.
Team-owned rules, not hardcoded prompts
One of the more consequential decisions in building this kind of agent is where the QA rules live. Embedding them in code means every update needs an engineering change. Putting them in version-controlled Markdown files that the QA team edits directly turns them into a team artifact — maintained by the people who actually know the quality standards, not whoever happened to write the original prompt.
These rules cover everything: naming conventions, how many test cases to generate based on AC complexity, the ratio of positive to negative to edge scenarios, and what format the platform expects for steps. When something needs updating, you edit a Markdown file — not source code.
Generation that the engineer reviews before it ships
AI-generated test cases shouldn't go straight into the platform. The review step isn't just a safety net — it's how you build trust in the system over time. When engineers can read every generated card, adjust what needs adjusting, and decide exactly what gets pushed, they develop a real sense of where the agent performs well and where it still needs a human call.
03 — THE DESIGN PRINCIPLES
Six Principles for Building a QA Agent Worth Trusting
- Coverage by type must be non-negotiable - The agent needs to generate positive, negative, and edge scenarios on every run — not as an option you can toggle, but as a hard output constraint. Make coverage type optional and it gets skipped the moment a sprint gets tight. A rough split of 40% positive, 35% negative, 25% edge tends to land well for most feature stories.
- Count should be derived from complexity, not set manually - A minimal story with no AC warrants fewer test cases than one with six conditions. The agent should figure out the right count from what it's reading — using the number of AC conditions, edge-triggering language, and the scope implied by the description. Three as a floor and twelve as a ceiling covers most stories you'll encounter.
- Per-step expected results, not just a final outcome - A single expected result at the end of a test case tells you whether it passed. Expected results on every step tell you exactly where it failed. That granularity speeds up defect reporting significantly — the tester doesn't need to re-run the whole scenario just to find the failure point.
- Platform format knowledge must be exact - Azure DevOps and Jira each have their own requirements for how test steps need to be structured. ADO uses a ValidateStep XML format with HTML-encoded content — not clearly documented anywhere, something you only discover by capturing actual network traffic. Xray needs its own GraphQL mutation rather than the standard Jira issue creation API. The agent has to know all of this precisely.
- Traceability should be automatic, not optional - Every test case the agent creates should be linked back to its parent story at creation time. Manually, that link gets skipped constantly. When it's just part of the workflow, the traceability chain from requirement to test case is something you can actually rely on.
- The engineer stays in the loop before anything is created - Trust in AI-generated output builds gradually. A workflow that generates test cases locally, shows them for review and editing, and only pushes them after explicit approval gets to adoption much faster than one that just creates directly — because engineers can see what they're approving.
04 — WHAT CHANGES
The Shift in How QA Time Gets Spent
When the agent is working well, the documentation phase of a sprint — which used to eat the first day or two — shrinks to a review activity. The engineer skims what was generated, tweaks anything that's off, and moves on.
But the bigger shift isn't the time saved — it's the consistency. When test case writing is manual, coverage quality depends on how much time is available, how experienced the engineer is, and how clearly the story was written. With agent-driven generation and team-owned rules, the last story of a sprint gets the same coverage as the first.
There's also a less obvious benefit to running the agent during backlog refinement rather than waiting for sprint start. A story that produces only three vague test cases is a story that needs better AC. The agent's output becomes a useful signal to the product team — and that conversation is much cheaper to have before the sprint than after it.
05 — PRACTICAL GUIDANCE
What Teams Getting This Right Have in Common
They treat the AI rules as a team artifact
The teams that get the most out of AI-driven test generation are the ones that treat the rulebook as a living document the QA team owns and maintains. When a naming convention changes, someone updates the Markdown file. The rules evolve with the team’s standards, and the agent’s output evolves with them.
They review before they trust
The review workflow is not bureaucracy. It is the mechanism by which the team develops calibrated trust in the agent’s output. Teams that skip the review phase either over-trust and push poor test cases, or under-trust and abandon the tool. The teams that persist through the review phase build a habit of editing sparingly and reach a point where the output requires very little adjustment most of the time.
They use the output as a quality signal, not just a time saver
Running the agent during backlog refinement is where things get interesting. A story that generates three thin test cases is one that needs more AC. The agent output starts working as a proxy for story quality — and it doesn't require anyone to carefully read every story to surface those gaps.
They keep the engineer in control
The agent isn't a replacement for QA judgement — it's a first draft. The engineer reviewing the output brings domain knowledge, context, and experience that the model doesn't have. That's where the real QA work lives. The agent handles the transcription. The thinking is still yours.
CONCLUSION
What We Built at Origin
At Origin Digital, we built TC Agent — a tool that reads user stories directly from Azure DevOps or Jira, uses Claude AI to generate structured test cases following our team’s QA rules, and creates them in the target platform fully formatted with steps, expected results, and issue links. The before and after is measurable: stories that previously took thirty to sixty minutes of manual test case writing now take around sixty seconds end to end.
It supports both Azure DevOps and Jira + Xray Cloud from a single interface, uses four Markdown skill files as the AI’s QA rulebook, and keeps the engineer in control through a review-before-push workflow where every generated test case can be read, edited, and selectively approved before anything is created in the platform.
The direction is clear enough: structured AI assistance in test case creation cuts documentation overhead, raises the floor on coverage consistency, and gives QA engineers more time to actually test.
.svg)