Test Automation

AI generates, validates, and helps maintain software tests inside developer workflows.

Business impact

• automated testing capacity — DoorDash reported a 50x increase in capacity.
• success rate of generating viable passing tests that increase code or scenario coverage — Uber reported about 20% for Java, 40% for Go, and 80% for Python.
• productivity in manufacturing and unit testing — Hitachi reported a 30% productivity figure for manufacturing and unit testing, though the article notes the measurement was Hitachi-reported.
• efficiency and quality gains — Hitachi also reported approximately 5 billion yen in gains from GenAI applied internally in FY2024.

Data requirements

• ground-truth data (tabular) — DoorDash's framework evaluates outputs against ground-truth data.
• source code (text) — The systems operate on newly written code or software artifacts to generate tests.
• development workflow context (text) — The tools are used inside engineering workflows where code, tests, and release tasks are produced.

AI methods and techniques

• test generation — Used to create software test cases or unit tests from developer workflows.
• validation against ground truth — Used in DoorDash's test-and-evaluation framework to check responses against ground-truth data.
• multi-agent workflow integration — Used in Uber's AutoCover system to generate and validate tests within engineers' development workflows.

AI models and model families

LLM-based multi-agent system, GenAI

Industries

Real-world evidence

7 documented case studies on record.

Companies using this: CNH Industrial, DoorDash, Expedia, Hitachi, Lockheed Martin, Qualcomm, Uber.