New benchmark study confirms Diffblue's advantages over LLM coding assistants realized through its reinforcement learning-powered agentic capabilities
Diffblue today announced the release of the next generation of its flagship product, Diffblue Cover, to address the unmet need for automated, high quality unit test generation at scale. Focused on empowering software development teams to systematically break through typical test coverage barriers, the next generation of the company's signature offering includes important new capabilities that help enterprises achieve critical application modernization, compliance, and productivity goals. This bold claim is supported by the release of a new benchmark study that shows how Diffblue's reinforcement learning approach to agentic unit test code generation is 20x more productive than LLM-based coding assistants such as Claude Code and GitHub Copilot.
Three Capabilities Transform Enterprise Test Coverage
The next generation of Diffblue Cover introduces three new capabilities, each of which offers quick and easy ways to improve test coverage and quality: Test Asset Insights, LLM-Augmented Intelligence, and Guided Coverage Improvement. Collectively, they extend Diffblue's market-leading capabilities far beyond what LLM-based coding assistants can deliver.
Test Asset Insights enables Diffblue Cover to analyze pre-existing unit test coverage and extract intelligence that it leverages to write additional tests. The agent learns from existing patterns, reuses established helper methods and fixtures, and builds upon domain-specific test infrastructure to generate high quality, idiomatic test coverage. Tests are clearly annotated to distinguish user-written, Diffblue-contributed, and Diffblue-managed tests, empowering hybrid workflows where the Diffblue agent autonomously maintains coverage as production code evolves.
LLM-Augmented Intelligence fuses the power of customer-approved large language models (LLMs) with Diffblue's reinforcement learning-based test generation to deliver better coverage and quality than any single generative AI technique can deliver in isolation. This eliminates months of security reviews and compliance assessments needed for the introduction of new models and immediately unlocks a new level of performance. By tapping into the capabilities of in situ LLMs, Diffblue Cover is able to generate even higher quality test coverage for complex business logic, while maintaining the product's signature promise: deterministically generated, high quality tests that always compile and capture the behavior of the application. A soon-to-be-released MCP server provides an optional standardized approach to connect to LLMs transparently and automate agentic workflows.
Guided Coverage Improvement dramatically speeds up the rate at which Diffblue Cover is used to improve coverage levels beyond the agent's first pass over a code base. It accelerates the resolution of coverage-blocking issues by automatically identifying and prioritizing the underlying problems and generating an actionable turnkey coverage improvement plan. Internal testing on several repositories has shown that this new capability can, in just one hour, raise coverage by 50% beyond out-of-box coverage.
"Together, these three new capabilities represent a fundamental shift in how enterprises can deal with under-tested applications," said Peter Schrammel, CTO and Co-founder of Diffblue. "We're not just generating more tests faster; we're orchestrating a complete solution that understands existing test infrastructure, leverages enterprise-approved AI models, and provides intelligent guidance to systematically achieve coverage goals faster and more efficiently than any alternative on the market."
20x Productivity Advantage Validated
Diffblue recently released a new benchmark study comparing the unit test generation performance of Diffblue Cover against three LLM-based code assistants: GitHub Copilot, Claude Code, and Qodo. Using three open source Java repositories, an experienced developer measured the rate at which the different tools could generate test coverage, the quality of the coverage, and the overall developer experience.
The study revealed profound differences between the different tools. Even when running in their so-called 'agent mode', all of the LLM-based code assistants required on-going developer oversight in the form of prompt generation, results inspection, re-prompting, and manual output retrofitting. Consequently, they could not run unattended over large code bases for long stretches, causing them to be ill-suited for test generation at scale. Conversely, the Diffblue Cover agent, activated with a single command, can be left unattended to generate coverage for hours on large code repositories.
Because of the fundamentally different levels of developer oversight needed to generate test coverage using a LLM-based code assistant, the study concludes that Diffblue's autonomous agent delivers 20 times more code coverage. Coupled with Diffblue's fixed price licensing model, Diffblue Cover provides cost certainty at any scale, a feature notably lacking given the unpredictable token consumption-based variable costs inherent in LLMs.
The next generation of Diffblue Cover is available immediately for existing customers and to new prospects through a self-service trial or a supported proof-of-value evaluation.
About Diffblue
Diffblue is the leading provider of agentic unit test generation solutions for Java. The company's flagship product, Diffblue Cover, uses reinforcement learning to automatically generate and maintain comprehensive test suites that enable organizations to accelerate development and modernize applications. Diffblue serves innovative development teams in financial services, insurance, technology, and government sectors. Founded by award-winning researchers from the University of Oxford, Diffblue is backed by leading investors including Albion, Parkwalk, IP Group, and Citi.
View source version on businesswire.com: https://www.businesswire.com/news/home/20251104720918/en/
Contacts:
Media Contact:
Ömer Tumer
Head of Marketing
press@diffblue.com