ExpertCrowd and HUD Collaborate to Advance AI Training and Evaluation, Release SWE Benchmark
SAN FRANCISCO, Nov. 6, 2026 /PRNewswire/ -- ExpertCrowd, Toptal's platform for training and improving AI models and systems, today announced a collaboration with HUD, a Y Combinator-backed startup that provides tools for building AI training data through its DataVendor marketplace. Together, the teams released ExpertCrowd-SWE, a software engineering benchmark that evaluates the performance of frontier-grade LLMs and coding agents.
As AI systems take on increasingly complex work, measuring their capabilities requires tasks that reflect professional practice and grading criteria that capture what successful execution actually means. This collaboration brings together ExpertCrowd's expert-led task development and HUD's infrastructure for running complex RL environments.
ExpertCrowd and HUD co-developed ExpertCrowd-SWE, bringing together engineers from Toptal's global network of highly skilled domain experts with HUD's experience collaborating with frontier research teams. After Toptal's experts authored tasks and grading rubrics, HUD's infrastructure hosted task-run environments and verified the benchmarks against internal quality assurance pipelines. HUD and Toptal's AI research teams, composed of more than a dozen PhDs and researchers with educational or professional experience at Harvard, EPFL, the University of Liège, DeepMind, Qwen, and ByteDance, reviewed and assessed the methodology and results to build one of the most robust software engineering benchmarks in the industry.
"ExpertCrowd puts practitioners at the center of defining the work AI systems should perform and judging how well they perform it. By combining that specialized expertise with HUD's testing infrastructure, we're advancing what AI systems can achieve and setting a higher standard for real-world performance," said Taso Du Val, founder and CEO of Toptal.
ExpertCrowd-SWE is the first release of an ongoing Toptal-HUD collaboration, with additional initiatives in development and further announcements coming soon. Building on this foundation, ExpertCrowd and HUD will bring more specialized human expertise into the tasks, environments, and evaluations that support the next generation of AI.
"We built HUD's infrastructure to give research teams the tools to create realistic RL training environments, run evaluations across AI models, and inspect model performance. ExpertCrowd put that infrastructure to work, bringing the professional expertise to translate complex, real-world workflows into rigorous tasks, reference outputs, and evaluation frameworks that test the limits of today's leading AI models," said Jay Ram, founder and CEO of HUD.
About ExpertCrowd
ExpertCrowd is Toptal's platform for training and improving AI models and systems through RLHF, RL data generation, data labeling, and model evaluation. It is powered by Toptal's global talent network, with more than 30,000 specialized domain experts across 140+ countries, including more than 2,000 AI and big data specialists, 12,000 professionals with advanced degrees, and 1,000 PhD-level AI specialists, available on demand. For more information, please visit ExpertCrowd.com.
About HUD
HUD is a platform for building reinforcement learning environments, used by startups, Fortune 500 companies, and frontier labs. It provides infrastructure for training models, evaluating AI agents, and creating the post-training data that aligns AI to organizations' work. HUD's mission is to build reliable, fair, and open infrastructure for AI data with the goal of making data valuable for the people who create it and trustworthy for the labs that train on it. DataVendor is HUD's marketplace, which allows AI labs to license training and evaluation data directly from the companies that created it. For more information, please visit HUD.ai.
Contact
Luiz Baia
SVP of Marketing Strategy
luiz.baia@toptal.com
View original content to download multimedia: https://www.prnewswire.co.uk/news-releases/expertcrowd-and-hud-collaborate-to-advance-ai-training-and-evaluation-release-swe-benchmark-302931733.html
