Axel, you're the Research Lead, and your own beat is hospitality technology. How did you end up running research standards for a benchmark-driven platform?
Axel Eklund: Hospitality is a surprisingly good training ground for skepticism. I have a Master's in Hospitality Management and a Bachelor's in Business Administration, and I spent six years as a hospitality technology researcher at an independent hotel industry advisory firm in Amsterdam. Revenue management platforms are sold almost entirely on performance claims: this tool lifts revenue per room by some percentage. Very few of those claims survive a careful look at the baseline. Lift compared to what? Over which season? In which kind of property? You learn quickly that a number without its measurement conditions is not evidence. At Axiobench I still cover hospitality technology, revenue management platforms and travel booking economics, but my bigger job is the pipeline: what counts as a primary source, what has to be reproduced before it is published, and how we label what we could not reproduce.
Connor, you cover cloud capacity and data center markets, and you fact-check a large share of the Best Lists. What's your background?
Connor Wardell: I have a Master's in Information Systems and a Bachelor's in Computer Science. I spent seven years as a cloud infrastructure researcher at an independent enterprise technology advisory firm in Barcelona, then worked freelance as a cloud infrastructure analyst. Infrastructure is the most benchmark-literate market there is, and also the one where benchmarks get gamed most creatively. A vendor publishes throughput figures on hardware nobody runs in production, or latency at a load level nobody operates at. My job at Axiobench is to ask under what conditions a number was produced and whether those conditions resemble what a buyer will actually face. As a fact-checker, I'm the person who sends a list back when a description says "fast" where it should say what was measured.
Andrei, enterprise AI is probably the hardest market to benchmark right now. How do you approach it?
Andrei Horváth: Carefully. I hold a Master's in Machine Learning and a Bachelor's in Computer Science, and I spent six years as an AI and machine learning researcher at an independent artificial intelligence advisory firm in Paris before freelancing as an AI analyst. At Axiobench I cover enterprise AI adoption, machine learning platforms and AI infrastructure spending. The challenge is that the field moves faster than its measurement. Leaderboards saturate, evaluation sets leak into training data, and adoption surveys often define "using AI" so loosely that a team trying a chatbot once counts the same as a company running models in production. So I separate three things in every report: what a vendor claims, what an independent evaluation measured, and what an adoption survey actually asked. Those are three different kinds of evidence, and blending them is how most misleading AI statistics are made.
Antje, you come from energy economics and chemistry, and you cover hydrogen and electrolyzers. That's a long way from software.
Antje Sauer: Further than it looks, and closer than it looks. I have a Master's in Energy Economics and a Bachelor's in Chemistry, and I spent seven years as a hydrogen economy researcher at an independent clean energy advisory firm in Accra. Hydrogen is a market built on announced capacity. Projects get announced, re-announced, resized and quietly cancelled, and a lot of published pipeline figures add up gigawatts that will never be built. Chemistry trains you to care about conditions: an electrolyzer efficiency figure means nothing until you know the operating point, the system boundary and whether the balance of plant is included. That is exactly the discipline Axiobench applies to software. A benchmark without its conditions is just a marketing number with decimals.
---
"Benchmark-driven" is the core of the Axiobench brand. What does it mean in practice?
Axel: Three steps, in that order. First, source collection: analysts pull figures from primary materials, meaning official releases, vendor documentation and established databases, before anything enters review. Second, benchmark and reproduce: we re-run the numbers where we can, test vendor claims against their own documentation, and check consistency across sources. AI does a lot of the heavy lifting on retrieval and extraction at that stage, and cross-model checks flag discrepancies, but AI doesn't decide anything. Third, editorial sign-off: a senior editor reviews the structure, the sourcing and the measurement notes, and nothing ships without a human decision.
Connor: And we don't pretend everything reproduces. Every figure carries a confidence band. "Verified" means it held up through measurement and review. "Directional" means the sources agree on the direction but not precisely on the magnitude. "Single source" means exactly that. Our target mix is roughly 70 percent verified, 15 percent directional and 15 percent single source. A reader deserves to know which of those they are looking at.
Andrei: That labelling changed how I write. If I know a figure will carry a single-source band, I have to decide whether it is still worth including. Sometimes it is, because it's the only data that exists on an emerging question. But then the text has to treat it as a signal, not a fact.
Your Best Lists avoid superlatives as a rule. Why?
Connor: Because "industry-leading" can't be checked. "Handles this many requests per second under these conditions, per the vendor's published load test" can. If a tool publishes benchmarks, performance documentation or load-test results, we can describe its value in measured terms: throughput, latency, capacity. If it doesn't, we say that too. The absence of published performance data is itself useful information for a buyer.
Antje: It's also a discipline for us. Once you ban the adjectives, you find out very quickly whether you actually know anything about a product.
---
Some of your research is funded by software vendors. How does that work without compromising the findings?
Axel: The rules are written down, and they are designed so the money can't reach the parts that matter. Vendors fund studies because they want rigorous, comparable research on their market to exist. Each study is co-funded by several of the vendors it assesses, never by a single vendor. Funders can propose the market to be studied. That's it. They don't choose which vendors are included, they don't set or weight criteria, and they don't approve findings.
Andrei: The part I would compare to good scientific practice is the protocol. Before research begins, an editor fixes the market definition, the inclusion rules that determine which vendors are in scope, and the evaluation criteria and their weights. That protocol is published with the study and doesn't change once research starts. It is essentially pre-registration. You can't move the goalposts after you've seen who is winning.
Connor: Every vendor in scope, funder or not, is assessed against the same criteria and gets the same factual-accuracy check on its own products before publication. We correct things only where documentation supports the correction. What vendors can't do is review findings, scores or statements about other vendors.
Axel: And every completed study is published as found. It is labelled as vendor-funded at the top of the page and carries a funding statement in its methodology. Studies go through the same pipeline and source checks as everything else, but they are kept separate: they are never cited as evidence in our statistics reports or Best Lists, and they have no effect on rankings.
---
What's the most common way you see numbers mislead in your areas?
Antje: Announced capacity presented as installed capacity. It is the single biggest distortion in clean energy data.
Andrei: Pilot adoption presented as production adoption. A headline saying most companies "use AI" usually means most respondents said someone in the company tried something.
Connor: Benchmarks run under conditions no customer will ever see. Peak numbers on idealised hardware, presented as if they were typical.
Axel: Lift figures without baselines. If you can't tell me what the comparison was, the percentage is decoration.
Last question: what should readers understand about Axiobench that they can't see on the page?
Connor: That the most important edits are deletions. A lot of what we review never gets published because it doesn't hold up.
Antje: That the confidence band is a promise, not small print. If we say "directional," we mean it, and we would rather you knew.
Andrei: That we're comfortable saying the data doesn't exist yet. In a fast market, that is often the most honest finding.
Axel: That reproduction is slow, and we do it anyway. The headline number takes seconds to find. Checking whether it holds takes the rest of the day.
---
Axiobench.com publishes more than 10,000 free research reports and software Best Lists across over 110 industries. Explore the library at axiobench.com/topics/ and read the full methodology at axiobench.com/editorial-process/.
Part of our series: The People Behind the Research