Insider Brief
- The QUOPS benchmark measures how much computation a quantum computer can complete successfully and how quickly it can perform the work.
- Tests found that Quantinuum’s Helios handled larger circuits, while Google’s Willow and IBM’s Boston completed operations at higher rates.
- The study estimates that current quantum computing capability must increase about 100,000-fold to reach the scientific workloads examined.
A new benchmark could give researchers and buyers a more practical way to compare quantum computers by measuring how much computation they can complete successfully and how quickly they can do it.
The new benchmark also indicates that future quantum computers will need a significant performance boost — in the 100,000-fold range — to tackle high-value scientific problems.
The Quantum Universal Operations Performance System, or QUOPS, measures the largest useful-sized test circuit a quantum computer can run above a set accuracy threshold. It also records the rate at which the machine performs those operations.
Researchers tested the benchmark on processors from Quantinuum, Google and IBM, according to a study posted on the preprint server arXiv. They also applied it to a small fault-tolerant processor built with up to eight logical qubits on Quantinuum’s Helios-1 system.
The results exposed a trade-off that basic hardware specifications often obscure. Google’s Willow and IBM’s Boston superconducting processors ran operations faster, while Quantinuum’s Helios trapped-ion processor successfully handled larger test circuits, according to a Quantinuum blog post about the study.
The study also found that current systems remain about five orders of magnitude below the estimated computational capability needed for several recognized scientific problems. In more simple terms, that means companies would need to boost performance by roughly 100,000-fold to reach the target range examined by the researchers.
The finding lends support to the ongoing efforts to achieve fault-tolerant quantum computing, which uses error correction to protect computations from the failures that accumulate as circuits grow. Today’s machines can run increasingly sophisticated experiments, but noise still limits the length and scale of the calculations they can complete reliably.
QUOPS was developed by Sandia National Laboratories with contributions from researchers at Quantinuum and NVIDIA, according to the study and Quantinuum’s post. The paper has not yet undergone peer review.
Measuring Completed Work
Quantum computer developers commonly describe their systems with figures such as qubit count, gate accuracy and gate speed. Those measurements remain important for understanding the quality of individual components, but they don’t always show how a complete machine will perform on a substantial calculation.
In quantum computing, a lot can go wrong even when everything else is going smoothly. A system with many qubits, for example, may not be able to use all of them effectively if its error rates are high. A machine with fast gates may lose some of that advantage if limited connections between qubits require it to carry out extra operations. A smaller processor with more accurate qubits and better connectivity may complete a larger circuit despite operating more slowly.
Fault tolerance adds further complications. Physical qubits must be combined to form logical qubits, which store quantum information in a way that allows errors to be detected and corrected. The system must repeatedly measure errors, interpret those measurements and apply corrections while the calculation continues.
Other requirements include moving information across the processor, preparing special quantum states for certain operations and coordinating quantum hardware with conventional computers. The resulting performance cannot be described fully by the speed or accuracy of a single physical gate.
QUOPS approaches the problem by treating the quantum computer as a complete system. Rather than estimating performance from component specifications, the benchmark gives machines randomized circuits of different widths and depths.
Width refers broadly to the number of qubits used. Depth reflects how many layers of operations the circuit must execute in sequence. A wide but shallow circuit uses many qubits for relatively few steps, while a narrow and deep circuit uses fewer qubits for a longer calculation.
The benchmark then checks whether the processor’s output passes a predefined success threshold. By testing circuits with different shapes, researchers map a capability region showing which combinations of width and depth the machine can handle.
QUOPS reduces that region to two main measurements. The first, Q, represents the largest qualifying circuit within a range chosen to resemble computationally relevant workloads. Circuit size is calculated as twice the width multiplied by the depth.
The second measurement, represented by the Greek letter omega, records the effective number of quantum operations completed per second at that point. Together, the two figures show both the scale of the calculation and the speed at which the machine runs it.
A high Q score without a corresponding rate could describe a system that completes large circuits but requires substantial time. A high rate paired with a smaller Q score could describe a fast machine that is limited to shorter or narrower calculations.
Different Machines Show Different Strengths
The researchers applied QUOPS directly to physical qubits on leading processors from Quantinuum, Google and IBM. This cross-platform testing is important because the three companies use systems with different designs and operating characteristics.
Google’s Willow and IBM’s Boston use superconducting qubits. These machines perform gates quickly, but each qubit typically connects directly to only a limited number of neighboring qubits. Calculations involving distant qubits can require added routing operations, increasing the circuit’s length and opportunities for error.
Quantinuum’s Helios uses trapped ions and a design known as a quantum charge-coupled device. Its gates operate more slowly than those of superconducting processors, but the system provides effective all-to-all connectivity, allowing operations between any pair of qubits without the same routing burden.
Quantinuum said in its blog post that Willow and Boston produced higher QUOPS rates but smaller capability regions and Q scores. Helios reached larger circuits and a higher Q score but operated at a lower rate.
The comparison does not identify one architecture as universally superior. Instead, it shows that systems can reach performance through different combinations of accuracy, speed and connectivity. Those trade-offs may matter differently depending on the workload.
The benchmark also accounts for the cost of error mitigation, according to Quantinuum. Error mitigation uses repeated runs and additional data processing to estimate what a quantum computer would have produced with less noise. It can improve the quality of a result, but the extra sampling lowers the effective rate of computation.
The study’s rate analysis quantified this exchange between speed and circuit capability. All three systems could potentially handle larger circuits with mitigation, but doing so reduced their effective operations per second.
Testing Logical Qubits
The researchers went beyond tests on uncorrected physical qubits by applying QUOPS to a basic fault-tolerant processor implemented on Helios-1. The experiment used as many as eight logical qubits encoded with a seven-qubit error-correcting code.
In this arrangement, groups of physical qubits work together to form each logical qubit. The code can detect and correct a limited number of errors, making the stored quantum information more resistant to noise than information held by a single physical qubit.
It’s important to note that this was not a demonstration on a fault-tolerant computer, but an experiment on a small implementation intended to show that the same benchmark can measure both present-day physical-qubit machines and emerging logical-qubit systems.
That continuity could become increasingly important because raw physical-qubit counts will become less useful for comparing machines as developers adopt different error-correction codes and fault-tolerant designs. One company may require far more physical qubits than another to produce a logical qubit of similar quality.
QUOPS instead measures the circuits the resulting system can execute. In principle, that allows a comparison without requiring buyers to resolve every technical difference in the underlying error-correction schemes.
The researchers also modeled how the benchmark could track successive generations of fault-tolerant machines. The goal is to connect current experimental performance with the scale of circuits expected to address scientific problems.
Not a Final Verdict — Some Limitations
The study and its estimates should be read as just that — estimates and not be read as a fixed finish line for the entire industry. Resource requirements can change as researchers improve algorithms, error-correction codes, compilers and hardware. Different applications also demand different circuit structures, data movement and accuracy.
QUOPS uses randomized circuits as a general stress test rather than running complete chemistry, materials or optimization applications. Passing the benchmark therefore does not prove that a machine can deliver a commercially useful result on a particular problem.
The benchmark’s summary score also depends on choices such as the success threshold and the region of circuit widths and depths considered relevant. A single number cannot preserve every feature of a system’s capability region, particularly when two machines have substantially different strengths.
The logical-qubit demonstration was limited to eight encoded qubits and a relatively simple error-correction setup. Projections for future fault-tolerant systems remain dependent on assumptions about error rates, code performance and system scaling that have not yet been demonstrated at application scale.
There are also practical questions about independent testing. Vendors control access to many advanced quantum processors, and results can be influenced by compilation, calibration and operating conditions. Reproducible comparisons will require clear test rules, access to underlying data and repeated measurements as systems change.
Quantinuum said QUOPS is not intended to replace component measurements, application-specific tests or benchmarks for quantum systems linked to high-performance computers. The company is calling on hardware developers to report Q and omega alongside existing specifications and encouraging agencies and buyers to consider QUOPS thresholds in procurement requests.
If widely adopted and independently verified, the benchmark could give government agencies, computing centers and commercial customers a clearer way to evaluate competing systems. A buyer seeking a machine capable of running a trillion reliable operations, for example, could specify a system-level target without prescribing a particular qubit technology or error-correction design.
For a deeper, more technical dive, please review the paper on arXiv. It’s important to note that arXiv is a pre-print server, which allows researchers to receive quick feedback on their work. However, it is not — nor is this article, itself — official peer-review publications. Peer-review is an important step in the scientific process to verify results.
