Insider Brief
- Measuring the progress towards the quantum advantage with objectivity and reliability, is of high interest for potential end-users and crucial for quantum’s future development.
- The challenges comes from the diversity of the hardware platforms, their specificities in terms of physical characteristics and applications, their maturity and its potential rapid evolution.
- The BACQ project precisely aims at developing, exploiting and promoting a reliable, objective and long-lasting measurement instrument of the quantum computing practical performance, by considering a set of benchmarks at a level close to real applications.
PRESS RELEASE — Quantum computing is expected to revolutionize multiple technical fields and activity sectors, from optimization in logistics or in finance, to simulation for research, engineering or industry, passing through cryptography.
Measuring the progress towards the quantum advantage and the realization of such promises, with objectivity and reliability, is of high interest for potential end-users and crucial for the future development of the domain, now subject of hype and high competition.
The challenges, especially to achieve comparable measurements, comes from the diversity of the hardware platforms, their specificities in terms of physical characteristics and applications, their maturity that can still be low, and the potential rapid evolution of the technologies.
The BACQ project precisely aims at developing, exploiting and promoting a reliable, objective and long-lasting measurement instrument of the quantum computing practical performance, by considering a set of benchmarks at a level close to real applications. The project targets high-level evaluation, meaningful for industrial users. The proposed methodology consists in the aggregation of low-level technical metrics and a multi-criteria analysis providing operational performance indicators of the different quantum computing solutions and pointing out the service qualities of interest to the end-users. Dedicated to the whole value chain setting up from quantum hardware development to industrial use cases, BACQ is complementary to the benchmarking initiatives only focusing on low-level hardware physical criteria. Sharing of the benchmark suite, as broadly as possible, is an important objective to establish common reference measurement methods and ensure no bias with inclusion of all the technologies.
The envisioned benchmark suite will be based on the resolution of several classes of problems covering important application domains of quantum computing:
- Optimization
- Linear systems solving
- Quantum physics simulation
- Prime factorization
Relevant criteria will be defined for the resolution of each problem, some being hardware-agnostic and others dependent (low level): computation time, latency, problem size, approximation rate, resolution probability, accuracy, fidelity… Importantly, the project also considers energetic criteria for the evaluation of the energetic performance of the machines. The aggregation of the criteria and multi-criteria analysis will allow fully explainable and transparent notations, allowing comparisons between different quantum machines and with classical computers, as well as determination of the practical advantages of each quantum machine with respect to specific applications. The project will address both analog machines (quantum simulators and annealers) and gate-based machines, Noisy Intermediate Scale Quantum (NISQ) and Fault Tolerant Quantum Computing (FTQC). The followed practical approach is to have a suite of benchmarks, adaptative to some extent, appropriate to the capabilities of the available machines and able to demonstrate their respective advantages, including, in the longer term, exponential speedup of specific algorithms on FTQC machines.
Consultations of the technology providers, as well as end-users is key to develop an instrument that meet the needs. Once specified and developed, the benchmark suite will be available to the community for its use. To achieve a universal tool, the project will seek to establish cooperation with similar initiatives worldwide. Standardization will be another way to get consensus and adoption at large scale, considering the European committees CEN-CENELEC and the international ones, ISO, IEC and IEEE.
As part of the project, a first action has already been launched regarding Q-Score, which Eviden developed on the MAXCUT optimization problem, to test and validate the benchmark on different types of quantum machines. Within this first action to establish the universality of the benchmark, discussions and collaborations have already been engaged with European and international technology providers regarding various hardware platforms, superconducting, photonic, spins, neutral atoms, trapped ions, and NV centres in diamond.
The BACQ project, with 3-year duration and 4 M€ budget, including 2.5 M€ funding by France 2030 via ANR within MetriQs-France, gathers a consortium composed of Thales as coordinator, Eviden, an Atos business, CEA, CNRS, Teratec and LNE.
A consortium of quantum experts
Thales is the coordinator of the BACQ project and will participate in the development of benchmark criteria related to linear systems solving and optimization, with particular focus on noise and errors, on both NISQ and FTQC machines. Furthermore, Thales will be involved in the aggregation of the criteria developed in the project based on the multi-criteria decision tool MYRIAD developed for capturing consensus preferences and providing an explicit global rating.
Eviden will participate in the development of technical benchmark criteria related to optimization through the adaptation of the existing QScore/MAXCUT for a large variety of quantum machines. They will also contribute on the development of a new QScore/Many-body based for simulation of quantum physics.
CEA will contribute with three teams. Two of them will work on benchmark criteria related to simulations of many-body quantum physics problems with gate-based and analog machines (e.g., accuracy of the preparation, evolution and measurement of entangled states tested against analytical solutions). The third team will be involved in the evaluation of analog and gate-based machines on several classes of optimization problems (from easy-sparse, to difficult-dense problems), linear systems (at different sizes), and integer factorizations (with an increasing complexity).
CNRS will define, test and promote the energetic efficiency of simple gate-based quantum machines, focusing on the hardware-agnostic algorithmic level. The contribution includes developing a simulation module and use it to optimize energetic efficiencies. The promotion and acceptance of the benchmark will directly benefit from the strong implication of the CNRS team, which is leading the IEEE newly launched Quantum Energy Initiative Working Group, to propose a standard on energy efficiency of quantum computing.
Teratec will participate in the establishment of the relations to the ecosystem to promote the project: consultations with industrial stake-holders, end-users and technology providers, dialogue and cooperation in Europe and at international, with e.g., similar benchmarking initiatives.
LNE is the coordinator of MetriQs-France. The public institution will participate in BACQ for developing the relations with the stakeholders, for promotion and exploitation, notably in metrology and standardization, taking advantage of the connection with different organizations AFNOR, CEN-CENELEC, ISO/IEC. LNE will make particular use of its expertise in the development and implementation of reference measurement methods. In addition to this, by its trusted third party status, LNE will guarantee the impartiality and the independence of the benchmarks.
MetriQs-France
As part of the National Quantum Strategy of France, MetriQs-France is the national program on measurements, standards, test & evaluation of quantum technologies. With the ambition to support innovation and establishment of a sound and thriving quantum industry, the program aims at developing, exploiting and promoting reference measurement capabilities, validated and harmonized, for reliable, comparable, and impartial characterizations and performance evaluations of these emerging technologies. Such measurement are required to demonstrate indisputably the possible advantage of the quantum technologies and to enable their adoption by industry and market. Coordinated by LNE the program addresses the measurement needs on the national priorities, including quantum computing, quantum sensing, quantum communication and the enabling technologies. It comprises both deployment of an infrastructure of reference for measurement & test, and R&D collaborative projects.
For more market insights, check out our latest quantum computing news here.
