Guest Post:  What a High School Student Found When He Stress-Tested a Quantum Benchmark

Abstract textured waves in warm orange and brown hues
Hub Hub

Guest Post By Jithesh Mithra, Independent Researcher, Jacksonville, Florida

Here is a result that should not happen. Take a quantum error correction (QEC) code, fix the physical error rate, and ask a simple question. Does adding more qubits make it work better? Under a single noise assumption, the answer would be yes. However under a different assumption at the exact same error rate, the answer would be no. This is the same code with the same error rate, yet there are opposite conclusions. The only thing that changed was the assumption about the structure of the noise.

I found this while building an open-source tool to test something the QEC field took mostly for granted.

The quantity I explore is the pseudo-threshold. For finite-size codes, it is the physical error rate at which the logical error curves for two adjacent code distances cross. Below it, increasing distance helps. Above it, increasing distance hurts. Pseudo-thresholds are used constantly as comparative indicators, basically how good a code is under a given noise model. They are almost always reported as single point values, with no error bars and no sensitivity analysis attached.

That seemed strange. A pseudo-threshold is estimated from Monte Carlo data, which means it carries statistical uncertainty like any other estimate. Yet the numbers get passed around like it was exact. So I started asking a narrower question. If you change the assumed noise model slightly, how much does the pseudo-threshold move, and is that movement large enough to change the conclusions people draw from it?

To check, I built QECops, an open-source Monte Carlo framework that simulates repetition codes under several phenomenological noise models. This includes: independent bit-flip, depolarizing, biased, and a correlated model where errors can propagate to neighboring qubits. The design principle was to hold everything else fixed and vary only the noise assumption. This makes it so any change in the threshold could be traced to that assumption. For every threshold estimate, I computed bootstrap confidence intervals, so uncertainty is packaged with the number instead of being thrown away.

The results were sharp.

Under correlated noise, the pseudo-threshold fell monotonically as correlation strength increased, from 0.4609 at weak correlation down to 0.2372 at the strongest correlation I tested. That’s a 52.2 percent reduction from the independent bit-flip baseline of 0.4961. Threshold suppression under correlated noise is not itself new. It has been understood since Klesse and Frank’s 2005 work. What I was after was different. I wanted to know whether the estimate itself stays trustworthy as a comparative tool when the noise assumption shifts.

Two diagnostics suggested it does not. Not always at least. The first is a sensitivity metric, a finite-difference measure of how strongly the threshold responds to a change in correlation strength. For one distance pair, that sensitivity roughly doubled between the weakest and strongest correlation regimes. The endpoints were statistically distinguishable, though I want to be clear that the intermediate points overlapped within uncertainty and could not be cleanly ordered. The second is a crossing-consistency check, where a stable threshold should not depend much on which pair of code distances you use to estimate it. At two correlation strengths, the estimates from different distance pairs disagreed by more than their combined bootstrap uncertainty. The threshold, in other words, stopped behaving like a single well-defined number.

The most concrete finding is the inversion that I opened with. At a physical error rate of 0.400, independent bit-flip noise says increasing the code distance improves the logical error rate. Correlated noise at moderate strength says it does not. If a hardware group characterized their device assuming independent noise, but the real noise was correlated, a threshold based decision could point the wrong way in its qualitative conclusion.

I want to be careful about what this does and doesn’t show. This is a study of repetition codes, which are classical codes, under a small set of finite distances and simplified phenomenological noise models. It is not a claim about surface codes, about asymptotic thresholds, or about real hardware behavior. The correlated noise model I used is one-directional and simple. Three code distances are not enough to separate a genuine effect from a finite-size artifact with full confidence, and I say so in the paper. 

But the methodology is not specific to the repetition code. The recipe I use can be applied to genuine quantum codes, to more realistic noise, and to larger distances. The point is that reporting a pseudo-threshold as a bare number hides exactly the information you need to judge whether that number can be trusted.

I didn’t come to this from inside a lab. I am a high school student in Jacksonville, Florida, and I built this on standard CPU hardware. I used no cluster, no cloud allocation, no institutional compute, I taught myself quantum error correction from open courses and papers, and I cold-emailed researchers until some of them wrote back. Researchers including Haining Pan at the University of Florida, Zebo Yang from the Florida Atlantic University, and Daniel Strano at the Unitary Foundation gave early feedback on the simulator. None of that required access I could not get for free, which is part of why I think the gap was there to be found in the first place. Uncertainty-aware reporting is not expensive or costly in the slightest, just not standard. 

That’s the part I want to keep coming back to. This was not a hard problem in the sense of needing exotic resources. It required asking whether a widely used number means what people assume it means, and then doing the statistics to check. QECops is open-source and reproducible from a fixed seed, so anyone can run it, break it, or extend it. I would rather it be useful than impressive.

As quantum error correction moves from proof-of-concept towards decisions made on real hardware, the reliability of our benchmarks start to matter as much as their values. A threshold that looks good but is not stable under realistic noise assumptions is worse than an honestly uncertain one, because it sends confidence in the wrong direction. Attaching uncertainty and sensitivity to these estimates is a small change in practice. Based on what I found, it may be a change worth making.

The paper is openly available on Zenodo (https://doi.org/10.5281/zenodo.19410503), and the code is on GitHub (https://github.com/JitheshMithra/QECops). I’d genuinely like to see someone push on it, including in the places where it might break.

BIO:

Jithesh Mithra is an independent high school computational researcher from Jacksonville, Florida. He is the developer of QECops, an open-source Monte Carlo framework for quantum error correction (QEC) diagnostics, and is currently extending this work to study threshold stability in quantum codes under spatial noise disorder. He also conducts research in computational biophysics through Bhargava Systems Research and has worked on precision frequency measurements at Florida State University’s Precision Penning Trap Lab.

LinkedIn: https://www.linkedin.com/in/jitheshmithra/

GitHub: https://github.com/JitheshMithra

Image: Photo by Zach M on Unsplash

Keep track of everything going on in the Quantum Technology Market. In one place.

Share

Stay Ahead of Quantum

Get the latest research, company news, and market intelligence every week.

MENTIONED IN THE ARTICLE

More in Research

Related Articles