feat: implement error-budgeting #76

nelimee · 2025-09-26T15:19:54Z

🔗 Closed Issues

Closes #66.

📝 Description

This PR implements error budgeting. A lot of code in this PR is about:

building the correct quantum circuits and decoding graphs in parallel,
running simulations in parallel,
post-processing results from parallel simulations,
numerical code to fit polynomials and propagate standard deviations.

🚦 Status

Working error budgeting
Code documentation
Tutorial for the public documentation
Extensive tests (might be tricky)

🛠️ Future Work

None

➕️ Additional Information

None

🧾 Release Note

PR title

…computation

…mputation

rolandriver

Thanks @nelimee. Quite a fair amount of comments as I think we should define data structures that'll help simplify the whole computation and reduce the number of passed arguments. Happy to discuss this in detail.

deltakit-explorer/src/deltakit_explorer/analysis/budget/__init__.py

rolandriver · 2026-01-08T13:48:28Z

deltakit-explorer/src/deltakit_explorer/analysis/budget/_budget.py

+)
+
+
+def get_error_budget(


I would strongly advise to avoid functions with more than 3 or 4 parameters. Having 13 parameters here is practically unusable. At first glance, a lot of them can be grouped under dedicated custom datatypes. For instance, noise_model_parameters is present in NoiseInterface. We should think of passing instances instead of types here I think.

Agreed with this - I think a lot of parameters can be nicely encapsulated into datatypes. Also, does this function need to be able to generate the circuits or can we pass the circuits to it? I'm thinking a dictionary of {distance: circuit}. You can apply different noises to this circuit once it is generated. Ideally, you also extract the number of rounds from the circuit (e.g. from detector coordinates).

It is unclear to me what will be gained by that. To me, this is a 4-parameter function for the average user, and the power user is given optional knobs to turn, approximately like a sinter.collect would do.

Also, in my opinion, it would be even more cumbersome for a user to use data types just for grouping parameters, because then he/she needs:

to find which data types (easy with type hints),

import these data types,

create an instance of these data types and give them to the function.

For the circuit part, a callable is a strictly more capable way of doing. If you have a dictionary of {distance: {num_round: circuit}}, you can just wrap it in a function that will return the entry if it exists, or fail with a meaningful error message if the circuit required is not in the mapping.
But I am happy to also allow a dictionary if you think that is a nice-to-have.

So as soon as you expose parameters publicly, it means they can be used. Defaulting them doesn't mean that a user can't try and fiddle with them. Even a power user wouldn't be able to humanly understand what are the possiblilties of tweaking 10 parameters, least to remember what they are supposed to do. If these parameters are to be understood as experiment configuration, then they should fall under a datastructure that represents that concept.

rolandriver · 2026-01-08T13:52:07Z

deltakit-explorer/src/deltakit_explorer/analysis/budget/_budget.py

+    """
+    # We will compute the gradient at the half point following the methodology outlined
+    # in "Exponential suppression of bit or phase errors with cyclic error correction".
+    point = np.asarray(noise_model_parameters) / 2


Suggested change

point = np.asarray(noise_model_parameters) / 2

point = np.asarray(noise_model_parameters) / 2.

Does that change anything? Or is that just a personal stylistic preference?

deltakit-explorer/src/deltakit_explorer/analysis/budget/_budget.py

rolandriver · 2026-01-08T13:56:45Z

deltakit-explorer/src/deltakit_explorer/analysis/budget/_budget.py

+    # in "Exponential suppression of bit or phase errors with cyclic error correction".
+    point = np.asarray(noise_model_parameters) / 2
+    # Evaluate the gradient.
+    gradient, gradient_stddev = compute_1_over_lambda_gradient_at(


Same, it is really hard for a user to use such functions, especially when they are exposed publicly.

deltakit-explorer/src/deltakit_explorer/analysis/budget/_generation.py

rolandriver · 2026-01-11T17:35:22Z

deltakit-explorer/src/deltakit_explorer/analysis/budget/_gradient.py

+    derivative = float(
+        sum(
+            coefficient * (power + 1) * gradient_approximation_point**power
+            for power, coefficient in enumerate(coefficients[1:])


Why excluding the first element here ?

Because in the derivative, the constant coefficient (associated with x**0, so the first one) is not needed.

deltakit-explorer/src/deltakit_explorer/analysis/budget/_gradient.py

rolandriver · 2026-01-11T20:56:35Z

deltakit-explorer/src/deltakit_explorer/analysis/budget/_gradient.py

+
+
+def compute_1_over_lambda_gradient_at(
+    noise_model_type: type[NoiseInterface],


Same: this signature should be simplified. It is unusable in state.

deltakit-explorer/src/deltakit_explorer/analysis/budget/_memory.py

github-actions · 2026-01-12T10:52:54Z

File	Coverage		Missing
All files	`84%`	❌
deltakit-explorer/src/deltakit_explorer/analysis/budget/_budget.py	`65%`	❌	`35` `40` `125-150`
deltakit-explorer/src/deltakit_explorer/analysis/budget/_discretisation.py	`78%`	❌	`89-95`
deltakit-explorer/src/deltakit_explorer/analysis/budget/_generation.py	`82%`	❌	`73` `127` `132-133` `171-172`
deltakit-explorer/src/deltakit_explorer/analysis/budget/_gradient.py	`46%`	❌	`253-344`
deltakit-explorer/src/deltakit_explorer/analysis/budget/_lambda.py	`42%`	❌	`76-110`
deltakit-explorer/src/deltakit_explorer/analysis/budget/_memory.py	`52%`	❌	`22-23` `26-33`
deltakit-explorer/src/deltakit_explorer/analysis/budget/_post_processing.py	`31%`	❌	`61-73` `80-89` `95-115`
deltakit-explorer/src/deltakit_explorer/analysis/budget/_rounds.py	`0%`	❌	`1-111`
deltakit-explorer/src/deltakit_explorer/analysis/budget/_visualisation.py	`20%`	❌	`43-49` `53-54` `64-72` `81-91` `157-282`

Minimum allowed coverage is 95%

Generated by 🐒 cobertura-action against 6673bb8

Skoricius · 2026-01-13T13:33:10Z

Thanks for including me, happy to have a look at this.

Skoricius

Thanks for this - it's going to be very useful for the project work.
For now, mainly commenting on the interface. Will look more closely into plotting and maths tomorrow.

Skoricius · 2026-01-16T17:15:21Z

deltakit-explorer/src/deltakit_explorer/analysis/budget/_budget.py

+)
+
+
+def get_error_budget(


Agreed with this - I think a lot of parameters can be nicely encapsulated into datatypes. Also, does this function need to be able to generate the circuits or can we pass the circuits to it? I'm thinking a dictionary of {distance: circuit}. You can apply different noises to this circuit once it is generated. Ideally, you also extract the number of rounds from the circuit (e.g. from detector coordinates).

Skoricius · 2026-01-20T16:26:36Z

deltakit-explorer/src/deltakit_explorer/analysis/budget/_budget.py

+    noise_model_type: type[NoiseInterface],
+    noise_model_parameters: npt.NDArray[np.floating] | Sequence[float],
+    num_rounds_by_distances: Mapping[int, Sequence[int]],
+    noise_parameters_exploration_bounds: list[tuple[float, float]],


I don't think this needs to be a mandatory input because it's a bit hard for the user to determine what's sensible here. I'd say this can be optional and should be some relative to the size of the parameters. E.g. default to 30% or something like that?

That's a part where I spent a lot of time trying. The problem is that because some parameters are way more sensitive than others (e.g., 2-qubit gate noise is more sensitive than 1-qubit gate noise), I was not able to find a fixed rule that would be fine for all the parameters. Either the bounds are too tight and estimation of the gradient on non-sensitive parameters (e.g., 1-qubit gate noise) are bad, or the bounds are too large and we end up with LEP that are at 50% for some variations and at 0% (i.e., no error witnessed) for others. Happy to re-test if you want.

deltakit-explorer/src/deltakit_explorer/analysis/budget/_budget.py

deltakit-explorer/src/deltakit_explorer/analysis/budget/_discretisation.py

Skoricius · 2026-01-20T16:56:16Z

deltakit-explorer/src/deltakit_explorer/analysis/budget/_generation.py

+    """
+    circuit = memory_generator(distance, num_rounds)
+    noisy_circuit = noise_model.apply(circuit)
+    decoder, decoder_circuit = PyMatchingDecoder.construct_decoder_and_stim_circuit(


I would imagine it would be important to select the decoder we are using for lambda modelling.

Yes it is, but for the moment at least there is no interface in deltakit for the construction of an arbitrary decoder. Said differently, deltakit_decode.DecoderProtocol does not include in its interface any generic way to construct a decoder.

Because I need to construct the decoder for various circuits and noise, I need such a capability. And as far as I have seen, PyMatchingDecoder is the only offline decoder that provides that for the moment.

AndrewPattersonRL

Looks good - I've ran the notebook too.

The main takeaway from me is that it could save time if you generate noiseless circuits first, then apply noise to each one. Happy to chat about that if you like

deltakit-explorer/src/deltakit_explorer/analysis/budget/__init__.py

AndrewPattersonRL · 2026-01-15T14:04:53Z

deltakit-explorer/src/deltakit_explorer/analysis/budget/__init__.py

I think budget is a bit too vague of a name for a module - error_budgeting?

I find error_budgeting a little bit long. Maybe error_budget?

docs/guide/error_budgeting.ipynb

src/deltakit/explorer/analysis/budget/__init__.py

deltakit-explorer/tests/analysis/budget/test_post_processing.py

deltakit-explorer/src/deltakit_explorer/analysis/budget/_generation.py

AndrewPattersonRL · 2026-01-15T17:07:20Z

docs/guide/error_budgeting.ipynb

+     "output_type": "display_data"
+    }
+   ],
+   "source": [


First time I ran this cell I got a warning that no fail had been found, then an error that my data was of the incorrect format. I presume this is because the no fails found entry had been removed. Should we make the lower bounds a bit larger so that this is very unlikely to happen?

(stupidly I re-ran the cell without saving the error message so I can't repeat it here)

AndrewPattersonRL · 2026-01-23T10:19:51Z

deltakit-explorer/src/deltakit_explorer/analysis/budget/_rounds.py

I think it's useful - in your experience does it generally produce round numbers that are greater or less than the distance? Or is that noise-dependent?

rolandriver

Same as before. I really think these functions need to be refactored into smtg more useable. In the interest of time, I approve but this shouldbe changed quickly.

rolandriver · 2026-01-30T17:28:14Z

deltakit-explorer/src/deltakit_explorer/analysis/budget/_budget.py

+)
+
+
+def get_error_budget(


So as soon as you expose parameters publicly, it means they can be used. Defaulting them doesn't mean that a user can't try and fiddle with them. Even a power user wouldn't be able to humanly understand what are the possiblilties of tweaking 10 parameters, least to remember what they are supposed to do. If these parameters are to be understood as experiment configuration, then they should fall under a datastructure that represents that concept.

rolandriver · 2026-01-30T18:13:32Z

deltakit-explorer/src/deltakit_explorer/analysis/budget/_discretisation.py

+    def __call__(
+        self, a: float, b: float, c: float, num_points: int, degree: int, /
+    ) -> npt.NDArray[np.floating]:
+        match self:


This is not supported in Python 3.10

The match? It seems like it is: https://peps.python.org/pep-0636/. Also, tests are passing for 3.10 so I think this is fine.

deltakit-explorer/src/deltakit_explorer/analysis/budget/_discretisation.py

deltakit-explorer/src/deltakit_explorer/analysis/budget/_memory.py

nelimee added 10 commits September 25, 2025 19:22

feat: better logical error probability per round

03a3ff2

feat: better lambda computation

766bf1d

Fix documentation and error analysis in lambda

713b2e9

Initial replication of private PR

25d32e7

Introduce reporters

8623185

Format

05ba2d0

Add temporary main.py

f9c2b0f

Add scipy dependency to code-explorer

16e4056

Update outdated documentation

1041435

Add docstring

4cfaa06

nelimee self-assigned this Sep 26, 2025

nelimee added deltakit-explorer feature PR that implements a new feature labels Sep 26, 2025

nelimee added 17 commits October 3, 2025 11:59

ruff format

ee74ad7

Computing stddevs directly instead of having variances

e23e262

Fixing typo in standard deviation computation

7512ac7

Fix real-world test

f90b491

Directly compute stddev

e1b15bf

Merge branch 'feat/better_leppr_computation' into feat/better_lambda_…

b899b7d

…computation

LEPPRResults -> LogicalErrorRatePerRoundResults

e93afde

Fix outdated return type in docstring

e5c2894

Use new numpy.random interface

6a0b238

Merge branch 'feat/better_leppr_computation' into feat/better_lambda_…

43335c7

…computation

Merge branch 'main' into feat/better_leppr_computation

b89e330

Merge branch 'feat/better_leppr_computation' into feat/better_lambda_…

88dd3b1

…computation

Merge remote-tracking branch 'origin/main' into feat/better_lambda_co…

26c22c0

…mputation

Adding a note on the fit parameters

0f8b898

Adding a note on the fit parameters

0b20a60

Fix mypy

d045f47

Ignoring gradient tests as unused

0cd94d2

rolandriver reviewed Jan 11, 2026

View reviewed changes

nelimee added 2 commits January 12, 2026 10:20

Rename internal variable

a267e7d

DecoderManager instances are never None

741f8d6

nelimee requested a review from Skoricius January 13, 2026 13:53

Skoricius reviewed Jan 20, 2026

View reviewed changes

AndrewPattersonRL reviewed Jan 23, 2026

View reviewed changes

nelimee added 17 commits January 28, 2026 11:24

Allow circuits to be pre-provided

35ce638

Improve comments about paper

2786ab0

Introduce a new error budget result class

3cde2ca

Rename internal variable

b188b96

compute_1_over_lambda_at -> inverse_lambda_at

d7cae40

compute_1_over_lambda_gradient_at -> inverse_lambda_gradient_at

7bc9580

Use enum instead of protocol for discretisation generation

f3acab3

Improve docstring for error budgeting module

c9e454c

Address comments by having dynamic __all__

d761cca

Address comment

a5ddfaf

Merge branch 'main' into feat/error_budgeting

afe25e9

Fix inheritance with enum

ad06edd

Rename is_valid

ad53b22

Remove unused test

907fb71

Remove NoiseInterface

460359a

Fix tests

d9c9dc8

Fix test

c5ae709

rolandriver self-requested a review January 30, 2026 17:21

rolandriver previously approved these changes Jan 30, 2026

View reviewed changes

nelimee added 2 commits February 3, 2026 11:51

Address code review

80fb77d

Merge remote-tracking branch 'origin/main' into feat/error_budgeting

6673bb8

nelimee dismissed rolandriver’s stale review via 6673bb8 February 3, 2026 11:52

	point = np.asarray(noise_model_parameters) / 2
	point = np.asarray(noise_model_parameters) / 2.



		def compute_1_over_lambda_gradient_at(
		noise_model_type: type[NoiseInterface],

feat: implement error-budgeting #76

Are you sure you want to change the base?

feat: implement error-budgeting #76

Uh oh!

Conversation

nelimee commented Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Closed Issues

📝 Description

🚦 Status

🛠️ Future Work

➕️ Additional Information

🧾 Release Note

Uh oh!

rolandriver left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Skoricius commented Jan 13, 2026

Uh oh!

Skoricius left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AndrewPattersonRL left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

nelimee commented Sep 26, 2025 •

edited

Loading

github-actions bot commented Jan 12, 2026 •

edited

Loading