make_beverland_et_al() uses Beverland's a=0.03 for FifteenToOne's error model instead of Litinski's own a=0.1 (arXiv:1808.02892)

[kkoci]

FifteenToOne's qubit and cycle count formulas are an exact match to their cited source (arXiv:1905.06903, page 11), and its internal noisy-circuit error simulation is a faithful reimplementation of the paper's protocol. I confirmed this directly: running FifteenToOne(9,3,3) at physical_error=1e-4 with the paper's own logical-error constant (a=0.1, from the paper's Eq. 7) reproduces the paper's own published result almost exactly, p_out = 9.300e-10 against the paper's stated 9.3e-10 for the same (15-to-1)_{9,3,3} configuration.

However, PhysicalCostModel.make_beverland_et_al() builds the LogicalErrorModel from QECScheme.make_beverland_et_al(), which uses a=0.03 (from Beverland et al., arXiv:2211.07629, Table V), and hands that into FifteenToOne instead of Litinski's own a=0.1. Running the same factory with this substituted constant gives p_out = 1.894e-10, about 4.9x smaller than Litinski's own paper predicts for identical (d_X, d_Z, d_m) and physical error rate.

Repro:

from qualtran.surface_code import FifteenToOne, LogicalErrorModel

factory = FifteenToOne(9, 3, 3)

# Litinski's own constant, reproduces the paper's Table 1 (p_out=9.3e-10)
litinski_model = LogicalErrorModel(error_rate_scaler=0.1, error_rate_threshold=0.01)

# what make_beverland_et_al() actually uses
beverland_model = LogicalErrorModel(error_rate_scaler=0.03, error_rate_threshold=0.01)

# compare factory.p_fail / factory error output under each, physical_error=1e-4

This is not a bug in the sense that FifteenToOne does something other than what it says. It correctly implements the paper's circuit and takes a QEC-scheme parameter as designed. The issue is that make_beverland_et_al() silently mixes Beverland's threshold assumption into Litinski's circuit rather than using Litinski's own constant, with nothing in the code or docstring flagging this as an intentional choice.

This may be deliberate, in which case a comment in the source noting that make_beverland_et_al() is a hybrid of Beverland (physical and logical error model) and Litinski (factory and data block layout) rather than a straight implementation of either would help, since the name currently reads as "this reproduces Beverland et al.'s procedure," which is not quite what is happening (see also #1943 for a related discrepancy in the data block formula from one of the same Litinski papers).

Source location: qualtran/surface_code/physical_cost_model.py, PhysicalCostModel.make_beverland_et_al(), lines 160-188 (constructs the mismatched LogicalErrorModel); the affected formula itself is correctly implemented in qualtran/surface_code/fifteen_to_one_factory.py, lines 52-54 and 94.

[mpharrigan]

I'm a little confused. it's called make_beverland_et_al. The constants are from Beverland et al. I acknowledge that those are different constants than Litinski; but it should reflect Beverland et al.

"this reproduces Beverland et al.'s procedure," which is not quite what is happening

can you explain this? What part does not reproduce beverland's procedure?

[kkoci]

Fair question, let me be precise about what I meant. Beverland et al. (arXiv:2211.07629) describes its own resource-estimation procedure end-to-end, including its own magic-state factory / distillation and physical-qubit-layout approach in Appendix D/E (the PSSPC method, factory selection via Eq. E2-E6, etc.). make_beverland_et_al() doesn't use that layout or factory-selection procedure. Instead it uses Litinski's FifteenToOne factory (arXiv:1905.06903) and Litinski's CompactDataBlock layout (arXiv:1808.02892) for those two components, and only applies Beverland's constants (a=0.03 threshold model, physical parameters) on top of them.

So the physical-parameter and logical-error-rate assumptions are Beverland's, as the name suggests, but the factory design and data-block layout, arguably the parts of "the procedure" that determine most of the qubit count, are Litinski's, not Beverland's own Appendix D/E method. That's what I meant by "not quite reproducing Beverland's procedure", it's accurate to the paper's threshold/error-rate assumptions, but not to its resource-layout method.

Given that, I think this is a defensible design choice (reuse Litinski's more concrete, directly-implementable factory/layout math rather than reimplement Beverland's own, which is more abstract), not a bug, just maybe worth a docstring note somewhere so someone comparing this preset's output against Beverland's own worked examples knows why it won't match exactly, and someone comparing against Litinski's paper knows the error constant used isn't his. Up to you whether that's worth adding or whether this is clear enough as-is; happy to close this out either way.

[mpharrigan]

Thanks for following up. Yes, I think it's a real issue if the code in the repository doesn't respect the choices made in the beverland paper and it's worth noting and ultimately fixing! From a quick skim, it looks like beverland is using a lot of litinski architecture but it's a worthy goal to recapture the original intent

1. From a glance, it looks like they're using the FastDataBlock, not the CompactDataBlock (by default) c.f. fig 11.
2. The FifteenToOne factory is one of the options (c.f. table VI and VII), but they have a gridsearch protocol for selecting the minimum-volume choice

On the second point, this parameter search over the design options is something that can be applied to any composition of architecture-model choices and it was a conscious design choice to have the classes do the "forward-case" cost modeling and then layer on "design optimization" on top of it, see e.g. get_ccz2t_costs_from_grid_search

---

an interested contributor could go carefully through the beverland paper to fix any discrepancies like the data block, implement the distillation-unit based factory construction, and layer on a design-sweep helper function to fully capture the intent of beverland et al.