[WIP] Add single-particle Worm and partition improved estimators

[weiyiguo9]

Summary

This work-in-progress PR adds two single-particle improved-estimator paths:

• a Worm-sector estimator for F_tau and F_l;
• a partition-space estimator for F_tau and F_l, with fused accumulation and a configurable measurement stride. This strategy follows the general approach used in ComCTQMC.

It also includes the associated solver plumbing, post-processing for the improved self-energy, HDF5 serialization, generated Python bindings, documentation, and focused tests. Time-translation-invariant density-matrix changes are based from Casitel‘s PR, but this PR doesn’t depend on it. (So feature per PR)

Status

This is an early Draft PR. The implementation and tests are still preliminary, and there’re some validation but not that complete. I am publishing it now because these estimator paths may already be useful for discussion and testing by others.

Production use should be cross-checked carefully against independent implementations and against the original Fourier-basis measurement of G(tau).

Feedback on the API, estimator conventions, and validation strategy is welcome.

Further context on the scope of this PR

The Worm implementation in this draft is deliberately minimal. The admissible single-particle F-worm components are enumerated during setup, while the relative visitation of the Z and F sectors still needs to be tuned in short pilot runs through worm_eta and, in practice, the Worm-move proposal weight worm_prob before production runs.

The implementation adds estimator-specific Z/F-sector bookkeeping, moves, and measurement gating on top of the existing mc_generic workflow. It does not attempt to introduce a unified multi-sector Worm framework that could support arbitrary correlators. A fully general solution would require substantially broader changes to the Monte Carlo core, including generic sector/component handling and potentially adaptive sector-weight determination, for example through a Wang–Landau-type procedure.

Such a general framework would be valuable future work, but it would considerably increase the scope and review workload of this PR. The present approach is intentionally less invasive: it keeps the existing partition-sector workflow as the default and makes the new estimators opt-in. In my view, this may make this feature useful to existing CT-HYB users sooner (and most of users are only measure the self-energy then), while allowing a unified Monte Carlo-core design to be developed and evaluated later as an alternative path rather than as a prerequisite replacement.

Base and cherry-pick

This work is based on TRIQS/cthyb@fe3a94a3adb36cbf6f6e09ac427d7d853b05b708, which was the live unstable head when this branch was created.

The changes are organized as 18 logical commits, from e6c0a54 through 1685974. The series separates the Worm trace/moves, estimator containers and post-processing, partition and Legendre paths, stabilization and fused measurement work, focused tests, and binding regeneration.

To cherry-pick the complete series after fetching this branch:

git cherry-pick fe3a94a3adb36cbf6f6e09ac427d7d853b05b708..168597464c23e7467ac4785148ff55d0cccf08f9

Preliminary validation

• the local C++/Python extension build completes successfully;
• impurity_trace_op_insert and worm_F_atomic pass, including simultaneous ordinary density-matrix and Worm-F measurements;
• Py_h5_read_write_more passes;
• additional in-house validation has also been performed;
• full validation against a current, consistently c2py-built unstable TRIQS stack is still pending.

Generative AI Use Declaration

This PR was co-authored with an OpenAI Codex agent.