Nexus: update supported QMCPACK inputs

[jtkrogel]

This PR updates QMCPACK input attributes, parameters, and XML elements supported by Nexus.

Inputs that QMCPACK no longer supports (possibly read but are non-functional) are stored in a new unsupported category.

At present, the added inputs are available only via direct instantiation of the related QIxml elements. Support for a subset of these will be added to generate_qmcpack* functions in a later PR.

Prior to merge, these inputs need to be screened for relevance: should some always be excluded? Are some that are marked non-functional/unsupported miscategorized?

Also helpful would be to note which are prominent/useful enough to surface to users at the generate level in a follow-on PR.

Below, I make a simple summary to aid reviewers in this process; the line-by-line changes in Nexus are less relevant than the corresponding information in this list.

Made with assistance from GPT-5.6 Sol based on the QMCPACK manual and the direct source code.

[jtkrogel]

Here is the summary referred to above:

____________________________________________________________
<simulation>
  elements   : walkerlogs

____________________________________________________________
<mcwalkerset>
  unsupported: target  walkers

____________________________________________________________
<qmcsystem>
  unsupported: dim

____________________________________________________________
<simulationcell>
  parameters : ewald_grid  vacuum
  unsupported: name  reciprocal  tilematrix  uc_grid

____________________________________________________________
<particleset>
  attributes : gpu  id  role
  unsupported: charge  role  simulationcell  source

____________________________________________________________
<group>
  attributes : id
  parameters : inner_cutoff
  unsupported: id  mass

____________________________________________________________
<sposet>
  attributes : twist
  elements   : coefs_mem

____________________________________________________________
<rotated_sposet>
  elements   : opt_vars

____________________________________________________________
<coefs_mem>
  attributes : distributed_ranks  shared_ranks

____________________________________________________________
<bspline_builder>
  attributes : check_orb_norm  save_coefs  skip_checks

____________________________________________________________
<molecular_orbital_builder>
  attributes : key  keyword

____________________________________________________________
<wavefunction>
  elements   : ionwf

____________________________________________________________
<ionwf>
  attributes : name  source  width

____________________________________________________________
<atomicbasisset>
  attributes : expm

____________________________________________________________
<multideterminant>
  attributes : algorithm  spo_0  spo_1

____________________________________________________________
<detlist>
  attributes : opt_coeffs  sortby  zero_cutoff  zerocutoff

____________________________________________________________
<rpa_jastrow>
  attributes : longrange  shortrange
  parameters : kc  rs

____________________________________________________________
<hamiltonian>
  attributes : wavefunction
  elements   : extpot

____________________________________________________________
<coulomb>
  attributes : gpu  pbc

____________________________________________________________
<pseudopotential>
  attributes : pbc  physicalso  spin_integrator

____________________________________________________________
<pseudo>
  attributes : disable_randomize_grid  symbol
  unsupported: cutoff

____________________________________________________________
<mpc>
  attributes : cutoff
  unsupported: ecut

____________________________________________________________
<harmonic_extpot>
  attributes : center  frequency  mass  source  target  type
               energy  length     name

____________________________________________________________
<grid_extpot>
  attributes : dataset_name  file_name  num  source  target
               end           name       pbc  start   type

____________________________________________________________
<origin>
  attributes : fraction

____________________________________________________________
<structurefactor>
  attributes : hdf5  source  target  writeionion  writerho
  unsupported: hdf5  report  writeionion  writerho

____________________________________________________________
<force>
  attributes : add_ion_ion_  pbc
  parameters : weight_exp
  unsupported: addionion  weightexp

____________________________________________________________
<dm1b>
  parameters : timestep  use_drift  warmup
  unsupported: basis_size  reuse

____________________________________________________________
<onebodydensitymatrices>
  parameters : corner  timestep  use_drift
  unsupported: basis_size  reuse

____________________________________________________________
<paircorrelation>
  attributes : debug  dr  name  num_bin  rmax  sources  type
  unsupported: debug  sources

____________________________________________________________
<specieskinetic>
  attributes : hdf5  name  type

____________________________________________________________
<latticedeviation>
  attributes : hdf5  per_xyz  source  target  tgroup  type
               name  sgroup

____________________________________________________________
<perparticlehamiltonianlogger>
  attributes : name       type  validate_per_particle_sum
               to_stdout
  unsupported: validate_per_particle_sum

____________________________________________________________
<init>
  attributes : use_volume

____________________________________________________________
<record>
  attributes : period
  unsupported: name

____________________________________________________________
<walkerlogs>
  attributes : max     min       quantiles    verbose
               median  particle  step_period

____________________________________________________________
<qmc_system_selector>
  attributes : hamiltonian  wavefunction

____________________________________________________________
<optimize>
  attributes : method
  parameters : finite_diff_delta  output_param_file

____________________________________________________________
<hybrid_optimizer>
  attributes : num_updates
  parameters : minmethod

____________________________________________________________
<optimize_qmc>
  unsupported: completed

____________________________________________________________
<linear>
  attributes : append  hdf5  walkerlog
  parameters : blocks_between_recompute  measure_imbalance
               ci_eta                    nsamp_comp
               collection_step           num_shifts
               compute_step              omega
               cost_increase_tol         orb_eta
               debug_checks              output_matrices_csv
               drift_modifier            print_derivs
               drift_unr_a               ramp_eta
               f_eta                     ramp_num
               flavor                    stored_vectors
               freeze_parameters         target_shift_i
               gauss_eta                 targetexcited
               max_relative_cost_change  tjf_1body_eta
               maxdisplsq                tjf_2body_eta
  elements   : optimize  optimizer  qmc_system_selector
  unsupported: max_relative_change

____________________________________________________________
<cslinear>
  elements   : qmc_system_selector

____________________________________________________________
<vmc>
  attributes : append  walkerlog
  parameters : blocks_between_recompute  drift_unr_a
               crowd_serialize_walkers   maxdisplsq
               debug_checks              measure_imbalance
               drift_modifier
  elements   : qmc_system_selector
  unsupported: completed

____________________________________________________________
<dmc>
  attributes : append  walkerlog
  parameters : blocks_between_recompute
               debug_checks
               debug_disable_branching
               drift_modifier
               drift_unr_a
               energyupdateinterval
               maxcopy
               maxdisplsq
               measure_imbalance
               refenergy
               refenergy_update_scheme
               target_walkers
               warmupbyreconfiguration
  elements   : qmc_system_selector  record
  unsupported: completed

____________________________________________________________
<rmc>
  attributes : append      profiling  trace  walkerlog
               checkpoint
  parameters : action  equilsteps  usescaleddrift
               beads   resize      vmcpresteps
               beta    usedrift    warmupsteps
               drift
  elements   : estimator  qmc_system_selector
  unsupported: bounce  collect  mass      observables
               chains  cuts     maxtouch  trunclength
               clone

____________________________________________________________
<vmc_batch>
  attributes : append  walkerlog
  parameters : blocks_between_recompute  estimator_period
               crowd_serialize_walkers   maxdisplsq
               debug_checks              measure_imbalance
               drift_modifier            spin_mass
               drift_unr_a
  elements   : qmc_system_selector  record

____________________________________________________________
<dmc_batch>
  attributes : append  walkerlog
  parameters : blocks_between_recompute
               debug_checks
               debug_disable_branching
               drift_modifier
               drift_unr_a
               energyupdateinterval
               estimator_period
               maxcopy
               maxdisplsq
               measure_imbalance
               refenergy
               refenergy_update_scheme
               spin_mass
               target_walkers
               warmupbyreconfiguration
  elements   : qmc_system_selector  record

____________________________________________________________
<linear_batch>
  attributes : append  checkpoint  hdf5  walkerlog
  parameters : blocks_between_recompute
               ci_eta
               collection_step
               compute_step
               debug_checks
               deriv_threshold
               drift_modifier
               drift_unr_a
               eigensolver
               f_eta
               filter_info
               filter_param
               flavor
               freeze_parameters
               gauss_eta
               line_search
               maxdisplsq
               measure_imbalance
               omega
               options_lmy_.block_lm
               options_lmy_.cost_increase_tol
               options_lmy_.max_param_change
               options_lmy_.max_relative_cost_change
               options_lmy_.nblocks
               options_lmy_.nkept
               options_lmy_.nolds
               options_lmy_.nsamp_comp
               options_lmy_.num_shifts
               options_lmy_.target_shift_i
               options_lmy_.targetexcited
               orb_eta
               output_matrices_csv
               output_matrices_hdf
               print_derivs
               ramp_eta
               ramp_num
               spin_mass
               store_samples
               stored_vectors
               tjf_1body_eta
               tjf_2body_eta
  elements   : optimize  optimizer  qmc_system_selector
  unsupported: max_relative_change

[jtkrogel]

Here is the same list provided as a text file for added convenience:

qmcpack_param_update_summary.txt

[prckent]

Are you able to get your robot to remove the unsupported tags from the actual source? That would be far more helpful :-)

( I find the list to be basically unreviewable not due to any fault of the PR or Nexus but simply that we don't know what is actually used. Obviously all the legacy stuff is headed for the wood chipper. One way to prove what is not used is to remove it. If there are any removals, unless anyone spots anything I suggest we check the full test set (more than it run in CI) and then go ahead with this good faith update. )

[jtkrogel]

Here is a more complete listing of all the XML attributes, parameters and child/parent elements. At present, I think this is the closest thing we have to a complete input schema for QMCPACK.

qmcpack_deep_param_search_listing.txt

I also attach a file that explains the functioning within QMCPACK of each XML element and its contents (actually in rst format due to some odd upload restriction). The attributes/etc are marked as required/optional and allowed values are listed for each. Batched and legacy functionalities are listed separately. I expect this document will be helpful to reference and validate against when we update the QMCPACK manual.

qmcpack_param_update_docs.txt

[jtkrogel]

I've made a deeper pass revising some of the earlier assignments. Attached is a document noting the QMCPACK source file location of each non-functioning input.

qmcpack_deep_unsupported_param_locations.txt

I haven't taken a close look at this yet, but I plan to use it in another PR that attempts removal of these inputs.

[PDoakORNL]

I'm not 100% sure these are completely accurate, it might be better to have the llm write a script but it did find an issue in the structure factor input. Just not 100% sure its the one implied.

<structurefactor>
  attributes : hdf5  source  target  writeionion  writerho
  unsupported: hdf5  report  writeionion  writerho

Between this and the deep_unsupported_param I noticed that there is disagreement in whether the attribute name is "hdf5" or "writehdf5"

The input grammar as represented in the input class is "hdf5" and that is also what the test program has for valid input xml, but the class actually reads "writehdf5" if present. This is an issue with the whole setIfInInput scheme which should have been backstopped by the unit test. I need to do a scan of all the input classes to see if more of these mismatches are getting through.

[PDoakORNL]

Ok to avoid burying the lede:
I haven't seen a false positive i.e. something it marked as unsupported that really is supported.

I do see many inputs marked unsupported that were never a part of the input grammar for a batched estimator input or its children. If I understand what this is about I think that is ok. So if you can tolerate false negatives i.e. something marked as unsupported that was never considering by the estimator then I see no trouble here.

Ok the TLDR part:
There do seem to be some errors wrt estimators that aren't being labeled as legacy that are, I don't think that has any impact on the python of the PR.

I now feel like a one to one comparison between the input class defined grammar of the new estimators and nexus needs to be done either by a human or with a deterministic tool. I think that would be a next step and would focus on removing particular inputs for particular estimators completely or the smaller possibility there could be input options that nexus doesn't know about.

I've got a couple of threads to pull from the surprising unsupported inputs with respect to cleanup of input classes that don't reflect design decisions that were made later a particular port. Since the PR sequence was InputClass+Testing->Estimator+Testing->Application+Estimator+Testing I can see now that there was a couple of missing steps ReviseInputClass and UpdateNexus.

[ye-luo]

I asked myself where I should be able to locate the up-to-date list of input tags as a user. My immediate answer is the user guide. However, we all know that our user-guide is not up-to-date and complete.

Finding all the input parameters from the source code is an easy task by AI. Once we have a list, enforcing tests and user-guide consistency will be feasible. For example, if a class interacts with xml, corresponding unit test should be required. Not necessarily testing the feature but at least testing the input should be required.

[ye-luo]

____________________________________________________________
<simulation>
  elements   : walkerlogs

walkerlogs is a driver input.

<group>
  attributes : id
  parameters : inner_cutoff
  unsupported: id  mass

id mentioned twice.

input options for bspline sposet is missing. meshfactor, tilematrix...

I didn't read the whole list. Quick spot checks indicate the current list in poor quality.

[jtkrogel]

@ye-luo

These lists were the result of a few deep passes of the source code with AI along with cross-checks against the entire example/test/lab set of known input files as well. The aim was to connect an input tag with evidence (or not) to "live" functionality in QMCPACK. Many times the inputs were simply read and not used at all.

<walkerlogs/> is supplied as a child to <simulation/>, see https://qmcpack.readthedocs.io/en/develop/methods.html#walker-data-logging

The categorizations of the attributes/parameters are not all exclusive. For example, id is supplied as an attribute, but its presence in the unsupported list means that it does absolutely nothing as far as QMCPACK is concerned, i.e. no attached/"live" functionality.

Also, the production and technical categories are meant to identify inputs useful to users in production, and separately for developers in implementation testing/etc (technical).