Skip to content

Nexus: Add check for Python errors and PySCF errors in PySCF runs#6023

Merged
prckent merged 5 commits into
QMCPACK:developfrom
brockdyer03:catch-pyscf-fails
Jul 8, 2026
Merged

Nexus: Add check for Python errors and PySCF errors in PySCF runs#6023
prckent merged 5 commits into
QMCPACK:developfrom
brockdyer03:catch-pyscf-fails

Conversation

@brockdyer03

Copy link
Copy Markdown
Contributor

Proposed changes

This PR adds a basic check for Python errors and PySCF errors in PySCF runs. This should be able to catch errors in the case that you are running a PySCF calculation, as long as Nexus recognizes that the process has finished.

This may not fix #5989, as there is a difference between checking if the run has failed versus checking if the run is finished. I can't reproduce the behavior in #5989 locally, so it is hard to work out a fix.

What type(s) of changes does this code introduce?

  • New feature

Does this introduce a breaking change?

  • No

What systems has this change been tested on?

Laptop, Fedora Linux 43 (KDE Plasma Desktop Edition)
AMD Ryzen 7 PRO 7840U (8 cores, 16 logical processors)

Python        3.14.5
uv            0.11.26
cif2cell      2.1.0
coverage      7.15.0
h5py          3.16.0
matplotlib    3.11.0
numpy         2.5.1
pycifrw       4.4.6
pydot         4.0.1
pytest        9.1.1
pytest-cov    7.1.0
pytest-order  1.5.0
scipy         1.18.0
seekpath      2.2.1
spglib        2.7.0
sphinx        9.1.0

Checklist

    • I have read the pull request guidance and develop docs
    • This PR is up to date with the current state of 'develop'

@brockdyer03 brockdyer03 requested review from jtkrogel and prckent July 6, 2026 17:43
@brockdyer03 brockdyer03 self-assigned this Jul 6, 2026
@brockdyer03 brockdyer03 added enhancement nexus python Pull requests that update python code labels Jul 6, 2026

@prckent prckent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it practical to test an actual failure case?
e.g. in the template I was expecting to see something like "from pyscf import nonexistentfeature"

A real world case would be someone requesting one of the optional pyscf features but not having it installed. e.g. One of the many vdw packages or the geometry optimizers.

@brockdyer03

Copy link
Copy Markdown
Contributor Author

@prckent I've updated the test to use the case that a user has installed PySCF, but not installed pyscf-dispersion, and then tries to import it.

@jtkrogel jtkrogel left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like it will catch most raised Python errors. I'm all for it.

Much harder to catch all the ways pyscf can fail to produce expected output (convergence failures, etc), but this is a good step.

Comment thread nexus/nexus/tests/test_pyscf_simulation.py Outdated
Comment thread nexus/nexus/pyscf_sim.py Outdated

@prckent prckent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. Jaron's indentation requests have been addressed; will double check with him before merging. Edit: OK received.

I also tested this "by hand". It is a big improvement, thanks Brock! This will significantly improve both user friendliness and robustness.

e.g. It we try running nexus/nexus/examples/pyscf/01_h2o_hf/h2o_ae_hf.py without any pyscf available:

$ ./h2o_ae_hf.py

_____________________________________________________

                     Nexus 2.3.9

        (c) Copyright 2012-  Nexus developers

                     Please cite:
  J. T. Krogel Comput. Phys. Commun. 198 154 (2016)
     https://doi.org/10.1016/j.cpc.2015.08.012
_____________________________________________________


Checking current machine for Nexus dependencies...

  Currently Available Nexus Dependencies:
    Python     = 3.12.3
    numpy      = 1.26.4      (required)
    scipy      = 1.11.4      (optional)
    h5py       = 3.10.0      (optional)
    matplotlib = 3.6.3       (optional)
    spglib     = Unavailable (optional)
    cif2cell   = Unavailable (optional)
    pydot      = Unavailable (optional)
    seekpath   = Unavailable (optional)

...
Project starting
  checking for file collisions
  loading cascade images
    cascade 0 checking in
  checking cascade dependencies
    all simulation dependencies satisfied

  starting runs:
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  elapsed time 0.0 s  memory 107.61 MB
    Entering ./runs/h2o_ae_hf 0
      writing input files  0 scf
    Entering ./runs/h2o_ae_hf 0
      sending required files  0 scf
      submitting job  0 scf
    Entering ./runs/h2o_ae_hf 0
      Executing:
        export OMP_NUM_THREADS=1
        python3 scf.py

  elapsed time 3.0 s  memory 107.61 MB

        PySCF run failed!
        Error File Contents:

          Traceback (most recent call last):
            File "... qmcpack/nexus/nexus/examples/pyscf/01_h2o_hf/runs/h2o_ae_hf/scf.py", line 3, in <module>
              from pyscf import scf
          ModuleNotFoundError: No module named 'pyscf'


    Entering ./runs/h2o_ae_hf 0
      copying results  0 scf
    Entering ./runs/h2o_ae_hf 0
      analyzing  0 scf

Project finished

Very importantly given e.g. #5605 , the job is correctly seen as Failed:

$ ./h2o_ae_hf.py --status_only
...
  cascade status
    setup, sent_files, submitted, finished, got_output, analyzed, failed
    111111  1  3749      scf     ./runs/h2o_ae_hf
    setup, sent_files, submitted, finished, got_output, analyzed, failed.

and if we run the workflow again nothing is done:

$ ./h2o_ae_hf.py
...
Project starting
  checking for file collisions
  loading cascade images
    cascade 0 checking in
  checking cascade dependencies
    all simulation dependencies satisfied

  starting runs:
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  elapsed time 0.0 s  memory 107.84 MB
Project finished

@prckent

prckent commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Test this please

@prckent prckent dismissed jtkrogel’s stale review July 8, 2026 17:08

OK given elsewhere.

@prckent prckent merged commit 0a885b3 into QMCPACK:develop Jul 8, 2026
50 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement nexus python Pull requests that update python code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nexus: unavailable pyscf in h2o_ae_hf_qmc.py not caught. Runs forever.

3 participants