Defacing module should write out each rating as it happens

[ericearl]

• visualqc version: 0.6.6
• Python version: 3.10.8
• Operating System: Linux-4.18.0-553.30.1.el8_10.x86_64-x86_64-with-glibc2.28

Description

Describe what you were trying to get done.
Tell us what happened, what went wrong, and what you expected to happen.

I was trying to visually inspect 459 defaced vs original images. This time and many times before, "something" happens which closes the program prematurely before selecting "Quit". All ratings performed since last opening are lost and not written to the running log of visually inspected defacings.

The "something" has been among one of these things in the past. I'm sure there's more possibilities, but I'm just speaking from my experience.

1. I lose my SSH connection to the server where I'm running VisualQC. It might be from an unreliable VPN connection or a server hiccup or a short (or long) lapse in my computer's internet connection.
2. VisualQC encounters an error in rendering the two images overlaid on one another, such as a volume that is 4D instead of strictly 3D.
3. I end the VisualQC session incorrectly by not hitting "Quit" to close it and record my last rating.

What I Did

> vqcdeface -u /data/defacing_QC -i /data/defacing_QC/present.txt -m orig.nii.gz -d defaced.nii.gz -r defaced_render

Defacing module
Time stamp : 2024-12-27 10:59:57

version info: visualqc 0.6.6
numpy 1.24.4 / scipy 1.11.1 / matplotlib 3.6.1
python 3.10.8 | packaged by conda-forge | (main, Nov 22 2022, 08:23:14) [GCC 10.4.0]
platform Linux-4.18.0-553.30.1.el8_10.x86_64-x86_64-with-glibc2.28
#1 SMP Fri Nov 15 03:46:25 EST 2024


        Linux distribution: {'NAME': 'Red Hat Enterprise Linux', 'ID': 'rhel', 'PRETTY_NAME': 'Red Hat Enterprise Linux 8.10 (Ootpa)', 'VERSION': '8.10 (Ootpa)', 'ID_LIKE': 'fedora', 'VERSION_ID': '8.10', 'PLATFORM_ID': 'platform:el8', 'ANSI_COLOR': '0;31', 'CPE_NAME': 'cpe:/o:redhat:enterprise_linux:8::baseos', 'HOME_URL': 'https://www.redhat.com/', 'DOCUMENTATION_URL': 'https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8', 'BUG_REPORT_URL': 'https://issues.redhat.com/', 'REDHAT_BUGZILLA_PRODUCT': 'Red Hat Enterprise Linux 8', 'REDHAT_BUGZILLA_PRODUCT_VERSION': '8.10', 'REDHAT_SUPPORT_PRODUCT': 'Red Hat Enterprise Linux', 'REDHAT_SUPPORT_PRODUCT_VERSION': '8.10'}
459 subjects are usable for review.
Input folder: /data/defacing_QC
Output folder: /data/defacing_QC/visualqc

Restoring ratings from previous session(s), if they exist ..

Ratings for 434 sessions were restored
To be reviewed : 25

...

At one point there is a crash, and I keep having to re-open VisualQC, rate 5-10 defacings, and Quit VisualQC to retain my ratings and avoid re-encountering the failed 4D volume:

Traceback (most recent call last):
  File "/home/earlea/.local/bin/vqcdeface", line 8, in <module>
    sys.exit(main())
  File "/home/earlea/.local/lib/python3.10/site-packages/visualqc/__defacing__.py", line 18, in main
    defacing.cli_run()
  File "/home/earlea/.local/lib/python3.10/site-packages/visualqc/defacing.py", line 702, in cli_run
    wf.run()
  File "/home/earlea/.local/lib/python3.10/site-packages/visualqc/workflows.py", line 98, in run
    self.loop_through_units()
  File "/home/earlea/.local/lib/python3.10/site-packages/visualqc/workflows.py", line 256, in loop_through_units
    skip_subject = self.load_unit(unit_id)
  File "/home/earlea/.local/lib/python3.10/site-packages/visualqc/defacing.py", line 419, in load_unit
    self.defaced_img, self.defaced_hdr = read_image(
  File "/home/earlea/.local/lib/python3.10/site-packages/visualqc/utils.py", line 47, in read_image
    img = check_image_is_3d(img)
  File "/home/earlea/.local/lib/python3.10/site-packages/visualqc/utils.py", line 318, in check_image_is_3d
    raise ValueError('Input volume is 4D with more than one volume!')
ValueError: Input volume is 4D with more than one volume!

Suggested solution

I think the solution is to write out ratings as they occur, so no matter if you walk away and lose connection or encounter an error part-way through, the work you've done rating is preserved after each defacing QC rating.

As for the 4D volume I encountered, that's my bad and I will resolve it. But if a show-stopping error happens in VisualQC's defacing module, I do think it would be better to write out a rating file as "Errored" or something like that and include the Error message in the "Notes" of that rating.

Thanks again for your awesome support of this awesome tool! And a soon happy new year to you!

[raamana]

thanks Eric - its a problem indeed, esp. when you do it over SSH, which can increase the chances of something going wrong.

I thought about writing ratings to disk as they happen, but haven't had a chance to fully implement and test them. But just to make sure you noticed, I do print them out in the terminal, and when VisualQC does quit unexpectedly, you can always take the log from the terminal and do some regex to recover the ratings and notes you need! its not ideal - but still a reasonable fallback. It's smarter to quit it every few hours or so (when doing it locally) to ensure ratings are saved to disk.

I will look into this and see if I can get it done quickly

[ericearl]

Awesome! Thanks @raamana ! And good advice to just pull the ratings from the terminal if you have to (for now)... Good luck making that fix. If you don't want to make it alone, I'd be happy to meet sometime and learn the codebase from you enough to contribute better than just raising issues at you?