Close resources in h5dumpgentest properly

[mattjala]

Several resources not being closed until library shutdown is the root cause of the Cache VOL CI failures.

---

Important

Ensure proper resource closure in h5dumpgentest.c to prevent Cache VOL CI failures, with updated function signatures and error handling.

• Resource Management:
  • Ensure proper closure of resources in gent_array1_big, gent_nested_compound_dt, and gent_intscalars in h5dumpgentest.c.
  • Add error handling and cleanup logic to prevent resource leaks.
• Function Signature Changes:
  • Change return type of gent_nested_compound_dt and gent_intscalars to herr_t in h5dumpgentest.c and h5dumpgentest.h.
• Misc:
  • Update gen_h5dump_files in h5gentest.c to handle new return types for error counting.

^{This description was created by for f065c35. You can customize this summary. It will automatically update as commits are pushed.}