Fix issue with handling of failure during discard of metadata cache entries

[jhendersonHDF]

When discarding a metadata cache entry after flushing it, errors during the discard process could cause the library to skip calling the 'free_icr' callback for the entry. This could result in resource leaks and the inability of the cache to be fully flushed and closed due to issues such as pinned entries remaining in the cache. This has been fixed by noting errors during the discard process, but attempting to fully free a cache entry before signalling that an error has occurred.

Fixes CVE-2025-7068

Fixes #5578
Also fixes #4586

---

Important

Fixes resource leak issue in metadata cache discard process, addressing CVE-2025-7068 by ensuring proper cleanup in H5Centry.c.

• Behavior:
  • Fixes issue in H5C__flush_single_entry() in H5Centry.c where errors during discard could skip free_icr callback, causing resource leaks.
  • Introduces H5C__discard_single_entry() to handle entry discard, ensuring cleanup even on errors.
• Security:
  • Fixes CVE-2025-7068, preventing resource leaks and ensuring cache can be fully flushed and closed.
• Documentation:
  • Updates RELEASE.txt to include fix for CVE-2025-7068 and GitHub issue [BUG] Memory leaks in H5FL__malloc at src/H5FL.c:211:30 #5578.

^{This description was created by for 749a292. You can customize this summary. It will automatically update as commits are pushed.}

[jhendersonHDF]

Context:
When opening a corrupted HDF5 file with read-write access, the process of reading the file superblock checks if a message exists inside the superblock extension object header using H5O_msg_exists, which causes H5O_protect to load an object header continuation chunk into the cache and deserialize it, increasing the reference count on the superblock extension object header and pinning it in the cache as part of H5O__inc_rc. A bit later during the file superblock reading process, the library finds a message in the superblock extension indicating that there is a metadata cache image to load and calls H5AC_load_cache_image_on_next_protect to have the metadata cache load it on the next H5C_protect call. When attempting to open a dataset in the corrupted file, the library eventually tries to load the cache image as part of H5G__obj_lookup, but fails. During the process of trying to load the cache image, the library is unable to delete the cache image by removing the cache image message from the superblock extension due to corruption in the file free space manager. When the metadata cache fails to free the file space for the relevant object header chunk, it throws an error and skips over calling the free_icr callback for the chunk, eventually leaving the superblock extension object header pinned in the cache with an increased reference count. When the file is closed later on, the superblock extension object header can't be flushed, leaking the memory for the chunk proxy structure allocated for the continuation chunk during H5O_protect.

[fortnern]

We should make sure fsf_size is set to 0 if this fails, since it gets passed to H5MM_xfree. Either set it here or initialize it to 0.

[fortnern]

Or simply skip calling H5MM_xfree if fsf_size fails

[jhendersonHDF]

Do you mean the call to H5MF_xfree below? If so, I added the (ret_value >= 0) for that case. A little hacky, but figured it was best to skip.

[fortnern]

See comment but looks good otherwise

[bmribler]

Is there any chance cache_ptr->last_entry_removed_ptr is pointing to something else?

[jhendersonHDF]

I would like to say no, but I didn't validate any of the comments here.

[bmribler]

See my other comments

[brtnfld]

@fortnern @bmribler @mattjala, can we get this approved as soon as possible, as it should resolve other CVE issues? If you can not approve it, please be clear as to why.

[bmribler]

@fortnern @mattjala Could you please re-review and re-approve this when you get the chance?