[MacOS] Godot crashes upon close when using Metal due to non-existent cache entry

[nikitalita]

Tested versions

master @ f964fa7

Not present in 4.7-stable

System information

Godot v4.8.dev - macOS Tahoe (26.4.1) - Multi-window, 3 monitors - Metal (Forward+) - integrated Apple M4 Max (Apple9) - Apple M4 Max (16 threads) - 64.00 GiB memory - CoreAudio (44100 Hz, Surround 3.1)

Issue description

Godot crashes upon close when using the Metal driver and any shaders were instantiated (i.e. anything other than the project picker).

I suspect that this happened in #119123.

The culprit seems to be here; MDLibrary is hanging onto a pointer to a freed ShaderCacheEntry:

Call stack:

HashMap<SHA256Digest, ShaderCacheEntry*, HashMapHasherDefault, HashMapComparatorDefault<SHA256Digest, void>, DefaultTypedAllocator<HashMapElement<SHA256Digest, ShaderCacheEntry*>>>::_lookup_idx(SHA256Digest const&, unsigned int&) const (/Users/nikita/Workspace/godot-ws/godot-master/core/templates/hash_map.h:111)
HashMap<SHA256Digest, ShaderCacheEntry*, HashMapHasherDefault, HashMapComparatorDefault<SHA256Digest, void>, DefaultTypedAllocator<HashMapElement<SHA256Digest, ShaderCacheEntry*>>>::getptr(SHA256Digest const&) (/Users/nikita/Workspace/godot-ws/godot-master/core/templates/hash_map.h:312)
RenderingDeviceDriverMetal::shader_cache_free_entry(SHA256Digest const&) (/Users/nikita/Workspace/godot-ws/godot-master/drivers/metal/rendering_device_driver_metal.cpp:1047)
ShaderCacheEntry::notify_free() const (/Users/nikita/Workspace/godot-ws/godot-master/drivers/metal/metal_objects_shared.cpp:645)
MDLibrary::~MDLibrary() (/Users/nikita/Workspace/godot-ws/godot-master/drivers/metal/metal_objects_shared.cpp:663)
MDImmediateLibrary::~MDImmediateLibrary() (/Users/nikita/Workspace/godot-ws/godot-master/drivers/metal/metal_objects_shared.cpp:806)
MDImmediateLibrary::~MDImmediateLibrary() (/Users/nikita/Workspace/godot-ws/godot-master/drivers/metal/metal_objects_shared.cpp:803)
void std::__1::__destroy_at[abi:ne200100]<MDImmediateLibrary, 0>(MDImmediateLibrary*) (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX26.2.sdk/usr/include/c++/v1/__memory/construct_at.h:66)
void std::__1::allocator_traits<std::__1::allocator<MDImmediateLibrary>>::destroy[abi:ne200100]<MDImmediateLibrary, void, 0>(std::__1::allocator<MDImmediateLibrary>&, MDImmediateLibrary*) (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX26.2.sdk/usr/include/c++/v1/__memory/allocator_traits.h:329)
void std::__1::__shared_ptr_emplace<MDImmediateLibrary, std::__1::allocator<MDImmediateLibrary>>::__on_zero_shared_impl[abi:ne200100]<std::__1::allocator<MDImmediateLibrary>, 0>() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX26.2.sdk/usr/include/c++/v1/__memory/shared_ptr.h:180)
std::__1::__shared_ptr_emplace<MDImmediateLibrary, std::__1::allocator<MDImmediateLibrary>>::__on_zero_shared() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX26.2.sdk/usr/include/c++/v1/__memory/shared_ptr.h:183)
std::__1::__shared_count::__release_shared[abi:ne200100]() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX26.2.sdk/usr/include/c++/v1/__memory/shared_count.h:91)
std::__1::__shared_weak_count::__release_shared[abi:ne200100]() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX26.2.sdk/usr/include/c++/v1/__memory/shared_count.h:120)
std::__1::shared_ptr<MDLibrary>::~shared_ptr[abi:ne200100]() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX26.2.sdk/usr/include/c++/v1/__memory/shared_ptr.h:558)
std::__1::shared_ptr<MDLibrary>::~shared_ptr[abi:ne200100]() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX26.2.sdk/usr/include/c++/v1/__memory/shared_ptr.h:556)
MDRenderShader::~MDRenderShader() (/Users/nikita/Workspace/godot-ws/godot-master/drivers/metal/metal_objects_shared.h:926)
MDRenderShader::~MDRenderShader() (/Users/nikita/Workspace/godot-ws/godot-master/drivers/metal/metal_objects_shared.h:926)
MDRenderShader::~MDRenderShader() (/Users/nikita/Workspace/godot-ws/godot-master/drivers/metal/metal_objects_shared.h:926)
RenderingDeviceDriverMetal::shader_free(RenderingDeviceDriver::ShaderID) (/Users/nikita/Workspace/godot-ws/godot-master/drivers/metal/rendering_device_driver_metal.cpp:1262)
RenderingDevice::_free_pending_resources(int) (/Users/nikita/Workspace/godot-ws/godot-master/servers/rendering/rendering_device.cpp:7965)
RenderingDevice::_begin_frame(bool) (/Users/nikita/Workspace/godot-ws/godot-master/servers/rendering/rendering_device.cpp:8066)
RenderingDevice::_flush_and_stall_for_all_frames(bool) (/Users/nikita/Workspace/godot-ws/godot-master/servers/rendering/rendering_device.cpp:8303)
RenderingDevice::screen_free(int) (/Users/nikita/Workspace/godot-ws/godot-master/servers/rendering/rendering_device.cpp:5492)
DisplayServerMacOS::window_destroy(int) (/Users/nikita/Workspace/godot-ws/godot-master/platform/macos/display_server_macos.mm:786)
-[GodotWindowDelegate windowWillClose:] (/Users/nikita/Workspace/godot-ws/godot-master/platform/macos/godot_window_delegate.mm:83)
__CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__ (__CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__:38)
___CFXRegistrationPost_block_invoke (___CFXRegistrationPost_block_invoke:24)
_CFXRegistrationPost (_CFXRegistrationPost:111)
_CFXNotificationPost (_CFXNotificationPost:186)
-[NSNotificationCenter postNotificationName:object:userInfo:] (-[NSNotificationCenter postNotificationName:object:userInfo:]:23)
-[NSWindow _finishClosingWindow] (-[NSWindow _finishClosingWindow]:77)
-[NSWindow _close] (-[NSWindow _close]:96)
DisplayServerMacOS::~DisplayServerMacOS() (/Users/nikita/Workspace/godot-ws/godot-master/platform/macos/display_server_macos.mm:3948)
DisplayServerMacOS::~DisplayServerMacOS() (/Users/nikita/Workspace/godot-ws/godot-master/platform/macos/display_server_macos.mm:3931)
void memdelete<DisplayServer>(DisplayServer*) (/Users/nikita/Workspace/godot-ws/godot-master/core/os/memory.h:163)
finalize_display() (/Users/nikita/Workspace/godot-ws/godot-master/main/main.cpp:417)
Main::cleanup(bool) (/Users/nikita/Workspace/godot-ws/godot-master/main/main.cpp:5332)
OS_MacOS_NSApp::cleanup() (/Users/nikita/Workspace/godot-ws/godot-master/platform/macos/os_macos.mm:1201)
-[GodotApplicationDelegate applicationWillTerminate:] (/Users/nikita/Workspace/godot-ws/godot-master/platform/macos/godot_application_delegate.mm:315)
__CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__ (__CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__:38)

Steps to reproduce

Open literally any project in the editor
Open any scene and open the 3D panel to ensure that a shader is instantiated (which the 3D panel always does, regardless of scene)
Close the editor
Observe crash

Minimal reproduction project (MRP)

Happens with any project if you use the Metal rendering driver.

[nikitalita]

cc @blueskythlikesclouds

[nikitalita]

Confirmed that it is #119123 , reverting that commit fixes the issue.

Also getting the same crash when loading a scene that contains a shader with load_threaded_get()

[stuartcarnie]

It is a bug in the Metal driver.

We should be protecting access to the _shader_cache, which is likely getting corrupted if multiple threads are calling shader_create_from_container concurrently.