New features available in Metal 4

[squidbus]

Metal 4 was just announced at WWDC 2025 with some new features that may be of interest to MoltenVK.

Updated feature set table: https://developer.apple.com/metal/Metal-Feature-Set-Tables.pdf

Of particular interest:

• Placement sparse textures and buffers
• Command barriers

[spnda]

This site seems to give a good overview of all new Metal 4 features and API changes: https://developer.apple.com/documentation/metal/understanding-the-metal-4-core-api
The opening sentence of that website makes me happy somehow, since Apple acknowledges the existence of DX and Vulkan explicitly:

Metal 4 improves runtime performance and memory efficiency through its underlying implementations, while making it easier to adapt your apps and games from other platforms, such as DirectX and Vulkan.

• Metal 4 is based on C++17, and allows lambdas
• Most new types have been moved into a MTL4 namespace, and some newer versions of existing types have been moved here, e.g. MTL4RenderPass, or MTL4Pipeline.
• Asynchronous compilation using MTL4Compiler
• MTL4Archive looks like a replacement for MTLBinaryArchive, however this still seems to take in a URL instead of bytes, sadly.
• Tensors, which are used with the MTL4MachineLearningPipeline
• All of these types seem to make it easier to model the Vulkan command model on top of Metal, in every way shape and form. It gets rid of specialized encoders such as MTLBlitCommandEncoder and MTLAccelerationStructureCommandEncoder and fuses them together into one MTL4ComputeCommandEncoder.
• The command buffer model also now more closely follows the Vulkan model, with a MTL4CommandBuffer, MTL4CommandQueue, and MTL4CommandAllocator, with the last serving effectively as a VkCommandPool, while the other two are VkCommandBuffer and VkQueue, respectively.
• The concept of descriptor sets now exists with MTL4ArgumentTable, which avoids having to set the bindings every time you encode a command buffer
• Automatic hazard tracking has been removed for all the MTL4 pipelines/command encoders. People now need to do perform manual synchronization using the barrier methods
• Actual texture views using MTLTextureViewPool
• Metal uses "macOS 16" and "iOS 19" everywhere lol

[K0bin]

This would be a great opportunity to clean up MoltenVK:

• Remove support for x86 Macs (so all the managed resource mess can be removed)
• Always use argument buffers
• Remove support for automatic resource tracking
• Move over to Metal 4
• Always encode when the equivalent Vulkan functions get called rather than offering 3 different modes (encode immediately, encode in vkCmdEndCommandBuffer, encode at submission time)

I don't know how feasible this is for MoltenVK users though.

[squidbus]

I don’t know if non-Apple GPU needs to be supported further or we could just drop it and direct to the last supported version.

But for Metal 4, I think the challenge will be continuing to support a reasonable range of OS versions. So there may be some mess supporting both paths for a while.

[spnda]

Also I wanted to mention that it seems like beyond residency sets there's no other option to convey residency information to the Metal driver, meaning residency sets are now required to be used with any of the MTL4 stuff.

Chiming in on the cleanup/rewrite thing: I could very well see the time coming where we drop support for Metal 1 and 2 and Intel GPUs in a newer version, but keep support for Metal 3. All Apple Silicon chips on macOS will support Metal 4, so it's only a question of supporting old mobile devices.

As I mentioned before, I am generally very much for a MoltenVK that is based on the newest Metal version by default, and only falls back to using Metal 3 (or older) for OSes or devices that do not support these. This should allow us to be able to provide the best possible performance and compatibility for up-to-date devices. Especially in this case, I believe if we do not switch to the new Metal 4 command and binding APIs, MoltenVK could have major performance issues from using the older APIs, especially on any new OSs or Apple GPUs that are likely tailored towards this API over the old ones. Furthermore, the day when Metal 1 and Metal 2 are deprecated or even unusable is probably not too far away given the recent reworks of Metal 3 and Metal 4. I think it's important to remember that Metal 1 is from 2014 and carries a lot of technical debt with it.

[squidbus]

Metal 3 sounds like a reasonable target to me at this point. Since you seem to have looked into the new APIs a fair bit, do you have an idea of how challenging it will be to support both Metal 3 and 4 in the same codebase? In terms of how much logic can be reused between the two versus completely separate code paths and resource management, for example. As with all the new structures and ways of doing things more aligned with other APIs I’m not sure how much overlap there is now.

[K0bin]

do you have an idea of how challenging it will be to support both Metal 3 and 4 in the same codebase

Metal 4 has new protocols for:

• MTL4CommandQueue replaces MTLCommandQueue
• MTL4CommandBuffer replaces MTLCommandBuffer
• MTL4RenderCommandEncoder replaces MTLRenderCommandEncoder
• MTL4ComputeCommandEncoder replaces MTLComputeCommandEncoder, MTLBlitCommandEncoder, MTLAccelerationStructureCommandEncoder.

So it would come with significant maintenance overhead to have two code paths for all of that.
Probably easier and cleaner to even keep those in different code bases or branches.

[squidbus]

Ah, I kind of expected the new versions to at least subclass the old versions, but it appears not.

[K0bin]

Should be doable though as long as you'd go all-in on Argument buffers. That would be shared between the Metal 3 command buffer implementation and the Metal 4 one.

Besides that MoltenVK could easily make up for the extra maintenance overhead by removing the encode-at-endCmdBuffer and encode-at-submit modes + all the command serialization.

[squidbus]

Related to the Intel GPU conversation, it appears that in the Platform State of the Union, the announced that Tahoe would be the final major update for any Intel Mac.

[cdavis5e]

There's more. Metal 4 doesn't support D3D11-style tessellation. You have to use mesh shaders. We need to finish up the implementation of vertex-pipeline shaders on top of Mesh shaders to be able to use Metal 4.

[squidbus]

There's more. Metal 4 doesn't support D3D11-style tessellation. You have to use mesh shaders. We need to finish up the implementation of vertex-pipeline shaders on top of Mesh shaders to be able to use Metal 4.

The docs seem to indicate you can mix MTL and MTL4 usage to incrementally migrate, so I assume at worst you could still fall back to older constructs to build a tessellation pipeline, while still using newer stuff where possible.

[K0bin]

The docs seem to indicate you can mix MTL and MTL4 usage to incrementally migrate, so I assume at worst you could still fall back to older constructs to build a tessellation pipeline, while still using newer stuff where possible.

You'd have to end the Metal 4 command buffer, start a Metal 3 command buffer, do the tessellated draw, end the Metal 3 command buffer, start a new Metal 4 command buffer.

[billhollings]

There's more. Metal 4 doesn't support D3D11-style tessellation. You have to use mesh shaders. We need to finish up the implementation of vertex-pipeline shaders on top of Mesh shaders to be able to use Metal 4.

Finish up?

Has it been started somewhere? 👀

[billhollings]

I'm wondering if the new flexible pipeline state feature would allow us to get an approximation of VK_EXT_graphics_pipeline_library or even VK_EXT_shader_object? That would be really helpful.

From my quick scan, in both cases, it seems the main sticking point will mostly be the vertex input state info content in VkPipelineVertexInputStateCreateInfo.

And it looks like final compiled pipelines are always required, so we'd be looking at some kind of internal hash-n-cache design of fully-compiled pipelines. But at least the final compilations would be much faster, and cause fewer hitches.

[billhollings]

Thinking out loud on the topic of backwards compatibility...

Metal4 is supported on all Apple Silicon Macs, and as far back as A14 on iOS, which is pretty impressive. And non-AS Macs are now 5 years old. So I don't think hardware is an issue.

Some level of backwards compatibility would be required to support older OS's for the next several years I think. Although game and engine devs could certainly opt to abandon their customers who don't upgrade.

The good news is that the overall class structure within MoltenVK is effectively based on Vulkan, not Metal, which gives us a couple of options.

1. My first choice would be to keep one main branch, and create *Metal4 subclasses of key objects such as MVKQueue, MVKGraphicsPipeline, MVKCommandEncoder, and maybe MVKDescriptorSet.

   We'd have to require that any improvements and Vulkan extensions and that don't explicitly need Metal4 tech be added to the parent subclasses.

   And maybe its a MVKConfiguration option to force not to use Metal4, say for an app that needs tessellation shader support.

2. As a second choice (and its a much lower preference for me), we could possibly spawn a Metal4 branch, on which the underlying Metal objects are replaced with Metal4 objects, but would still be generally synchronizable with the main branch.

   We'd have to require that any improvements and Vulkan extensions and that don't explicitly need Metal4 support be added to the main branch, constantly sync main into Metal4, and make the Metal4 branch only about Metal objects.

   And as @cdavis5e points out, to support tessellation shaders, we'd need to figure out mesh shaders as part of it, and build mesh shaders from tessellation shaders. Or maybe that's part of legacy management choice between the two branches that an app dev makes.

   And at some point (years) down the road, when we don't care about legacy at all, we could swap the branches, and rename main to legacy, and Metal4 to main.