In the prequel, we introduced Neptune, a Direct3D protocol forwarding layer for VirtIO. Neptune allowed us to serialize Direct3D API calls across the hypervisor boundary and this allowed us to run Wine games on a Linux guest with a Linux host faster than with DXVK directly in the guest. Admittedly, the payoff there was not that exciting but it laid the groundwork for our real goal: modern graphics acceleration for Windows guests. We have now achieved this by building a brand new Windows driver called Triton which along with Neptune brings full DirectX 11 support to QEMU virtual machines.

You might be wondering: if Neptune can serialize Direct3D API calls and Windows uses Direct3D, then aren’t we already done? If Direct3D works in Wine, it should also work in Windows, right? After all, what is Wine, if not a Windows emulator? The short answer is: you sort of can. The Neptune Mesa drivers build a d3d11.dll and dxgi.dll which fully implements the Direct3D API set and so if you just put those files next to the game’s executable, it should load them instead of Windows’ own drivers and you can get some games to run that way. This is also the approach done by previous attempts which used DXVK→Vulkan→Venus to run Direct3D locally inside an application. There are a few disadvantages to this approach. First and most importantly, you cannot get good performance because the window compositor (DWM) “sees” your frame as an image so it needs to use CPU blitting to copy the GPU image buffer to the correct window location. You might be able to do some tricks for full-screen applications to scanout natively but you will never get a smooth desktop experience. Second, because d3d11.dll and dxgi.dll are core components of Windows, you cannot replace the system files themselves and expect Windows to still work. Even if you do manage to get it to work, you will not be able to play many games with anti-cheats which specifically detect this kind of modification. That is why it is only possible to get the DLL to load on a per-application basis (and compatibility varies). Which brings us to the last point: needing to copy files to every application you want working graphics acceleration is not a user friendly experience. The correct approach is not to implement the DirectX APIs but to implement the DirectX DDIs (Device Driver Interface).

In Windows, the application communicates with the system Direct3D and DXGI libraries. The d3d11.dll (as well as older versions) do the complicated work of state tracking and send a more sanitized stream of commands to the user-mode driver (UMD) which implements the DDI. The application also talks to dxgi.dll to initialize the graphics adapters, set up the swapchain, etc. The UMD also goes through the DXGI to talk with the kernel-mode driver (KMD). The KMD is implemented by the graphics vendor (us) to drive the actual hardware (or in our case the virtual hardware). For Wine, we implemented a custom d3d11.dll and dxgi.dll to intercept the API calls and now for Windows, we need to instead implement the UMD and KMD.

So that’s the challenge: implement the UMD with the DirectX DDI interface and also set up a private interface with the KMD which communicates with the VirtIO device.

Lucky for us, the second part has already been solved. Both anonymix007 and arehnman had independently been working on a KMD for Venus (Vulkan). Since Vulkan is a completely independent graphics API, it does not need to implement the DirectX DDI and its UMD is similar to the “replace d3d11.dll” approach in that it talks directly with the KMD to drive commands to QEMU. Since Neptune is modelled after Venus, the high level kernel interfaces (for DMA, command buffers, etc) is very similar and the interface between UMD and KMD is exactly the same. Ultimately, we chose to use anonymix007’s branch as the base because their implementation had more features working on the KMD side.

That leaves us with the hard part. We have to implement the DDI for DirectX 11. When you are designing a new system, it is always wise to understand how others who have come before have solved similar problems. Unfortunately, there are not many open source DDI implementations to draw inspiration from. Windows graphics drivers is a very niche subject and most of the experts work in one of the handful of graphics hardware vendors. This is one of the reasons that QEMU has never got far with Windows GPU acceleration.

Fortunately for us, there are two working open source implementation that we can learn from. First, Mesa has a DirectX 10 UMD. If you did not read the last article, the short version is that Mesa implements OpenGL for Linux. Mesa performs state tracking for OpenGL and emits Gallium API calls. The Mesa DirectX 10 UMD is an alternative to OpenGL that emits the same Gallium API calls. Then the Gallium backend driver (AMD, Intel, VirGL, etc) converts them into native graphics driver APIs. The upstream Mesa only supports the software rasterization backend for DirectX 10 but there was some recent work to get it working with VirGL. Unfortunately, the macOS virglrenderer lacks support for many of the features that this UMD requires so it is not a viable way to get graphics acceleration for macOS hosts. However, the integration with the Mesa codebase provides us with a clean example for integrating Triton.

VirtualBox has the only working open source DirectX 11 UMD. However, this driver cannot really be “adopted” for our use. The way their driver works is that they translate the DDI calls into an intermediate bytecode and then on the host side, they interpret the bytecode into DirectX API calls. While it would be easy (with AI help) to just port this bytecode emitter and interpreter into QEMU, we decided against it for a couple of reasons. First, we think this conversion of DDI into a bytecode and then lifting that bytecode back to DirectX API can result in bugs that limit compatibility with games. Indeed, reading forum threads online, it seems like many games do not run in VirtualBox for this reason. If there is a missing feature or bug in the translation engine, it would require a lot of active maintenance effort to fix and we do not want to depend on Oracle for that. Second, there is a licence incompatibility between VirtualBox’s GPLv3 and virglrenderer’s MIT License or QEMU’s LGPLv2. VirtualBox’s code can’t be integrated but we did learn some valuable insight from it. Their list of which DDI prototypes were implemented and which ones returned error is used as the minimal requirements for a working implementation. This information isn’t readily available from MSDN documentation and trying to implement every single prototype would be massive in scope. Their DXBC signature algorithm is also helpful to understand because Microsoft does not publish it anywhere.

Since we didn’t want to take the VirtualBox approach of using an intermediate transport format for DDI calls, we can do something better. If you imagine d3d11.dll as a component that roughly transforms DirectX API calls into UMD DDI calls, then what we want our UMD to do is to transform the DDI call back to DirectX API calls. Why is this useful? Because then we can use our tested and working Neptune protocol without having to invent a new transport for serializing DDI calls. On the host side, we do not have to do any extra work to execute those calls. VirtualBox needs an emitter and transport layer on the guest as well as an interpreter and dispatcher on the host. Each step adds latency and the chance to introduce errors and incompatibility. We still need an emitter and transport on the guest but on the host side we do not need an interpreter because the deserialized Neptune commands ARE DirectX 11 API calls and can be dispatched without any additional parsing. One less transform step means less opportunity for mistakes. Another advantage of a DDI→API transform is that most DDI calls in D3D11 have an API equivalent meaning that the transform is as simple as mapping some API handles to device handles and sometimes doing a lookup for API→DDI enum differences. The biggest win however, also turns out to be the most complicated part of the story, which is the DXBC shader code.

DXBC (DirectX Byte Code) is the IR code that Microsoft’s shader compiler (FXC) emits. Specifically, it is the older (pre-DirectX 12) format and is compiled from HLSL, the shader language that DirectX uses. Since Triton acts as a reverse transform from DDI to API, it does not need to disassemble and convert this shader bytecode. This is a huge win for us in terms of complexity and compatibility. Unfortunately, it is not as simple as passing the bytecode unmodified to the host.

The compiler (FXC) emits the DXBC bytecode along with other metadata. d3d11.dll expects to see this metadata and consumes it. When the DDI is called, only the bytecode is passed. That means for us to make a valid “inverse transform” back to the API call, we need to re-construct all that metadata by interpreting the bytecode. In the end we still pass through the bytecode unmodified but since we don’t see the original DXContainer file, we have to synthesize fields that the host DirectX renderer expects. This was a lot of trial and error that the AI assistant handled but it is the weakest and most error prone part of our implementation.

Here’s what we have so far:

1. Application makes DirectX and DXGI API calls to the system libraries.
2. System libraries invoke Triton through DDI calls.
3. Triton DDI converts raw DXBC bytecode back into DXContainer and makes DirectX and DXGI API calls to Neptune.
4. Neptune UMD serializes the API calls and passes them through a ring buffer managed by the KMD.
5. KMD uses the VirtIO interface to send commands to the host.
6. QEMU host handles the command and passes Neptune calls to virglrenderer.
7. Neptune host module in virglrenderer deserializes the API calls and forwards them to the host side DirectX implementation.
8. Host DirectX implementation renders the frame.

Let’s zoom in on the last point. Once the DirectX API calls make it to the host, we still need to render it. When we brought up Neptune for Wine on Linux, we forked DXVK to support exporting the swapchain images as DMAbuf resources. At the time, we decided to implement the swapchain on the host side in order to sidestep the issue of shared textures. Internally, swapchain images are represented as textures but these textures are special in that the host needs to be able to find them and use them to show the final frame on screen. Our Wine DXGI library forwards all the swapchain API calls directly to the host and therefore the host “knows” which textures will be used as a backbuffer. Then separate VirGL commands can be used to scan out the texture blob to the VM window. This trades simplicity in the guest driver and minimal changes in DXVK for the complexity of swapchain logic in the host virglrenderer process.

When bringing up Triton, we realized that host side swapchain handling was a mistake. On Windows, DXGI is a system component which talks to the UMD. DXGI handles the backbuffer creation, frame pacing, mode switching, etc. The UMD (mostly) doesn’t give special treatment to DXGI and so our method of doing an “inverse transform” of DDI calls back to API calls does not really work with DXGI. That means all the swapchain logic we added to the host side is largely bypassed. The desktop compositor (DWM) operates on shared textures. One process’s DXGI renders content to its own backbuffer and that backbuffer is shared with the DWM process which draws the desktop, window chrome, etc. The final frame that DWM constructs is set for scan out. This means that in addition to DMAbuf exports, we also need to implement DMAbuf imports in DXVK as well (separate guest contexts map to separate host contexts). Once we have both import and export implemented, there is no need for host side swapchain logic anymore and so to make the Wine driver more unified, we moved all the swapchain logic into the guest Neptune driver. An added benefit of this move is that it more closely tracks with how Venus is designed and so virglrenderer is kept clean.

There’s some challenge in getting virglrenderer working on macOS but since Venus now runs in macOS, most of the backend challenges have been fleshed out. The remaining task is to connect Neptune to a host side DirectX renderer. There are three major projects that can handle the goal of DirectX on macOS. However, they all been designed with running Wine as the main target. They lack the shared textures and shared fences features that Neptune and Triton requires.

DXVK is the project we used on Linux hosts. It translates D3D11 API to Vulkan API and then uses the host Vulkan driver to do the rendering. On Linux, this works great because Vulkan is a first class citizen and all modern graphics hardware have a good Vulkan driver at this point. On macOS though, Vulkan is handled by another translation layer, MoltenVK, which translates Vulkan API to Metal API. In the last post, we talked about the unique challenge of getting DXVK + MoltenVK working and the short version is: it is unstable and requires a lot more work for compatibility.

DXMT sidesteps the Vulkan issue by translating D3D11 directly to Metal (with D3D12 coming soon). Just like DXVK, the project is designed primarily with Wine in mind so the first step was to implement a native variant of the library. With our fork, DXMT can be built as a macOS shared library and exposes some additional exports for import/export of textures and fences.

One major hurdle in the design of dxmt-native is in the implementation of shared textures that can cross the process boundary. We need to cross the process boundary because virglrenderer spawns helper processes for each renderer context so roughly every guest D3D context corresponds to a separate virgl_render_server process. This strict process isolation allows a renderer to crash without taking down the entire VM. Now, we can force the older thread based isolation model and use standard MTLTexture handles across renderer context boundary (and we will have to for the eventual iOS port), but upstream maintainers do not want to support this. Sharing Metal resources across processes can be tricky but there are a few “well supported” way of doing it.

1. MTLSharedTextureHandle + XPC: This is the Apple preferred way but it requires bringing in XPC which is its own can of worms. Both QEMU and virglrenderer use file descriptors and SCM_RIGHTS to pass handles between processes but MTLSharedTextureHandle does not support this. For the best performance though, we should eventually adopt this across virglrenderer, QEMU, and SPICE but for this initial bringup, we want to not make major architectural changes across projects.
2. IOSurface: You can render to an IOSurface and share the global handle with any other process. This is how we implement the accelerated rendering on UTM but it is using technology long deprecated by Apple. You also need to pay the penalty of an additional GPU blit into the IOSurface and that also means having to set up a render pipeline in virglrenderer which adds complexity.
3. CALayerHost: A private API that is used by Chrome and other older macOS apps where rendering is done in a separate process. This is strictly worse than IOSurface in terms of latency (CoreAnimation is higher up the graphics stack) and the reverse (going from CALayer back to MTLTexture) is even more complicated. This technique may work for offline rendering but will not work for shared textures that must be composited.

None of these give us a good way to share textures across different processes using SCM_RIGHTS but a new idea was brought up (during the Venus discussion) by @Drakulix (who was working on bringing Wayland to macOS). Their idea is to use shm_open() to create a shared memory object (which can be represented as a file descriptor that works across SCM_RIGHTS) and then map it to a MTLBuffer using newBufferWithBytesNoCopy:length:options:deallocator:. That gives us a single memory region that can be seen by the CPU and GPU and you can repeat this in the other process as well. All of this works in Apple Silicon because of UMA (Unified Memory Architecture) meaning that CPU and GPU share a single physical address space. The only downside is that you can only do this with linear textures which is not memory efficient. However, as long as the number of shared textures is small, this should not be an issue.

Sharing textures between different contexts in different processes is one half of the equation. The other half is synchronization. When you have producer process A drawing to a shared texture and consumer process B compositing all the shared textures into the final scanout image, you will run into tearing if A is in middle of drawing when B starts compositing. To prevent this, you need fences which allows process A to block while B is drawing and B to block while A is drawing. To make matters more complicated, you need GPU fences because the GPU executes asynchronously to the CPU. Ideally, B’s GPU process can consume A’s GPU fence without any polling by A’s CPU process. The only way to achieve this is to use MTLSharedEventHandle which requires XPC. However, we can get most of the way there with emulated fences by relying on two facts.

1. Most of our shared fence events happen at frame completion boundaries. That means the added latency of a CPU side wait is limited to one fence per completed frame.
2. The fence event producer can execute on the GPU while the fence consumer must wait on the CPU. That means we only need to waste CPU cycles on one side.

The way our emulated fence works is as follows: the producer calls ID3D11DeviceContext::ClearUnorderedAccessViewUint with an address in a shared memory buffer mapped by the consumer process. This API call lets us write an arbitrary integer to shared memory and we use it to write a timeline value. The write is done by the GPU so it is ordered with A’s other draw calls meaning that when the timeline value write occurs in the GPU, we know that all the draws are complete. The consumer must poll on the shared memory (this must be done on the CPU because there is no memory value polling instruction on Apple GPUs). Once it sees the updated timeline value, it knows the draws are complete and that it is safe to consume the texture. The consumer CPU cannot queue its draw calls until it sees the fence event and therefore there is introduced latency while the GPU is potentially idle waiting for the next submission.

The last DirectX API implementation for macOS is made by Apple for the Game Porting Toolkit. Originally designed for developers to test their Windows game on Apple Silicon, the GPT includes D3DMetal.framework, an implementation of D3D11 and D3D12 on top of Metal as well as a transpiler from DXBC/DXIL (Microsoft’s proprietary GPU shader bytecode format) to AIR (Apple’s proprietary GPU shader bytecode format). Just like DXMT, it is designed to work with Wine. Just like DXMT, it does not support shared textures or shared fences so we need to emulate them in the same way. However, unlike DXMT, it is not open source so we need to use swizzling and vtable patching to intercept API calls and change the output.

That is what we have done with d3dmetal-native. It is a wrapper around D3DMetal.framework that allows it to work outside of Wine and support these additional features. Since we designed the API interface to be compatible with DXMT, we can easily switch between the two interfaces in virglrenderer. The result is a significant improvement in performance over DXMT.

Note that since D3DMetal only has an x86_64 slice (as it was intended for use with Rosetta + Wine), we have to run the entire virgl_render_server process in Rosetta. Even then it still outperforms DXMT running on native ARM64.

Unfortunately, D3DMetal’s licence terms explicitly prohibits usage outside of the “sole purpose of developing, testing, or evaluating video games for use on Apple-branded products” and that it can only be distributed “solely for non-commercial purposes.” That means we cannot include D3DMetal as part of a bundled application. Curiously, CrossOver, a commercial Wine distribution, does bundle D3DMetal and I was told that they have a special agreement with Apple in order to do this. If anyone is familiar with this arrangement, please contact us because we would love to include D3DMetal in UTM due to the improvement in performance.

All the work described here is open source so if you like tinkering, you can try it out and give us your feedback. We are actively working to upstream as much of these changes as possible and we will update UTM soon to support these features so anyone can try it without having to compile multiple projects.

Hint: Point your AI to this page and ask it to set it up for you.

You need a Windows machine (or VM) to build the Windows drivers. Full instructions are here and you may want to use build-mesa for the UMD which simplifies the build process. If you just want to test the driver, we have pre-built signed drivers available. Note these drivers are still very unstable and you should not install them on any VM that you care about!