I’ve always wondered why we don’t have a GPU debugger similar to the one used for CPUs. A tool that allows pausing execution and examining the current state. This capability feels essential, especially since the GPU’s concurrent execution model is much harder to reason about. After searching for solutions, I came across rocgdb, a debugger for AMD’s ROCm environment. Unfortunately, its scope is limited to that environment. Still, this shows it’s technically possible. I then found a helpful series of blog posts by Marcell Kiss, detailing how he achieved this, which inspired me to try to recreate the process myself.

The best place to start learning about this is RADV. By tracing what it does, we can find how to do it. Our goal here is to run the most basic shader nop 0 without using Vulkan, aka RADV in our case.

First of all, we need to open the DRM file to establish a connection with the KMD, using a simple open(“/dev/dri/cardX”), then we find that it’s calling amdgpu_device_initialize, which is a function defined in libdrm, which is a library that acts as middleware between user mode drivers(UMD) like RADV and and kernel mode drivers(KMD) like amdgpu driver, and then when we try to do some actual work we have to create a context which can be achieved by calling amdgpu_cs_ctx_create from libdrm again, next up we need to allocate 2 buffers one of them for our code and the other for writing our commands into, we do this by calling a couple of functions, here’s how I do it:

void bo_alloc(amdgpu_t* dev, size_t size, u32 domain, bool uncached, amdgpubo_t* bo) {
 s32    ret         = -1;
 u32    alignment   = 0;
 u32    flags       = 0;
 size_t actual_size = 0;

 amdgpu_bo_handle bo_handle = NULL;
 amdgpu_va_handle va_handle = NULL;
 u64              va_addr   = 0;
 void*            host_addr = NULL;

Here we’re choosing the domain and assigning flags based on the params, some buffers we will need uncached, as we will see:

 if (
   domain != AMDGPU_GEM_DOMAIN_GWS && domain != AMDGPU_GEM_DOMAIN_GDS &&
   domain != AMDGPU_GEM_DOMAIN_OA) {
  actual_size = (size + 4096 - 1) & 0xFFFFFFFFFFFFF000ULL;
  alignment   = 4096;
  flags       = AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED | AMDGPU_GEM_CREATE_VRAM_CLEARED |
          AMDGPU_GEM_CREATE_VM_ALWAYS_VALID;
  flags |=
    uncached ? (domain == AMDGPU_GEM_DOMAIN_GTT) * AMDGPU_GEM_CREATE_CPU_GTT_USWC : 0;
 } else {
  actual_size = size;
  alignment   = 1;
  flags       = AMDGPU_GEM_CREATE_NO_CPU_ACCESS;
 }

 struct amdgpu_bo_alloc_request req = {
  .alloc_size     = actual_size,
  .phys_alignment = alignment,
  .preferred_heap = domain,
  .flags          = flags,
 };

 // memory aquired!!
 ret = amdgpu_bo_alloc(dev->dev_handle, &req, &bo_handle);
 HDB_ASSERT(!ret, "can't allocate bo");

Now we have the memory, we need to map it. I opt to map anything that can be CPU-mapped for ease of use. We have to map the memory to both the GPU and the CPU virtual space. The KMD creates the page table when we open the DRM file, as shown here.

So map it to the GPU VM and, if possible, to the CPU VM as well. Here, at this point, there’s a libdrm function that does all of this setup for us and maps the memory, but I found that even when specifying AMDGPU_VM_MTYPE_UC, it doesn’t always tag the page as uncached, not quite sure if it’s a bug in my code or something in libdrm anyways, the function is amdgpu_bo_va_op, I opted to do it manually here and issue the IOCTL call myself:

 u32 kms_handle = 0;
 amdgpu_bo_export(bo_handle, amdgpu_bo_handle_type_kms, &kms_handle);

 ret = amdgpu_va_range_alloc(
   dev->dev_handle,
   amdgpu_gpu_va_range_general,
   actual_size,
   4096,
   0,
   &va_addr,
   &va_handle,
   0);
 HDB_ASSERT(!ret, "can't allocate VA");

 u64 map_flags =
   AMDGPU_VM_PAGE_EXECUTABLE | AMDGPU_VM_PAGE_READABLE | AMDGPU_VM_PAGE_WRITEABLE;
 map_flags |= uncached ? AMDGPU_VM_MTYPE_UC | AMDGPU_VM_PAGE_NOALLOC : 0;

 struct drm_amdgpu_gem_va va = {
  .handle       = kms_handle,
  .operation    = AMDGPU_VA_OP_MAP,
  .flags        = map_flags,
  .va_address   = va_addr,
  .offset_in_bo = 0,
  .map_size     = actual_size,

 };

 ret = drm_ioctl_write_read(dev->drm_fd, DRM_AMDGPU_GEM_VA, &va, sizeof(va));
 HDB_ASSERT(!ret, "can't map bo in GPU space");
 // ret = amdgpu_bo_va_op(bo_handle, 0, actual_size, va_addr, map_flags,
 // AMDGPU_VA_OP_MAP);

 if (flags & AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED) {
  ret = amdgpu_bo_cpu_map(bo_handle, &host_addr);
  HDB_ASSERT(!ret, "can't map bo in CPU space");

  // AMDGPU_GEM_CREATE_VRAM_CLEARED doesn't really memset the memory to 0 anyways for
  // debug I'll just do it manually for now
  memset(host_addr, 0x0, actual_size);
 }

 *bo = (amdgpubo_t){
  .bo_handle = bo_handle,
  .va_handle = va_handle,
  .va_addr   = va_addr,
  .size      = actual_size,
  .host_addr = host_addr,
 };
}

Now we have the context and 2 buffers. Next, fill those buffers and send our commands to the KMD, which will then forward them to the Command Processor (CP) in the GPU for processing.

Let’s compile our code. We can use clang assembler for that, like this:

# https://gitlab.freedesktop.org/martty/radbg-poc/-/blob/master/ll-as.sh
clang -c -x assembler -target amdgcn-amd-amdhsa -mcpu=gfx1100 -o asm.o "$1"
objdump -h asm.o | grep .text | awk '{print "dd if='asm.o' of='asmc.bin' bs=1 count=$[0x" $3 "] skip=$[0x" $6 "] status=none"}' | bash
#rm asm.o

The bash script compiles the code, and then we’re only interested in the actual machine code, so we use objdump to figure out the offset and the size of the section and copy it to a new file called asmc.bin, then we can just load the file and write its bytes to the CPU-mapped address of the code buffer.

Next up, filling in the commands. This was extremely confusing for me because it’s not well documented. It was mostly learning how RADV does things and trying to do similar things. Also, shout-out to the folks on the Graphics Programming Discord server for helping me, especially Picoduck. The commands are encoded in a special format called PM4 Packets, which has multiple types. We only care about Type 3: each packet has an opcode and the number of bytes it contains.

The first thing we need to do is program the GPU registers, then dispatch the shader. Some of those registers are rsrc[1-3]; those registers are responsible for a number of configurations, pgm_[lo/hi], which hold the pointer to the code buffer and num_thread_[x/y/z]; those are responsible for the number of threads inside a work group. All of those are set using the set shader register packets, and here is how to encode them:

It’s worth mentioning that we can set multiple registers in 1 packet if they’re consecutive.

void pkt3_set_sh_reg(pkt3_packets_t* packets, u32 reg, u32 value) {
 HDB_ASSERT(
   reg >= SI_SH_REG_OFFSET && reg < SI_SH_REG_END,
   "can't set register outside sh registers span");

 // packet header
 da_append(packets, PKT3(PKT3_SET_SH_REG, 1, 0));
 // offset of the register
 da_append(packets, (reg - SI_SH_REG_OFFSET) / 4);
 da_append(packets, value);
}

Then we append the dispatch command:

// we're going for 1 thread since we want the simplest case here.

da_append(&pkt3_packets, PKT3(PKT3_DISPATCH_DIRECT, 3, 0) | PKT3_SHADER_TYPE_S(1));
da_append(&pkt3_packets, 1u);
da_append(&pkt3_packets, 1u);
da_append(&pkt3_packets, 1u);
da_append(&pkt3_packets, dispatch_initiator);

Now we want to write those commands into our buffer and send them to the KMD:

void dev_submit(
  amdgpu_t*         dev,
  pkt3_packets_t*   packets,
  amdgpu_bo_handle* buffers,
  u32               buffers_count,
  amdgpu_submit_t*  submit
) {
 s32        ret = -1;
 amdgpubo_t ib  = { 0 };

 bo_alloc(dev, pkt3_size(packets), AMDGPU_GEM_DOMAIN_GTT, false, &ib);
 bo_upload(&ib, packets->data, pkt3_size(packets));

 amdgpu_bo_handle* bo_handles = // +1 for the indirect buffer
   (amdgpu_bo_handle*)malloc(sizeof(amdgpu_bo_handle) * (buffers_count + 1));

 bo_handles[0] = ib.bo_handle;
 for_range(i, 0, buffers_count) {
  bo_handles[i + 1] = buffers[i];
 }

 amdgpu_bo_list_handle bo_list = NULL;
 ret =
   amdgpu_bo_list_create(dev->dev_handle, buffers_count + 1, bo_handles, NULL, &bo_list);
 HDB_ASSERT(!ret, "can't create a bo list");
 free(bo_handles);

 struct amdgpu_cs_ib_info ib_info = {
  .flags         = 0,
  .ib_mc_address = ib.va_addr,
  .size          = packets->count,
 };

 struct amdgpu_cs_request req = {
  .flags                  = 0,
  .ip_type                = AMDGPU_HW_IP_COMPUTE,
  .ip_instance            = 0,
  .ring                   = 0,
  .resources              = bo_list,
  .number_of_dependencies = 0,
  .dependencies           = NULL,
  .number_of_ibs          = 1,
  .ibs                    = &ib_info,
  .seq_no                 = 0,
  .fence_info             = { 0 },
 };

 ret = amdgpu_cs_submit(dev->ctx_handle, 0, &req, 1);
 HDB_ASSERT(!ret, "can't submit indirect buffer request");

 *submit = (amdgpu_submit_t){
    .ib = ib,
    .bo_list = bo_list,
    .fence = {
      .context = dev->ctx_handle,
      .ip_type = AMDGPU_HW_IP_COMPUTE,
      .ip_instance = 0,
      .ring = 0,
      .fence = req.seq_no,
    },
  };
}

Here is a good point to make a more complex shader that outputs something. For example, writing 1 to a buffer.

No GPU hangs ?! nothing happened ?! cool, cool, now we have a shader that runs on the GPU, what’s next? Let’s try to hang the GPU by pausing the execution, aka make the GPU trap.

The RDNA3’s ISA manual does mention 2 registers, TBA, TMA; here’s how they describe them respectively:

Holds the pointer to the current trap handler program address. Per-VMID register. Bit [63] indicates if the trap handler is present (1) or not (0) and is not considered part of the address (bit[62] is replicated into address bit[63]). Accessed via S_SENDMSG_RTN.

Temporary register for shader operations. For example, it can hold a pointer to memory used by the trap handler.

You can configure the GPU to enter the trap handler when encountering certain exceptions listed in the RDNA3 ISA manual.

We know from Marcell Kiss’s blog posts that we need to compile a trap handler, which is a normal shader the GPU switches to when encountering a s_trap. The TBA register has a special bit that indicates whether the trap handler is enabled.

Since these are privileged registers, we cannot write to them from user space. To bridge this gap for debugging, we can utilize the debugfs interface. Luckily, we have UMR, which uses that debugfs interface, and it’s open source; we copy AMD’s homework here which is great.

The amdgpu KMD has a couple of files in debugfs under /sys/kernel/debug/dri/{PCI address}; one of them is regs2, which is an interface to a amdgpu_debugfs_regs2_write in the kernel that writes to the registers. It works by simply opening the file, seeking the register’s offset, and then writing; it also performs some synchronisation and writes the value correctly. We need to provide more parameters about the register before writing to the file, tho and do that by using an ioctl call. Here are the ioctl arguments:

typedef struct amdgpu_debugfs_regs2_iocdata_v2 {
 __u32 use_srbm, use_grbm, pg_lock;
 struct {
  __u32 se, sh, instance;
 } grbm;
 struct {
  __u32 me, pipe, queue, vmid;
 } srbm;
 __u32 xcc_id;
} regs2_ioc_data_t;

The 2 structs are because there are 2 types of registers, GRBM and SRBM, each of which is banked by different constructs; you can learn more about some of them here in the Linux kernel documentation.

Turns out our registers here are SBRM registers and banked by VMIDs, meaning each VMID has its own TBA and TMA registers. Cool, now we need to figure out the VMID of our process. As far as I understand, VMIDs are a way for the GPU to identify a specific process context, including the page table base address, so the address translation unit can translate a virtual memory address. The context is created when we open the DRM file. They get assigned dynamically at dispatch time, which is a problem for us; we want to write to those registers before dispatch.

We can obtain the VMID of the dispatched process by querying the HW_ID2 register with s_getreg_b32. I do a hack here, by enabling the trap handler in every VMID, and there are 16 of them, the first being special, and used by the KMD and the last 8 allocated to the amdkfd driver. We loop over the remaining VMIDs and write to those registers. This can cause issues to other processes using other VMIDs by enabling trap handlers in them and writing the virtual address of our trap handler, which is only valid within our virtual memory address space. It’s relatively safe tho since most other processes won’t cause a trap¹.

Now we can write to TMA and TBA, here’s the code:

void dev_op_reg32(
  amdgpu_t* dev, gc_11_reg_t reg, regs2_ioc_data_t ioc_data, reg_32_op_t op, u32* value) {
 s32 ret = 0;

 reg_info_t reg_info     = gc_11_regs_infos[reg];
 uint64_t   reg_offset   = gc_11_regs_offsets[reg];
 uint64_t   base_offset  = dev->gc_regs_base_addr[reg_info.soc_index];
 uint64_t   total_offset = (reg_offset + base_offset);

 // seems like we're multiplying by 4 here because the registers database in UMRs
 // source has them in indexes rather than bytes.
 total_offset *= (reg_info.type == REG_MMIO) ? 4 : 1;

 ret = hdb_ioctl(dev->regs2_fd, AMDGPU_DEBUGFS_REGS2_IOC_SET_STATE_V2, &ioc_data);
 HDB_ASSERT(!ret, "Failed to set registers state");

 size_t size = lseek(dev->regs2_fd, total_offset, SEEK_SET);
 HDB_ASSERT(size == total_offset, "Failed to seek register address");

 switch (op) {
 case REG_OP_READ : size = read(dev->regs2_fd, value, 4); break;
 case REG_OP_WRITE: size = write(dev->regs2_fd, value, 4); break;
 default          : HDB_ASSERT(false, "unsupported op");
 }

 HDB_ASSERT(size == 4, "Failed to write/read the values to/from the register");
}

And here’s how we write to TMA and TBA: If you noticed, I’m using bitfields. I use them because working with them is much easier than macros, and while the byte order is not guaranteed by the C spec, it’s guaranteed by System V ABI, which Linux adheres to.

void dev_setup_trap_handler(amdgpu_t* dev, u64 tba, u64 tma) {
 reg_sq_shader_tma_lo_t tma_lo = { .raw = (u32)(tma) };
 reg_sq_shader_tma_hi_t tma_hi = { .raw = (u32)(tma >> 32) };

 reg_sq_shader_tba_lo_t tba_lo = { .raw = (u32)(tba >> 8) };
 reg_sq_shader_tba_hi_t tba_hi = { .raw = (u32)(tba >> 40) };

 tba_hi.trap_en = 1;

 regs2_ioc_data_t ioc_data = {
  .use_srbm = 1,
  .xcc_id   = -1,
 };

 // NOTE(hadi):
 // vmid's get assigned when code starts executing before hand we don't know which vmid
 // will get assigned to our process so we just set all of them
 for_range(i, 1, 9) {
  ioc_data.srbm.vmid = i;
  dev_op_reg32(dev, REG_SQ_SHADER_TBA_LO, ioc_data, REG_OP_WRITE, &tba_lo.raw);
  dev_op_reg32(dev, REG_SQ_SHADER_TBA_HI, ioc_data, REG_OP_WRITE, &tba_hi.raw);

  dev_op_reg32(dev, REG_SQ_SHADER_TMA_LO, ioc_data, REG_OP_WRITE, &tma_lo.raw);
  dev_op_reg32(dev, REG_SQ_SHADER_TMA_HI, ioc_data, REG_OP_WRITE, &tma_hi.raw);
 }
}

Anyway, now that we can write to those registers, if we enable the trap handler correctly, the GPU should hang when we launch our shader if we added s_trap instruction to it, or we enabled the TRAP_ON_START bit in rsrc3² register.

Now, let’s try to write a trap handler.

If you wrote a different shader that outputs to a buffer, u can try writing to that shader from the trap handler, which is nice to make sure it’s actually being run.

We need 2 things: our trap handler and some scratch memory to use when needed, which we will store the address of in the TMA register.

The trap handler is just a normal program running in privileged state, meaning we have access to special registers like TTMP[0-15]. When we enter a trap handler, we need to first ensure that the state of the GPU registers is saved, just as the kernel does for CPU processes when context-switching, by saving a copy of the stable registers and the program counter, etc. The problem, tho, is that we don’t have a stable ABI for GPUs, or at least not one I’m aware of, and compilers use all the registers they can, so we need to save everything.

AMD GPUs’ Command Processors (CPs) have context-switching functionality, and the amdkfd driver does implement some context-switching shaders. The problem is they’re not documented, and we have to figure them out from the amdkfd driver source and from other parts of the driver stack that interact with it, which is a pain in the ass. I kinda did a workaround here since I didn’t find luck understanding how it works, and some other reasons I’ll discuss later in the post.

The workaround here is to use only TTMP registers and a combination of specific instructions to copy the values of some registers, allowing us to use more instructions to copy the remaining registers. The main idea is to make use of the global_store_addtid_b32 instruction, which adds the index of the current thread within the wave to the writing address, aka