I have been into the vulnerability research field for a while now, and VirtualBox is my very first target. I have learned a lot along the way and I hope that anyone who are interested in escaping hypervisors can find something useful from these notes. I assume that you have some basic knowledge on memory corruption, hypervisor architecture and device I/O.

The code snippets here are based on VirtualBox 6.1.4.

VirtualBox Compilation on Windows

The VirtualBox user-mode processes can be used to compromise the kernel so they implement a kernel module to protect the user-mode processes and the driver itself. So we can’t simply attach any debugger to the VM process and debug it. Thanks to @_niklasb, I know there are two ways to work around with it:

  • Using nested VMs, run VirtualBox inside a VMWare VM then use a remote kernel debugger.
  • Build a custom VirtualBox version without process hardening. We also have the debug symbols this way.

Follow the steps in this guide to build VirtualBox – it would save you a lot of time.

Finding Interesting Code Parts

One of the most frequently question that people asked me is where to find bugs in such a large codebase. For me, the answer is to look at the code parts that process the guest OS supplied data. In a nutshell, VirtualBox (or hypervisor) has a set of many emulated devices, some are common standard devices (Intel 82540EM Ethernet Controller, Intel HD Audio Controller, Open Host Controller Interface, etc), some are VirtualBox specific devices (VMM/Host communication device). Those emulated devices communicate with the Guest OS to process data using the device-specific protocols, our goal is to manipulate those communications, break their protocols and put the devices into a weird state.

So to exploit an emulated device, you would need to write a kernel module to interact with it then set the device’s registers properly and trigger the data processing. Here are some suggestions where the devices process guest supplied data (mostly is a binary blob which presents some data structures):

RT_UNTRUSTED_VOLATILE_GUEST Keyword

From the definition:

VirtualBox-6.1.4\include\iprt\cdefs.h:1678

/** @def RT_UNTRUSTED_VOLATILE_GUEST
 * For marking volatile data shared with the guest as untrusted.
 * This is more than just documentation as it specifies the 'volatile' keyword,
 * because the guest could modify the data at any time. */
#define RT_UNTRUSTED_VOLATILE_GUEST             volatile

So any variable declared with the RT_UNTRUSTED_VOLATILE_GUEST keyword should contain volatile data shared with the guest OS and the guest could modify the data at any time. Search for this keyword in the source code directory should reveal many functions that use untrusted guest data:

VirtualBox-6.1.4\src\VBox\Devices\Graphics\DevVGA-SVGA.cpp:3430

static DECLCALLBACK(int) vmsvgaR3FifoLoop(PPDMDEVINS pDevIns, PPDMTHREAD pThread)
{
	...    
    uint32_t RT_UNTRUSTED_VOLATILE_GUEST * const pFIFO = pThisCC->svga.pau32FIFO;
	...
}

VirtualBox-6.1.4\src\VBox\Devices\Graphics\DevVGA_VBVA.cpp:206

static bool vbvaFetchCmd(VBVADATA *pVBVAData, VBVACMDHDR RT_UNTRUSTED_VOLATILE_GUEST **ppHdr, uint32_t *pcbCmd)
{
	...
}

VirtualBox-6.1.4\src\VBox\Devices\Graphics\HGSMI\HGSMIHost.cpp:516

static void RT_UNTRUSTED_VOLATILE_GUEST *hgsmiHostHeapBufferAlloc(HGSMIHOSTHEAP *pHeap, HGSMISIZE cbBuffer)
{
	...
}
```s

## I/O Ports & Mapped Memory Handlers

I/O Ports and Mapped Memory are two complementary methods perform input/output between the CPUs and other devices, to find where those handlers are registered, we can search for the following functions call:

```cpp
DECLINLINE(int) PDMDevHlpIoPortCreate(PPDMDEVINS pDevIns, 
                                      RTIOPORT cPorts, 
                                      PPDMPCIDEV pPciDev, 
                                      uint32_t iPciRegion,
                                      PFNIOMIOPORTNEWOUT pfnOut, 
                                      PFNIOMIOPORTNEWIN pfnIn, 
                                      void *pvUser, 
                                      const char *pszDesc,
                                      PCIOMIOPORTDESC paExtDescs,
                                      IOMIOPORTHANDLE phIoPorts)

DECLINLINE(int) PDMDevHlpPCIIORegionCreateIo(PPDMDEVINS pDevIns,
                                             uint32_t iPciRegion,
                                             RTIOPORT cPorts,
                                             PFNIOMIOPORTNEWOUT pfnOut,
                                             PFNIOMIOPORTNEWIN pfnIn,
                                             void *pvUser,
                                             const char *pszDesc,
                                             PCIOMIOPORTDESC paExtDescs,
                                             PIOMIOPORTHANDLE phIoPorts)

pfnIn and pfnOut are the callback functions to handle guest’s CPU IN and OUT instructions.

DECLINLINE(int) PDMDevHlpMmioCreateEx(PPDMDEVINS pDevIns,
                                      RTGCPHYS cbRegion,
                                      uint32_t fFlags,
                                      PPDMPCIDEV pPciDev,
                                      uint32_t iPciRegion,
                                      PFNIOMMMIONEWWRITE pfnWrite,
                                      PFNIOMMMIONEWREAD pfnRead,
                                      PFNIOMMMIONEWFILL pfnFill,
                                      void *pvUser,
                                      const char *pszDesc,
                                      PIOMMMIOHANDLE phRegion)

DECLINLINE(int) PDMDevHlpPCIIORegionCreateMmio(PPDMDEVINS pDevIns,
                                               uint32_t iPciRegion,
                                               RTGCPHYS cbRegion,
                                               PCIADDRESSSPACE enmType,
                                               PFNIOMMMIONEWWRITE pfnWrite,
                                               PFNIOMMMIONEWREAD pfnRead,
                                               void *pvUser,
                                               uint32_t fFlags,
                                               const char *pszDesc,
                                               PIOMMMIOHANDLE phRegion)

pfnWrite and pfnRead are the callback functions to handle guest’s mapped memory read/write operation.

Search for those functions references and we can find many places that directly handle data from guest OS.

Guest’s Physical Memory Read/Write Handlers

If the emulated devices want to read/write data from/to the guest’s memory, they would use the following functions:

DECLINLINE(int) PDMDevHlpPhysRead(PPDMDEVINS pDevIns, 
                                  RTGCPHYS GCPhys, 
                                  void *pvBuf, 
                                  size_t cbRead)
  • GCPhys is the guest’s physical address of the buffer to read from.
  • pvBuf is the host process buffer to store the read buffer.
  • cbRead is the size of the buffer.
DECLINLINE(int) PDMDevHlpPhysWrite(PPDMDEVINS pDevIns,
                                   RTGCPHYS GCPhys,
                                   const void *pvBuf,
                                   size_t cbWrite)
  • GCPhys is the guest’s physical address of the buffer to write to.
  • pvBuf is the host process buffer to write to the guest’s memory.
  • cbWrite is the size of the buffer.

We can think about those functions like memmove() but from the guest OS to the host process and vice versa, so if the size of those operations is miscalculated:

  • For PDMDevHlpPhysRead, it could overwrite the memory adjacent to the destination host buffer and trigger a memory out-of-bounds write. With this, we can control over an object vtable, properties, etc.
  • For PDMDevHlpPhysWrite, it could overread the memory adjacent to the source buffer and move it to the guest physical memory. The guest OS can read this physical memory and obtain an information leakage

Finding Bugs

You can start to audit the VirtualBox source code with the above notes, the following ideas on fuzzing can be applied into QEMU, VMWare, etc also.

Dumb Fuzzing

I used to find bugs in VirtualBox like this:

  • Write a kernel driver that keep doing:
    • Generate random device’s data structures.
    • Save the generated data to a shared folder.
    • Setup device states, registers using I/O ports or mapped memory.
  • Enable Page Heap on the host VM process.
  • Start the VM then run the driver and wait until the VM crash.
    • Inspect the VM log files and the saved testcase in shared folder.
    • Try to reproduce and find the root cause.

It’s quite easy to use this way, I’ll generate simple random data structures then start the fuzzer first. While waiting for it to find anything, I can keep reading the source code to optimize the generator, make more sophisticate structures and pass more constraints.

It actually worked and I won my first Pwn2Own with this method. It took a few days for me to research and write a proper generator but 5 minutes to find this vulnerability after I ran my fuzzer. The exploit details can be found here.

Coverage Guided Fuzzing

After dumb fuzzing those emulated devices for a while, I realize that most data processed by the devices is read from the guest’s physical memory using PDMDevHlpPhysRead() function. Since our driver can fully control that memory range then I had an idea.

Firstly, I modify the VirtualBox’s source code to check if the VM process is running in fuzzing mode. If we’re in fuzzing mode, the devices will read the data from a file located on the host OS instead of guest’s memory using PDMDevHlpPhysRead() function.

For example, let’s pick the E1000 network adapter since we have some published research on it here. When it wants to send an ethernet frame, it’ll read the frame from the guest’s memory using e1kTxDLoadMore() function:

DECLINLINE(unsigned) e1kTxDLoadMore(PPDMDEVINS pDevIns, PE1KSTATE pThis)
{
	...    
    E1KTXDESC* pFirstEmptyDesc = &pThis->aTxDescriptors[pThis->nTxDFetched];
    PDMDevHlpPhysRead(pDevIns,
                      ((uint64_t)TDBAH << 32) + TDBAL + nFirstNotLoaded * sizeof(E1KTXDESC),
                      pFirstEmptyDesc, nDescsInSingleRead * sizeof(E1KTXDESC));

	...    
    
    if (nDescsToFetch > nDescsInSingleRead)
    {
        PDMDevHlpPhysRead(pDevIns,
                          ((uint64_t)TDBAH << 32) + TDBAL,
                          pFirstEmptyDesc + nDescsInSingleRead,
                          (nDescsToFetch - nDescsInSingleRead) * sizeof(E1KTXDESC));
	...        
    }
    pThis->nTxDFetched += (uint8_t)nDescsToFetch;
    return nDescsToFetch;
}

We’ll modify it like this:

DECLINLINE(unsigned) e1kTxDLoadMore(PPDMDEVINS pDevIns, PE1KSTATE pThis)
{
	...    
    E1KTXDESC* pFirstEmptyDesc = &pThis->aTxDescriptors[pThis->nTxDFetched];
    if(is_fuzzing())
    {
        read_from_testcase(pFirstEmptyDesc, nDescsInSingleRead * sizeof(E1KTXDESC));
    }
    else
    {
        PDMDevHlpPhysRead(pDevIns,
                      ((uint64_t)TDBAH << 32) + TDBAL + nFirstNotLoaded * sizeof(E1KTXDESC),
                      pFirstEmptyDesc, nDescsInSingleRead * sizeof(E1KTXDESC));

        if(is_write_seed_file())
        {
            write_seed_file(pFirstEmptyDesc, nDescsInSingleRead * sizeof(E1KTXDESC));
        }
    }

	...

    if (nDescsToFetch > nDescsInSingleRead)
    {   
        if(is_fuzzing())
        {
            read_from_testcase(pFirstEmptyDesc + nDescsInSingleRead,
                               (nDescsToFetch - nDescsInSingleRead) * sizeof(E1KTXDESC));
        }
        else
        {
            PDMDevHlpPhysRead(pDevIns,
                          ((uint64_t)TDBAH << 32) + TDBAL,
                          pFirstEmptyDesc + nDescsInSingleRead,
                          (nDescsToFetch - nDescsInSingleRead) * sizeof(E1KTXDESC));

                            
            if(is_write_seed_file())
            {
                write_seed_file(pFirstEmptyDesc + nDescsInSingleRead, 
                                (nDescsToFetch - nDescsInSingleRead) * sizeof(E1KTXDESC));
            }
        }

	...        
    }
    pThis->nTxDFetched += (uint8_t)nDescsToFetch;
    return nDescsToFetch;
}
  • Function is_fuzzing() is to check if the process is in fuzzing mode or not.
  • Function read_from_testcase() is to read the testcase’s content to the destination buffer.
  • Function is_write_seed_file() is to check if we are generating the seeds file.
  • Function write_seed_file() is to write the device’s data to the seed file.

Keep modifying every location where the device uses PDMDevHlpPhysRead() to read data, then we need to read the code to find out how the device process one data unit, for example:

  • How E1000 network adapter sends an Ethernet frame
  • How OHCI USB controller sends an URB.
  • How HDA transfers a stream, etc

With this knowledge, we can write a function to quickly call necessary functions to perform the data processing, remembering to reset the devices each time to make sure that the code coverage is stable. Let’s name this function fuzz_entry().

Choose a proper place to put the call to fuzz_entry() function in the device’s initialization routine then every time the VM is booting up in fuzzing mode, fuzz_entry() will be called and process the data from the test case file. Now, VirtualBox becomes an application that can read an input file and process it, so we can use AFL to fuzz it. That how I found the vulnerabilities for HITB Driven2Pwn 2019 and Pwn2Own 2020, the write-ups are coming soon.

Exploit Primitives

Heap Spraying

So when you want to perform a heap spray, you’ll need to allocate many buffers with controlled content in the host VM process, here’s how I did it.

Allocation size < 0x3FA0: Send An Ethernet Frame With E1000 Network Adapter

With a proper device’s config, the E1000 network adapter will allocate the frame buffer with RTMemAlloc() which is a wrapper for malloc(). The maximum size of the Ethernet frame is 0x3FA0.

static DECLCALLBACK(int) drvNATNetworkUp_AllocBuf(PPDMINETWORKUP pInterface, size_t cbMin,
                                                  PCPDMNETWORKGSO pGso, PPPDMSCATTERGATHER ppSgBuf)
{
	...    
    PPDMSCATTERGATHER pSgBuf = (PPDMSCATTERGATHER)RTMemAlloc(sizeof(*pSgBuf));
    if (!pSgBuf)
        return VERR_NO_MEMORY;
    if (!pGso)
    {
	...        
    }
    else
    {
	...        
        pSgBuf->pvUser      = RTMemDup(pGso, sizeof(*pGso));
        pSgBuf->pvAllocator = NULL;
        pSgBuf->aSegs[0].cbSeg = RT_ALIGN_Z(cbMin, 16);
        pSgBuf->aSegs[0].pvSeg = RTMemAlloc(pSgBuf->aSegs[0].cbSeg);  // Allocate Frame Buffer
        if (!pSgBuf->pvUser || !pSgBuf->aSegs[0].pvSeg)
        {
            RTMemFree(pSgBuf->aSegs[0].pvSeg);
            RTMemFree(pSgBuf->pvUser);
            RTMemFree(pSgBuf);
            return VERR_TRY_AGAIN;
        }
    }

Allocation Size > 0x1000: Send A VMMDev Request

When we send a VMMDev request, the VM will make a copy of the command in the host process. Leveraging this feature, we can allocate a buffer in the host process like this:

static DECLCALLBACK(VBOXSTRICTRC)
vmmdevRequestHandler(PPDMDEVINS pDevIns, void *pvUser, RTIOPORT offPort, uint32_t u32, unsigned cb)
{
  ...    
  if (   pThis->fu32AdditionsOk
      || requestHeader.requestType == VMMDevReq_ReportGuestInfo2
      || requestHeader.requestType == VMMDevReq_ReportGuestInfo
      || requestHeader.requestType == VMMDevReq_WriteCoreDump
      || requestHeader.requestType == VMMDevReq_GetHostVersion
     )
  {
  	...            
      
      if (   requestHeader.size <= _4K
          && iCpu < RT_ELEMENTS(pThisCC->apReqBufs))
      {
  	...
      }
      else
      {
          Assert(iCpu != NIL_VMCPUID);
          STAM_REL_COUNTER_INC(&pThisCC->StatReqBufAllocs);
          pRequestHeaderFree = pRequestHeader = 
              (VMMDevRequestHeader *)RTMemAlloc(RT_MAX(requestHeader.size, 512));  // Allocate request buffer
      }
      if (pRequestHeader)
      {
          ...                
          if (cbLeft)
          {
              ...                    
              else
                  PDMDevHlpPhysRead(pDevIns,
                                    (RTGCPHYS)u32             + sizeof(VMMDevRequestHeader),
                                    (uint8_t *)pRequestHeader + sizeof(VMMDevRequestHeader),
                                    cbLeft);	// Copy command's body
          }
          ...                
}

To allocate the buffer this way, we need to install Guest Additions in the Guest OS.

Information Leakage

By observing the memory layout of the host process VirtualBoxVM.exe in Windows, it looks like the range of the Video RAM buffer (VRAM) is quite similar each run due to weak randomization. This is shared memory which can be accessed by both the host process and guest OS. Its default size is 128MB for a Windows 10 64bit guest.

I wrote a script to dump the VRAM address and its size, after running it ~500 times I got the result like this:

vram: 0xbc60000, size: 0x8000000
vram: 0xbb50000, size: 0x8000000
vram: 0xc580000, size: 0x8000000
vram: 0xc7d0000, size: 0x8000000
vram: 0xc7a0000, size: 0x8000000
vram: 0xbc80000, size: 0x8000000
vram: 0xc0e0000, size: 0x8000000
vram: 0xb690000, size: 0x8000000
vram: 0xbfb0000, size: 0x8000000
vram: 0xc200000, size: 0x8000000
vram: 0xb7f0000, size: 0x8000000
vram: 0xc0c0000, size: 0x8000000
vram: 0xb350000, size: 0x8000000
vram: 0xca00000, size: 0x8000000
vram: 0xc7d0000, size: 0x8000000
vram: 0xbef0000, size: 0x8000000
vram: 0xb490000, size: 0x8000000
...

We can see that the VRAM addresses are slightly different but the memory range 0xcb10000 - 0x13220000 always falls within this buffer. So it’s safe to use the address 0x10000000 as a leaked address.

With this tricky information leak, I have escaped VirtualBox with a single UAF vulnerability at HITB Driven2Pwn 2019.

The RWX Page

On a Windows host, this page is very close to the VRAM buffer and its size is big enough for us to make a good guess of its address. So we have the address of an RWX page for free, what could be better? :D

This page used to be allocated by the recompiler module, but unfortunately for us, this module has been removed since VirtualBox 6.1.0.

Upcoming Posts

In the upcoming posts, we will be covering the exploits that were found by the methods described here.

Other Research on VirtualBox