Description of the vulnerability

VirtualBox is a x86 and AMD64/Intel64 virtualization product for enterprise as well as home use. It is a solution commercially supported by Oracle, in addition to being made available as open source software. It runs on various host platforms like Windows, Linux, Mac and Solaris and also supports a large number of guest operating systems.

When we create a pixel shader using the CreatePixelShader() method, the shader data buffer will be validated by CBaseShaderValidator::ValidateShader() function. The lack of proper validation can result in a heap Out-of-Bounds write in the process that uses the runtime library. The crash context with page heap enabled:

FAULTING_IP: 
d3d9!CPShaderValidator30::Rule_ValidDCLInstruction+69b
00007ffa`de6311f7 4108040e        or      byte ptr [r14+rcx],al

EXCEPTION_RECORD:  ffffffffffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 00007ffade6311f7 (d3d9!CPShaderValidator30::Rule_ValidDCLInstruction+0x000000000000069b)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 0000000000000001
   Parameter[1]: 000002149f9ad161
Attempt to write to address 000002149f9ad161

CONTEXT:  0000000000000000 -- (.cxr 0x0;r)
rax=0000000000000000 rbx=000002149f7df880 rcx=000002149f9acfe0
rdx=0000003234f8f479 rsi=00007ffade6b7b20 rdi=000002149f7df900
rip=00007ffade6311f7 rsp=0000003234f8f3d0 rbp=0000000000000000
 r8=000002149f9aafd8  r9=0000000000000000 r10=000002149f9cce00
r11=000002149f9cce80 r12=0000000000000004 r13=00000000000f0000
r14=0000000000000181 r15=0000000000000000
iopl=0         nv up ei pl nz na pe nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010202
d3d9!CPShaderValidator30::Rule_ValidDCLInstruction+0x69b:
00007ffa`de6311f7 4108040e        or      byte ptr [r14+rcx],al ds:00000214`9f9ad161=??

FAULTING_THREAD:  0000000000000948

DEFAULT_BUCKET_ID:  INVALID_POINTER_WRITE

PROCESS_NAME:  d3d9_ShaderDefine.exe

ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.

EXCEPTION_PARAMETER1:  0000000000000001

EXCEPTION_PARAMETER2:  000002149f9ad161

WRITE_ADDRESS:  000002149f9ad161 

FOLLOWUP_IP: 
d3d9!CPShaderValidator30::Rule_ValidDCLInstruction+69b
00007ffa`de6311f7 4108040e        or      byte ptr [r14+rcx],al

NTGLOBALFLAG:  2000000

APPLICATION_VERIFIER_FLAGS:  0

APP:  d3d9_shaderdefine.exe

ANALYSIS_VERSION: 6.3.9600.17237 (debuggers(dbg).140716-0327) amd64fre

PRIMARY_PROBLEM_CLASS:  INVALID_POINTER_WRITE

BUGCHECK_STR:  APPLICATION_FAULT_INVALID_POINTER_WRITE

LAST_CONTROL_TRANSFER:  from 00007ffade62f8de to 00007ffade6311f7

STACK_TEXT:  
00000032`34f8f3d0 00007ffa`de62f8de : 00000214`9f7df800 00000214`9f859000 00000000`ffff0300 00000214`00000000 : d3d9!CPShaderValidator30::Rule_ValidDCLInstruction+0x69b
00000032`34f8f460 00007ffa`de589afa : 00000214`9f7df880 00000214`9f859008 00000000`ffff0301 00000214`9f7df900 : d3d9!CPShaderValidator30::ApplyPerInstructionRules+0x9e
00000032`34f8f490 00007ffa`de589a04 : 00000214`9f7df880 00000000`00000014 00000000`ffff0300 00000214`9f7df880 : d3d9!CBaseShaderValidator::Instruction+0xc6
00000032`34f8f4d0 00007ffa`de589bc5 : 00000000`ffff0300 00000214`9f859004 00000000`00000000 00000032`34f8f5e0 : d3d9!CBaseShaderValidator::ValidateShader+0x34
00000032`34f8f510 00007ffa`de588c58 : 00000214`9f7ddf20 00000214`9f32f4a0 00000000`00000014 00007ffa`dd91f3a9 : d3d9!ValidatePixelShaderInternal+0x31
00000032`34f8f540 00007ff7`666210cf : 00000214`9f7ddf20 00007ffa`dd9cc500 00000214`9f32f4a0 00000214`9f7ddf20 : d3d9!CD3DBase::CreatePixelShader+0xc8
00000032`34f8f5e0 00007ff7`666212bd : 00000032`34f8f6b9 00000000`00001388 00000214`92349f10 00000000`00000000 : d3d9_ShaderDefine!shaderdefine+0xcf
00000032`34f8f610 00007ff7`66621583 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : d3d9_ShaderDefine!main+0x18d
00000032`34f8f720 00007ffb`14947974 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : d3d9_ShaderDefine!__tmainCRTStartup+0x10f
00000032`34f8f750 00007ffb`1725a271 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : KERNEL32!BaseThreadInitThunk+0x14
00000032`34f8f780 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x21


STACK_COMMAND:  .cxr 0x0 ; kb

SYMBOL_STACK_INDEX:  0

SYMBOL_NAME:  d3d9!CPShaderValidator30::Rule_ValidDCLInstruction+69b

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: d3d9

IMAGE_NAME:  d3d9.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  0

FAILURE_BUCKET_ID:  INVALID_POINTER_WRITE_c0000005_d3d9.dll!CPShaderValidator30::Rule_ValidDCLInstruction

BUCKET_ID:  APPLICATION_FAULT_INVALID_POINTER_WRITE_d3d9!CPShaderValidator30::Rule_ValidDCLInstruction+69b

ANALYSIS_SOURCE:  UM

FAILURE_ID_HASH_STRING:  um:invalid_pointer_write_c0000005_d3d9.dll!cpshadervalidator30::rule_validdclinstruction

FAILURE_ID_HASH:  {96c761fc-78c2-bbbb-d6cf-7c1572bff300}

Followup: MachineOwner
---------

0:000> lmvm d3d9
start             end                 module name
00007ffa`de560000 00007ffa`de6fe000   d3d9       (pdb symbols)          c:\symbols\d3d9.pdb\C52C34FEAD16F0483B42E69D4760E2191\d3d9.pdb
    Loaded symbol image file: C:\WINDOWS\SYSTEM32\d3d9.dll
    Image path: C:\WINDOWS\SYSTEM32\d3d9.dll
    Image name: d3d9.dll
    Timestamp:        ***** Invalid (FC1B9E5E)
    CheckSum:         001A7AF3
    ImageSize:        0019E000
    File version:     10.0.17763.1
    Product version:  10.0.17763.1
    File flags:       0 (Mask 3F)
    File OS:          40004 NT Win32
    File type:        2.0 Dll
    File date:        00000000.00000000
    Translations:     0409.04b0
    CompanyName:      Microsoft Corporation
    ProductName:      Microsoft® Windows® Operating System
    InternalName:     D3D9.dll
    OriginalFilename: D3D9.dll
    ProductVersion:   10.0.17763.1
    FileVersion:      10.0.17763.1 (WinBuild.160101.0800)
    FileDescription:  Direct3D 9 Runtime
    LegalCopyright:   © Microsoft Corporation. All rights reserved.
0:000> dt ntdll!_DPH_BLOCK_INFORMATION 00000214`9f9acf90
   +0x000 StartStamp       : 0xabcdbbbb
   +0x008 Heap             : 0x00000214`91c61000 Void
   +0x010 RequestedSize    : 0x22
   +0x018 ActualSize       : 0x1000
   +0x020 FreeQueue        : _LIST_ENTRY [ 0x00000000`00000000 - 0x00000000`00000000 ]
   +0x020 FreePushList     : _SLIST_ENTRY
   +0x020 TraceIndex       : 0
   +0x030 StackTrace       : 0x00000214`90566420 Void
   +0x038 Padding          : 0
   +0x03c EndStamp         : 0xdcbabbbb

When validating the DCL shader instruction, we come to the function CPShaderValidator30::Rule_ValidDCLInstruction():

char __fastcall CPShaderValidator30::Rule_ValidDCLInstruction(struct_CBaseShaderValidator *this)
{

....

  if ( !this->a37 )
  {
    CBaseShaderValidator::Spew(this, 2u, 0x129u, "def and dcl instructions must appear before other instructions.");
    ins = this->ins;
    ++this->a15;
    v58 = ins;
  }
  regNum = ins->dstParam.RegNum;

....

  v13 = this->a33;
  v57 = v13;

....

  count = 4i64;
  
....
  
  v49 = &v62;
  v50 = (v57 + 12);
  do
  {
    reg = *v50;
    ++v50;
    value = *v49++;
    reg[regNum] |= value;		// regNum controlled, reg buffer only 0x22 bytes
    --count;
  }
  while ( count );
  return 1;
}

regNum is assigned by this->ins->dstParam.RegNum which was initialized in CBaseShaderValidator::DecodeDstParam():

void __fastcall CBaseShaderValidator::DecodeDstParam(CBaseShaderValidator *this, struct_dstParam *ins, unsigned int token)
{
  ins->gap0 = 1;
  RegNum = D3DSI_GETREGNUM_RESOLVING_CONSTANTS(token);
  ins->RegNum = RegNum;
  ins->dword8 = v5 & 0xF0000;
  ins->dwordC = v5 & 0xF00000;
  ins->dword10 = (v5 >> 24) & 0xF;
  v6 = D3DSI_GETREGTYPE_RESOLVING_CONSTANTS(v5);
  ins->RegType = v6;
  ins->dword1C = v7 & 0x2000;
}

RegNum is calculated from token value, which comes from the input shader buffer:

__int64 __fastcall D3DSI_GETREGNUM_RESOLVING_CONSTANTS(unsigned int a1)
{
  __int64 result; // rax

  result = a1 & 0x7FF;
  switch ( (a1 & 0x1800 | (a1 >> 20) & 0x700) >> 8 )
  {
    case 0xBu:
      return (result + 2048);
    case 0xCu:
      return (result + 4096);
    case 0xDu:
      result = (result + 6144);
      break;
  }
  return result;
}

So we can control the value of ins->dstParam.RegNum with out any validation and trigger a heap out-of-bounds write in PShaderValidator30::Rule_ValidDCLInstruction():

char __fastcall CPShaderValidator30::Rule_ValidDCLInstruction(struct_CBaseShaderValidator *this)
{

....

  do
  {
    reg = *v50;
    ++v50;
    value = *v49++;
    reg[regNum] |= value;		// regNum controlled, reg buffer only 0x22 bytes
    --count;
  }
  while ( count );
  return 1;
}

reg buffer was allocated with size 0x22:

0:000> dt ntdll!_DPH_BLOCK_INFORMATION 00000214`9f9acf90
   +0x000 StartStamp       : 0xabcdbbbb
   +0x008 Heap             : 0x00000214`91c61000 Void
   +0x010 RequestedSize    : 0x22
   +0x018 ActualSize       : 0x1000
   +0x020 FreeQueue        : _LIST_ENTRY [ 0x00000000`00000000 - 0x00000000`00000000 ]
   +0x020 FreePushList     : _SLIST_ENTRY
   +0x020 TraceIndex       : 0
   +0x030 StackTrace       : 0x00000214`90566420 Void
   +0x038 Padding          : 0
   +0x03c EndStamp         : 0xdcbabbbb

Timeline:

2019-12-04 Disclosed via ZDI
2021-07-24 Vendor patched

Vendor Response

The vendor has acknowledged the issue and released an update to address it.

The vendor advisory can be found here: https://www.oracle.com/security-alerts/cpuapr2020.html.

Description of the vulnerability#

Timeline:#

Vendor Response#

Description of the vulnerability

Timeline:

Vendor Response