Process of getting debug data in a managed environment

Question

I'm trying to figure out where the debugger gets debug information from when working with managed code. What I understood (probably):

Firstly. For .NET applications, the lack of debug symbols is not a problem since the components are self-described using metadata. From ECMA-335:

Each CLI component carries the metadata for declarations, implementations, and references specific to that component. Therefore, the component-specific metadata is referred to as component metadata, and the resulting component is said to be self-describing.

---

Secondly. In the process of executing a managed .NET application, one of the first to enter into business is the type loader, whose responsibility is to create internal data structures that are subsequently used by the JIT Compiler.

Internal structures include: TypeDesc (and derivatives), MethodTable, EEClass, MethodDesc, and so on. The structures contain information required by the JIT compiler, which it uses when compiling IL code into machine code.

---

Thirdly. Type loader uses tokens to find type information in executable files (from metadata tables):

Token is a 4-byte unsigned integer whose most significant byte carries a zero-based table index (the same as the internal metadata RID type). The remaining 3 bytes are left for the RID.

---

Fourthly. After JIT Complier has performed jitting and compiled the native code in structures such as MethodDesc, addresses pointing to the IL of the code are changed to the address of the generated code -a second call to the method no longer launches the JIT Compiler. Eventually the Execution Engine executes the compiled code.

---

This sequence can be expressed by the following diagram:

Let's say we have created a .NET application dump and want to analyze it. We know that WinDbg is a native debugger and doesn't know anything about managed code. How does the reverse process take place?

How does the data from the dump compare with information about types and so on? How exactly is SOS and DAC involved?

---

I know that we must to load the SOS extension and DAC:

The SOS Debugging Extension (SOS.dll) helps you debug managed programs in Visual Studio and in the Windows debugger (WinDbg.exe) by providing information about the internal Common Language Runtime (CLR) environment.

Debugging managed code requires special knowledge of managed objects and constructs. For example, objects have various kinds of header information in addition to the data itself. Objects may move in memory as the garbage collector does its work. Getting type information may require help from the loader. Retrieving the correct version of a function that has undergone an edit-and-continue or getting information for a function emitted through reflection requires the debugger to be aware of EnC version numbers and metadata. The debugger must be able to distinguish AppDomains and assemblies. The code in the VM directory embodies the necessary knowledge of these managed constructs. This essentially means that APIs to retrieve information about managed code and data must run some of the same algorithms that the execution engine itself runs.

In short, it seems that the DAC's task is to extract information from the dump, compare it with information at the CLR level, that is, an exact description of types and other things, and then transfer this information to the debugger through marshaling.

Is it correct?

Both SOS and the CLR debugging APIs use the Data Access Component (DAC) to implement out-of-process debugging.

Is SOS a DBI interface? Where does it fit in the diagram?

Answer 1

thats a very elaborately written question
so i think my answer wont do justice to every little detail mentioned out there

sos is a windbg extension
as such it uses the same dbghelp and dbgeng apis to retrieve information from
either a live or dump .net target.

if you retrieve its file version info

you can see it advertises itself as such using powershell

:\>powershell -c "(get-item -path \"C:/Windows/Microsoft.NET/Framework64/v4.0.30319/SOS.dll\").VersionInfo.FileDescription"
Microsoft NTSD extension for .NET Runtime

or with ghidra

Executable Format:  Portable Executable (PE)
Executable Location: C:/Windows/Microsoft.NET/Framework64/v4.0.30319/SOS.dll
Executable MD5: 4b631b100706ea6a804e229b3f885f29
FileDescription:    Microsoft NTSD extension for .NET Runtime

the decompilation of command !dumpheap is like this

int DumpHeap(IDebugClient *param_1,char *param_2)

{
  code *pcVar1;
  IXCLRDataProcess *pIVar2;
  ISOSDacInterface *pIVar3;
  int iVar4;
  char *pcVar5;
  DumpHeapImpl local_b8;
  
  pcVar5 = param_2;
                    /* 0x3f3c0  16  DumpHeap
                       0x3f3c0  115  dumpheap */
  iVar4 = ExtQuery(param_1);
  if ((iVar4 == 0) && (iVar4 = ArchQuery(), iVar4 == 0)) {
    ControlC = 0;
    g_bDacBroken = 1;
    iVar4 = CheckEEDll();
    if (iVar4 == 0) {
      iVar4 = LoadClrDebugDll();
      pIVar3 = g_sos;
      pIVar2 = g_clrData;
      if (iVar4 != 0) {
        DACMessage(iVar4);
        ExtRelease();
        return iVar4;
      }
      g_bDacBroken = 0;
      ResetGlobals();
      iVar4 = IsMiniDumpFile();
      if (iVar4 != 0) {
        ExtOut("This command is not supported in a minidump without full memory\n");
        ExtOut("To try the command anyway, run !MinidumpMode 0\n");
        if (pIVar3 != (ISOSDacInterface *)0x0) {
          pcVar1 = *(code **)((longlong)*pIVar3 + 0x10);
          _guard_check_icall(pcVar1,pcVar5);
          (*pcVar1)(pIVar3);
        }
        if (pIVar2 != (IXCLRDataProcess *)0x0) {
          pcVar1 = *(code **)((longlong)*pIVar2 + 0x10);
          _guard_check_icall(pcVar1,pcVar5);
          (*pcVar1)(pIVar2);
        }
        ExtRelease();
        return 0;
      }
      iVar4 = GCHeapSnapshot::Build(&g_snapshot);
      if (iVar4 == 0) {
        ExtOut("Unable to build snapshot of the garbage collector state\n");
        if (pIVar3 != (ISOSDacInterface *)0x0) {
          pcVar1 = *(code **)((longlong)*pIVar3 + 0x10);
          _guard_check_icall(pcVar1,pcVar5);
          (*pcVar1)(pIVar3);
        }
        if (pIVar2 != (IXCLRDataProcess *)0x0) {
          pcVar1 = *(code **)((longlong)*pIVar2 + 0x10);
          _guard_check_icall(pcVar1,pcVar5);
          (*pcVar1)(pIVar2);
        }
        ExtRelease();
        return -0x7fffbffb;
      }
      DumpHeapImpl::DumpHeapImpl();
      DumpHeapImpl::Run(&local_b8);
      DumpHeapImpl::~DumpHeapImpl(&local_b8);
      if (pIVar3 != (ISOSDacInterface *)0x0) {
        pcVar1 = *(code **)((longlong)*pIVar3 + 0x10);
        _guard_check_icall(pcVar1,param_2);
        (*pcVar1)(pIVar3);
      }
      if (pIVar2 != (IXCLRDataProcess *)0x0) {
        pcVar1 = *(code **)((longlong)*pIVar2 + 0x10);
        _guard_check_icall(pcVar1,param_2);
        (*pcVar1)(pIVar2);
      }
      ExtRelease();
      return 0;
    }
    ExtOut("Failed to find runtime DLL (clr.dll), 0x%08x\n");
    ExtOut("Extension commands need clr.dll in order to have something to do.\n");
  }
  ExtRelease();
  return iVar4;
}