drew sicard
drew sicard

Reputation: 1

Lifting an obfuscated stack machine to LLVM IR

I am researching a code virtualization solution named VMProtect, which creates a custom stack machine with a virtual instruction set, to obfuscate code. What I want to do, is lift every virtual machine instruction handler, most of which consists of stack based operations, to LLVM IR, and then apply optimization and recompile.

I will be using llvm's IRBuilder for this.

Some of my questions are:

  1. How can push/pop be represented in IR
  2. What would a handler like this look like in IR
  mov     rdx, [rbp] ; get virtual stack operand
  add     rbp, 8         ; add to stack pointer
  mov     [rax+rdi], rdx ; move to scratch space

Upvotes: 0

Views: 707

Answers (0)

Related Questions