CODEWITHSUNDEEP
assemblyvirtualboxx86-64avx
poby
poby

Reputation: 1746

vmovdqa not working in virtualbox?

I'm using a late model i7 cpu that supports avx and avx2, and supposedly virtualbox supports avx and avx2 so that being the case why does the following code hang?

vmovdqa    ymm0, qqword[testmem]

testmem is defined elsewhere as

align 32
testmem:   rb   128

If I use 

movdqa    xmm0, dqword[testmem]

It works fine.

FASM 1.72, windows 10, i7-7700hq, virtualbox 5.2.6

EDIT: It's a UEFI application (so obviously running in 64 bit mode) that works fine except if the above instruction appears.

EDIT

Tried adding

  mov     rcx, 0
  xgetbv
  or      rax, 0007h
  xsetbv

At the start of the code, but didn't help.

Upvotes: 4

Views: 400

Answers (1)

poby
poby

Reputation: 1746

Ok found the answer. I know this is a rather esoteric question but just in case it helps someone else here is how to enable AVX

mov rax, cr4
or eax, 0x40000              ; bit 18 for oxsave bit
mov cr4, rax

xor     rcx, rcx
xgetbv
or      rax, 6
xsetbv

What I was missing was setting bit 18 of the CR4 register which enables OSXSAVE, a requirement prior to enabling AVX.

Upvotes: 4

Related Questions