Testing an ARM CPU emulator

Question

I'm currently writing a Gameboy Advance emulator, purely for fun. It has an ARM7TDMI CPU, and I've written most of the CPU emulation code. My question is, how does one go about testing it?

Testing it out on random GBA software doesn't hugely help - the programs might crash for reasons unrelated to the CPU, and it's best to test things piece by piece, rather than all together.

Are there any available programs which test an ARM CPU, especially the instruction set? I've looked in the source for a couple of ARM simulators, but haven't found what I'm looking for. If not, I shall write my own program, but it will be hard to make the testing comprehensive.

Thanks in advance!

Answer 1

I agree with R. test the cpu first, and/or separately from the emulated hardware. Have a solid foundation to start with.

I have both written instruction set simulators and tested them as well as real processors under development.

There are the obvious directed tests, target specific instructions to see that they work. Carefully look at the ARM documentation for example, shifts have if-then-else definitions, if the shift is equal to 32 then dont shift or something like that, create a test that verifies each of those conditions in the if-then-else.

Flags are always tricky no matter what instruction set (except those without flags of course). Get your signed and unsigned overflows correct (carry = unsigned overflow) for addition. Then figure out how your target processor handles subtraction. Typically you want to invert the carry in and the second operand (into the adder), where one architecture varies from another is the carry out, some invert the carry out for a subtract, some dont. If there is a subtract with borrow instruction then whatever the carry out rule affects the subtract with borrow whether it inverts or not on the carry in. Fortunately there are C compilers which can help with this assuming their output works against real processors (part of this job is sorting out compiler bugs from processor bugs, and there are compiler bugs that you will hit). The special cases of all zeros, all ones, and 1 with all zeros (0x80000000) should get directed testing to make sure their (signed) overflows work properly. Some operations affect only some flags or some operations zero some flags, etc. Make sure you create situations to test that. ARM also makes it worse by allowing any instruction to be conditional and alu instructions to optionally modify flags.

I like to take things like zlib and jpeg decoders, mp3 decoders, anything that is open source and can be compiled in a bare metal fashion (yes these already mentioned can!), meaning in this case without operating system. Now zlib for example is a nightmare when it comes to mixing longs and ints, so depending on how your compiler is working that day on the host that day (some compilers, some times use the host architecture to size variables rather than the target architecture when cross compiling, it is quite annoying). I would go so far as to call zlib buggy for all the type jumping it does, so I have cut back on using it for testing because of the time spent debugging the compiling rather than the value it returns pushing the processor as it doesnt push the processor very hard (Although I have found processor bugs using it). for the jpeg and mp3 decoders which are lossy, you need the pre-computed answer, computed on your host. and you need integer solutions not floating point unless you want to use a soft float.

When using a compiler to test, understand a few things, first you want to try all the optimizations, for each compiled test have a -O0, -O1, -O2, and -O3 version, because it creates different instruction mixes and pushes the processor in different ways. Be careful though this is where you may start to hit compiler bugs. Different versions of the same compiler and different compilers produce different output, different instruction mixes, it is desirable if possible to get as many different compilers as you can, use evals or whatever. At one point while evaluating a compiler I found a number of compiler bugs to the point that they gave me a free copy of the full version for a while. YMMV. Also, which may not be possible for you, different individuals (not heavily governed by coding standards) produce different mixes of instructions (asm or higher), after you have each instruction, in theory, working when isolated, you may find that there are still bugs when a certain instruction follows another, so trying to create instruction mixes is useful, even if you never find a bug it is a good validation. Compiled code as I have been mentioning, but you may also want to for example create a sequence of alu operations using a randomizer to pick the operation and source and destination registers (with operations that sample the flags mixed in) and if possible run that code on a real processor then run it on yours to see if you get the same results.

One processor I worked on we built a test fixture in hardware where I could talk to a peripheral in the test fixture that would 1) allow me to send interrupts at some rate 2) create data/prefetch aborts for particular addresses. So I took a zlib test, while it was running I pounded it with interrupts, then in the interrupt I looked at the return address of the instruction and armed a prefetch abort for a few words later, sometimes they hit sometimes not, that was fine, the point was to have the main application run flawless despite the interruptions (I would have the interrupt continue as normal, and the abort would disarm, back up and let it try again). At a minimum you should be able to pound on your test apps with interrupts and insure they keep on truckin.

Unfortunately the bulk of the logic in a GBA is the graphics side, so even if you put a lot of effort into the processor you still have to debug the graphics. you can isolate that code by putting a test interface on your pseudo memory decoder and have host applications hit the virtual peripherals rather than always have to simulate arm code to do it. At the end of the day though I think it is playing off the shelf games that are going to push that the hardest, and yes debugging the problems can be extremely difficult. I would argue having a solid arm core will make the debugging easier, one less thing to worry about, one less source of the problem.

Another issue, is that because of how the gba was designed, there is a significant performance boost by running the arm in thumb mode, so you will see many thumb mode programs, which is more arm core stuff to validate/debug. mixed mode, etc.

Answer 2

QEMU is great for ARM CPU/instruction set. Balau is fine article to get started. QEMU can do Linux for many hardware platforms which are widely used, and even do U-Boot / bare metal code for a few ARM hardware platforms.

yFor your Gameboy emulator, how will your graphics be shown?

Answer 3

If your purpose is to write a whole-system emulator, I would figure out the other systems you need to support at least minimally and implement them before you worry about whether the CPU emulation works. Graphics and memory controller come to mind as the most important. They're also easier to write unit tests for, and if you can convince yourself that they're working okay, you'll have the right tools in place to test your CPU emulation.

With that said, if you DO want to test the CPU emulation first, I would add some debugging code to disassemble the instruction and print the values of all registers (and perhaps the values in memory that they potentially point to) before each instruction. Then try running some games. Even though they won't get very far, you'll have a HUGE corpus of data to look at to evaluate whether the emulation code is making the right changes to register/memory state.