Custom Instruction Sets

Question

I recently saw a video that explains how to make a Custom Instruction Set (just the concept ... how it would work). This video was part of a series that in total would have 3 videos: "How to make a Custom Instruction Set", "How to make an Assembler for that Custom Intruction Set", and the third would be "How to emulate the Custom Instruction Set". Unfortunately the third video was never released, so I went to find out if I could do it myself. After my research, I was quite inconclusive. Is it feasible to make Custom Instruction Sets? How do I emulate them? Do I need specific hardware if I really want to run it? How do I get this hardware?

PS: The videos that I've watched: 1st video, 2nd video. (click on them)

Answer 1

Is it feasible to make Custom Instruction Sets?

Yes, of course! You can make anything you like! (Seriously) :) I assume that you have specific definitions in mind for "custom" and "instruction set". By "custom," to me that implies it's an instruction set not implemented on a known piece of hardware. A custom instruction set probably really implies a custom CPU architecture. (This leads to the answer to your last question, which I'll get to.) If you have a different definition of "custom," then you should share it. By "instruction set" I assume you mean commands that are similar in function and scope to existing CPU instruction sets (as opposed to something more extremely different or exotic). But in either case, the answer is still "yes".

How do I emulate them?

Erik's very informative answer has offered verious perspectives on this question. I will focus on a specific aspect. Assuming you are thinking of a custom instruction set that operates in the context of a standard von Neumann architecture that runs in a linear array of memory with a program counter, then your emulate would be a piece of software that pretends to be the CPU. As such, you would:

• Define variables that represent the state of the CPU: a PC (program counter), registers (if your custom instruction set supports them), CPU status bits (if your custom instruction set and architecture uses them - likely it would), stack pointer (if your instruction set and architecture has a stack), etc.
• Simulate the operation of your instructions and architecture. To do this you have to think about how your instruction set gets started. For example, some CPUs start at a specific memory location when they power up. Maybe yours would do something similar. You would define an array in your emulation software that represents the memory of your custom CPU. When your emulation starts (your custom CPU powers on) you would initialize your program counter and other state variables to their initial condition (defined by you) and start accessing instructions from memory (your array) just like a CPU would (fetch/execute cycle).

Do I need specific hardware if I really want to run it? How do I get this hardware?

If it's a custom instruction set, then I would think the hardware to execute it directly doesn't exist, at least by my definition of "custom" (otherwise, in what way is it custom?). My definition of "instruction set" is that it consists of instructions that are "close to the hardware", i.e., in one-to-one correspondence with the low level operations that the hardware directly knows how to execute. But as already noted, you can execute the instructions with your emulator. :)

---

In response to additional question in the comments.

But is there any way to make my own hardware? (easily if possible :)

Yes, there is of course a way to design and create the hardware needed to run a custom instruction set. Is it easy? It depends upon your skill level. You can find resources online on how to design a basic computer from scratch. Although the individual concepts are not difficult to understand for a basic CPU, it's not a trivial undertaking since there are lots of pieces to implement. You would be basically designing your own computer, from scratch.

If you're interested in this topic from a hardware perspective, I highly recommend looking up Ben Eater on YouTube. He has numerous excellent instructional videos discussing the creation of the various pieces of a CPU.

Answer 2

Emulation can be done several ways:

1. Interpreter loop

An interpreter loop for that processor would look something like this:

uint16_t memory [ 65536 ];
uint16_t registerFile [ 7 ];
uint16_t pc = 0;
for(;;) {
    uint32_t ix = (memory [ pc ] << 16) + memory [ pc + 1 ]; // instruction fetch (32-bits)
    pc += 2;  // advance two words worth for next instruction as default
    switch ( ix >> 24 ) {
    case 0x00 : // Load R, Immediate
        uint16_t imm = ix & 0xFFFF;
        int regNumber = (ix >> 16) & 0x7;
        registerFile [ regNumber ] = imm;
        break;
    // other opcodes handled by cases
        
    }
}

You would make adjustments for the custom processor, such as whether the processor is byte addressable vs. only word addressable; whether big endian or little endian, etc..

2. Various forms of Translation

We can translate machine code of one form into machine code of another form, i.e. from a custom instruction set to an existing instruction set.  The actual translation requires a mapping of the stateful features of the custom instruction set into stateful features of the existing instruction set, along with mapping of machine code operations over those stateful resources.

Because machine code programs can do indirect branches, whose operations are difficult to predict statically, a translating solution may include mapping tables to find where to go in the translation given the simulated program counter.

Machine code programs may also load (indirectly) from code memory, so the original machine code program may also be kept in the translation.

Sometimes the translation is done more or less dynamically as well, which is probably necessary if the custom processor allows for writing to code memory and later executing that new code.