CODEWITHSUNDEEP
gem5
Ciro Santilli
Ciro Santilli

Reputation: 4073

How to run a gem5 arm aarch64 full system simulation with fs.py with more than 8 cores?

If I try to use more than --num-cpus=8 cores, e.g. 16, 32 or 64, the terminal just stays blank.

Tested with gem5 at commit 2a9573f5942b5416fb0570cf5cb6cdecba733392 and Linux kernel 4.16.

Related thread: https://www.mail-archive.com/[email protected]/msg15469.html

Upvotes: 1

Views: 1743

Answers (2)

Ciro Santilli
Ciro Santilli

Reputation: 4073

Newer method: GICv3

Since GICv3 was implemented in February 2019 at https://gem5-review.googlesource.com/c/public/gem5/+/13436 you can just use it instead.

The GICv3 hardware natively supports more than 8 CPUs, so it just works.

As of July 2020 gem5 3ca404da175a66e0b958165ad75eb5f54cb5e772, GICv3 is the default GIC for the VExpress_GEM5_V2 but the default fs.py machine is VExpress_GEM5_V2 at that commit, so you just have to select it with:

fs.py --machine-type VExpress_GEM5_V2

Once I did that, it just worked, Atomic boot took about 6x on 16 cores compared to a single CPU. Tested with this setup: https://github.com/cirosantilli/linux-kernel-module-cheat/tree/d0ada7f58c6624662bbfa3446c7c26422d1c5afb#gem5-arm-full-system-with-more-than-8-cores

Older method: GICv2 extensions

As mentioned at: https://www.mail-archive.com/[email protected]/msg24593.html gem5 has a GICv2 extension + kernel patch that allows this:

Tested with this setup: https://github.com/cirosantilli/linux-kernel-module-cheat/tree/bab029f60656913b5dea629a220ae593cc16147d#gem5-arm-full-system-with-more-than-8-cores (gem5 4c8efdbef45d98109769cf675ee3411393e8ed06, Linux kernel fork v4.15, aarch64).

Upvotes: 1

Cherry
Cherry

Reputation: 73

If I add more to Ciro's answer, the current GICv2 model in gem5 supports single core by default because of this line of code. Without enabling gem5ExtensionsEnabled, it won't update the highest_int with the receiving interrupt number, and as a result the received interrupt won't get posted to a specified cpu to invoke a handler. That is, there is no jump to interrupt handler. In addition, even when we turn on gem5ExtensionsEnabled, I think that it will support up to 4 cores because the default values of INT_BITS_MAX and itLines are 32 and 128, respectively (see this); it checks 32 interrupt lines per core across 4 cores. For example, imagine that a system features 16 cores and cpu 5 executes the loop. Also, suppose that the other core (say core 11) already has an interrupt with higher priority than this. Then, the loop will ignore the other interrupt from core 11 because the loop index x can grow at most to 3.

To turn on gem5ExtensionsEnabled, you can pass an option --param='system.realview.gic.gem5_extensions=True' to your command as Ciro stated. However, note that the parameter is used to set haveGem5Extensions variable at here, not setting gem5ExtensionsEnabled, which is enabled only when a firmware code writes some data (0x200) to GIC distributor register at GICD_TYPER offset (see this).

Upvotes: 2

Related Questions