CODEWITHSUNDEEP
profilingperfamd-processormemory-bandwidth
smz
smz

Reputation: 505

Tracking DRAM traffic in AMD Zen 2 (Rome)

I want to track the number of read/write accesses at each of the Unified Memory Controllers (UMCs) in my AMD EPYC processor (family: 0x17 and model: 0x31). The AMDuProfPcm tool, when used with the -m memory option, provides the read and write bandwidth per channel for 8 memory channels (Mem Ch-A to H). Since this is a dual-socket system, I assume that these channels correspond only to the memory channels available on one socket.

Upon inspecting the configuration files for AMDuProfPcm, I found the following entries:

# cat AMDuProf_Linux_x64_4.2.850/bin/Data/Events/0x17_0x3.xml | grep Umc

        <event value="0x0000000000403007" name="DFCS0Umc0ReqRd"/>
        <event value="0x0000000000400807" name="DFCS0Umc0ReqWr"/>
        <event value="0x0000000000403047" name="DFCS1Umc1ReqRd"/>
        <event value="0x0000000000400847" name="DFCS1Umc1ReqWr"/>
        <event value="0x0000000000403087" name="DFCS2Umc2ReqRd"/>
        <event value="0x0000000000400887" name="DFCS2Umc2ReqWr"/>
        <event value="0x00000000004030C7" name="DFCS3Umc3ReqRd"/>
        <event value="0x00000000004008C7" name="DFCS3Umc3ReqWr"/>
        <event value="0x0000000100403007" name="DFCS4Umc4ReqRd"/>
        <event value="0x0000000100400807" name="DFCS4Umc4ReqWr"/>
        <event value="0x0000000100403047" name="DFCS5Umc5ReqRd"/>
        <event value="0x0000000100400847" name="DFCS5Umc5ReqWr"/>
        <event value="0x0000000100403087" name="DFCS6Umc6ReqRd"/>
        <event value="0x0000000100400887" name="DFCS6Umc6ReqWr"/>
        <event value="0x00000001004030C7" name="DFCS7Umc7ReqRd"/>
        <event value="0x00000001004008C7" name="DFCS7Umc7ReqWr"/>

However, I am unable to find these specific events in the AMD PPR document. The closest event I could locate pertains to Combined DRAM B/bytes of all channels on an NPS1 node (die), which is listed as:

(DfEvent[0x0000000000403807] + DfEvent[0x0000000000403847] + DfEvent[0x0000000000403887] + DfEvent[0x00000000004038C7] + DfEvent[0x0000000100403807] + DfEvent[0x0000000100403847] + DfEvent[0x0000000100403887] + DfEvent[0x00000001004038C7]) * 64B

I get the following with perf list:

# perf list | grep dram

  ...

  nps1_die_to_dram

Strangely, I get the following when I run perf stat with -e nps1_die_to_dram:

# sudo perf stat -B -e nps1_die_to_dram ls
event syntax error: 'nps1_die_to_dram'
                     \___ Bad event name

And get the following with the -M option:


# sudo perf stat -M nps1_die_to_dram -a ls
event syntax error: 'dram_channel_data_controller_4/metric-id=dram_channel_data_controller_4/,dram_channel_data_controller_1/metric-id=dram_channel_data_controller_1/,dram_channel_data_controller_6/metric-id=dram_channel_data_controller_6/,dram_channel_data_controller_3/metric-id=dram_channel_data_controller_3/,dram_chan..'
                     \___ Bad event or PMU

Unable to find PMU or event on a PMU of 'dram_channel_data_controller_4'

Initial error:
event syntax error: 'dram_channel_data_controller_4/metric-id=dram_channel_data_controller_4/,dram_channel_data_controller_1/metric-id=dram_channel_data_controller_1/,dram_channel_data_controller_6/metric-id=dram_channel_data_controller_6/,dram_channel_data_controller_3/metric-id=dram_channel_data_controller_3/,dram_chan..'
                     \___ Cannot find PMU `dram_channel_data_controller_4'. Missing kernel support?

I'm not sure why perf is not working, and also, is there any way to determine how AMDuProfPcm is calculating the bandwidth for each memory channel?

Upvotes: 2

Views: 85

Answers (1)

Александр Маслов
Александр Маслов

Reputation: 46

I had the same issue with perf on my Zen3. My answer is not about AMDuProf, but I hope it helps with perf error.

First I looked for dram_channel_data_controller_ and found linux/tools/perf/pmu-events/arch/x86/amdzen3/data-fabric.json file which shows that this counter belongs to DFPMC unit. DFPMC resolved to amd_df in jevents.py. Finally arch/x86/events/Kconfig claims that amd_df PMU support implemented in amd-uncore kernel module.

It turned out that this module is compiled but not loaded by default, so

sudo modprobe amd-uncore

fixed the issue for me.

Upvotes: 3

Related Questions