Reputation: 25
Where are Kernel global variables stored?
As i know global variables(in userspace programs) are stored in coressponding DATA (and BSS) segment. But how and where are they stored in case of kernel code?
Upvotes: 2
Views: 1236
Answers (1)
Reputation: 2884
For an example let's look at one of Linux's global variable. Basically, Linux is an ELF image compressed with gzip or other compression tools. As stated on Wikipedia (https://en.wikipedia.org/wiki/Vmlinux):
Traditionally, when creating a bootable kernel image, the kernel is also compressed using gzip, or, since Linux 2.6.30,[3] using LZMA or bzip2, which requires a very small decompression stub to be included in the resulting image. The stub decompresses the kernel code, on some systems printing dots to the console to indicate progress, and then continues the boot process. Support for LZO,[4] xz[5] and LZ4[6] compression was added later.
On Ubuntu, there is a special script called extract-vmlinux which comes with your version of the Linux kernel (https://blog.packagecloud.io/eng/2016/03/08/how-to-extract-and-disassmble-a-linux-kernel-image-vmlinuz/). The Linux ELF file is /boot/vmlinuz. To decompress it, you must use the extract-vmlinux script which can be installed with
sudo apt-get install linux-headers-$(uname -r)
The script is safe to use because it is a standard script which is found in the official repository of packages for your distribution (most packages there are open source and safe). Now use
sudo /usr/src/kernels/$(uname -r)/scripts/extract-vmlinux /boot/vmlinuz-$(uname -r) > vmlinux
to extract the Linux kernel's ELF file from the compressed vmlinux image to your home repository. Now we can look at the ELF file of the Linux kernel. Use readelf -a vmlinux to have a summary of the content. The output for me is this:
user@user-System-Product-Name:~$ readelf -a vmlinux
ELF Header:
Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
Class: ELF64
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: Advanced Micro Devices X86-64
Version: 0x1
Entry point address: 0x1000000
Start of program headers: 64 (bytes into file)
Start of section headers: 46137728 (bytes into file)
Flags: 0x0
Size of this header: 64 (bytes)
Size of program headers: 56 (bytes)
Number of program headers: 5
Size of section headers: 64 (bytes)
Number of section headers: 35
Section header string table index: 34
Section Headers:
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[ 0] NULL 0000000000000000 00000000
0000000000000000 0000000000000000 0 0 0
[ 1] .text PROGBITS ffffffff81000000 00200000
0000000000e025f7 0000000000000000 AX 0 0 4096
[ 2] .rodata PROGBITS ffffffff82000000 01200000
00000000004e46f2 0000000000000000 WA 0 0 4096
[ 3] .pci_fixup PROGBITS ffffffff824e4700 016e4700
00000000000032b0 0000000000000000 A 0 0 16
[ 4] .tracedata PROGBITS ffffffff824e79b0 016e79b0
0000000000000078 0000000000000000 A 0 0 1
[ 5] __ksymtab PROGBITS ffffffff824e7a28 016e7a28
0000000000010a34 0000000000000000 A 0 0 4
[ 6] __ksymtab_gpl PROGBITS ffffffff824f845c 016f845c
00000000000121ec 0000000000000000 A 0 0 4
[ 7] __kcrctab PROGBITS ffffffff8250a648 0170a648
00000000000058bc 0000000000000000 A 0 0 4
[ 8] __kcrctab_gpl PROGBITS ffffffff8250ff04 0170ff04
00000000000060a4 0000000000000000 A 0 0 4
[ 9] __ksymtab_strings PROGBITS ffffffff82515fa8 01715fa8
0000000000037bea 0000000000000001 AMS 0 0 1
[10] __init_rodata PROGBITS ffffffff8254dba0 0174dba0
00000000000002a8 0000000000000000 A 0 0 32
[11] __param PROGBITS ffffffff8254de48 0174de48
00000000000038e0 0000000000000000 A 0 0 8
[12] __modver PROGBITS ffffffff82551728 01751728
00000000000005a0 0000000000000000 WA 0 0 8
[13] __ex_table PROGBITS ffffffff82551cd0 01751cd0
0000000000001af4 0000000000000000 A 0 0 4
[14] .notes NOTE ffffffff825537c4 017537c4
00000000000001ec 0000000000000000 A 0 0 4
[15] .data PROGBITS ffffffff82600000 01800000
00000000003657c0 0000000000000000 WA 0 0 8192
[16] __bug_table PROGBITS ffffffff829657c0 01b657c0
00000000000178d4 0000000000000000 WA 0 0 1
[17] .vvar PROGBITS ffffffff8297e000 01b7e000
0000000000001000 0000000000000000 WA 0 0 16
[18] .data..percpu PROGBITS 0000000000000000 01c00000
000000000002f000 0000000000000000 WA 0 0 4096
[19] .init.text PROGBITS ffffffff829ae000 01dae000
00000000000772e5 0000000000000000 AX 0 0 16
[20] .altinstr_aux PROGBITS ffffffff82a252e5 01e252e5
0000000000002d9b 0000000000000000 AX 0 0 1
[21] .init.data PROGBITS ffffffff82a2a000 01e2a000
00000000001c74f0 0000000000000000 WA 0 0 8192
[22] .x86_cpu_dev.init PROGBITS ffffffff82bf14f0 01ff14f0
0000000000000028 0000000000000000 A 0 0 8
[23] .parainstructions PROGBITS ffffffff82bf1518 01ff1518
0000000000022b7c 0000000000000000 A 0 0 8
[24] .altinstructions PROGBITS ffffffff82c14098 02014098
0000000000006660 0000000000000000 A 0 0 1
[25] .altinstr_replace PROGBITS ffffffff82c1a6f8 0201a6f8
0000000000001789 0000000000000000 AX 0 0 1
[26] .iommu_table PROGBITS ffffffff82c1be88 0201be88
00000000000000f0 0000000000000000 A 0 0 8
[27] .apicdrivers PROGBITS ffffffff82c1bf78 0201bf78
0000000000000040 0000000000000000 WA 0 0 8
[28] .exit.text PROGBITS ffffffff82c1bfb8 0201bfb8
0000000000001f4b 0000000000000000 AX 0 0 1
[29] .smp_locks PROGBITS ffffffff82c1e000 0201e000
000000000000a000 0000000000000000 A 0 0 4
[30] .data_nosave PROGBITS ffffffff82c28000 02028000
0000000000001000 0000000000000000 WA 0 0 4
[31] .bss NOBITS ffffffff82c29000 02029000
00000000005d7000 0000000000000000 WA 0 0 4096
[32] .brk NOBITS ffffffff83200000 02029000
000000000002c000 0000000000000000 WA 0 0 1
[33] .init.scratch PROGBITS ffffffff83400000 02800000
0000000000400000 0000000000000000 WA 0 0 32
[34] .shstrtab STRTAB 0000000000000000 02c00000
0000000000000180 0000000000000000 0 0 1
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
L (link order), O (extra OS processing required), G (group), T (TLS),
C (compressed), x (unknown), o (OS specific), E (exclude),
l (large), p (processor specific)
There are no section groups in this file.
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
LOAD 0x0000000000200000 0xffffffff81000000 0x0000000001000000
0x00000000015539b0 0x00000000015539b0 R E 0x200000
LOAD 0x0000000001800000 0xffffffff82600000 0x0000000002600000
0x000000000037f000 0x000000000037f000 RW 0x200000
LOAD 0x0000000001c00000 0x0000000000000000 0x000000000297f000
0x000000000002f000 0x000000000002f000 RW 0x200000
LOAD 0x0000000001dae000 0xffffffff829ae000 0x00000000029ae000
0x0000000000e52000 0x0000000000e52000 RWE 0x200000
NOTE 0x00000000017537c4 0xffffffff825537c4 0x00000000025537c4
0x00000000000001ec 0x00000000000001ec 0x4
Section to Segment mapping:
Segment Sections...
00 .text .rodata .pci_fixup .tracedata __ksymtab __ksymtab_gpl __kcrctab __kcrctab_gpl __ksymtab_strings __init_rodata __param __modver __ex_table .notes
01 .data __bug_table .vvar
02 .data..percpu
03 .init.text .altinstr_aux .init.data .x86_cpu_dev.init .parainstructions .altinstructions .altinstr_replacement .iommu_table .apicdrivers .exit.text .smp_locks .data_nosave .bss .brk .init.scratch
04 .notes
There is no dynamic section in this file.
There are no relocations in this file.
The decoding of unwind sections for machine type Advanced Micro Devices X86-64 is not currently supported.
No version information found in this file.
Displaying notes found in: .notes
Owner Data size Description
Xen 0x00000006 Unknown note type: (0x00000006)
description data: 6c 69 6e 75 78 00
Xen 0x00000004 Unknown note type: (0x00000007)
description data: 32 2e 36 00
Xen 0x00000008 Unknown note type: (0x00000005)
description data: 78 65 6e 2d 33 2e 30 00
Xen 0x00000008 Unknown note type: (0x00000003)
description data: 00 00 00 80 ff ff ff ff
Xen 0x00000008 Unknown note type: (0x0000000f)
description data: 00 00 00 00 80 00 00 00
Xen 0x00000008 NT_VERSION (version)
description data: c0 e1 9a 82 ff ff ff ff
Xen 0x00000008 NT_ARCH (architecture)
description data: 00 20 00 81 ff ff ff ff
Xen 0x00000029 Unknown note type: (0x0000000a)
description data: 21 77 72 69 74 61 62 6c 65 5f 70 61 67 65 5f 74 61 62 6c 65 73 7c 70 61 65 5f 70 67 64 69 72 5f 61 62 6f 76 65 5f 34 67 62
Xen 0x00000004 Unknown note type: (0x00000011)
description data: 01 88 00 00
Xen 0x00000004 Unknown note type: (0x00000009)
description data: 79 65 73 00
Xen 0x00000008 Unknown note type: (0x00000008)
description data: 67 65 6e 65 72 69 63 00
Xen 0x00000010 Unknown note type: (0x0000000d)
description data: 01 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00
Xen 0x00000004 Unknown note type: (0x0000000e)
description data: 01 00 00 00
Xen 0x00000004 Unknown note type: (0x00000010)
description data: 01 00 00 00
Xen 0x00000008 Unknown note type: (0x0000000c)
description data: 00 00 00 00 00 80 ff ff
Xen 0x00000008 Unknown note type: (0x00000004)
description data: 00 00 00 00 00 00 00 00
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: c0263a3075bc0a9388365ddf35ab5422da3356a9
Linux 0x00000001 OPEN
description data: 00
Xen 0x00000008 Unknown note type: (0x00000012)
description data: 40 07 00 01 00 00 00 00
The important section for the question is the Program headers section:
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
LOAD 0x0000000000200000 0xffffffff81000000 0x0000000001000000
0x00000000015539b0 0x00000000015539b0 R E 0x200000
LOAD 0x0000000001800000 0xffffffff82600000 0x0000000002600000
0x000000000037f000 0x000000000037f000 RW 0x200000
LOAD 0x0000000001c00000 0x0000000000000000 0x000000000297f000
0x000000000002f000 0x000000000002f000 RW 0x200000
LOAD 0x0000000001dae000 0xffffffff829ae000 0x00000000029ae000
0x0000000000e52000 0x0000000000e52000 RWE 0x200000
NOTE 0x00000000017537c4 0xffffffff825537c4 0x00000000025537c4
0x00000000000001ec 0x00000000000001ec 0x4
I would guess that the RW sections are the data/bss segments of the static and freestanding kernel image. There isn't much bss data (uninitialized data) in this kernel image. You can see that by looking at the filesize versus the memsize of the different segments. In bss data, only the size is written in the executable. The loader will then assign zeroes to these segments. Since we have only memsizes which are equal to the filesizes, then we don't have much bss here.
Also, we can look at the different virtual addresses at which the kernel will load itself after decompression. This will be useful to find the segment in which a certain global variable resides.
Now, to determine where a global variable is loaded one can look at the System.map file in the /boot directory. For example, we can take a random global variable in kernel/sched/core.c called sysctl_sched_rt_period and find its address using
user@user-System-Product-Name:~$ sudo grep "sysctl_sched_rt_period" /boot/System.map-$(uname -r)
ffffffff8266eb24 D sysctl_sched_rt_period
The address is thus ffffffff8266eb24. This address resides in the RW segment loaded at ffffffff82600000 because the size of this segment is 37f000 and because ffffffff82600000 + 37f000 > ffffffff8266eb24. You can clearly see that this global variable is stored in the data segment of the kernel's executable.
Upvotes: 6
Related Questions
- Are envirnoment variables part of the kernel space?
- Location of OS Kernel Data
- Linux: where are environment variables stored?
- Where are global variables located in the elf file
- Where are global (statically compiled) variables located?
- Where are global variables which are static stored in memory?
- Why are global variables static in the linux kernel?
- Where is kernel configuration stored on Linux target?
- How to declare global variables used by all the kernel code in kernel area?
- where is the global and static variables stored in memory segment?