-
upstream/1.2125ad41e7 · ·
DWARF loader: - Handle DWARF5 DW_OP_addrx properly Part of the effort to support the subset of DWARF5 that is generated when building the kernel. - Handle subprogram ret type with abstract_origin properly Adds a second pass to resolve abstract origin DWARF description of functions to aid the BTF encoder in getting the right return type. - Check .notes section for LTO build info When LTO is used, currently only with clang, we need to do extra steps to handle references from one object (compile unit, aka CU) to another, a way for DWARF to avoid duplicating information. - Check .debug_abbrev for cross-CU references When the kernel build process doesn't add an ELF note in vmlinux indicating that LTO was used and thus intra-CU references are present and thus we need to use a more expensive way to resolve types and (again) thus to encode BTF, we need to look at DWARF's .debug_abbrev ELF section to figure out if such intra-CU references are present. - Permit merging all DWARF CU's for clang LTO built binary Allow not trowing away previously supposedly self contained compile units (objects, aka CU, aka Compile Units) as they have type descriptions that will be used in later CUs. - Permit a flexible HASHTAGS__BITS So that we can use a more expensive algorithm when we need to keep previously processed compile units that will then be referenced by later ones to resolve types. - Use a better hashing function, from libbpf Enabling patch to combine compile units when using LTO. BTF encoder: - Add --btf_gen_all flag A new command line to allow asking for the generation of all BTF encodings, so that we can stop adding new command line options to enable new encodings in the kernel Makefile. - Match ftrace addresses within ELF functions To cope with differences in how DWARF and ftrace describes function boundaries. - Funnel ELF error reporting through a macro To use libelf's elf_error() function, improving error messages. - Sanitize non-regular int base type Cope with clang with dwarf5 non-regular int base types, tricky stuff, see yhs full explanation in the relevant cset. - Add support for the floating-point types S/390 has floats'n'doubles in its arch specific linux headers, cope with that. Pretty printer: - Honour conf_fprintf.hex when printing enumerations If the user specifies --hex in the command line, honour it when printing enumerations. Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
-
upstream/v1.2125ad41e7 · ·
DWARF loader: - Handle DWARF5 DW_OP_addrx properly Part of the effort to support the subset of DWARF5 that is generated when building the kernel. - Handle subprogram ret type with abstract_origin properly Adds a second pass to resolve abstract origin DWARF description of functions to aid the BTF encoder in getting the right return type. - Check .notes section for LTO build info When LTO is used, currently only with clang, we need to do extra steps to handle references from one object (compile unit, aka CU) to another, a way for DWARF to avoid duplicating information. - Check .debug_abbrev for cross-CU references When the kernel build process doesn't add an ELF note in vmlinux indicating that LTO was used and thus intra-CU references are present and thus we need to use a more expensive way to resolve types and (again) thus to encode BTF, we need to look at DWARF's .debug_abbrev ELF section to figure out if such intra-CU references are present. - Permit merging all DWARF CU's for clang LTO built binary Allow not trowing away previously supposedly self contained compile units (objects, aka CU, aka Compile Units) as they have type descriptions that will be used in later CUs. - Permit a flexible HASHTAGS__BITS So that we can use a more expensive algorithm when we need to keep previously processed compile units that will then be referenced by later ones to resolve types. - Use a better hashing function, from libbpf Enabling patch to combine compile units when using LTO. BTF encoder: - Add --btf_gen_all flag A new command line to allow asking for the generation of all BTF encodings, so that we can stop adding new command line options to enable new encodings in the kernel Makefile. - Match ftrace addresses within ELF functions To cope with differences in how DWARF and ftrace describes function boundaries. - Funnel ELF error reporting through a macro To use libelf's elf_error() function, improving error messages. - Sanitize non-regular int base type Cope with clang with dwarf5 non-regular int base types, tricky stuff, see yhs full explanation in the relevant cset. - Add support for the floating-point types S/390 has floats'n'doubles in its arch specific linux headers, cope with that. Pretty printer: - Honour conf_fprintf.hex when printing enumerations If the user specifies --hex in the command line, honour it when printing enumerations. Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
-
v1.2125ad41e7 · ·
DWARF loader: - Handle DWARF5 DW_OP_addrx properly Part of the effort to support the subset of DWARF5 that is generated when building the kernel. - Handle subprogram ret type with abstract_origin properly Adds a second pass to resolve abstract origin DWARF description of functions to aid the BTF encoder in getting the right return type. - Check .notes section for LTO build info When LTO is used, currently only with clang, we need to do extra steps to handle references from one object (compile unit, aka CU) to another, a way for DWARF to avoid duplicating information. - Check .debug_abbrev for cross-CU references When the kernel build process doesn't add an ELF note in vmlinux indicating that LTO was used and thus intra-CU references are present and thus we need to use a more expensive way to resolve types and (again) thus to encode BTF, we need to look at DWARF's .debug_abbrev ELF section to figure out if such intra-CU references are present. - Permit merging all DWARF CU's for clang LTO built binary Allow not trowing away previously supposedly self contained compile units (objects, aka CU, aka Compile Units) as they have type descriptions that will be used in later CUs. - Permit a flexible HASHTAGS__BITS So that we can use a more expensive algorithm when we need to keep previously processed compile units that will then be referenced by later ones to resolve types. - Use a better hashing function, from libbpf Enabling patch to combine compile units when using LTO. BTF encoder: - Add --btf_gen_all flag A new command line to allow asking for the generation of all BTF encodings, so that we can stop adding new command line options to enable new encodings in the kernel Makefile. - Match ftrace addresses within ELF functions To cope with differences in how DWARF and ftrace describes function boundaries. - Funnel ELF error reporting through a macro To use libelf's elf_error() function, improving error messages. - Sanitize non-regular int base type Cope with clang with dwarf5 non-regular int base types, tricky stuff, see yhs full explanation in the relevant cset. - Add support for the floating-point types S/390 has floats'n'doubles in its arch specific linux headers, cope with that. Pretty printer: - Honour conf_fprintf.hex when printing enumerations If the user specifies --hex in the command line, honour it when printing enumerations. Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
-
acmel/1.200d415f68 · ·
v1.20: BTF encoder: - Improve ELF error reporting using elf_errmsg(elf_errno()). - Improve objcopy error handling. - Fix handling of 'restrict' qualifier, that was being treated as a 'const'. - Support SHN_XINDEX in st_shndx symbol indexes, to handle ELF objects with more than 65534 sections, for instance, which happens with kernels built with 'KCFLAGS="-ffunction-sections -fdata-sections", Other cases may include when using FG-ASLR, LTO. - Cope with functions without a name, as seen sometimes when building kernel images with some versions of clang, when a SEGFAULT was taking place. - Fix BTF variable generation for kernel modules, not skipping variables at offset zero. - Fix address size to match what is in the ELF file being processed, to fix using a 64-bit pahole binary to generate BTF for a 32-bit vmlinux image. - Use kernel module ftrace addresses when finding which functions to encode, which increases the number of functions encoded. libbpf: - Allow use of packaged version, for distros wanting to dynamically link with the system's libbpf package instead of using the libbpf git submodule shipped in pahole's source code. DWARF loader: - Support DW_AT_data_bit_offset This appeared in DWARF4 but is supported only in gcc's -gdwarf-5, support it in a way that makes the output be the same for both cases. $ gcc -gdwarf-5 -c examples/dwarf5/bf.c $ pahole bf.o struct pea { long int a:1; /* 0: 0 8 */ long int b:1; /* 0: 1 8 */ long int c:1; /* 0: 2 8 */ /* XXX 29 bits hole, try to pack */ /* Bitfield combined with next fields */ int after_bitfield; /* 4 4 */ /* size: 8, cachelines: 1, members: 4 */ /* sum members: 4 */ /* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */ /* last cacheline: 8 bytes */ }; - DW_FORM_implicit_const in attr_numeric() and attr_offset() - Support DW_TAG_GNU_call_site, its the standardized rename of the previously supported DW_TAG_GNU_call_site. build: - Fix compilation on 32-bit architectures. Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> -
upstream/1.200d415f68 · ·
v1.20: BTF encoder: - Improve ELF error reporting using elf_errmsg(elf_errno()). - Improve objcopy error handling. - Fix handling of 'restrict' qualifier, that was being treated as a 'const'. - Support SHN_XINDEX in st_shndx symbol indexes, to handle ELF objects with more than 65534 sections, for instance, which happens with kernels built with 'KCFLAGS="-ffunction-sections -fdata-sections", Other cases may include when using FG-ASLR, LTO. - Cope with functions without a name, as seen sometimes when building kernel images with some versions of clang, when a SEGFAULT was taking place. - Fix BTF variable generation for kernel modules, not skipping variables at offset zero. - Fix address size to match what is in the ELF file being processed, to fix using a 64-bit pahole binary to generate BTF for a 32-bit vmlinux image. - Use kernel module ftrace addresses when finding which functions to encode, which increases the number of functions encoded. libbpf: - Allow use of packaged version, for distros wanting to dynamically link with the system's libbpf package instead of using the libbpf git submodule shipped in pahole's source code. DWARF loader: - Support DW_AT_data_bit_offset This appeared in DWARF4 but is supported only in gcc's -gdwarf-5, support it in a way that makes the output be the same for both cases. $ gcc -gdwarf-5 -c examples/dwarf5/bf.c $ pahole bf.o struct pea { long int a:1; /* 0: 0 8 */ long int b:1; /* 0: 1 8 */ long int c:1; /* 0: 2 8 */ /* XXX 29 bits hole, try to pack */ /* Bitfield combined with next fields */ int after_bitfield; /* 4 4 */ /* size: 8, cachelines: 1, members: 4 */ /* sum members: 4 */ /* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */ /* last cacheline: 8 bytes */ }; - DW_FORM_implicit_const in attr_numeric() and attr_offset() - Support DW_TAG_GNU_call_site, its the standardized rename of the previously supported DW_TAG_GNU_call_site. build: - Fix compilation on 32-bit architectures. Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> -
upstream/v1.200d415f68 · ·
v1.20: BTF encoder: - Improve ELF error reporting using elf_errmsg(elf_errno()). - Improve objcopy error handling. - Fix handling of 'restrict' qualifier, that was being treated as a 'const'. - Support SHN_XINDEX in st_shndx symbol indexes, to handle ELF objects with more than 65534 sections, for instance, which happens with kernels built with 'KCFLAGS="-ffunction-sections -fdata-sections", Other cases may include when using FG-ASLR, LTO. - Cope with functions without a name, as seen sometimes when building kernel images with some versions of clang, when a SEGFAULT was taking place. - Fix BTF variable generation for kernel modules, not skipping variables at offset zero. - Fix address size to match what is in the ELF file being processed, to fix using a 64-bit pahole binary to generate BTF for a 32-bit vmlinux image. - Use kernel module ftrace addresses when finding which functions to encode, which increases the number of functions encoded. libbpf: - Allow use of packaged version, for distros wanting to dynamically link with the system's libbpf package instead of using the libbpf git submodule shipped in pahole's source code. DWARF loader: - Support DW_AT_data_bit_offset This appeared in DWARF4 but is supported only in gcc's -gdwarf-5, support it in a way that makes the output be the same for both cases. $ gcc -gdwarf-5 -c examples/dwarf5/bf.c $ pahole bf.o struct pea { long int a:1; /* 0: 0 8 */ long int b:1; /* 0: 1 8 */ long int c:1; /* 0: 2 8 */ /* XXX 29 bits hole, try to pack */ /* Bitfield combined with next fields */ int after_bitfield; /* 4 4 */ /* size: 8, cachelines: 1, members: 4 */ /* sum members: 4 */ /* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */ /* last cacheline: 8 bytes */ }; - DW_FORM_implicit_const in attr_numeric() and attr_offset() - Support DW_TAG_GNU_call_site, its the standardized rename of the previously supported DW_TAG_GNU_call_site. build: - Fix compilation on 32-bit architectures. Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> -
v1.200d415f68 · ·
v1.20: BTF encoder: - Improve ELF error reporting using elf_errmsg(elf_errno()). - Improve objcopy error handling. - Fix handling of 'restrict' qualifier, that was being treated as a 'const'. - Support SHN_XINDEX in st_shndx symbol indexes, to handle ELF objects with more than 65534 sections, for instance, which happens with kernels built with 'KCFLAGS="-ffunction-sections -fdata-sections", Other cases may include when using FG-ASLR, LTO. - Cope with functions without a name, as seen sometimes when building kernel images with some versions of clang, when a SEGFAULT was taking place. - Fix BTF variable generation for kernel modules, not skipping variables at offset zero. - Fix address size to match what is in the ELF file being processed, to fix using a 64-bit pahole binary to generate BTF for a 32-bit vmlinux image. - Use kernel module ftrace addresses when finding which functions to encode, which increases the number of functions encoded. libbpf: - Allow use of packaged version, for distros wanting to dynamically link with the system's libbpf package instead of using the libbpf git submodule shipped in pahole's source code. DWARF loader: - Support DW_AT_data_bit_offset This appeared in DWARF4 but is supported only in gcc's -gdwarf-5, support it in a way that makes the output be the same for both cases. $ gcc -gdwarf-5 -c examples/dwarf5/bf.c $ pahole bf.o struct pea { long int a:1; /* 0: 0 8 */ long int b:1; /* 0: 1 8 */ long int c:1; /* 0: 2 8 */ /* XXX 29 bits hole, try to pack */ /* Bitfield combined with next fields */ int after_bitfield; /* 4 4 */ /* size: 8, cachelines: 1, members: 4 */ /* sum members: 4 */ /* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */ /* last cacheline: 8 bytes */ }; - DW_FORM_implicit_const in attr_numeric() and attr_offset() - Support DW_TAG_GNU_call_site, its the standardized rename of the previously supported DW_TAG_GNU_call_site. build: - Fix compilation on 32-bit architectures. Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> -
acmel/1.19dd15aa4b · ·
v1.19: - Support split BTF, where a main BTF file, vmlinux, can be used to find types and then a kernel module, for instance, can have just what is unique to it. For instance, looking for a type in the main vmlinux BTF info: $ pahole wmi_notify_handler pahole: type 'wmi_notify_handler' not found $ If we look at the 'wmi' module BTF info that is in: $ ls -la /sys/kernel/btf/wmi -r--r--r--. 1 root root 2866 Nov 18 13:35 /sys/kernel/btf/wmi $ $ pahole /sys/kernel/btf/wmi -C wmi_notify_handler typedef void (*wmi_notify_handler)(u32, void *); $ '--btf_base=/sys/kernel/btf/vmlinux' was automatically added in this last example, an option that was also introduced in this version where types used in the wmi.ko module but present in vmlinux can be found so that there is no duplicity of types. - Update libbpf to get the split BTF support and use some of its functions to load BTF and speed up DWARF loading and BTF encoding. - Support cross-compiled ELF binaries with different endianness - Support showing typedefs for anonymous types, like structs, unions and enums, see the "Align enumerators" entry below for an example, another: $ pahole rwlock_t typedef struct { arch_rwlock_t raw_lock; /* 0 8 */ /* size: 8, cachelines: 1, members: 1 */ /* last cacheline: 8 bytes */ } rwlock_t; $ - Align enumerators: $ pahole ZSTD_strategy typedef enum { ZSTD_fast = 0, ZSTD_dfast = 1, ZSTD_greedy = 2, ZSTD_lazy = 3, ZSTD_lazy2 = 4, ZSTD_btlazy2 = 5, ZSTD_btopt = 6, ZSTD_btopt2 = 7, } ZSTD_strategy; $ - Workaround bugs in the generation of DWARF records for functions in some gcc versions that were causing breakage in the encoding of BTF: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97060 "Missing DW_AT_declaration=1 in dwarf data" - Ignore zero-sized ELF symbols instead of erroring out. - Handle union forward declaration properly in the BTF loader. - Introduce --numeric_version for use in scripts and Makefiles: $ pahole --version v1.19 $ pahole --numeric_version 119 $ To avoid things like this in the kernel's scripts/link-vmlinux.sh: pahole_ver=$(${PAHOLE} --version | sed -E 's/v([0-9]+)\.([0-9]+)/\1\2/') - Try sole pfunct argument as a function name, just like pahole with type names: $ pfunct tcp_v4_rcv int tcp_v4_rcv(struct sk_buff * skb); $ - Speed up pfunct using some of the load techniques used in pahole. - Discard CUs after BTF encoding as they're not used anymore, greatly reducing memory usage and speeding up vmlinux BTF encoding. - Revamp how per-CPU variables are encoded in BTF. - Include BTF info for static functions. - Use BTF's string APIs for strings management, greatly improving performance over the tsearch(). - Increase size of DWARF lookup hash table, shaving off about 1 second out of about 20 seconds total for Linux BTF dedup. - Stop BTF encoding when errors are found in some DWARF CU. - Implement --packed, to show just packed structures, for instance, here are the top 5 packed data structures in the Linux kernel: $ pahole --sizes --packed | sort -k2 -nr | head -5 e820_table 64004 0 boot_params 4096 0 efi_variable 2084 0 snd_soc_tplg_pcm 912 0 ntb_info_regs 800 0 $ And here is one of them: $ pahole efi_variable struct efi_variable { efi_char16_t VariableName[512]; /* 0 1024 */ /* --- cacheline 16 boundary (1024 bytes) --- */ efi_guid_t VendorGuid; /* 1024 16 */ long unsigned int DataSize; /* 1040 8 */ __u8 Data[1024]; /* 1048 1024 */ /* --- cacheline 32 boundary (2048 bytes) was 24 bytes ago --- */ efi_status_t Status; /* 2072 8 */ __u32 Attributes; /* 2080 4 */ /* size: 2084, cachelines: 33, members: 6 */ /* last cacheline: 36 bytes */ } __attribute__((__packed__)); $ - Fix bug in distros such as OpenSUSE:15.2 where DW_AT_alignment isn't defined. Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> -
upstream/1.19dd15aa4b · ·
v1.19: - Support split BTF, where a main BTF file, vmlinux, can be used to find types and then a kernel module, for instance, can have just what is unique to it. For instance, looking for a type in the main vmlinux BTF info: $ pahole wmi_notify_handler pahole: type 'wmi_notify_handler' not found $ If we look at the 'wmi' module BTF info that is in: $ ls -la /sys/kernel/btf/wmi -r--r--r--. 1 root root 2866 Nov 18 13:35 /sys/kernel/btf/wmi $ $ pahole /sys/kernel/btf/wmi -C wmi_notify_handler typedef void (*wmi_notify_handler)(u32, void *); $ '--btf_base=/sys/kernel/btf/vmlinux' was automatically added in this last example, an option that was also introduced in this version where types used in the wmi.ko module but present in vmlinux can be found so that there is no duplicity of types. - Update libbpf to get the split BTF support and use some of its functions to load BTF and speed up DWARF loading and BTF encoding. - Support cross-compiled ELF binaries with different endianness - Support showing typedefs for anonymous types, like structs, unions and enums, see the "Align enumerators" entry below for an example, another: $ pahole rwlock_t typedef struct { arch_rwlock_t raw_lock; /* 0 8 */ /* size: 8, cachelines: 1, members: 1 */ /* last cacheline: 8 bytes */ } rwlock_t; $ - Align enumerators: $ pahole ZSTD_strategy typedef enum { ZSTD_fast = 0, ZSTD_dfast = 1, ZSTD_greedy = 2, ZSTD_lazy = 3, ZSTD_lazy2 = 4, ZSTD_btlazy2 = 5, ZSTD_btopt = 6, ZSTD_btopt2 = 7, } ZSTD_strategy; $ - Workaround bugs in the generation of DWARF records for functions in some gcc versions that were causing breakage in the encoding of BTF: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97060 "Missing DW_AT_declaration=1 in dwarf data" - Ignore zero-sized ELF symbols instead of erroring out. - Handle union forward declaration properly in the BTF loader. - Introduce --numeric_version for use in scripts and Makefiles: $ pahole --version v1.19 $ pahole --numeric_version 119 $ To avoid things like this in the kernel's scripts/link-vmlinux.sh: pahole_ver=$(${PAHOLE} --version | sed -E 's/v([0-9]+)\.([0-9]+)/\1\2/') - Try sole pfunct argument as a function name, just like pahole with type names: $ pfunct tcp_v4_rcv int tcp_v4_rcv(struct sk_buff * skb); $ - Speed up pfunct using some of the load techniques used in pahole. - Discard CUs after BTF encoding as they're not used anymore, greatly reducing memory usage and speeding up vmlinux BTF encoding. - Revamp how per-CPU variables are encoded in BTF. - Include BTF info for static functions. - Use BTF's string APIs for strings management, greatly improving performance over the tsearch(). - Increase size of DWARF lookup hash table, shaving off about 1 second out of about 20 seconds total for Linux BTF dedup. - Stop BTF encoding when errors are found in some DWARF CU. - Implement --packed, to show just packed structures, for instance, here are the top 5 packed data structures in the Linux kernel: $ pahole --sizes --packed | sort -k2 -nr | head -5 e820_table 64004 0 boot_params 4096 0 efi_variable 2084 0 snd_soc_tplg_pcm 912 0 ntb_info_regs 800 0 $ And here is one of them: $ pahole efi_variable struct efi_variable { efi_char16_t VariableName[512]; /* 0 1024 */ /* --- cacheline 16 boundary (1024 bytes) --- */ efi_guid_t VendorGuid; /* 1024 16 */ long unsigned int DataSize; /* 1040 8 */ __u8 Data[1024]; /* 1048 1024 */ /* --- cacheline 32 boundary (2048 bytes) was 24 bytes ago --- */ efi_status_t Status; /* 2072 8 */ __u32 Attributes; /* 2080 4 */ /* size: 2084, cachelines: 33, members: 6 */ /* last cacheline: 36 bytes */ } __attribute__((__packed__)); $ - Fix bug in distros such as OpenSUSE:15.2 where DW_AT_alignment isn't defined. Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> -
upstream/v1.19dd15aa4b · ·
v1.19: - Support split BTF, where a main BTF file, vmlinux, can be used to find types and then a kernel module, for instance, can have just what is unique to it. For instance, looking for a type in the main vmlinux BTF info: $ pahole wmi_notify_handler pahole: type 'wmi_notify_handler' not found $ If we look at the 'wmi' module BTF info that is in: $ ls -la /sys/kernel/btf/wmi -r--r--r--. 1 root root 2866 Nov 18 13:35 /sys/kernel/btf/wmi $ $ pahole /sys/kernel/btf/wmi -C wmi_notify_handler typedef void (*wmi_notify_handler)(u32, void *); $ '--btf_base=/sys/kernel/btf/vmlinux' was automatically added in this last example, an option that was also introduced in this version where types used in the wmi.ko module but present in vmlinux can be found so that there is no duplicity of types. - Update libbpf to get the split BTF support and use some of its functions to load BTF and speed up DWARF loading and BTF encoding. - Support cross-compiled ELF binaries with different endianness - Support showing typedefs for anonymous types, like structs, unions and enums, see the "Align enumerators" entry below for an example, another: $ pahole rwlock_t typedef struct { arch_rwlock_t raw_lock; /* 0 8 */ /* size: 8, cachelines: 1, members: 1 */ /* last cacheline: 8 bytes */ } rwlock_t; $ - Align enumerators: $ pahole ZSTD_strategy typedef enum { ZSTD_fast = 0, ZSTD_dfast = 1, ZSTD_greedy = 2, ZSTD_lazy = 3, ZSTD_lazy2 = 4, ZSTD_btlazy2 = 5, ZSTD_btopt = 6, ZSTD_btopt2 = 7, } ZSTD_strategy; $ - Workaround bugs in the generation of DWARF records for functions in some gcc versions that were causing breakage in the encoding of BTF: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97060 "Missing DW_AT_declaration=1 in dwarf data" - Ignore zero-sized ELF symbols instead of erroring out. - Handle union forward declaration properly in the BTF loader. - Introduce --numeric_version for use in scripts and Makefiles: $ pahole --version v1.19 $ pahole --numeric_version 119 $ To avoid things like this in the kernel's scripts/link-vmlinux.sh: pahole_ver=$(${PAHOLE} --version | sed -E 's/v([0-9]+)\.([0-9]+)/\1\2/') - Try sole pfunct argument as a function name, just like pahole with type names: $ pfunct tcp_v4_rcv int tcp_v4_rcv(struct sk_buff * skb); $ - Speed up pfunct using some of the load techniques used in pahole. - Discard CUs after BTF encoding as they're not used anymore, greatly reducing memory usage and speeding up vmlinux BTF encoding. - Revamp how per-CPU variables are encoded in BTF. - Include BTF info for static functions. - Use BTF's string APIs for strings management, greatly improving performance over the tsearch(). - Increase size of DWARF lookup hash table, shaving off about 1 second out of about 20 seconds total for Linux BTF dedup. - Stop BTF encoding when errors are found in some DWARF CU. - Implement --packed, to show just packed structures, for instance, here are the top 5 packed data structures in the Linux kernel: $ pahole --sizes --packed | sort -k2 -nr | head -5 e820_table 64004 0 boot_params 4096 0 efi_variable 2084 0 snd_soc_tplg_pcm 912 0 ntb_info_regs 800 0 $ And here is one of them: $ pahole efi_variable struct efi_variable { efi_char16_t VariableName[512]; /* 0 1024 */ /* --- cacheline 16 boundary (1024 bytes) --- */ efi_guid_t VendorGuid; /* 1024 16 */ long unsigned int DataSize; /* 1040 8 */ __u8 Data[1024]; /* 1048 1024 */ /* --- cacheline 32 boundary (2048 bytes) was 24 bytes ago --- */ efi_status_t Status; /* 2072 8 */ __u32 Attributes; /* 2080 4 */ /* size: 2084, cachelines: 33, members: 6 */ /* last cacheline: 36 bytes */ } __attribute__((__packed__)); $ - Fix bug in distros such as OpenSUSE:15.2 where DW_AT_alignment isn't defined. Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> -
v1.19dd15aa4b · ·
v1.19: - Support split BTF, where a main BTF file, vmlinux, can be used to find types and then a kernel module, for instance, can have just what is unique to it. For instance, looking for a type in the main vmlinux BTF info: $ pahole wmi_notify_handler pahole: type 'wmi_notify_handler' not found $ If we look at the 'wmi' module BTF info that is in: $ ls -la /sys/kernel/btf/wmi -r--r--r--. 1 root root 2866 Nov 18 13:35 /sys/kernel/btf/wmi $ $ pahole /sys/kernel/btf/wmi -C wmi_notify_handler typedef void (*wmi_notify_handler)(u32, void *); $ '--btf_base=/sys/kernel/btf/vmlinux' was automatically added in this last example, an option that was also introduced in this version where types used in the wmi.ko module but present in vmlinux can be found so that there is no duplicity of types. - Update libbpf to get the split BTF support and use some of its functions to load BTF and speed up DWARF loading and BTF encoding. - Support cross-compiled ELF binaries with different endianness - Support showing typedefs for anonymous types, like structs, unions and enums, see the "Align enumerators" entry below for an example, another: $ pahole rwlock_t typedef struct { arch_rwlock_t raw_lock; /* 0 8 */ /* size: 8, cachelines: 1, members: 1 */ /* last cacheline: 8 bytes */ } rwlock_t; $ - Align enumerators: $ pahole ZSTD_strategy typedef enum { ZSTD_fast = 0, ZSTD_dfast = 1, ZSTD_greedy = 2, ZSTD_lazy = 3, ZSTD_lazy2 = 4, ZSTD_btlazy2 = 5, ZSTD_btopt = 6, ZSTD_btopt2 = 7, } ZSTD_strategy; $ - Workaround bugs in the generation of DWARF records for functions in some gcc versions that were causing breakage in the encoding of BTF: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97060 "Missing DW_AT_declaration=1 in dwarf data" - Ignore zero-sized ELF symbols instead of erroring out. - Handle union forward declaration properly in the BTF loader. - Introduce --numeric_version for use in scripts and Makefiles: $ pahole --version v1.19 $ pahole --numeric_version 119 $ To avoid things like this in the kernel's scripts/link-vmlinux.sh: pahole_ver=$(${PAHOLE} --version | sed -E 's/v([0-9]+)\.([0-9]+)/\1\2/') - Try sole pfunct argument as a function name, just like pahole with type names: $ pfunct tcp_v4_rcv int tcp_v4_rcv(struct sk_buff * skb); $ - Speed up pfunct using some of the load techniques used in pahole. - Discard CUs after BTF encoding as they're not used anymore, greatly reducing memory usage and speeding up vmlinux BTF encoding. - Revamp how per-CPU variables are encoded in BTF. - Include BTF info for static functions. - Use BTF's string APIs for strings management, greatly improving performance over the tsearch(). - Increase size of DWARF lookup hash table, shaving off about 1 second out of about 20 seconds total for Linux BTF dedup. - Stop BTF encoding when errors are found in some DWARF CU. - Implement --packed, to show just packed structures, for instance, here are the top 5 packed data structures in the Linux kernel: $ pahole --sizes --packed | sort -k2 -nr | head -5 e820_table 64004 0 boot_params 4096 0 efi_variable 2084 0 snd_soc_tplg_pcm 912 0 ntb_info_regs 800 0 $ And here is one of them: $ pahole efi_variable struct efi_variable { efi_char16_t VariableName[512]; /* 0 1024 */ /* --- cacheline 16 boundary (1024 bytes) --- */ efi_guid_t VendorGuid; /* 1024 16 */ long unsigned int DataSize; /* 1040 8 */ __u8 Data[1024]; /* 1048 1024 */ /* --- cacheline 32 boundary (2048 bytes) was 24 bytes ago --- */ efi_status_t Status; /* 2072 8 */ __u32 Attributes; /* 2080 4 */ /* size: 2084, cachelines: 33, members: 6 */ /* last cacheline: 36 bytes */ } __attribute__((__packed__)); $ - Fix bug in distros such as OpenSUSE:15.2 where DW_AT_alignment isn't defined. Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> -
acmel/1.18fdc63918 · ·
v1.18: - Use type information to pretty print raw data from stdin, all documented in the man pages, further information in the csets. TLDRish: this almost completely deciphers a perf.data file: $ pahole ~/bin/perf --header=perf_file_header \ -C 'perf_file_attr(range=attrs),perf_event_header(range=data,sizeof,type,type_enum=perf_event_type+perf_user_event_type)' < perf.data What the above command line does: This will state that a 'struct perf_file_header' is available in BTF or DWARF in the ~/bin/perf file and that at the start of stdin it should be used to decode sizeof(struct perf_file_header) bytes, pretty printing it according to its members. Furthermore, that header can be referenced later in the command line, for instance that 'range=data' means that in the header, it expects a 'range' member in 'struct perf_file_header' to be used: $ pahole ~/bin/perf --header=perf_file_header < perf.data { .magic = 3622385352885552464, .size = 104, .attr_size = 136, .attrs = { .offset = 296, .size = 136, }, .data = { .offset = 432, .size = 14688, }, .event_types = { .offset = 0, .size = 0, }, .adds_features = { 376537084, 0, 0, 0 }, }, $ That 'range' field is expected to have 'offset' and 'size' fields, so that it can go on decoding a number of 'struct perf_event_header' entries. That 'sizeof' in turn expects that in 'struct perf_event_header' there is a 'size' field stating how long that particular record is, one can also use 'sizeof=some_other_member_name'. This supports variable sized records and then the 'type' field expects there is a 'struct perf_event_type' member named 'type' (again, type=something_else may be used. Finally, the value in the 'type' field is used to lookup an entry in the set formed by the two enumerations specified in the 'type_enum=' argument. If we look at these enums we'll see that its entries have names that can be, when lowercased, point to structs containing the layout for the variable sized record, which allows it to cast and produce the right pretty printed output. I.e. using the kernel BTF info we get: $ pahole perf_event_type enum perf_event_type { PERF_RECORD_MMAP = 1, PERF_RECORD_LOST = 2, PERF_RECORD_COMM = 3, PERF_RECORD_EXIT = 4, PERF_RECORD_THROTTLE = 5, PERF_RECORD_UNTHROTTLE = 6, PERF_RECORD_FORK = 7, PERF_RECORD_READ = 8, PERF_RECORD_SAMPLE = 9, PERF_RECORD_MMAP2 = 10, PERF_RECORD_AUX = 11, PERF_RECORD_ITRACE_START = 12, PERF_RECORD_LOST_SAMPLES = 13, PERF_RECORD_SWITCH = 14, PERF_RECORD_SWITCH_CPU_WIDE = 15, PERF_RECORD_NAMESPACES = 16, PERF_RECORD_KSYMBOL = 17, PERF_RECORD_BPF_EVENT = 18, PERF_RECORD_CGROUP = 19, PERF_RECORD_TEXT_POKE = 20, PERF_RECORD_MAX = 21, }; $ That is the same as in ~/bin/perf, and, if we get one of these and ask for that struct: $ pahole -C perf_record_mmap ~/bin/perf struct perf_record_mmap { struct perf_event_header header; /* 0 8 */ __u32 pid; /* 8 4 */ __u32 tid; /* 12 4 */ __u64 start; /* 16 8 */ __u64 len; /* 24 8 */ __u64 pgoff; /* 32 8 */ char filename[4096]; /* 40 4096 */ /* size: 4136, cachelines: 65, members: 7 */ /* last cacheline: 40 bytes */ }; $ Many other options were introduced to work with this, including --count, --skip, etc, look at the man page for details. - Store percpu variables in vmlinux BTF. This can be disabled when debugging kernel features being developed to use it. - pahole now should be segfault free when handling gdb test suit DWARF files, including ADA, FORTRAN, rust and dwp compressed files, the later being just flatly refused, that got left for v1.19. - Bail out on partial units for now, avoiding segfaults and providing warning to user, hopefully will be addressed in v1.19. Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> -
upstream/1.18fdc63918 · ·
v1.18: - Use type information to pretty print raw data from stdin, all documented in the man pages, further information in the csets. TLDRish: this almost completely deciphers a perf.data file: $ pahole ~/bin/perf --header=perf_file_header \ -C 'perf_file_attr(range=attrs),perf_event_header(range=data,sizeof,type,type_enum=perf_event_type+perf_user_event_type)' < perf.data What the above command line does: This will state that a 'struct perf_file_header' is available in BTF or DWARF in the ~/bin/perf file and that at the start of stdin it should be used to decode sizeof(struct perf_file_header) bytes, pretty printing it according to its members. Furthermore, that header can be referenced later in the command line, for instance that 'range=data' means that in the header, it expects a 'range' member in 'struct perf_file_header' to be used: $ pahole ~/bin/perf --header=perf_file_header < perf.data { .magic = 3622385352885552464, .size = 104, .attr_size = 136, .attrs = { .offset = 296, .size = 136, }, .data = { .offset = 432, .size = 14688, }, .event_types = { .offset = 0, .size = 0, }, .adds_features = { 376537084, 0, 0, 0 }, }, $ That 'range' field is expected to have 'offset' and 'size' fields, so that it can go on decoding a number of 'struct perf_event_header' entries. That 'sizeof' in turn expects that in 'struct perf_event_header' there is a 'size' field stating how long that particular record is, one can also use 'sizeof=some_other_member_name'. This supports variable sized records and then the 'type' field expects there is a 'struct perf_event_type' member named 'type' (again, type=something_else may be used. Finally, the value in the 'type' field is used to lookup an entry in the set formed by the two enumerations specified in the 'type_enum=' argument. If we look at these enums we'll see that its entries have names that can be, when lowercased, point to structs containing the layout for the variable sized record, which allows it to cast and produce the right pretty printed output. I.e. using the kernel BTF info we get: $ pahole perf_event_type enum perf_event_type { PERF_RECORD_MMAP = 1, PERF_RECORD_LOST = 2, PERF_RECORD_COMM = 3, PERF_RECORD_EXIT = 4, PERF_RECORD_THROTTLE = 5, PERF_RECORD_UNTHROTTLE = 6, PERF_RECORD_FORK = 7, PERF_RECORD_READ = 8, PERF_RECORD_SAMPLE = 9, PERF_RECORD_MMAP2 = 10, PERF_RECORD_AUX = 11, PERF_RECORD_ITRACE_START = 12, PERF_RECORD_LOST_SAMPLES = 13, PERF_RECORD_SWITCH = 14, PERF_RECORD_SWITCH_CPU_WIDE = 15, PERF_RECORD_NAMESPACES = 16, PERF_RECORD_KSYMBOL = 17, PERF_RECORD_BPF_EVENT = 18, PERF_RECORD_CGROUP = 19, PERF_RECORD_TEXT_POKE = 20, PERF_RECORD_MAX = 21, }; $ That is the same as in ~/bin/perf, and, if we get one of these and ask for that struct: $ pahole -C perf_record_mmap ~/bin/perf struct perf_record_mmap { struct perf_event_header header; /* 0 8 */ __u32 pid; /* 8 4 */ __u32 tid; /* 12 4 */ __u64 start; /* 16 8 */ __u64 len; /* 24 8 */ __u64 pgoff; /* 32 8 */ char filename[4096]; /* 40 4096 */ /* size: 4136, cachelines: 65, members: 7 */ /* last cacheline: 40 bytes */ }; $ Many other options were introduced to work with this, including --count, --skip, etc, look at the man page for details. - Store percpu variables in vmlinux BTF. This can be disabled when debugging kernel features being developed to use it. - pahole now should be segfault free when handling gdb test suit DWARF files, including ADA, FORTRAN, rust and dwp compressed files, the later being just flatly refused, that got left for v1.19. - Bail out on partial units for now, avoiding segfaults and providing warning to user, hopefully will be addressed in v1.19. Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> -
upstream/v1.18fdc63918 · ·
v1.18: - Use type information to pretty print raw data from stdin, all documented in the man pages, further information in the csets. TLDRish: this almost completely deciphers a perf.data file: $ pahole ~/bin/perf --header=perf_file_header \ -C 'perf_file_attr(range=attrs),perf_event_header(range=data,sizeof,type,type_enum=perf_event_type+perf_user_event_type)' < perf.data What the above command line does: This will state that a 'struct perf_file_header' is available in BTF or DWARF in the ~/bin/perf file and that at the start of stdin it should be used to decode sizeof(struct perf_file_header) bytes, pretty printing it according to its members. Furthermore, that header can be referenced later in the command line, for instance that 'range=data' means that in the header, it expects a 'range' member in 'struct perf_file_header' to be used: $ pahole ~/bin/perf --header=perf_file_header < perf.data { .magic = 3622385352885552464, .size = 104, .attr_size = 136, .attrs = { .offset = 296, .size = 136, }, .data = { .offset = 432, .size = 14688, }, .event_types = { .offset = 0, .size = 0, }, .adds_features = { 376537084, 0, 0, 0 }, }, $ That 'range' field is expected to have 'offset' and 'size' fields, so that it can go on decoding a number of 'struct perf_event_header' entries. That 'sizeof' in turn expects that in 'struct perf_event_header' there is a 'size' field stating how long that particular record is, one can also use 'sizeof=some_other_member_name'. This supports variable sized records and then the 'type' field expects there is a 'struct perf_event_type' member named 'type' (again, type=something_else may be used. Finally, the value in the 'type' field is used to lookup an entry in the set formed by the two enumerations specified in the 'type_enum=' argument. If we look at these enums we'll see that its entries have names that can be, when lowercased, point to structs containing the layout for the variable sized record, which allows it to cast and produce the right pretty printed output. I.e. using the kernel BTF info we get: $ pahole perf_event_type enum perf_event_type { PERF_RECORD_MMAP = 1, PERF_RECORD_LOST = 2, PERF_RECORD_COMM = 3, PERF_RECORD_EXIT = 4, PERF_RECORD_THROTTLE = 5, PERF_RECORD_UNTHROTTLE = 6, PERF_RECORD_FORK = 7, PERF_RECORD_READ = 8, PERF_RECORD_SAMPLE = 9, PERF_RECORD_MMAP2 = 10, PERF_RECORD_AUX = 11, PERF_RECORD_ITRACE_START = 12, PERF_RECORD_LOST_SAMPLES = 13, PERF_RECORD_SWITCH = 14, PERF_RECORD_SWITCH_CPU_WIDE = 15, PERF_RECORD_NAMESPACES = 16, PERF_RECORD_KSYMBOL = 17, PERF_RECORD_BPF_EVENT = 18, PERF_RECORD_CGROUP = 19, PERF_RECORD_TEXT_POKE = 20, PERF_RECORD_MAX = 21, }; $ That is the same as in ~/bin/perf, and, if we get one of these and ask for that struct: $ pahole -C perf_record_mmap ~/bin/perf struct perf_record_mmap { struct perf_event_header header; /* 0 8 */ __u32 pid; /* 8 4 */ __u32 tid; /* 12 4 */ __u64 start; /* 16 8 */ __u64 len; /* 24 8 */ __u64 pgoff; /* 32 8 */ char filename[4096]; /* 40 4096 */ /* size: 4136, cachelines: 65, members: 7 */ /* last cacheline: 40 bytes */ }; $ Many other options were introduced to work with this, including --count, --skip, etc, look at the man page for details. - Store percpu variables in vmlinux BTF. This can be disabled when debugging kernel features being developed to use it. - pahole now should be segfault free when handling gdb test suit DWARF files, including ADA, FORTRAN, rust and dwp compressed files, the later being just flatly refused, that got left for v1.19. - Bail out on partial units for now, avoiding segfaults and providing warning to user, hopefully will be addressed in v1.19. Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> -
v1.18fdc63918 · ·
v1.18: - Use type information to pretty print raw data from stdin, all documented in the man pages, further information in the csets. TLDRish: this almost completely deciphers a perf.data file: $ pahole ~/bin/perf --header=perf_file_header \ -C 'perf_file_attr(range=attrs),perf_event_header(range=data,sizeof,type,type_enum=perf_event_type+perf_user_event_type)' < perf.data What the above command line does: This will state that a 'struct perf_file_header' is available in BTF or DWARF in the ~/bin/perf file and that at the start of stdin it should be used to decode sizeof(struct perf_file_header) bytes, pretty printing it according to its members. Furthermore, that header can be referenced later in the command line, for instance that 'range=data' means that in the header, it expects a 'range' member in 'struct perf_file_header' to be used: $ pahole ~/bin/perf --header=perf_file_header < perf.data { .magic = 3622385352885552464, .size = 104, .attr_size = 136, .attrs = { .offset = 296, .size = 136, }, .data = { .offset = 432, .size = 14688, }, .event_types = { .offset = 0, .size = 0, }, .adds_features = { 376537084, 0, 0, 0 }, }, $ That 'range' field is expected to have 'offset' and 'size' fields, so that it can go on decoding a number of 'struct perf_event_header' entries. That 'sizeof' in turn expects that in 'struct perf_event_header' there is a 'size' field stating how long that particular record is, one can also use 'sizeof=some_other_member_name'. This supports variable sized records and then the 'type' field expects there is a 'struct perf_event_type' member named 'type' (again, type=something_else may be used. Finally, the value in the 'type' field is used to lookup an entry in the set formed by the two enumerations specified in the 'type_enum=' argument. If we look at these enums we'll see that its entries have names that can be, when lowercased, point to structs containing the layout for the variable sized record, which allows it to cast and produce the right pretty printed output. I.e. using the kernel BTF info we get: $ pahole perf_event_type enum perf_event_type { PERF_RECORD_MMAP = 1, PERF_RECORD_LOST = 2, PERF_RECORD_COMM = 3, PERF_RECORD_EXIT = 4, PERF_RECORD_THROTTLE = 5, PERF_RECORD_UNTHROTTLE = 6, PERF_RECORD_FORK = 7, PERF_RECORD_READ = 8, PERF_RECORD_SAMPLE = 9, PERF_RECORD_MMAP2 = 10, PERF_RECORD_AUX = 11, PERF_RECORD_ITRACE_START = 12, PERF_RECORD_LOST_SAMPLES = 13, PERF_RECORD_SWITCH = 14, PERF_RECORD_SWITCH_CPU_WIDE = 15, PERF_RECORD_NAMESPACES = 16, PERF_RECORD_KSYMBOL = 17, PERF_RECORD_BPF_EVENT = 18, PERF_RECORD_CGROUP = 19, PERF_RECORD_TEXT_POKE = 20, PERF_RECORD_MAX = 21, }; $ That is the same as in ~/bin/perf, and, if we get one of these and ask for that struct: $ pahole -C perf_record_mmap ~/bin/perf struct perf_record_mmap { struct perf_event_header header; /* 0 8 */ __u32 pid; /* 8 4 */ __u32 tid; /* 12 4 */ __u64 start; /* 16 8 */ __u64 len; /* 24 8 */ __u64 pgoff; /* 32 8 */ char filename[4096]; /* 40 4096 */ /* size: 4136, cachelines: 65, members: 7 */ /* last cacheline: 40 bytes */ }; $ Many other options were introduced to work with this, including --count, --skip, etc, look at the man page for details. - Store percpu variables in vmlinux BTF. This can be disabled when debugging kernel features being developed to use it. - pahole now should be segfault free when handling gdb test suit DWARF files, including ADA, FORTRAN, rust and dwp compressed files, the later being just flatly refused, that got left for v1.19. - Bail out on partial units for now, avoiding segfaults and providing warning to user, hopefully will be addressed in v1.19. Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>