Ensure that all of our artificially-created subprocess.CalledProcessError instances have `output` instances that are bytes objects, not str.

Gbp-Dch: ignore

Skip calls to unsquashfs when we are not root or fakeroot. (Closes: reproducible-builds/diffoscope#63)

This is a little unfortunate but currently easier than recreating the
test1.squashfs and test2.squashfs test input files to remove the character
device.

See also 4689755d so this general issue is
visible to end-users.

Include either standard error or standard output (and not just the latter) when an external command fails.

Avoid a possible traceback caused by a str/bytes confusion when handling the output of failing external commands.

Gbp-Dch: ignore

Test for "4.4" in the output of "squashfs -version", even though the Debian package version is "1:4.3+git190823-1". Thanks, Mattia. (Re: reproducible-builds/diffoscope#62)

Gbp-Dch: ignore

Signed-off-by: Mattia Rizzolo <mattia@debian.org>

Gbp-Dch: ignore

Apply patch from László Böszörményi to update the squashfs test output and additionally bump the required version for the test itself. (Closes: reproducible-builds/diffoscope#62, Debian:#935684)

Gbp-Dch: Ignore
Signed-off-by: Mattia Rizzolo <mattia@debian.org>

Gbp-Dch: ignore

Correct/revert some in-process debugging code that was accidentally committed in d3c7ac8e.

Don't fallback to a (useless) raw hexdump when, for example, readelf(1) reports an minor issue in a section in an ELF binary. For example, when the "frames" section is of the "NOBITS" type its contents are apparently "unreliable" and thus readelf(1) returns 1). (Closes: reproducible-builds/diffoscope#58, Debian:#931962)

Add support to Difference.from_command_exc and friends to ignore specific returncodes from the called program and treat them as "no" difference.

No need to parse the section group from the class name; we can pass it via type(..)'s kwargs argument.

Gbp-Dch: ignore

Reposition a comment regarding an exception within the indented block. This seems the more-common way of documentation such things, at least in Python.

Gbp-Dch: ignore

I fail at shell, clearly.

Gbp-Dch: ignore

Add an "eg." to the comment on stripping "original size modulo..." text to emphasise that the actual numbers are not fixed.

Gbp-Dch: ignore

Strip off trailing "original size modulo 2^32 671" (etc.) from gzip compressed data as this is just a symptom of the contents itself changing that will be reflected elsewhere. (Closes: reproducible-builds/diffoscope#61)

Not all versions of file(1) support detection of Jmod files yet, so perform a manual comparison instead. (re. reproducible-builds/diffoscope#60)

Thanks to Emmanuel Bourg (ebourg@apache.org) for the bug report.

Signed-off-by: Chris Lamb <lamby@debian.org>

Only override the exit code from our special-cased calls to zipinfo(1) if they are 1 or 2 to avoid potentially masking real errors. (re. reproducible-builds/diffoscope#60)

Factor out the ability to ignore the exit codes of "zipinfo" and "zipinfo -v" in the presence of non-standard headers. (re. reproducible-builds/diffoscope#60)

Marc Herbert authored Jul 16, 2019 and Chris Lamb committed Jul 17, 2019

When trying and failing to disassemble an ELF library from an architecture not
supported by objdump, catch that particular failure and don't let it revert the
*entire* diff to hexdump, throwing the entire kindergarten class with the
bathwater. Fallback on hexdump _only_ for the problematic text section and
rescue all the rest. Example:

Before:

diffoscope build?/app/libapp.a
--- build1/app/libapp.a
+++ build2/app/libapp.a
│┄ Command `objdump --line-numbers --disassemble --demangle --reloc
│┄  --section=.text build1/app/libapp.a` exited with return code 1. Output:
│┄     objdump: can't disassemble for architecture UNKNOWN!
│┄
│┄     objdump: section '.text' mentioned in a -j option, but not found
│┄     in any input file
│┄
@@ -1,22 +1,22 @@
 00000000: 213c 6172 6368 3e0a 2f20 2020 2020 2020  !<arch>./
 00000010: 2020 2020 2020 2020 3135 3633 3333 3638          15633368
-00000020: 3830 2020 3020 2020 2020 3020 2020 2020  80  0     0
+00000020: 3931 2020 3020 2020 2020 3020 2020 2020  91  0     0
 00000030: 3020 2020 2020 2020 3134 2020 2020 2020  0       14
 00000040: 2020 600a 0000 0001 0000 0052 6d61 696e    `........Rmain
 00000050: 0000 6d61 696e 2e63 2e6f 626a 2f20 2020  ..main.c.obj/
 00000060: 2020 3020 2020 2020 2020 2020 2020 3020    0           0
 00000070: 2020 2020 3020 2020 2020 3634 3420 2020      0     644
 00000080: 2020 3130 3332 2020 2020 2020 600a 7f45    1032      `..E
 00000090: 4c46 0101 0100 0000 0000 0000 0000 0100  LF..............
 000000a0: 2800 0100 0000 0000 0000 0000 0000 2802  (.............(.
 000000b0: 0000 0000 0005 3400 0000 0000 2800 0c00  ......4.....(...
 000000c0: 0b00 0149 0248 fff7 febf 0000 0000 0b00  ...I.H..........
 000000d0: 0000 6d70 7332 5f61 6e33 3835 0048 656c  ..mps2_an385.Hel
-000000e0: 6c6f 2057 6f72 6c64 2120 2573 0a00 0047  lo World! %s...G
+000000e0: 6c6f 2058 6f72 6c64 2120 2573 0a00 0047  lo Xorld! %s...G
 000000f0: 4343 3a20 2863 726f 7373 746f 6f6c 2d4e  CC: (crosstool-N
 00000100: 4720 312e 3234 2e30 2d72 6332 2d64 6972  G 1.24.0-rc2-dir
 00000110: 7479 2920 382e 332e 3000 412c 0000 0061  ty) 8.3.0.A,...a
 00000120: 6561 6269 0001 2200 0000 0537 2d4d 0006  eabi.."....7-M..
 00000130: 0a07 4d09 0212 0414 0115 0117 0318 0119  ..M.............
 00000140: 011a 011e 0422 0100 0000 0000 0000 0000  ....."..........
 00000150: 0000 0000 0000 0000 0000 0100 0000 0000  ................

- After:

 2019-07-17 04:15:40 E: diffoscope.comparators.elf:
 Command '['objdump', '--line-numbers', '--disassemble', '--demangle',
 '--reloc', '--section=.text.main', 'build1/app/libapp.a']' returned
 non-zero exit status 1.

2019-07-17 04:15:40 E: diffoscope.comparators.elf:
  Command '['objdump', '--disassemble', '--demangle',
  '--section=.text.main', 'build1/app/libapp.a']' returned non-zero exit
  status 1.

--- build1/app/libapp.a
+++ build2/app/libapp.a
├── file list
│ @@ -1,2 +1,2 @@
│ -----------   0        0        0       14 2019-07-17 04:14:40.000000 /
│ +----------   0        0        0       14 2019-07-17 04:14:51.000000 /
│  ?rw-r--r--   0        0        0     1032 1970-01-01 00:00:00.000000 main.c.obj
├── readelf --wide --decompress --string-dump=.rodata.main.str1.1 {}
│ @@ -1,7 +1,7 @@
│
│  File: lib.a(main.c.obj)
│
│  String dump of section '.rodata.main.str1.1':
│    [     0]  mps2_an385
│ -  [     b]  Hello World! %s^J
│ +  [     b]  Hello Xorld! %s^J

Signed-off-by: Marc Herbert <marc.herbert@gmail.com>
Signed-off-by: Chris Lamb <lamby@debian.org>