Skip to content

"NameError: name 'difference' is not defined" in comparing APKs due to recent change

Recent change (probably apksigner addition) cause NameError: name 'difference' is not defined error for diffing some APKs, example:

  1. compare below apks:

https://verification.f-droid.org/tmp/fi.harism.wallpaper.yinyang_9.apk

https://verification.f-droid.org/fi.harism.wallpaper.yinyang_9.apk

  1. Running diffoscope on apk files that will cause apktool to fail, and have same content except 1 additional file in one of them (more info in issue #125)

on updated version of diffoscope from git (latest commit 0293f49a) will fail with "NameError: name 'difference' is not defined", see:

diffoscope 106_039ea82f26db61aa480f9000620c78f1-copy.apk 106_039ea82f26db61aa480f9000620c78f1.apk --debug  --max-page-diff-block-lines 5 --max-page-diff-block-lines 5 --max-diff-input-lines 10 --max-container-depth 2 --max-diff-block-lines-saved 5
2020-05-18 22:43:24 D: diffoscope.main: Starting diffoscope 144
2020-05-18 22:43:24 D: diffoscope.main: Free space in temporary directory: 25.41 GiB
2020-05-18 22:43:24 D: diffoscope.presenters.formats: Will generate the following formats: text
2020-05-18 22:43:24 D: diffoscope.environ: Normalising locale, timezone, etc. Inheriting PATH of /srv/diffoscope/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2020-05-18 22:43:24 D: diffoscope.main: Starting comparison
2020-05-18 22:43:24 D: diffoscope.comparators: Loaded 76 comparator classes
2020-05-18 22:43:26 D: diffoscope.comparators.utils.specialize: Using diffoscope.comparators.apk.ApkFile for 106_039ea82f26db61aa480f9000620c78f1-copy.apk
2020-05-18 22:43:27 D: diffoscope.comparators.utils.specialize: Using diffoscope.comparators.apk.ApkFile for 106_039ea82f26db61aa480f9000620c78f1.apk
2020-05-18 22:43:27 D: diffoscope.comparators.utils.compare: Comparing 106_039ea82f26db61aa480f9000620c78f1-copy.apk (ApkFile) and 106_039ea82f26db61aa480f9000620c78f1.apk (ApkFile)
2020-05-18 22:43:27 D: diffoscope.comparators.utils.file: File.has_same_content: <<class 'abc.ApkFile'> 106_039ea82f26db61aa480f9000620c78f1-copy.apk> <<class 'abc.ApkFile'> 106_039ea82f26db61aa480f9000620c78f1.apk>
2020-05-18 22:43:30 D: diffoscope.comparators.utils.command: Executing zipinfo {}
2020-05-18 22:43:30 D: diffoscope.comparators.utils.command: Executing zipinfo {}
2020-05-18 22:43:30 D: diffoscope.tempfiles: Created top-level temporary directory: /tmp/diffoscope_h9teoaz4
2020-05-18 22:43:30 D: diffoscope.diff: Running diff -aU7 /tmp/diffoscope_h9teoaz4/tmp_nxbq005/fifo1 /tmp/diffoscope_h9teoaz4/tmp_nxbq005/fifo2
2020-05-18 22:43:30 D: diffoscope.diff: diff -aU7 /tmp/diffoscope_h9teoaz4/tmp_nxbq005/fifo1 /tmp/diffoscope_h9teoaz4/tmp_nxbq005/fifo2: returncode 1, parsed True
2020-05-18 22:43:30 D: diffoscope.comparators.utils.command: Executing java -jar /usr/bin/apksigner verify --verbose --print-certs {}
2020-05-18 22:43:31 D: diffoscope.comparators.utils.command: Executing java -jar /usr/bin/apksigner verify --verbose --print-certs {}
2020-05-18 22:43:35 D: diffoscope.diff: Running diff -aU7 /tmp/diffoscope_h9teoaz4/tmpbi7mdcyw/fifo1 /tmp/diffoscope_h9teoaz4/tmpbi7mdcyw/fifo2
2020-05-18 22:43:35 D: diffoscope.diff: diff -aU7 /tmp/diffoscope_h9teoaz4/tmpbi7mdcyw/fifo1 /tmp/diffoscope_h9teoaz4/tmpbi7mdcyw/fifo2: returncode 1, parsed True
Traceback (most recent call last):
  File "/srv/diffoscope/diffoscope/main.py", line 745, in main
    sys.exit(run_diffoscope(parsed_args))
  File "/srv/diffoscope/diffoscope/main.py", line 697, in run_diffoscope
    difference = compare_root_paths(path1, path2)
  File "/srv/diffoscope/diffoscope/comparators/utils/compare.py", line 74, in compare_root_paths
    difference = compare_files(file1, file2)
  File "/srv/diffoscope/diffoscope/comparators/utils/compare.py", line 128, in compare_files
    return file1.compare(file2, source)
  File "/srv/diffoscope/diffoscope/comparators/utils/file.py", line 453, in compare
    difference = self._compare_using_details(other, source)
  File "/srv/diffoscope/diffoscope/comparators/utils/file.py", line 371, in _compare_using_details
    details.extend(self.compare_details(other, source))
  File "/srv/diffoscope/diffoscope/comparators/apk.py", line 224, in compare_details
    difference.insert(0, x)
NameError: name 'difference' is not defined

on older diffoscope version 140 (image registry.salsa.debian.org/reproducible-builds/diffoscope) the result is:

diffoscope 106_039ea82f26db61aa480f9000620c78f1-copy.apk 106_039ea82f26db61aa480f9000620c78f1.apk --debug  --max-page-diff-block-lines 5 --max-page-diff-block-lines 5 --max-diff-input-lines 10 --max-container-depth 2 --max-diff-block-lines-saved 5
2020-05-18 22:55:27 D: diffoscope.main: Starting diffoscope 140
2020-05-18 22:55:27 D: diffoscope.presenters.formats: Will generate the following formats: text
2020-05-18 22:55:27 D: diffoscope.environ: Normalising locale, timezone, etc. Inheriting PATH of /srv/diffoscope/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2020-05-18 22:55:27 D: diffoscope.main: Starting comparison
2020-05-18 22:55:27 D: diffoscope.comparators: Loaded 74 comparator classes
2020-05-18 22:55:27 D: diffoscope.comparators.utils.specialize: Using diffoscope.comparators.apk.ApkFile for 106_039ea82f26db61aa480f9000620c78f1-copy.apk
2020-05-18 22:55:27 D: diffoscope.comparators.utils.specialize: Using diffoscope.comparators.apk.ApkFile for 106_039ea82f26db61aa480f9000620c78f1.apk
2020-05-18 22:55:27 D: diffoscope.comparators.utils.compare: Comparing 106_039ea82f26db61aa480f9000620c78f1-copy.apk (ApkFile) and 106_039ea82f26db61aa480f9000620c78f1.apk (ApkFile)
2020-05-18 22:55:27 D: diffoscope.comparators.utils.file: File.has_same_content: <<class 'abc.ApkFile'> 106_039ea82f26db61aa480f9000620c78f1-copy.apk> <<class 'abc.ApkFile'> 106_039ea82f26db61aa480f9000620c78f1.apk>
2020-05-18 22:55:27 D: diffoscope.comparators.utils.command: Executing zipinfo /dev/stdin
2020-05-18 22:55:27 D: diffoscope.comparators.utils.command: Executing zipinfo /dev/stdin
2020-05-18 22:55:28 D: diffoscope.tempfiles: Created top-level temporary directory: /tmp/diffoscope_cbwbdxhe
2020-05-18 22:55:28 D: diffoscope.diff: Running diff -aU7 /tmp/diffoscope_cbwbdxhe/tmp7_ht6wwx/fifo1 /tmp/diffoscope_cbwbdxhe/tmp7_ht6wwx/fifo2
2020-05-18 22:55:28 D: diffoscope.diff: diff -aU7 /tmp/diffoscope_cbwbdxhe/tmp7_ht6wwx/fifo1 /tmp/diffoscope_cbwbdxhe/tmp7_ht6wwx/fifo2: returncode 1, parsed True
2020-05-18 22:55:28 D: diffoscope.comparators.utils.file: Instantiating a diffoscope.comparators.apk.ApkContainer for 106_039ea82f26db61aa480f9000620c78f1-copy.apk
2020-05-18 22:55:28 D: diffoscope.comparators.apk: Extracting 106_039ea82f26db61aa480f9000620c78f1-copy.apk to /tmp/diffoscope_cbwbdxhe/tmpxlrajqox/106_039ea82f26db61aa480f9000620c78f1-copy.apk
W: Unknown data detected. Skipping: 80 byte(s)
W: Unknown data detected. Skipping: 48 byte(s)
W: Unknown data detected. Skipping: 1136 byte(s)
W: Unknown data detected. Skipping: 1296 byte(s)
W: Unknown data detected. Skipping: 1760 byte(s)
W: Cant find 9patch chunk in file: "x/ame.9.png". Renaming it to *.png.
W: Cant find 9patch chunk in file: "x/ame.9.png". Renaming it to *.png.
Exception in thread "main" java.lang.NullPointerException
	at brut.androlib.res.data.value.ResEnumAttr.serializeBody(ResEnumAttr.java:56)
	at brut.androlib.res.data.value.ResAttr.serializeToResValuesXml(ResAttr.java:64)
	at brut.androlib.res.AndrolibResources.generateValuesFile(AndrolibResources.java:742)
	at brut.androlib.res.AndrolibResources.decode(AndrolibResources.java:263)
	at brut.androlib.Androlib.decodeResourcesFull(Androlib.java:129)
	at brut.androlib.ApkDecoder.decode(ApkDecoder.java:124)
	at brut.apktool.Main.cmdDecode(Main.java:170)
	at brut.apktool.Main.main(Main.java:76)
2020-05-18 22:55:31 D: diffoscope.comparators.utils.command: Executing xxd {}
2020-05-18 22:55:38 D: diffoscope.comparators.utils.command: Executing xxd {}
2020-05-18 22:55:46 D: diffoscope.diff: Running diff -aU7 /tmp/diffoscope_cbwbdxhe/tmpft0tyaz5/fifo1 /tmp/diffoscope_cbwbdxhe/tmpft0tyaz5/fifo2
2020-05-18 22:55:50 D: diffoscope.diff: diff -aU7 /tmp/diffoscope_cbwbdxhe/tmpft0tyaz5/fifo1 /tmp/diffoscope_cbwbdxhe/tmpft0tyaz5/fifo2: returncode 1, parsed True
2020-05-18 22:55:50 D: diffoscope.presenters.formats: Generating 'text' output at '-'
--- 106_039ea82f26db61aa480f9000620c78f1-copy.apk
+++ 106_039ea82f26db61aa480f9000620c78f1.apk
│┄ Command `apktool d -k -m -o /tmp/diffoscope_cbwbdxhe/tmpxlrajqox/106_039ea82f26db61aa480f9000620c78f1-copy.apk 106_039ea82f26db61aa480f9000620c78f1-copy.apk` exited with exit code 1. (No output)
@@ -3,8 +3,8 @@
 00000020: 6472 6f69 644d 616e 6966 6573 742e 786d  droidManifest.xm
 00000030: 6c74 d60b 7857 6318 00f0 f724 3393 4932  lt..xWc....$3.I2
 00000040: cc64 3299 9924 9321 9999 4c66 6692 4ce6  .d2..$.!..Lff.L.
 00000050: 524d f197 9949 ee77 1272 bfdf 93dc 496e  RM...I.w.r....In
 00000060: b993 2121 f724 b94d c86d 9264 48fc 9e99  ..!!.$.M.m.dH...
 00000070: e579 d09e dfff 9cf3 9d73 beef fdde f7fb  .y.......s......
 00000080: ced3 4a91 1e65 ebac 1449 e444 d3d0 242e  ..J..e...I.D..$.
-[ Too much input for diff (SHA1: 2dc0af4cb4505a1d0ad15b5eda8c1ccfa0bf20bb) ]
+[ Too much input for diff (SHA1: 728073cd1b1820d958109680aa3630884eca6ca3) ]
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information