FileNotFound Errors in comparing APKs/IPAs
Comparing different versions of the same Android or IOS app, may return FileNotFound errors.
for example: Hamagen app APKs from:
https://github.com/MohGovIL/hamagen-react-native/releases/download/v1.3.8/Hamagen-1.3.8.apk
https://github.com/MohGovIL/hamagen-react-native/releases/download/v1.2.4-qa/app-qa-1.2.4.apk
on try.diffoscope.org return error message Whoops! An error occurred when comparing your files, full messages here https://try.diffoscope.org/pytpdkvcasyt
snippet of probably related messages:
Whoops! An error occurred when comparing your files
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
mkdir: cannot create directory ‘/var/www’: Read-only file system
ln: failed to create symbolic link '/var/www/.local/share/apktool/framework//1.apk': No such file or directory
S: WARNING: Could not write to (/var/www/.local/share/apktool/framework), using /tmp instead...
S: Please be aware this is a volatile directory and frameworks could go missing, please utilize --frame-path if the default storage directory is unavailable
W: Could not decode attr value, using undecoded value instead: ns=android, name=versionCode, value=0x0000002e
W: Could not decode attr value, using undecoded value instead: ns=android, name=versionName, value=0x00000020
Exception in thread "main" brut.androlib.AndrolibException: java.io.FileNotFoundException: /tmp/1.apk (Read-only file system)
at brut.androlib.res.AndrolibResources.getFrameworkApk(AndrolibResources.java:775)
at brut.androlib.res.AndrolibResources.loadFrameworkPkg(AndrolibResources.java:116)
at brut.androlib.res.data.ResTable.getPackage(ResTable.java:84)
at brut.androlib.res.data.ResTable.getResSpec(ResTable.java:67)
at brut.androlib.res.data.ResTable.getResSpec(ResTable.java:63)
at brut.androlib.res.data.value.ResReferenceValue.getReferent(ResReferenceValue.java:62)
at brut.androlib.res.data.value.ResReferenceValue.encodeAsResXml(ResReferenceValue.java:48)
at brut.androlib.res.data.value.ResScalarValue.encodeAsResXmlValue(ResScalarValue.java:58)
at brut.androlib.res.data.value.ResStyleValue.serializeToResValuesXml(ResStyleValue.java:78)
at brut.androlib.res.AndrolibResources.generateValuesFile(AndrolibResources.java:697)
at brut.androlib.res.AndrolibResources.decode(AndrolibResources.java:263)
at brut.androlib.Androlib.decodeResourcesFull(Androlib.java:131)
at brut.androlib.ApkDecoder.decode(ApkDecoder.java:124)
at brut.apktool.Main.cmdDecode(Main.java:164)
at brut.apktool.Main.main(Main.java:73)
Caused by: java.io.FileNotFoundException: /tmp/1.apk (Read-only file system)
at java.base/java.io.FileOutputStream.open0(Native Method)
at java.base/java.io.FileOutputStream.open(FileOutputStream.java:298)
at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:237)
at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:187)
at brut.androlib.res.AndrolibResources.getFrameworkApk(AndrolibResources.java:771)
... 14 morealso comparing .ipa files instead of apk for above app result in similar errors: https://try.diffoscope.org/qcsfrtxtfkke
which was comparing one of the following files:
https://github.com/MohGovIL/hamagen-react-native/releases/download/v1.3.8-qa/hamagen-1.3.8-qa.ipa
https://github.com/MohGovIL/hamagen-react-native/releases/download/v1.2.4-qa/hamagen-qa.ipa
https://github.com/MohGovIL/hamagen-react-native/releases/download/v1.2.2-qa.1/hamagen-qa.ipa
(as can seen in '--profile=- a/hamagen-qa.ipa b/hamagen-1.3.8-qa.ipa')
but rerun recently always return clean message:
Timeout Sorry, it took too long to process your files. Install and run diffoscope on your system to avoid this limit.
Whoops! An error occurred when comparing your files
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 418, in compare
difference = self._compare_using_details(other, source)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 336, in _compare_using_details
details.extend(self.compare_details(other, source))
File "/usr/lib/python3/dist-packages/diffoscope/comparators/macho.py", line 97, in compare_details
my_archs = MachoFile.get_arch_from_macho(self.path)
File "/usr/lib/python3/dist-packages/diffoscope/tools.py", line 92, in tool_check
raise RequiredToolNotFound(command)
diffoscope.exc.RequiredToolNotFound: lipo
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/diffoscope/diff.py", line 331, in diff
return run_diff(fifo1_path, fifo2_path, fifo1.end_nl_q, fifo2.end_nl_q)
File "/usr/lib/python3/dist-packages/diffoscope/diff.py", line 216, in __exit__
self.join()
File "/usr/lib/python3/dist-packages/diffoscope/diff.py", line 249, in join
raise self._exception
File "/usr/lib/python3/dist-packages/diffoscope/diff.py", line 241, in run
self.end_nl_q.put(end_nl)
BrokenPipeError: [Errno 32] Broken pipe
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/diffoscope", line 36, in <module>
main()
File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 726, in main
sys.exit(run_diffoscope(parsed_args))
File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 679, in run_diffoscope
difference = compare_root_paths(path1, path2)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 72, in compare_root_paths
difference = compare_files(file1, file2)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 124, in compare_files
return file1.compare(file2, source)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 418, in compare
difference = self._compare_using_details(other, source)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 366, in _compare_using_details
other.as_container, no_recurse=no_recurse
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/container.py", line 195, in compare_pair
file1, file2, source=None, diff_content_only=no_recurse
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 124, in compare_files
return file1.compare(file2, source)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 452, in compare
difference = self.compare_bytes(other, source=source)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 329, in compare_bytes
return compare_binary_files(self, other, source)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 146, in compare_binary_files
has_internal_linenos=True,
File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 254, in from_command
klass, path1, path2, *args, **kwargs
File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 287, in from_command_exc
feeder1, feeder2, path1, path2, *args, **kwargs
File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 211, in from_feeder
unified_diff = diff(feeder1, feeder2)
File "/usr/lib/python3/dist-packages/diffoscope/diff.py", line 331, in diff
return run_diff(fifo1_path, fifo2_path, fifo1.end_nl_q, fifo2.end_nl_q)
File "/usr/lib/python3/dist-packages/diffoscope/diff.py", line 216, in __exit__
self.join()
File "/usr/lib/python3/dist-packages/diffoscope/diff.py", line 249, in join
raise self._exception
File "/usr/lib/python3/dist-packages/diffoscope/diff.py", line 241, in run
self.end_nl_q.put(end_nl)
BrokenPipeError: [Errno 32] Broken pipe
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/", "total": 107712669, "current": 8193}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/_CodeSignature/", "total": 107712669, "current": 16385}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/_CodeSignature/CodeResources", "total": 107712669, "current": 24577}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/LaunchImage-1100-Portrait-2436h@3x.png", "total": 107712669, "current": 123061}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/SimplerPro_V3-Black.otf", "total": 107712669, "current": 312693}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/AppIcon20x20@2x.png", "total": 107712669, "current": 426645}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/Info-qa.plist", "total": 107712669, "current": 431249}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/SimplerPro_V3-Bold.otf", "total": 107712669, "current": 434572}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/AppIcon20x20@3x.png", "total": 107712669, "current": 552972}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/AppIcon60x60@2x.png", "total": 107712669, "current": 560632}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/AppIcon29x29@2x.png", "total": 107712668, "current": 578938}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/LaunchImage-800-667h@2x.png", "total": 107712669, "current": 586270}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/AppIcon40x40@3x.png", "total": 107712669, "current": 664252}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/AppIcon40x40@2x.png", "total": 107712669, "current": 683708}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/LaunchImage-1200-Portrait-2688h@3x.png", "total": 107712669, "current": 694870}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/AppIcon29x29@3x.png", "total": 107712669, "current": 887598}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/GoogleService-Info.plist", "total": 107712669, "current": 900466}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/AppIcon60x60@3x.png", "total": 107712669, "current": 902094}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/Base.lproj/", "total": 107712669, "current": 932110}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/Assets.car", "total": 107712669, "current": 940302}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/LaunchImage-800-Portrait-736h@3x.png", "total": 107712669, "current": 1506206}
{"msg": "Payload/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA.app/\u256b\u00f6\u256b\u20a7\u256b\u00c6\u256b\u0192 QA", "total": 107712669, "current": 1668424}
{"msg": "", "total": 107712669, "current": 42531560}
{"msg": "", "total": 107712669, "current": 107712669}
# Profiling output for: /usr/bin/diffoscope --status-fd=2 --html output.html --text output.txt --profile=- a/hamagen-qa.ipa b/hamagen-1.3.8-qa.ipa
## command (total time: 4.154s)
3.417s 8 calls diff
0.562s 28 calls xxd
0.162s 8 calls cmp
0.008s 11 calls zipnote
0.003s 11 calls zipinfo
0.001s 12 calls cmp (internal)
## compare_files (cumulative) (total time: 0.688s)
0.548s 2 calls diffoscope.comparators.utils.archive.ArchiveMember
0.140s 1 call abc.TextFile
0.000s 4 calls diffoscope.comparators.zip.ZipDirectory
## container_extract (total time: 1.902s)
1.902s 267 calls diffoscope.comparators.zip.ZipContainer
## has_same_content_as (total time: 0.170s)
0.163s 19 calls diffoscope.comparators.utils.archive.ArchiveMember
0.007s 1 call abc.ZipFile
0.000s 4 calls diffoscope.comparators.zip.ZipDirectory
## main (total time: 0.025s)
0.025s 1 call cleanup
## open_archive (total time: 0.005s)
0.005s 2 calls diffoscope.comparators.zip.ZipContainer
## recognizes (total time: 0.035s)
0.026s 652 calls diffoscope.comparators.utils.archive.ArchiveMember
0.007s 114 calls diffoscope.comparators.binary.FilesystemFile
0.002s 112 calls abc.ZipFile
Traceback (most recent call last):
File "/srv/trydiffoscope/trydiffoscope/compare/tasks.py", line 67, in execute_diffoscope
with open(html_output, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/srv/storage.try.diffoscope.org/qcsfrtxtfkke/output.html'so I guess it's related to #2 as the comparing will take time to complete, and it's not deterministic (at least for IPA compare), but maybe more issues are there (why clean error-message is not always shown?)