UnicodeDecodeError on ttf font file
Ran diffoscope against a zip containing font ttf files. Processing stopped when it encountered a ttf file:
2021-01-24 21:45:18 D: diffoscope.comparators.utils.compare: Comparing static/rapture/resources/fonts/ (ZipDirectory) and static/rapture/resources/fonts/ (ZipDirectory)
2021-01-24 21:45:18 D: diffoscope.comparators.directory: Excluding directory metadata for paths (static/rapture/resources/fonts/, static/rapture/resources/fonts/)
2021-01-24 21:45:18 D: diffoscope.comparators.utils.compare: Comparing static/rapture/resources/fonts/OpenSans-Bold.ttf (ArchiveMember) and static/rapture/resources/fonts/OpenSans-Bold.ttf (ArchiveMember)
2021-01-24 21:45:18 D: diffoscope.comparators.utils.specialize: trying <class 'diffoscope.comparators.directory.Directory'> on <<class 'diffoscope.comparators.utils.archive.ArchiveMember'> static/rapture/resources/fonts/OpenSans-Bold.ttf>
2021-01-24 21:45:18 D: diffoscope.comparators.utils.specialize: trying <class 'diffoscope.comparators.missing_file.MissingFile'> on <<class 'diffoscope.comparators.utils.archive.ArchiveMember'> static/rapture/resources/fonts/OpenSans-Bold.ttf>
2021-01-24 21:45:18 D: diffoscope.comparators.utils.specialize: trying <class 'diffoscope.comparators.symlink.Symlink'> on <<class 'diffoscope.comparators.utils.archive.ArchiveMember'> static/rapture/resources/fonts/OpenSans-Bold.ttf>
2021-01-24 21:45:18 D: diffoscope.comparators.utils.specialize: trying <class 'diffoscope.comparators.device.Device'> on <<class 'diffoscope.comparators.utils.archive.ArchiveMember'> static/rapture/resources/fonts/OpenSans-Bold.ttf>
2021-01-24 21:45:18 D: diffoscope.comparators.utils.specialize: trying <class 'diffoscope.comparators.debian_fallback.DotChangesFile'> on <<class 'diffoscope.comparators.utils.archive.ArchiveMember'> static/rapture/resources/fonts/OpenSans-Bold.ttf>
Traceback (most recent call last):
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/main.py", line 764, in main
sys.exit(run_diffoscope(parsed_args))
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/main.py", line 718, in run_diffoscope
difference = compare_root_paths(path1, path2)
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/compare.py", line 69, in compare_root_paths
difference = compare_files(file1, file2)
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/compare.py", line 125, in compare_files
return file1.compare(file2, source)
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/file.py", line 488, in compare
difference = self._compare_using_details(other, source)
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/file.py", line 423, in _compare_using_details
details.extend(
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/container.py", line 193, in compare_pair
difference = compare_files(
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/compare.py", line 125, in compare_files
return file1.compare(file2, source)
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/file.py", line 488, in compare
difference = self._compare_using_details(other, source)
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/file.py", line 423, in _compare_using_details
details.extend(
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/container.py", line 193, in compare_pair
difference = compare_files(
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/compare.py", line 96, in compare_files
specialize(file1)
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/specialize.py", line 54, in specialize
if try_recognize(file, cls, cls.recognizes):
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/specialize.py", line 37, in try_recognize
if not recognizes(file):
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/file.py", line 154, in recognizes
file.magic_file_type,
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/file.py", line 308, in magic_file_type
self._magic_file_type = File.guess_file_type(self.path)
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/file.py", line 92, in guess_file_type
return maybe_decode(cls._mimedb.from_file(path))
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/magic.py", line 99, in from_file
return maybe_decode(magic_file(self.cookie, filename))
File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/magic.py", line 228, in maybe_decode
return s.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa9 in position 107: invalid start byte
Workaround
Since I know ttf files are not changing contents between releases in my specific case, I just excluded ttf files from the comparison.
--exclude '*.ttf'
Attached is the file OpenSans-Bold.ttf