Skip to content

UnicodeDecodeError on ttf font file

Ran diffoscope against a zip containing font ttf files. Processing stopped when it encountered a ttf file:

2021-01-24 21:45:18 D: diffoscope.comparators.utils.compare: Comparing static/rapture/resources/fonts/ (ZipDirectory) and static/rapture/resources/fonts/ (ZipDirectory)
2021-01-24 21:45:18 D: diffoscope.comparators.directory: Excluding directory metadata for paths (static/rapture/resources/fonts/, static/rapture/resources/fonts/)
2021-01-24 21:45:18 D: diffoscope.comparators.utils.compare: Comparing static/rapture/resources/fonts/OpenSans-Bold.ttf (ArchiveMember) and static/rapture/resources/fonts/OpenSans-Bold.ttf (ArchiveMember)
2021-01-24 21:45:18 D: diffoscope.comparators.utils.specialize: trying <class 'diffoscope.comparators.directory.Directory'> on <<class 'diffoscope.comparators.utils.archive.ArchiveMember'> static/rapture/resources/fonts/OpenSans-Bold.ttf>
2021-01-24 21:45:18 D: diffoscope.comparators.utils.specialize: trying <class 'diffoscope.comparators.missing_file.MissingFile'> on <<class 'diffoscope.comparators.utils.archive.ArchiveMember'> static/rapture/resources/fonts/OpenSans-Bold.ttf>
2021-01-24 21:45:18 D: diffoscope.comparators.utils.specialize: trying <class 'diffoscope.comparators.symlink.Symlink'> on <<class 'diffoscope.comparators.utils.archive.ArchiveMember'> static/rapture/resources/fonts/OpenSans-Bold.ttf>
2021-01-24 21:45:18 D: diffoscope.comparators.utils.specialize: trying <class 'diffoscope.comparators.device.Device'> on <<class 'diffoscope.comparators.utils.archive.ArchiveMember'> static/rapture/resources/fonts/OpenSans-Bold.ttf>
2021-01-24 21:45:18 D: diffoscope.comparators.utils.specialize: trying <class 'diffoscope.comparators.debian_fallback.DotChangesFile'> on <<class 'diffoscope.comparators.utils.archive.ArchiveMember'> static/rapture/resources/fonts/OpenSans-Bold.ttf>
Traceback (most recent call last):
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/main.py", line 764, in main
    sys.exit(run_diffoscope(parsed_args))
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/main.py", line 718, in run_diffoscope
    difference = compare_root_paths(path1, path2)
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/compare.py", line 69, in compare_root_paths
    difference = compare_files(file1, file2)
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/compare.py", line 125, in compare_files
    return file1.compare(file2, source)
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/file.py", line 488, in compare
    difference = self._compare_using_details(other, source)
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/file.py", line 423, in _compare_using_details
    details.extend(
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/container.py", line 193, in compare_pair
    difference = compare_files(
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/compare.py", line 125, in compare_files
    return file1.compare(file2, source)
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/file.py", line 488, in compare
    difference = self._compare_using_details(other, source)
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/file.py", line 423, in _compare_using_details
    details.extend(
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/container.py", line 193, in compare_pair
    difference = compare_files(
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/compare.py", line 96, in compare_files
    specialize(file1)
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/specialize.py", line 54, in specialize
    if try_recognize(file, cls, cls.recognizes):
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/specialize.py", line 37, in try_recognize
    if not recognizes(file):
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/file.py", line 154, in recognizes
    file.magic_file_type,
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/file.py", line 308, in magic_file_type
    self._magic_file_type = File.guess_file_type(self.path)
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/diffoscope/comparators/utils/file.py", line 92, in guess_file_type
    return maybe_decode(cls._mimedb.from_file(path))
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/magic.py", line 99, in from_file
    return maybe_decode(magic_file(self.cookie, filename))
  File "/usr/local/Cellar/diffoscope/164/libexec/lib/python3.9/site-packages/magic.py", line 228, in maybe_decode
    return s.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa9 in position 107: invalid start byte

Workaround

Since I know ttf files are not changing contents between releases in my specific case, I just excluded ttf files from the comparison.

--exclude '*.ttf'

Attached is the file OpenSans-Bold.ttf

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information