Several tests fail on FreeBSD 12-STABLE
This one is s track tests failures on FreeBSD and act upon:
osipovmi@deblndw011x:~/var/Projekte/diffoscope (master=)
$ python3 setup.py test
running test
WARNING: Testing via this command is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to use tox.
running egg_info
writing diffoscope.egg-info/PKG-INFO
writing dependency_links to diffoscope.egg-info/dependency_links.txt
writing entry points to diffoscope.egg-info/entry_points.txt
writing requirements to diffoscope.egg-info/requires.txt
writing top-level names to diffoscope.egg-info/top_level.txt
reading manifest template 'MANIFEST.in'
writing manifest file 'diffoscope.egg-info/SOURCES.txt'
running build_ext
=============================================================================================== test session starts ===============================================================================================
platform freebsd12 -- Python 3.7.7, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /var/osipovmi/Projekte/diffoscope
collected 440 items
tests/test_difference.py ..... [ 1%]
tests/test_excludes.py ....... [ 2%]
tests/test_main.py ............ [ 5%]
tests/test_presenters.py ..........s........ [ 9%]
tests/test_progress.py s. [ 10%]
tests/test_quines.py ..... [ 11%]
tests/test_readers.py . [ 11%]
tests/test_source.py s. [ 12%]
tests/test_tools.py .. [ 12%]
tests/comparators/test_android.py ..ss [ 13%]
tests/comparators/test_apk.py ..ssssss [ 15%]
tests/comparators/test_archive.py . [ 15%]
tests/comparators/test_berkeley_db.py ..ss [ 16%]
tests/comparators/test_binary.py ..................s.. [ 21%]
tests/comparators/test_binwalk.py sssss [ 22%]
tests/comparators/test_bzip2.py ...... [ 23%]
tests/comparators/test_cbfs.py ssssss [ 25%]
tests/comparators/test_containers.py .. [ 25%]
tests/comparators/test_cpio.py ...... [ 26%]
tests/comparators/test_deb.py ............... [ 30%]
tests/comparators/test_debian.py .s.sss.s.ss.s.ss. [ 34%]
tests/comparators/test_device.py .FF [ 34%]
tests/comparators/test_dex.py ..ss [ 35%]
tests/comparators/test_directory.py ........ [ 37%]
tests/comparators/test_docx.py ..ss [ 38%]
tests/comparators/test_dtb.py ..ss [ 39%]
tests/comparators/test_elf.py ........Fss.. [ 42%]
tests/comparators/test_epub.py .... [ 43%]
tests/comparators/test_ffprobe.py ..ss [ 44%]
tests/comparators/test_fontconfig.py ... [ 44%]
tests/comparators/test_fonts.py ..ss [ 45%]
tests/comparators/test_fsimage.py ....ssss [ 47%]
tests/comparators/test_gettext.py ..... [ 48%]
tests/comparators/test_gif.py ....ss [ 50%]
tests/comparators/test_git.py ..F [ 50%]
tests/comparators/test_gnumeric.py ..ss [ 51%]
tests/comparators/test_gzip.py ........ [ 53%]
tests/comparators/test_haskell.py s.s [ 54%]
tests/comparators/test_hdf.py ssss [ 55%]
tests/comparators/test_icc.py ..s. [ 55%]
tests/comparators/test_ico_image.py ..ssss [ 57%]
tests/comparators/test_ipk.py ..... [ 58%]
tests/comparators/test_iso9660.py ..sssss [ 60%]
tests/comparators/test_java.py F.sssF [ 61%]
tests/comparators/test_javascript.py ..ss [ 62%]
tests/comparators/test_jpeg_image.py ..ssss [ 63%]
tests/comparators/test_json.py ...... [ 65%]
tests/comparators/test_kbx.py ..ss [ 65%]
tests/comparators/test_lz4.py ...... [ 67%]
tests/comparators/test_macho.py ..ss [ 68%]
tests/comparators/test_mono.py ..ss [ 69%]
tests/comparators/test_ocaml.py ssss [ 70%]
tests/comparators/test_odt.py ..ss [ 70%]
tests/comparators/test_ogg.py ..ss [ 71%]
tests/comparators/test_openssh_pub_key.py .... [ 72%]
tests/comparators/test_pcap.py .... [ 73%]
tests/comparators/test_pdf.py ...sss [ 75%]
tests/comparators/test_pgp.py ..ss.s [ 76%]
tests/comparators/test_png.py ..sss [ 77%]
tests/comparators/test_ppu.py s.ss [ 78%]
tests/comparators/test_ps.py ..... [ 79%]
tests/comparators/test_rdata.py ..ss [ 80%]
tests/comparators/test_rlib.py ......s. [ 82%]
tests/comparators/test_rpm.py ....... [ 83%]
tests/comparators/test_sqlite.py .... [ 84%]
tests/comparators/test_squashfs.py ...ssss [ 86%]
tests/comparators/test_symlink.py . [ 86%]
tests/comparators/test_tar.py ....... [ 88%]
tests/comparators/test_text.py ....... [ 89%]
tests/comparators/test_utils.py .ss..... [ 91%]
tests/comparators/test_wasm.py ..ss [ 92%]
tests/comparators/test_xml.py ...s [ 93%]
tests/comparators/test_xsb.py ..ss [ 94%]
tests/comparators/test_xz.py ...... [ 95%]
tests/comparators/test_zip.py ................ [ 99%]
tests/comparators/test_zst.py ... [100%]
==================================================================================================== FAILURES =====================================================================================================
____________________________________________________________________________________________________ test_diff ____________________________________________________________________________________________________
differences = <Difference /dev/null -- /var/osipovmi/Projekte/diffoscope/tests/data/text_ascii1 []>
@skip_unless_tools_exist('xxd')
def test_diff(differences):
if os.uname()[0] == 'FreeBSD':
expected_diff = get_data('device_expected_diff_freebsd')
else:
expected_diff = get_data('device_expected_diff')
> assert normalize_zeros(differences.unified_diff) == expected_diff
E AssertionError: assert '@@ -1,3 +1,2...t laborum..\n' == '@@ -1,3 +1,2...t laborum..\n'
E Skipping 162 identical leading characters in diff, use -v to show
E Skipping 1923 identical trailing characters in diff, use -v to show
E - 20: 2031 340a 14.
E ? ^ ^
E + 20: 2031 330a 13.
E ? ^ ^
E +000000
tests/comparators/test_device.py:59: AssertionError
________________________________________________________________________________________________ test_diff_reverse ________________________________________________________________________________________________
differences_reverse = <Difference /var/osipovmi/Projekte/diffoscope/tests/data/text_ascii1 -- /dev/null []>
@skip_unless_tools_exist('xxd')
def test_diff_reverse(differences_reverse):
if os.uname()[0] == 'FreeBSD':
expected_diff = get_data('device_expected_diff_reverse_freebsd')
else:
expected_diff = get_data('device_expected_diff_reverse')
> assert normalize_zeros(differences_reverse.unified_diff) == expected_diff
E AssertionError: assert '@@ -1,28 +1,... 13.\n' == '@@ -1,28 +1,... 14.\n'
E Skipping 2092 identical leading characters in diff, use -v to show
E - 20: 2031 340a 14.
E ? ^ ^
E + 20: 2031 330a 13.
E ? ^ ^
tests/comparators/test_device.py:68: AssertionError
_____________________________________________________________________________________________ test_libmix_differences _____________________________________________________________________________________________
libmix_differences = [<Difference file list -- file list []>, <Difference Mach-O.o -- Mach-O.o []>, <Difference return42_or_3_long_name.o -...emangle --reloc --section=.text {} -- objdump --line-numbers --disassemble --demangle --reloc --section=.text {} []>]>]
@skip_unless_tools_exist('xxd')
@skip_unless_tools_exist('readelf', 'objdump')
@skip_if_tool_version_is('readelf', readelf_version, '2.29')
@skip_if_binutils_does_not_support_x86()
def test_libmix_differences(libmix_differences):
assert len(libmix_differences) == 5
file_list, mach_o, x86_o, src_c, x_obj = libmix_differences
# Check order and basic identification
assert file_list.source1 == 'file list'
assert "Falling back to binary" in mach_o.comments[0]
x86_o = x86_o.details[0]
assert x86_o.source1.startswith('objdump ')
assert src_c.source1.endswith('.c')
x_obj = x_obj.details[0]
> assert x_obj.source1.startswith('readelf ')
E AssertionError: assert False
E + where False = <built-in method startswith of str object at 0x806ad6bb0>('readelf ')
E + where <built-in method startswith of str object at 0x806ad6bb0> = 'objdump --line-numbers --disassemble --demangle --reloc --section=.text {}'.startswith
E + where 'objdump --line-numbers --disassemble --demangle --reloc --section=.text {}' = <Difference objdump --line-numbers --disassemble --demangle --reloc --section=.text {} -- objdump --line-numbers --disassemble --demangle --reloc --section=.text {} []>.source1
tests/comparators/test_elf.py:183: AssertionError
____________________________________________________________________________________________________ test_diff ____________________________________________________________________________________________________
differences = [<Difference /var/osipovmi/Projekte/diffoscope/tests/data/test1.git-index -- /var/osipovmi/Projekte/diffoscope/tests/data/test2.git-index []>]
def test_diff(differences):
if get_current_os() == 'FreeBSD':
expected_diff = get_data('git_expected_diff_freebsd')
else:
expected_diff = get_data('git_expected_diff')
> assert differences[0].unified_diff == expected_diff
E AssertionError: assert '@@ -10,7 +10...: (8, 6)\n+\n' == '@@ -10,7 +10...0, 2054)\n+\n'
E Skipping 143 identical leading characters in diff, use -v to show
E - vice ID: (0, 2054)
E ? ^ ^^^^
E + vice ID: (8, 6)
E ? ^ ^
E
E +Path: b'b'...
E
E ...Full output truncated (14 lines hidden), use '-vv' to show
tests/comparators/test_git.py:50: AssertionError
_______________________________________________________________________________________________ test_identification _______________________________________________________________________________________________
class1 = <<class 'diffoscope.comparators.binary.FilesystemFile'> /var/osipovmi/Projekte/diffoscope/tests/data/Test1.class>
def test_identification(class1):
> assert isinstance(class1, ClassFile)
E AssertionError: assert False
E + where False = isinstance(<<class 'diffoscope.comparators.binary.FilesystemFile'> /var/osipovmi/Projekte/diffoscope/tests/data/Test1.class>, ClassFile)
tests/comparators/test_java.py:45: AssertionError
_________________________________________________________________________________________ test_compare_non_existing_javap _________________________________________________________________________________________
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x8075e5f90>, class1 = <<class 'diffoscope.comparators.binary.FilesystemFile'> /var/osipovmi/Projekte/diffoscope/tests/data/Test1.class>
@skip_unless_tools_exist('javap')
def test_compare_non_existing_javap(monkeypatch, class1):
> compare_non_existing(monkeypatch, class1, Javap)
tests/comparators/test_java.py:95:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x8075e5f90>, class1 = <<class 'diffoscope.comparators.binary.FilesystemFile'> /var/osipovmi/Projekte/diffoscope/tests/data/Test1.class>
decompiler = <class 'diffoscope.comparators.java.Javap'>
def compare_non_existing(monkeypatch, class1, decompiler):
monkeypatch.setattr(Config(), 'new_file', True)
> monkeypatch.setattr(class1, 'decompilers', [decompiler])
E AttributeError: <<class 'diffoscope.comparators.binary.FilesystemFile'> /var/osipovmi/Projekte/diffoscope/tests/data/Test1.class> has no attribute 'decompilers'
tests/comparators/test_java.py:72: AttributeError
============================================================================================= short test summary info =============================================================================================
FAILED tests/comparators/test_device.py::test_diff - AssertionError: assert '@@ -1,3 +1,2...t laborum..\n' == '@@ -1,3 +1,2...t laborum..\n'
FAILED tests/comparators/test_device.py::test_diff_reverse - AssertionError: assert '@@ -1,28 +1,... 13.\n' == '@@ -1,28 +1,... 14.\n'
FAILED tests/comparators/test_elf.py::test_libmix_differences - AssertionError: assert False
FAILED tests/comparators/test_git.py::test_diff - AssertionError: assert '@@ -10,7 +10...: (8, 6)\n+\n' == '@@ -10,7 +10...0, 2054)\n+\n'
FAILED tests/comparators/test_java.py::test_identification - AssertionError: assert False
FAILED tests/comparators/test_java.py::test_compare_non_existing_javap - AttributeError: <<class 'diffoscope.comparators.binary.FilesystemFile'> /var/osipovmi/Projekte/diffoscope/tests/data/Test1.class> has n...
============================================================================== 6 failed, 314 passed, 120 skipped in 91.27s (0:01:31) ==============================================================================I haven't looked into any of them yet, I am also not a daily Python programmer. Let's track failures and see how easy they can be resolved. I am also confused why so many are skipped. What does it take to make them work?