Skip to content

tests/comparators/test_ffprobe.py broken with ffmpeg 4.4

Upgrading from ffmpeg 4.3.1 to ffmpeg 4.4 breaks the ffprobe test:

=================================== FAILURES ===================================
__________________________________ test_diff ___________________________________

differences = [<Difference ffprobe -- ffprobe []>]

    @skip_unless_tools_exist("ffprobe")
    def test_diff(differences):
>       assert_diff(differences[0], "mp3_expected_diff")

tests/comparators/test_ffprobe.py:47: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

difference = <Difference ffprobe -- ffprobe []>, filename = 'mp3_expected_diff'

    def assert_diff(difference, filename):
>       assert difference.unified_diff == get_data(filename)
E       AssertionError

tests/utils/data.py:50: AssertionError
=========================== short test summary info ============================
FAILED tests/comparators/test_ffprobe.py::test_diff - AssertionError
====== 1 failed, 425 passed, 39 skipped, 5 deselected in 77.36s (0:01:17) ======
error: builder for '/nix/store/c8d40h71kqswh6nlcgyzqp5705sq8d7v-diffoscope-171.drv' failed with exit code 1;
       last 10 log lines:
       > difference = <Difference ffprobe -- ffprobe []>, filename = 'mp3_expected_diff'
       >
       >     def assert_diff(difference, filename):
       > >       assert difference.unified_diff == get_data(filename)
       > E       AssertionError
       >
       > tests/utils/data.py:50: AssertionError
       > =========================== short test summary info ============================
       > FAILED tests/comparators/test_ffprobe.py::test_diff - AssertionError
       > ====== 1 failed, 425 passed, 39 skipped, 5 deselected in 77.36s (0:01:17) ======

The issue appears to be that the last line of the ffprobe output containing the "Stream" info now has a different number of spaces: https://salsa.debian.org/reproducible-builds/diffoscope/-/blob/175/tests/data/mp3_expected_diff

Comparing the ffprobe output for test1.mp3 using both versions produces the following output:

$ diff <(/nix/store/5ndchrs9mcvpvvcl9mjphdq5jsg5ynpl-ffmpeg-4.3.1-bin/bin/ffprobe tests/data/test1.mp3 2>&1) <(/nix/store/kh3gy6zdrnbrsx74nd13ilqbzls39kpr-ffmpeg-4.4-bin/bin/ffprobe tests/data/test1.mp3 2>&1)
[ snipped irrelevant changes]
<     Stream #0:0: Audio: mp3, 8000 Hz, mono, fltp, 8 kb/s
---
>   Stream #0:0: Audio: mp3, 8000 Hz, mono, fltp, 8 kb/s

Downstream nixpkgs issue: https://github.com/NixOS/nixpkgs/pull/124175

(I could submit the trivial change to make it work with ffmpeg 4.4, but I wasn't sure exactly how you intended to handle small variations in output based on version issues like this.)

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information