Usage of FIFOs causes pair-comparisons to not run in parallel, wasting performance by about 50%
This bug was originally reported by Ximin Luo (infinity0@debian.org) in Debian bug #863636:
diff(1) first reads the contents of one file then the next one:
https://sources.debian.net/src/diffutils/1:3.5-3/src/io.c/#L552
This means that if the "files" are actually FIFOs connected to the output of a
process, as they are in many cases in diffoscope, the second process has to wait
for diff(1) to fully read the output of the first process, before it itself can
run. This prevents both processes from running in parallel.
An appropriate fix would be to store the output of at least one of the commands
into a temporary file, and have diff(1) read from this instead. This has to be
done carefully however, to make sure that diff(1) doesn't accidentally read it
before the process is finished.