GNU diff is unable to cope with very large files as its memory usage
grows too much. So we now have a maximum input size to avoid getting OOM.

Convert msgunfmt as an example.

This paves the way for huge memory savings.
Convert xxd as an example.

Difference() now takes an unified diff directly. Computing the diff is
moved to a new static method from_content() which returns None when there are
no differences.

This paves the way for passing file descriptors to from_content() to avoid
loading entire outputs in memory.

This reverts commit 303a31e9.

We've the current design, it means holding way too much data in memory.

The side effect of adding @tool_required('xxd') is that now
we get vim-common in Recommends as it should have been.

Helmut Grohne authored Mar 28, 2015 and Jérémy Bobbio committed Mar 28, 2015

Example traceback:

Traceback (most recent call last):
  File "/usr/bin/debbindiff", line 121, in <module>
    sys.exit(main())
  File "/usr/bin/debbindiff", line 107, in main
    parsed_args.file1, parsed_args.file2)
  File "/usr/lib/python2.7/dist-packages/debbindiff/comparators/__init__.py", line 123, in compare_files
    return comparator(path1, path2, source)
  File "/usr/lib/python2.7/dist-packages/debbindiff/comparators/utils.py", line 74, in with_fallback
    except RequiredToolNotFound as e:
NameError: global name 'RequiredToolNotFound' is not defined

Closes: #781374

Thanks Helmut Grohne for spotting the issue.

Instead of storing the full content twice when creating Difference objects, we
now directly run `diff` and store the unified diff. Large blocks in the diff
are still trimmed. This results in huge memory savings and debbindiff can now
happily compare changes for installation-guide.

As tool_required() is not only for comparators anymore, we move it to
debbindiff, together with logger.

Text output becomes really straightforward as we just have to write what
we've previously recorded.

For the HTML output, we stop using vim and instead borrow code from
diff2html.py found at <http://git.droids-corp.org/?p=diff2html.git>.

Closes: #772029
Closes: #779476