diff: Use bytes as much as possible, to prevent possible encoding issues

In practice, this prevents weird codepoints to behave like linebreaks of
sorts.
And while I believe it also decreses memory consume, especially for
larger diffs.

Signed-off-by: Mattia Rizzolo <mattia@debian.org>