Fix a regression introduced in diffoscope version 207 where diffoscope would...

Fix a regression introduced in diffoscope version 207 where diffoscope would crash if one directory contained a directory that wasn't in the other. Thanks to Alderico Gallo for the testcase. (Closes: #310)