Also skip the checks for build number one, otherwise we will never
produce reproducible packages. Fixes 97145223

 which introduces this
issue.

Signed-off-by: Jelle van der Waa <jelle@vdwaa.nl>