• Simon McVittie's avatar
    Implement superficial as a test restriction · 3c17c0c4
    Simon McVittie authored
    This feature was previously prototyped under the name 'trivial'.
    
    Superficial tests do not provide significant test coverage, so if they
    pass, that does not necessarily mean that the package under test
    is actually functional. If a superficial test fails, it will be
    treated like any other failing test, but if it succeeds, this is
    only a weak indication of success. Continuous integration systems
    should treat a package where all non-superficial tests are skipped as
    equivalent to a package where all tests are skipped.
    
    For example, debian/tests/build in packages like dbus, libinput and
    pango1.0 should be marked as superficial. For packages that have other
    tests (like dbus and pango1.0), this has no effect on migration to
    testing unless the non-superficial tests happen to be skipped. For
    packages where the only test is superficial (like libinput), marking
    the test as superficial removes the migration delay bonus on success
    (because the test doesn't actually confirm that the library works,
    so we have to rely on user testing to detect most regressions), while
    retaining the migration delay penalty on failure.
    
    Based on a patch from Rebecca N. Palmer.
    Signed-off-by: Simon McVittie's avatarSimon McVittie <smcv@debian.org>
    Closes: #904979
    3c17c0c4