Commit 6b29511c authored by Ximin Luo's avatar Ximin Luo

Add "auto" presets and --testbed-init

* Add a --testbed-init option to allow the user to install dependencies that
  are needed to make the variations in the first place.
* Add an "auto" feature to the CLI, plus a presets module so it's easier to
  use, and other non-Debian systems can start populating this too.
parent a1838c42
Command Line Interface
======================
reprotest's CLI takes two mandatory arguments, the build command to
run and the build artifact file/pattern to test after running the
build. Here are some sample invocations for running reprotest on
itself:
The easiest way to run reprotest is via our presets:
# Build the current directory in a null server (/tmp)
reprotest auto . -- null -d
# Build the given Debian source package in an schroot
# See https://wiki.debian.org/sbuild for instructions on setting that up.
reprotest auto reprotest_0.3.3.dsc -- schroot unstable-amd64-sbuild
Currently, we only support this for Debian packages, but are keen on adding
more. If we don't have knowledge on how to build your file or directory, you
can send a patch to us on adding this intelligence - see the reprotest.presets
python module, and adapt the existing logic.
Before that happens, you can use the more advanced CLI to build arbitrary
things. This takes two mandatory arguments, the build command to run and the
build artifact file/pattern to test after running the build. For example:
reprotest 'python3 setup.py bdist' 'dist/*.tar.gz'
reprotest 'debuild -b -uc -us' '../*.deb' -- null -d
When using reprotest from a shell:
When using this from a shell:
If the build command has spaces, you will need to quote them, e.g.
`reprotest "debuild -b -uc -us" [..]`.
......@@ -22,7 +34,10 @@ quote these twice, e.g. `'"a file with spaces.gz"'` for a single
artifact or `'"dir 1"/* "dir 2"/*'` for multiple patterns.
To get more help for the CLI, including documentation on optional
arguments and what they do, run `reprotest --help`.
arguments and what they do, run:
reprotest --help
reprotest --help schroot
Running in a virtual server
......@@ -38,24 +53,37 @@ full list. You run them like this:
You can also run `reprotest --help <virtual_server_name>` for a full list of
options for that particular virtual server.
Unfortunately we currently don't set up build dependencies inside the virtual
server so you will have to either do that yourself before running reprotest,
or by giving the set-up command to reprotest manually. For example:
reprotest --dont-vary=fileordering,kernel \
'PATH=/sbin:/usr/sbin:$PATH apt-get install --no-install-recommends -y devscripts equivs;\
PATH=/sbin:/usr/sbin:$PATH mk-build-deps -t "apt-get --no-install-recommends -y" -ir;\
debuild -b -uc -us' '../*.deb' \
schroot unstable-amd64-sbuild
TODO: fix this, e.g. by copying what sbuild does / running sbuild. In
particular, the above example command installs devscripts and other unnecessary
dependencies which might pollute the build, so it is not the ideal method.
TODO: also the command is run *after* setting up the variations, which is why
we need to disable the fileordering/kernel variations above - they use
disorderfs and /usr/bin/linux64 (from util-linux) which aren't available even
if we add "apt-get install disorderfs util-linux" into the build command.
You will probably have to give extra commands to reprotest, in order to set up
your build dependencies inside the virtual server. For example, to take you
through what the "Debian directory" preset would look like, if we ran it via
the advanced CLI:
reprotest auto . -- schroot unstable-amd64-sbuild
# equivalent to:
reprotest \
--testbed-init 'apt-get -y --no-install-recommends install \
util-linux disorderfs 2>/dev/null; \
test -c /dev/fuse || mknod -m 666 /dev/fuse c 10 229' \
'PATH=/sbin:/usr/sbin:$PATH apt-get -y --no-install-recommends build-dep ./; \
dpkg-buildpackage -uc -us -b' \
'../*.deb' \
-- \
schroot unstable-amd64-sbuild
The `--testbed-init` argument is needed to set up basic tools, which reprotest
needs in order to make the variations in the first place. This should be the
same regardless of what package is being built, but might differ depending on
what virtual_server is being used.
Next, we have the build_command. For our Debian directory, we install
build-dependencies using apt-get, then we run the actual build command itself
using dpkg-buildpackage(1).
Then, we have the artifact pattern. For reproducibility, we're only interested
in the binary packages.
Finally, we specify that this is to take place in the "schroot" virtual_server
with arguments "unstable-amd64-sbuild".
Config File
......
......@@ -3,6 +3,10 @@ reprotest (0.3.3) UNRELEASED; urgency=medium
* Document virtual servers and caveats better.
* Add a --help [virtual server] option.
* Add a --no-diffoscope option. (Closes: #844512)
* Add a --testbed-init option to allow the user to install dependencies that
are needed to make the variations in the first place.
* Add an "auto" feature to the CLI, plus a presets module so it's easier to
use, and other non-Debian systems can start populating this too.
-- Ximin Luo <infinity0@debian.org> Fri, 18 Nov 2016 17:13:49 +0100
......
......@@ -21,6 +21,7 @@ from reprotest.lib import adtlog
from reprotest.lib import adt_testbed
from reprotest import _contextlib
from reprotest import _shell_ast
from reprotest import presets
adtlog.verbosity = 1
......@@ -359,9 +360,6 @@ def check(build_command, artifact_pattern, virtual_server_args, source_root,
with tempfile.TemporaryDirectory() as temp_dir, \
start_testbed(virtual_server_args, temp_dir, no_clean_on_error) as testbed:
script = Script(build_command)
if testbed_init:
script = script.append_setup(_shell_ast.SimpleCommand(
"sh", "-ec", _shell_ast.Quote(testbed_init)))
script = Pair(script, script)
env = Pair(types.MappingProxyType(os.environ.copy()),
types.MappingProxyType(os.environ.copy()))
......@@ -378,13 +376,16 @@ def check(build_command, artifact_pattern, virtual_server_args, source_root,
source_root = new_source_root
testbed.command('copydown', (source_root + '/', tree.control))
testbed.command('copydown', (source_root + '/', tree.experiment))
if testbed_init:
testbed.check_exec(["sh", "-ec", testbed_init])
# print(source_root)
try:
with _contextlib.ExitStack() as stack:
for variation in variations:
# print('START')
# print(variation)
script, env, tree = stack.enter_context(VARIATIONS[variation](script, env, tree, testbed))
script, env, tree = stack.enter_context(
VARIATIONS[variation](script, env, tree, testbed))
# print(script)
# print(env)
# print(tree)
......@@ -456,6 +457,13 @@ COMMAND_LINE_OPTIONS = types.MappingProxyType(collections.OrderedDict([
'default': None, 'metavar': 'COMMANDS',
'help': 'Shell commands to run after starting the test bed, but before '
'applying variations. Used to e.g. install disorderfs in a chroot.'})),
('--auto-preset-expr', types.MappingProxyType({
'default': "_", 'metavar': 'PYTHON_EXPRESSION',
'help': 'This may be used to transform the presets returned by the '
'auto-detection feature. The value should be a python expression '
'that transforms the _ variable, which is a value of type '
'reprotest.presets.ReprotestPreset. See that class\'s documentation '
'for ways you can write this expression. Default: %(default)s'})),
('--diffoscope-arg', types.MappingProxyType({
'default': [], 'action': 'append',
'help': 'Give extra arguments to diffoscope when running it.'})),
......@@ -592,6 +600,16 @@ def main():
testbed_pre = command_line_options.get("testbed_pre")
testbed_init = command_line_options.get("testbed_init")
if build_command == 'auto':
auto_preset_expr = command_line_options.get("auto_preset_expr")
values = presets.get_presets(artifact, virtual_server_args[0])
values = eval(auto_preset_expr, {'_':values}, {})
print(values)
build_command = values.build_command
artifact = values.artifact
testbed_pre = values.testbed_pre
testbed_init = values.testbed_init
# print(build_command, artifact, virtual_server_args)
sys.exit(check(build_command, artifact, virtual_server_args, source_root,
no_clean_on_error, variations, diffoscope_args,
......
# Licensed under the GPL: https://www.gnu.org/licenses/gpl-3.0.en.html
# For details: reprotest/debian/copyright
import collections
import os
class AttributeFunctor(collections.namedtuple('_AttributeFunctor', 'x f')):
def __getattr__(self, name):
return lambda *args: self.x._replace(**{
name: self.f(getattr(self.x, name), *args)
})
class ReprotestPreset(collections.namedtuple('_ReprotestPreset',
'build_command artifact testbed_pre testbed_init')):
"""Named-tuple representing a reprotest command preset.
You can manipulate it like this:
>>> ReprotestPreset(None, None, None, None)
ReprotestPreset(build_command=None, artifact=None, testbed_pre=None, testbed_init=None)
>>> _.set.build_command("etc")
ReprotestPreset(build_command='etc', artifact=None, testbed_pre=None, testbed_init=None)
>>> _.append.build_command("; etc2")
ReprotestPreset(build_command='etc; etc2', artifact=None, testbed_pre=None, testbed_init=None)
>>> _.prepend.build_command("setup; ")
ReprotestPreset(build_command='setup; etc; etc2', artifact=None, testbed_pre=None, testbed_init=None)
>>> _.set.build_command("dpkg-buildpackage -us -uc -b")
ReprotestPreset(build_command='dpkg-buildpackage -us -uc -b', artifact=None, testbed_pre=None, testbed_init=None)
>>> _.str_replace.build_command(
... "dpkg-buildpackage", "DEB_BUILD_OPTIONS=nocheck dpkg-buildpackage -Pnocheck")
ReprotestPreset(build_command='DEB_BUILD_OPTIONS=nocheck dpkg-buildpackage -Pnocheck -us -uc -b', artifact=None, testbed_pre=None, testbed_init=None)
"""
@property
def set(self):
"""Set the given attribute to the given value."""
return AttributeFunctor(self, lambda x, y: y)
@property
def str_replace(self):
"""Do a substring-replace on the given attribute."""
return AttributeFunctor(self, str.replace)
@property
def prepend(self):
"""Prepend the given value to the given attribute."""
return AttributeFunctor(self, lambda a, b: b + a)
@property
def append(self):
"""Apppend the given value to the given attribute."""
return AttributeFunctor(self, lambda a, b: a + b)
PRESET_DEB_DIR = ReprotestPreset(
build_command = 'dpkg-buildpackage -uc -us -b',
artifact = '../*.deb',
testbed_pre = None,
testbed_init = None
)
def preset_deb_schroot(preset):
return preset.prepend.build_command(
'PATH=/sbin:/usr/sbin:$PATH apt-get -y --no-install-recommends build-dep ./; '
).set.testbed_init(
'apt-get -y --no-install-recommends install util-linux disorderfs 2>/dev/null; \
test -c /dev/fuse || mknod -m 666 /dev/fuse c 10 229'
)
def preset_deb_dsc(fn):
return ReprotestPreset(
build_command = 'dpkg-source -x "%s" build && cd build && dpkg-buildpackage -uc -us -b' % fn,
artifact = '*.deb',
testbed_pre = None,
testbed_init = None
)
def get_presets(buildfile, virtual_server):
fn = os.path.basename(buildfile)
parts = os.path.splitext(fn)
if os.path.isdir(buildfile):
if os.path.isdir(os.path.join(buildfile, "debian")):
if virtual_server == "null":
return PRESET_DEB_DIR
else:
return preset_deb_schroot(PRESET_DEB_DIR)
elif os.path.isfile(buildfile):
if parts[1] == '.dsc':
if virtual_server == "null":
return preset_deb_dsc(fn)
else:
return preset_deb_schroot(preset_deb_dsc(fn))
raise ValueError("unrecognised file type: %s" % buildfile)
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment