Loading .gitignore +2 −0 Original line number Diff line number Diff line Loading @@ -6,10 +6,12 @@ __pycache__ .eggs/ .cache/ .pytest_cache/ .idea/ /test-report.xml /test-report-*.xml /venv/ tmp/ /src/toil/test/cwl/spec /cwltool_deps/ /docs/generated_rst/ Loading Makefile +11 −6 Original line number Diff line number Diff line Loading @@ -23,8 +23,8 @@ The 'prepare' target installs Toil's build requirements into the current virtual The 'develop' target creates an editable install of Toil and its runtime requirements in the current virtualenv. The install is called 'editable' because changes to the source code immediately affect the virtualenv. Set the 'extras' variable to ensure that the 'develop' target installs support for extras. Consult setup.py for the list of supported extras. To install Toil in develop mode with all extras, run installs support for extras; some tests require extras to be installed. Consult setup.py for the list of supported extras. To install Toil in develop mode with all extras, run make develop extras=[all] Loading @@ -37,6 +37,9 @@ uploaded to PyPI. The 'docs' target uses Sphinx to create HTML documentation in the docs/_build directory Targets are provided to run Toil's tests. Note that these targets do *not* automatically install Toil's dependencies; it is recommended to 'make develop' before running any of them. The 'test' target runs Toil's unit tests serially with pytest. It will run some docker tests and setup. If you wish to avoid this, use the 'test_offline' target instead. Note: this target does not capture output from the terminal. For any of the test targets, set the 'tests' variable to run a Loading @@ -45,7 +48,9 @@ particular test, e.g. make test tests=src/toil/test/sort/sortTest.py::SortTest::testSort The 'test_offline' target is similar to 'test' but it skips the docker dependent tests and their setup. setup. It can also be used to invoke individual tests, e.g. make test_offline tests_local=src/toil/test/sort/sortTest.py::SortTest::testSort The 'integration_test_local' target runs toil's integration tests. These are more thorough but also more costly than the regular unit tests. For the AWS integration tests to run, the environment Loading Loading @@ -146,7 +151,7 @@ test_offline: check_venv check_build_reqs # The auto-deployment test needs the docker appliance test: check_venv check_build_reqs docker TOIL_APPLIANCE_SELF=$(docker_registry)/$(docker_base_name):$(docker_tag) \ $(python) -m pytest $(pytest_args_local) $(tests) $(python) -m pytest --cov=toil $(pytest_args_local) $(tests) # For running integration tests locally in series (uses the -s argument for pyTest) integration_test_local: check_venv check_build_reqs sdist push_docker Loading Loading @@ -262,12 +267,12 @@ check_build_reqs: prepare: check_venv $(pip) install sphinx==1.5.5 mock==1.0.1 pytest==2.8.3 stubserver==1.0.1 \ $(pip) install sphinx==1.5.5 mock==1.0.1 pytest==3.6.2 stubserver==1.0.1 \ pytest-timeout==1.2.0 cwltest check_venv: @$(python) -c 'import sys; sys.exit( int( not (hasattr(sys, "real_prefix") or ( hasattr(sys, "base_prefix") and sys.base_prefix != sys.prefix ) ) ) )' \ @$(python) -c 'import sys, os; sys.exit( int( 0 if "VIRTUAL_ENV" in os.environ else 1 ) )' \ || ( printf "$(red)A virtualenv must be active.$(normal)\n" ; false ) Loading README.rst +2 −4 Original line number Diff line number Diff line .. image:: https://badge.waffle.io/BD2KGenomics/toil.svg?label=ready&title=Ready :target: https://waffle.io/BD2KGenomics/toil :alt: 'Stories in Ready' ATTENTION: Toil has moved from https://github.com/BD2KGenomics/toil to https://github.com/DataBiosphere/toil as of July 5th, 2018. Toil is a scalable, efficient, cross-platform pipeline management system, written entirely in Python, and designed around the principles of functional Loading @@ -13,7 +11,7 @@ programming. * Google Groups discussion `forum`_ and videochat `invite list`_. .. _website: http://toil.ucsc-cgl.org/ .. _Read the Docs: http://toil.readthedocs.org/ .. _Read the Docs: https://toil.readthedocs.io/en/latest .. _forum: https://groups.google.com/forum/#!forum/toil-community .. _invite list: https://groups.google.com/forum/#!forum/toil-community-videochats .. _blog: https://toilpipelines.wordpress.com/ Loading attic/toil-sort-example.py +1 −0 Original line number Diff line number Diff line from __future__ import absolute_import from six.moves import xrange from argparse import ArgumentParser import os import logging Loading docs/appendices/environment_vars.rst +2 −0 Original line number Diff line number Diff line Loading @@ -52,6 +52,8 @@ There are several environment variables that affect the way Toil runs. | | deleted until all associated nodes have been | | | terminated. | +------------------------+----------------------------------------------------+ | TOIL_AZURE_ZONE | A specified region for provisioning instances. | +------------------------+----------------------------------------------------+ | TOIL_SLURM_ARGS | Arguments for sbatch for the slurm batch system. | | | Do not pass CPU or memory specifications here. | | | Instead, define resource requirements for the job. | Loading Loading
.gitignore +2 −0 Original line number Diff line number Diff line Loading @@ -6,10 +6,12 @@ __pycache__ .eggs/ .cache/ .pytest_cache/ .idea/ /test-report.xml /test-report-*.xml /venv/ tmp/ /src/toil/test/cwl/spec /cwltool_deps/ /docs/generated_rst/ Loading
Makefile +11 −6 Original line number Diff line number Diff line Loading @@ -23,8 +23,8 @@ The 'prepare' target installs Toil's build requirements into the current virtual The 'develop' target creates an editable install of Toil and its runtime requirements in the current virtualenv. The install is called 'editable' because changes to the source code immediately affect the virtualenv. Set the 'extras' variable to ensure that the 'develop' target installs support for extras. Consult setup.py for the list of supported extras. To install Toil in develop mode with all extras, run installs support for extras; some tests require extras to be installed. Consult setup.py for the list of supported extras. To install Toil in develop mode with all extras, run make develop extras=[all] Loading @@ -37,6 +37,9 @@ uploaded to PyPI. The 'docs' target uses Sphinx to create HTML documentation in the docs/_build directory Targets are provided to run Toil's tests. Note that these targets do *not* automatically install Toil's dependencies; it is recommended to 'make develop' before running any of them. The 'test' target runs Toil's unit tests serially with pytest. It will run some docker tests and setup. If you wish to avoid this, use the 'test_offline' target instead. Note: this target does not capture output from the terminal. For any of the test targets, set the 'tests' variable to run a Loading @@ -45,7 +48,9 @@ particular test, e.g. make test tests=src/toil/test/sort/sortTest.py::SortTest::testSort The 'test_offline' target is similar to 'test' but it skips the docker dependent tests and their setup. setup. It can also be used to invoke individual tests, e.g. make test_offline tests_local=src/toil/test/sort/sortTest.py::SortTest::testSort The 'integration_test_local' target runs toil's integration tests. These are more thorough but also more costly than the regular unit tests. For the AWS integration tests to run, the environment Loading Loading @@ -146,7 +151,7 @@ test_offline: check_venv check_build_reqs # The auto-deployment test needs the docker appliance test: check_venv check_build_reqs docker TOIL_APPLIANCE_SELF=$(docker_registry)/$(docker_base_name):$(docker_tag) \ $(python) -m pytest $(pytest_args_local) $(tests) $(python) -m pytest --cov=toil $(pytest_args_local) $(tests) # For running integration tests locally in series (uses the -s argument for pyTest) integration_test_local: check_venv check_build_reqs sdist push_docker Loading Loading @@ -262,12 +267,12 @@ check_build_reqs: prepare: check_venv $(pip) install sphinx==1.5.5 mock==1.0.1 pytest==2.8.3 stubserver==1.0.1 \ $(pip) install sphinx==1.5.5 mock==1.0.1 pytest==3.6.2 stubserver==1.0.1 \ pytest-timeout==1.2.0 cwltest check_venv: @$(python) -c 'import sys; sys.exit( int( not (hasattr(sys, "real_prefix") or ( hasattr(sys, "base_prefix") and sys.base_prefix != sys.prefix ) ) ) )' \ @$(python) -c 'import sys, os; sys.exit( int( 0 if "VIRTUAL_ENV" in os.environ else 1 ) )' \ || ( printf "$(red)A virtualenv must be active.$(normal)\n" ; false ) Loading
README.rst +2 −4 Original line number Diff line number Diff line .. image:: https://badge.waffle.io/BD2KGenomics/toil.svg?label=ready&title=Ready :target: https://waffle.io/BD2KGenomics/toil :alt: 'Stories in Ready' ATTENTION: Toil has moved from https://github.com/BD2KGenomics/toil to https://github.com/DataBiosphere/toil as of July 5th, 2018. Toil is a scalable, efficient, cross-platform pipeline management system, written entirely in Python, and designed around the principles of functional Loading @@ -13,7 +11,7 @@ programming. * Google Groups discussion `forum`_ and videochat `invite list`_. .. _website: http://toil.ucsc-cgl.org/ .. _Read the Docs: http://toil.readthedocs.org/ .. _Read the Docs: https://toil.readthedocs.io/en/latest .. _forum: https://groups.google.com/forum/#!forum/toil-community .. _invite list: https://groups.google.com/forum/#!forum/toil-community-videochats .. _blog: https://toilpipelines.wordpress.com/ Loading
attic/toil-sort-example.py +1 −0 Original line number Diff line number Diff line from __future__ import absolute_import from six.moves import xrange from argparse import ArgumentParser import os import logging Loading
docs/appendices/environment_vars.rst +2 −0 Original line number Diff line number Diff line Loading @@ -52,6 +52,8 @@ There are several environment variables that affect the way Toil runs. | | deleted until all associated nodes have been | | | terminated. | +------------------------+----------------------------------------------------+ | TOIL_AZURE_ZONE | A specified region for provisioning instances. | +------------------------+----------------------------------------------------+ | TOIL_SLURM_ARGS | Arguments for sbatch for the slurm batch system. | | | Do not pass CPU or memory specifications here. | | | Instead, define resource requirements for the job. | Loading