Loading .travis.yml +1 −1 Original line number Diff line number Diff line Loading @@ -10,7 +10,7 @@ python: - "3.5" - "3.6" - "3.7" - "3.8-dev" - "3.8" - "nightly" install: Loading CHANGES.rst +12 −0 Original line number Diff line number Diff line Loading @@ -2,6 +2,18 @@ Changes ======= v2.7 (2019-11-22) ----------------- * :issue:`427`: Multicore is now supported even when using ``--info-file``, ``--rest-file`` or ``--wildcard-file``. The only remaining feature that still does not work with multicore is now demultiplexing. * :issue:`290`: When running on a single core, Cutadapt no longer spawns external ``pigz`` processes for writing gzip-compressed files. This is a first step towards ensuring that using ``--cores=n`` uses only at most *n* CPU cores. * This release adds support for Python 3.8. v2.6 (2019-10-26) ----------------- Loading buildwheels.sh +9 −8 Original line number Diff line number Diff line #!/bin/bash # # Build manylinux1 wheels for cutadapt. Based on the example at # Build manylinux wheels. Based on the example at # <https://github.com/pypa/python-manylinux-demo> # # It is best to run this in a fresh clone of the repository! # # Run this within the repository root: # docker run --rm -v $(pwd):/io quay.io/pypa/manylinux1_x86_64 /io/buildwheels.sh # ./buildwheels.sh # # The wheels will be put into the wheelhouse/ subdirectory. # # For interactive tests: # docker run -it -v $(pwd):/io quay.io/pypa/manylinux1_x86_64 /bin/bash # docker run -it -v $(pwd):/io quay.io/pypa/manylinux2010_x86_64 /bin/bash set -xeuo pipefail MANYLINUX=quay.io/pypa/manylinux2010_x86_64 manylinux=quay.io/pypa/manylinux2010_x86_64 # For convenience, if this script is called from outside of a docker container, # it starts a container and runs itself inside of it. if ! grep -q docker /proc/1/cgroup; then # We are not inside a container docker pull ${MANYLINUX} exec docker run --rm -v $(pwd):/io ${MANYLINUX} /io/$0 docker pull ${manylinux} exec docker run --rm -v $(pwd):/io ${manylinux} /io/$0 fi # Strip binaries (copied from multibuild) Loading @@ -37,11 +37,12 @@ PYBINS="/opt/python/*/bin" HAS_CYTHON=0 for PYBIN in ${PYBINS}; do # ${PYBIN}/pip install -r /io/requirements.txt ${PYBIN}/pip wheel /io/ -w wheelhouse/ ${PYBIN}/pip wheel --no-deps /io/ -w wheelhouse/ done ls wheelhouse/ # Bundle external shared libraries into the wheels for whl in wheelhouse/cutadapt-*.whl; do for whl in wheelhouse/*.whl; do auditwheel repair "$whl" --plat manylinux1_x86_64 -w repaired/ done Loading doc/conf.py +1 −1 Original line number Diff line number Diff line Loading @@ -46,7 +46,7 @@ source_suffix = '.rst' master_doc = 'index' # General information about the project. project = u'cutadapt' project = u'Cutadapt' copyright = u'2010-2019, Marcel Martin' # The version info for the project you're documenting, acts as replacement for Loading doc/guide.rst +21 −32 Original line number Diff line number Diff line Loading @@ -129,22 +129,9 @@ Make also sure that you have ``pigz`` (parallel gzip) installed if you use multiple cores and write to a ``.gz`` output file. Otherwise, compression of the output will be done in a single thread and therefore be a bottleneck. There are some limitations at the moment: * The following command-line arguments are not compatible with multi-core: - ``--info-file`` - ``--rest-file`` - ``--wildcard-file`` - ``--format`` * Multi-core is not available when you use Cutadapt for demultiplexing. If you try to use multiple cores with an incompatible commandline option, you will get an error message. Some of these limitations will be lifted in the future, as time allows. Currently, multi-core support is not available when demultiplexing. You will get an error message if you try to use it. This limitations may be lifted in the future. .. versionadded:: 1.15 Loading @@ -154,6 +141,9 @@ Some of these limitations will be lifted in the future, as time allows. .. versionadded:: 2.5 Multicore works with ``--untrimmed/too-short/too-long-(paired)-output`` .. versionadded:: 2.7 Muticore works with ``--info-file``, ``--rest-file``, ``--wildcard-file`` Speed-up tricks --------------- Loading Loading @@ -1037,7 +1027,7 @@ each read. Steps not requested on the command-line are skipped. Filtering reads =============== By default, all processed reads, no matter whether they were trimmed are not, By default, all processed reads, no matter whether they were trimmed or not, are written to the output file specified by the ``-o`` option (or to standard output if ``-o`` was not provided). For paired-end reads, the second read in a pair is always written to the file specified by the ``-p`` option. Loading @@ -1047,23 +1037,22 @@ them entirely or by redirecting them to other files. When redirecting reads, the basic rule is that *each read is written to at most one file*. You cannot write reads to more than one output file. In the following, the term "processed read" refers to a read to which all modifications have been applied (adapter removal, quality trimming etc.). A processed read can be identical to the input read if no modifications were done. Filters are applied to *all* processed reads, no matter whether they have been modified by adapter- or quality trimming. ``--minimum-length LENGTH`` or ``-m LENGTH`` Discard processed reads that are shorter than LENGTH. Reads that are too short even before adapter removal are also discarded. Without this option, reads that have a length of zero (empty reads) are kept in the output. Discard processed reads that are shorter than LENGTH. If you do not use this option, reads that have a length of zero (empty reads) are kept in the output. Some downstream tools may have problems with zero-length sequences. In that case, specify at least ``-m 1``. ``--too-short-output FILE`` Instead of discarding the reads that are too short according to ``-m``, write them to *FILE* (in FASTA/FASTQ format). ``--maximum-length LENGTH`` or ``-M LENGTH`` Discard processed reads that are longer than LENGTH. Reads that are too long even before adapter removal are also discarded. Discard processed reads that are longer than LENGTH. ``--too-long-output FILE`` Instead of discarding reads that are too long (according to ``-M``), Loading @@ -1089,11 +1078,11 @@ The options ``--untrimmed-output``, ``--discard-trimmed`` and ``-discard-untrimm are mutually exclusive. The following filtering options do not have a corresponding option for redirecting reads. They always discard reads for which the filtering criterion applies. reads. They always discard those reads for which the filtering criterion applies. ``--max-n COUNT_or_FRACTION`` Discard reads with more than COUNT ``N`` bases. If ``COUNT_or_FRACTION`` is an number between 0 and 1, it is interpreted as a fraction of the read length Discard reads with more than COUNT ``N`` bases. If ``COUNT_or_FRACTION`` is a number between 0 and 1, it is interpreted as a fraction of the read length ``--discard-casava`` Discard reads that did not pass CASAVA filtering. Illumina’s CASAVA pipeline in Loading Loading
.travis.yml +1 −1 Original line number Diff line number Diff line Loading @@ -10,7 +10,7 @@ python: - "3.5" - "3.6" - "3.7" - "3.8-dev" - "3.8" - "nightly" install: Loading
CHANGES.rst +12 −0 Original line number Diff line number Diff line Loading @@ -2,6 +2,18 @@ Changes ======= v2.7 (2019-11-22) ----------------- * :issue:`427`: Multicore is now supported even when using ``--info-file``, ``--rest-file`` or ``--wildcard-file``. The only remaining feature that still does not work with multicore is now demultiplexing. * :issue:`290`: When running on a single core, Cutadapt no longer spawns external ``pigz`` processes for writing gzip-compressed files. This is a first step towards ensuring that using ``--cores=n`` uses only at most *n* CPU cores. * This release adds support for Python 3.8. v2.6 (2019-10-26) ----------------- Loading
buildwheels.sh +9 −8 Original line number Diff line number Diff line #!/bin/bash # # Build manylinux1 wheels for cutadapt. Based on the example at # Build manylinux wheels. Based on the example at # <https://github.com/pypa/python-manylinux-demo> # # It is best to run this in a fresh clone of the repository! # # Run this within the repository root: # docker run --rm -v $(pwd):/io quay.io/pypa/manylinux1_x86_64 /io/buildwheels.sh # ./buildwheels.sh # # The wheels will be put into the wheelhouse/ subdirectory. # # For interactive tests: # docker run -it -v $(pwd):/io quay.io/pypa/manylinux1_x86_64 /bin/bash # docker run -it -v $(pwd):/io quay.io/pypa/manylinux2010_x86_64 /bin/bash set -xeuo pipefail MANYLINUX=quay.io/pypa/manylinux2010_x86_64 manylinux=quay.io/pypa/manylinux2010_x86_64 # For convenience, if this script is called from outside of a docker container, # it starts a container and runs itself inside of it. if ! grep -q docker /proc/1/cgroup; then # We are not inside a container docker pull ${MANYLINUX} exec docker run --rm -v $(pwd):/io ${MANYLINUX} /io/$0 docker pull ${manylinux} exec docker run --rm -v $(pwd):/io ${manylinux} /io/$0 fi # Strip binaries (copied from multibuild) Loading @@ -37,11 +37,12 @@ PYBINS="/opt/python/*/bin" HAS_CYTHON=0 for PYBIN in ${PYBINS}; do # ${PYBIN}/pip install -r /io/requirements.txt ${PYBIN}/pip wheel /io/ -w wheelhouse/ ${PYBIN}/pip wheel --no-deps /io/ -w wheelhouse/ done ls wheelhouse/ # Bundle external shared libraries into the wheels for whl in wheelhouse/cutadapt-*.whl; do for whl in wheelhouse/*.whl; do auditwheel repair "$whl" --plat manylinux1_x86_64 -w repaired/ done Loading
doc/conf.py +1 −1 Original line number Diff line number Diff line Loading @@ -46,7 +46,7 @@ source_suffix = '.rst' master_doc = 'index' # General information about the project. project = u'cutadapt' project = u'Cutadapt' copyright = u'2010-2019, Marcel Martin' # The version info for the project you're documenting, acts as replacement for Loading
doc/guide.rst +21 −32 Original line number Diff line number Diff line Loading @@ -129,22 +129,9 @@ Make also sure that you have ``pigz`` (parallel gzip) installed if you use multiple cores and write to a ``.gz`` output file. Otherwise, compression of the output will be done in a single thread and therefore be a bottleneck. There are some limitations at the moment: * The following command-line arguments are not compatible with multi-core: - ``--info-file`` - ``--rest-file`` - ``--wildcard-file`` - ``--format`` * Multi-core is not available when you use Cutadapt for demultiplexing. If you try to use multiple cores with an incompatible commandline option, you will get an error message. Some of these limitations will be lifted in the future, as time allows. Currently, multi-core support is not available when demultiplexing. You will get an error message if you try to use it. This limitations may be lifted in the future. .. versionadded:: 1.15 Loading @@ -154,6 +141,9 @@ Some of these limitations will be lifted in the future, as time allows. .. versionadded:: 2.5 Multicore works with ``--untrimmed/too-short/too-long-(paired)-output`` .. versionadded:: 2.7 Muticore works with ``--info-file``, ``--rest-file``, ``--wildcard-file`` Speed-up tricks --------------- Loading Loading @@ -1037,7 +1027,7 @@ each read. Steps not requested on the command-line are skipped. Filtering reads =============== By default, all processed reads, no matter whether they were trimmed are not, By default, all processed reads, no matter whether they were trimmed or not, are written to the output file specified by the ``-o`` option (or to standard output if ``-o`` was not provided). For paired-end reads, the second read in a pair is always written to the file specified by the ``-p`` option. Loading @@ -1047,23 +1037,22 @@ them entirely or by redirecting them to other files. When redirecting reads, the basic rule is that *each read is written to at most one file*. You cannot write reads to more than one output file. In the following, the term "processed read" refers to a read to which all modifications have been applied (adapter removal, quality trimming etc.). A processed read can be identical to the input read if no modifications were done. Filters are applied to *all* processed reads, no matter whether they have been modified by adapter- or quality trimming. ``--minimum-length LENGTH`` or ``-m LENGTH`` Discard processed reads that are shorter than LENGTH. Reads that are too short even before adapter removal are also discarded. Without this option, reads that have a length of zero (empty reads) are kept in the output. Discard processed reads that are shorter than LENGTH. If you do not use this option, reads that have a length of zero (empty reads) are kept in the output. Some downstream tools may have problems with zero-length sequences. In that case, specify at least ``-m 1``. ``--too-short-output FILE`` Instead of discarding the reads that are too short according to ``-m``, write them to *FILE* (in FASTA/FASTQ format). ``--maximum-length LENGTH`` or ``-M LENGTH`` Discard processed reads that are longer than LENGTH. Reads that are too long even before adapter removal are also discarded. Discard processed reads that are longer than LENGTH. ``--too-long-output FILE`` Instead of discarding reads that are too long (according to ``-M``), Loading @@ -1089,11 +1078,11 @@ The options ``--untrimmed-output``, ``--discard-trimmed`` and ``-discard-untrimm are mutually exclusive. The following filtering options do not have a corresponding option for redirecting reads. They always discard reads for which the filtering criterion applies. reads. They always discard those reads for which the filtering criterion applies. ``--max-n COUNT_or_FRACTION`` Discard reads with more than COUNT ``N`` bases. If ``COUNT_or_FRACTION`` is an number between 0 and 1, it is interpreted as a fraction of the read length Discard reads with more than COUNT ``N`` bases. If ``COUNT_or_FRACTION`` is a number between 0 and 1, it is interpreted as a fraction of the read length ``--discard-casava`` Discard reads that did not pass CASAVA filtering. Illumina’s CASAVA pipeline in Loading