New upstream version 1.18

Changes

=======

v1.18 (2018-09-07)

------------------

Features

~~~~~~~~

* Close :issue:`327`: Maximum and minimum lengths can now be specified

  separately for R1 and R2 with ``-m LENGTH1:LENGTH2``. One of the

  lengths can be omitted, in which case only the length of the other

  read is checked (as in ``-m 17:`` or ``-m :17``).

* Close :issue:`322`: Use ``-j 0`` to auto-detect how many cores to run on.

  This should even work correctly on cluster systems when Cutadapt runs as

  a batch job to which fewer cores than exist on the machine have been

  assigned. Note that the number of threads used by ``pigz`` cannot be

  controlled at the moment, see :issue:`290`.

* Close :issue:`225`: Allow setting the maximum error rate and minimum overlap

  length per adapter. A new :ref:`syntax for adapter-specific

  parameters <trimming-parameters>` was added for this. Example:

  ``-a "ADAPTER;min_overlap=5"``.

* Close :issue:`152`: Using the new syntax for adapter-specific parameters,

  it is now possible to allow partial matches of a 3' adapter at the 5' end

  (and partial matches of a 5' adapter at the 3' end) by specifying the

  ``anywhere`` parameter (as in ``-a "ADAPTER;anywhere"``).

* Allow ``--pair-filter=first`` in addition to ``both`` and ``any``. If

  used, a read pair is discarded if the filtering criterion applies to R1;

  and R2 is ignored.

* Close :issue:`112`: Implement a ``--report=minimal`` option for printing

  a succinct two-line report in tab-separated value (tsv) format. Thanks

  to :user:`jvolkening` for coming up with an initial patch!

Bug fixes

~~~~~~~~~

* Fix :issue:`128`: The “Reads written” figure in the report incorrectly

  included both trimmed and untrimmed reads if ``--untrimmed-output`` was used.

Other

~~~~~

* The options ``--no-trim`` and ``--mask-adapter`` should now be written as

  ``--action=mask`` and ``--action=none``. The old options still work.

* This is the last release to support :ref:`colorspace data <colorspace>`.

* This is the last release to support Python 2.

v1.17 (2018-08-20)

------------------

* Close :issue:`53`: Implement adapters :ref:`that disallow internal matches <non-internal>`.

  This is a bit like anchoring, but less strict: The adapter sequence

  can appear at different lengths, but must always be at one of the ends.

  Use ``-a ADAPTERX`` (with a literal ``X``) to disallow internal matches

  for a 3' adapter. Use ``-g XADAPTER`` to disallow for a 5' adapter.

* :user:`klugem` contributed PR :issue:`299`: The ``--length`` option (and its

  alias ``-l``) can now be used with negative lengths, which will remove bases

  from the beginning of the read instead of from the end.

* Close :issue:`107`: Add a ``--discard-casava`` option to remove reads

  that did not pass CASAVA filtering (this is possibly relevant only for

  older datasets).

* Fix :issue:`318`: Cutadapt should now be installable with Python 3.7.

* Running Cutadapt under Python 3.3 is no longer supported (Python 2.7 or

  3.4+ are needed)

* Planned change: One of the next Cutadapt versions will drop support for

  Python 2 entirely, requiring Python 3.

v1.16 (2018-02-21)

------------------

Metadata-Version: 1.1

Name: cutadapt

Version: 1.16

Version: 1.18

Summary: trim adapters from high-throughput sequencing reads

Home-page: https://cutadapt.readthedocs.io/

Author: Marcel Martin

Author-email: marcel.martin@scilifelab.se

License: MIT

Description-Content-Type: UNKNOWN

Description: .. image:: https://travis-ci.org/marcelm/cutadapt.svg?branch=master

            :target: https://travis-ci.org/marcelm/cutadapt

=================

Algorithm details

=================

.. _adapter-alignment-algorithm:

Adapter alignment algorithm

===========================

Since the publication of the `EMBnet journal application note about

cutadapt <http://dx.doi.org/10.14806/ej.17.1.200>`_, the alignment algorithm

used for finding adapters has changed significantly. An overview of this new

algorithm is given in this section. An even more detailed description is

available in Chapter 2 of my PhD thesis `Algorithms and tools for the analysis

of high-throughput DNA sequencing data <http://hdl.handle.net/2003/31824>`_.

The algorithm is based on *semiglobal alignment*, also called *free-shift*,

*ends-free* or *overlap* alignment. In a regular (global) alignment, the

two sequences are compared from end to end and all differences occuring over

that length are counted. In semiglobal alignment, the sequences are allowed to

freely shift relative to each other and differences are only penalized in the

overlapping region between them::

      FANTASTIC

   ELEFANT

The prefix ``ELE`` and the suffix ``ASTIC`` do not have a counterpart in the

respective other row, but this is not counted as an error. The overlap ``FANT``

has a length of four characters.

Traditionally, *alignment scores* are used to find an optimal overlap aligment:

This means that the scoring function assigns a positive value to matches,

while mismatches, insertions and deletions get negative values. The optimal

alignment is then the one that has the maximal total score. Usage of scores

has the disadvantage that they are not at all intuitive: What does a total score

of *x* mean? Is that good or bad? How should a threshold be chosen in order to

avoid finding alignments with too many errors?

For cutadapt, the adapter alignment algorithm uses *unit costs* instead.

This means that mismatches, insertions and deletions are counted as one error, which

is easier to understand and allows to specify a single parameter for the

algorithm (the maximum error rate) in order to describe how many errors are

acceptable.

There is a problem with this: When using costs instead of scores, we would like

to minimize the total costs in order to find an optimal alignment. But then the

best alignment would always be the one in which the two sequences do not overlap

at all! This would be correct, but meaningless for the purpose of finding an

adapter sequence.

The optimization criteria are therefore a bit different. The basic idea is to

consider the alignment optimal that maximizes the overlap between the two

sequences, as long as the allowed error rate is not exceeded.

Conceptually, the procedure is as follows:

1. Consider all possible overlaps between the two sequences and compute an

   alignment for each, minimizing the total number of errors in each one.

2. Keep only those alignments that do not exceed the specified maximum error

   rate.

3. Then, keep only those alignments that have a maximal number of matches

   (that is, there is no alignment with more matches).

4. If there are multiple alignments with the same number of matches, then keep

   only those that have the smallest error rate.

5. If there are still multiple candidates left, choose the alignment that starts

   at the leftmost position within the read.

In Step 1, the different adapter types are taken into account: Only those

overlaps that are actually allowed by the adapter type are actually considered.

.. _quality-trimming-algorithm:

Quality trimming algorithm

--------------------------

The trimming algorithm implemented in cutadapt is the same as the one used by

BWA, but applied to both

ends of the read in turn (if requested). That is: Subtract the given cutoff

from all qualities; compute partial sums from all indices to the end of the

sequence; cut the sequence at the index at which the sum is minimal. If both

ends are to be trimmed, repeat this for the other end.

The basic idea is to remove all bases starting from the end of the read whose

quality is smaller than the given threshold. This is refined a bit by allowing

some good-quality bases among the bad-quality ones. In the following example,

we assume that the 3' end is to be quality-trimmed.

Assume you use a threshold of 10 and have these quality values:

42, 40, 26, 27, 8, 7, 11, 4, 2, 3

Subtracting the threshold gives:

32, 30, 16, 17, -2, -3, 1, -6, -8, -7

Then sum up the numbers, starting from the end (partial sums). Stop early if

the sum is greater than zero:

(70), (38), 8, -8, -25, -23, -20, -21, -15, -7

The numbers in parentheses are not computed (because 8 is greater than zero),

but shown here for completeness. The position of the minimum (-25) is used as

the trimming position. Therefore, the read is trimmed to the first four bases,

which have quality values 42, 40, 26, 27.

The `Cutadapt source code is on GitHub <https://github.com/marcelm/cutadapt/>`_.

Cutadapt is written in Python with some extension modules that are written

in Cython. Cutadapt uses a single code base that is compatible with both

Python 2 and 3. Python 2.7 is the minimum supported Python version. With

relatively little effort, compatibility with Python 2.6 could be restored.

Python 2 and 3. Python 2.7 is the minimum supported Python version.

Development installation

Making a release

----------------

Since version 1.17, Travis CI is used to automatically deploy a new Cutadapt release

(both as an sdist and as wheels) whenever a new tag is pushed to the Git repository.

Cutadapt uses `versioneer <https://github.com/warner/python-versioneer>`_ to automatically manage

version numbers. This means that the version is not stored in the source code but derived from

the most recent Git tag. The following procedure can be used to bump the version and make a new

release.

#. Update ``CHANGES.rst`` (version number and list of changes)

#. Ensure you have no uncommitted changes in the working copy.

#. Run a ``git pull``.

#. Run ``tox``, ensuring all tests pass.

#. Tag the current commit with the version number (there must be a ``v`` prefix)::

       git tag v0.1

   To release a development version, use a ``dev`` version number such as ``v1.17.dev1``.

   Users will not automatically get these unless they use ``pip install --pre``.

#. Push the tag::

       git push --tags

#. Wait for Travis to finish and to deploy to PyPI.

#. Update the `bioconda recipe <https://github.com/bioconda/bioconda-recipes/blob/master/recipes/cutadapt/meta.yaml>`_.

   It is probly easiest to edit the recipe via the web interface and send in a

   pull request. Ensure that the list of dependencies (the ``requirements:``

   section in the recipe) is in sync with the ``setup.py`` file.

   Since this is just a version bump, the pull request does not need a

   review by other bioconda developers. As soon as the tests pass and if you

   have the proper permissions, it can be merged directly.

Releases to bioconda still need to be made manually.

Making a release manually

-------------------------

.. note:

    This section is outdated, see the previous section!

If this is the first time you attempt to upload a distribution to PyPI, create a

configuration file named ``.pypirc`` in your home directory with the following

contents::