New upstream version 0.10.2

Software package for signal-level analysis of Oxford Nanopore sequencing data. Nanopolish can calculate an improved consensus sequence for a draft genome assembly, detect base modifications, call SNPs and indels with respect to a reference genome and more (see Nanopolish modules, below).

## Release notes

* 0.10.2: added new program `nanopolish polya` to estimate the length of poly-A tails on direct RNA reads (by @paultsw)

* 0.10.1: `nanopolish variants --consensus` now only outputs a VCF file instead of a fasta sequence. The VCF file describes the changes that need to be made to turn the draft sequence into the polished assembly. A new program, `nanopolish vcf2fasta`, is provided to generate the polished genome (this replaces `nanopolish_merge.py`, see usage instructions below). This change is to avoid issues when merging segments that end on repeat boundaries (reported by Michael Wykes and Chris Wright).

## Dependencies

A compiler that supports C++11 is needed to build nanopolish. Development of the code is performed using [gcc-4.8](https://gcc.gnu.org/gcc-4.8/).

```

git clone --recursive https://github.com/jts/nanopolish.git

cd nanopolish

git checkout v0.7.1

git checkout v0.9.2

make

```

The main subprograms of nanopolish are:

```

nanopolish extract: extract reads in FASTA or FASTQ format from a directory of FAST5 files

nanopolish call-methylation: predict genomic bases that may be methylated

nanopolish variants: detect SNPs and indels with respect to a reference genome

nanopolish variants --consensus: calculate an improved consensus sequence for a draft genome assembly

```

python nanopolish_makerange.py draft.fa | parallel --results nanopolish.results -P 8 \

    nanopolish variants --consensus polished.{1}.fa -w {1} -r reads.fa -b reads.sorted.bam -g draft.fa -t 4 --min-candidate-frequency 0.1

    nanopolish variants --consensus -o polished.{1}.vcf -w {1} -r reads.fa -b reads.sorted.bam -g draft.fa -t 4 --min-candidate-frequency 0.1

```

This command will run the consensus algorithm on eight 50kbp segments of the genome at a time, using 4 threads each. Change the ```-P``` and ```--threads``` options as appropriate for the machines you have available.

After all polishing jobs are complete, you can merge the individual 50kb segments together back into the final assembly:

```

python nanopolish_merge.py polished.*.fa > polished_genome.fa

nanopolish vcf2fasta -g draft.fa polished.*.vcf > polished_genome.fa

```

## Calling Methylation

    nanopolish variants: detect SNPs and indels with respect to a reference genome

    nanopolish variants --consensus: calculate an improved consensus sequence for a draft genome assembly

    nanopolish eventalign: align signal-level events to k-mers of a reference genome

    nanopolish phase-reads: Phase reads using heterozygous SNVs with respect to a reference genome 

|

extract

   * - ``-r``, ``--reads=FILE``

     - Y

     - NA

     - the 2D ONT reads are in fasta FILE

     - the ONT reads are in fasta FILE

   * - ``-b``, ``--bam=FILE``

     - Y

   * - ``-r``, ``--reads=FILE``

     - Y

     - NA

     - the 2D ONT reads are in fasta FILE

     - the ONT reads are in fasta FILE

   * - ``-b``, ``--bam=FILE``

     - Y

   * - ``-r, --reads=FILE``

     - Y

     - NA

     - the 2D ONT reads are in fasta FILE

     - the ONT reads are in fasta FILE

   * - ``-b, --bam=FILE``

     - Y

     - N

     - NA

     - read alternative k-mer models from FILE

phase-reads - (experimental)

--------------------

Overview

"""""""""""""""""""""""

Phase reads using heterozygous SNVs with respect to a reference genome 

Input

"""""""""""""""""""""""

    * basecalled reads

    * alignment information

    * assembled genome

    * variants (from nanopolish variants or from other sources eg. Illumina VCF)

Usage example

"""""""""""""""""""""""

::

   nanopolish phase-reads [OPTIONS] --reads reads.fa --bam alignments.bam --genome genome.fa variants.vcf

.. list-table::

   :widths: 20 10 20 50

   :header-rows: 1

   * - Argument name(s)

     - Required

     - Default value

     - Description

   * - ``-v``

     - N

     - NA

     - write verbose output

   * - ``-w, --window=STR``

     - N

     - NA

     - Only phase reads in the window STR (format : ctg:start_id-end_id)

   * - ``-r, --reads=FILE``

     - Y

     - NA

     - the ONT reads are in fasta FILE

   * - ``-b, --bam=FILE``

     - Y

     - NA

     - the reads aligned to the genome assembly are in bam FILE

   * - ``-g, --genome=FILE``

     - Y

     - NA

     - the genome we are computing a consensus for is in FILE

   * - ``variants.vcf``

     - Y

     - NA

     - the variants (from nanopolish variants or Illumina in VCF format) to be phased are in FILE

   * - ``-t, --threads=NUM``

     - N

     - 1

     - use NUM threads

    * - ``--progress``

     - N

     - NA

     - print out a progress message

.. _quickstart_consensus:

Quickstart - how to polish the consensus assembly

Quickstart - how to polish a genome assembly

===================================================

The original purpose for nanopolish was to improve the consensus assembly accuracy for Oxford Nanopore Technology sequencing reads. Here we provide a step-by-step tutorial to help you get started with our tool.

The original purpose of nanopolish was to improve the consensus accuracy of an assembly of Oxford Nanopore Technology sequencing reads. Here we provide a step-by-step tutorial to help you get started.

**Requirements**:

* `nanopolish v0.8.4 <installation.html>`_

* `samtools v1.2 <https://htslib.org>`_

* `bwa v0.7.12 <https://github.com/lh3/bwa>`_

* `nanopolish <installation.html>`_

* `samtools <https://htslib.org>`_

* `minimap2 <https://github.com/lh3/minimap2>`_

* `MUMmer <https://github.com/mummer4/mummer>`_

Download example dataset

* Region: "tig00000001:200000-202000"

* Note: Ligation-mediated PCR amplification performed

This is a subset of reads that aligned to a 2kb region in the E. coli draft assembly. To see how we generated these files please refer to the tutorial :ref:`creati

ng_example_dataset <here>`.

This is a subset of reads that aligned to a 2kb region in the E. coli draft assembly. To see how we generated these files please refer to the tutorial :ref:`creating_example_dataset <here>`.

You should find the following files:

For the evaluation step you will need the reference genome: ::

    wget -O ref.fa ftp://ftp.ncbi.nih.gov/genomes/archive/old_genbank/Bacteria/Escherichia_coli_K_12_substr__MG1655_uid225/U00096.ffn

    curl -o ref.fa https://ftp.ncbi.nih.gov/genomes/archive/old_genbank/Bacteria/Escherichia_coli_K_12_substr__MG1655_uid225/U00096.ffn

Analysis workflow

-------------------------------

The pipeline below describes the recommended analysis workflow for larger datasets. In this tutorial, we will run through the basic steps of the pipeline for this smaller (2kb) dataset.

.. figure:: _static/nanopolish-workflow.png

  :scale: 90%

  :alt: nanopolish-tutorial-workflow

Data preprocessing

-----------------------------------------------

As computing the draft genome assembly takes a few hours we have included the pre-assembled data for you (``draft.fa``).

We used the following parameters with `canu <canu.readthedocs.io>`_: ::

We used the following parameters with `canu <http://canu.readthedocs.io/en/latest/>`_: ::

    canu \

        -p ecoli -d outdir genomeSize=4.6m \

        -nanopore-raw albacore-2.0.1-merged.fastq \

        -nanopore-raw albacore-2.0.1-merged.fastq

Compute a new consensus sequence for a draft assembly

------------------------------------------------------------------------

Now that we have ``reads.fasta`` indexed with ``nanopolish index``, and have a draft genome assembly ``draft.fa``, we can begin to improve the assembly with nanopolish. Let us get started!

First step, is to index the draft genome assembly. We can do that with the following command: ::

First, we align the original reads (``reads.fasta``) to the draft assembly (``draft.fa``) and sort alignments: ::

    bwa index draft.fa

Next, we align the original reads (``reads.fasta``) to the draft assembly (``draft.fa``) and sort alignments: ::

    bwa mem -x ont2d -t 8 draft.fa reads.fasta | samtools sort -o reads.sorted.bam -T reads.tmp

    minimap2 -ax map-ont -t 8 draft.fa reads.fasta | samtools sort -o reads.sorted.bam -T reads.tmp

    samtools index reads.sorted.bam

**Checkpoint**: we can do a quick check to see if this step worked. The bam file should not be empty. ::

Then we run the consensus algorithm. For larger datasets we use ``nanopolish_makerange.py`` to split the draft genome assembly into 50kb segments, so that we can run the consensus algorithm on each segment in parallel. The output would be the polished segments in ``fasta`` format. 

Since our dataset is only covering a 2kb region, we skip this step and use the following command: ::

    nanopolish variants --consensus polished.fa \

    nanopolish variants --consensus -o polished.vcf \

        -w "tig00000001:200000-202000" \

        -r reads.fasta \

        -b reads.sorted.bam \

        -g draft.fa

We are left with our desired output: ``polished.fa``.

We are left with: ``polished.vcf``.

**Note**: As of v0.10.1, ``nanopolish variants --consensus`` only outputs a VCF file instead of a fasta sequence.

To generate the polished genome in fasta format: ::

    nanopolish vcf2fasta --skip-checks -g draft.fa polished.vcf > polished_genome.fa

We only polished a 2kb region, so let's pull that out: :: 

    samtools faidx polished_genome.fa

    samtools faidx polished_genome.fa "tig00000001:200000-202000" > polished.fa

Evaluate the assembly

---------------------------------

**Requirements**:

* `nanopolish v0.8.4 <installation.html>`_

* `samtools v1.2 <http://samtools.sourceforge.net/>`_

* `bwa v0.7.12 <https://github.com/lh3/bwa>`_

* `nanopolish <installation.html>`_

* `samtools <http://samtools.sourceforge.net/>`_

* `minimap2 <https://github.com/lh3/minimap2>`_

Download example dataset

------------------------------------

You will need the E. coli reference genome: ::

    wget -O ref.fa ftp://ftp.ncbi.nih.gov/genomes/archive/old_genbank/Bacteria/Escherichia_coli_K_12_substr__MG1655_uid225/U00096.ffn

    curl -o ref.fa https://ftp.ncbi.nih.gov/genomes/archive/old_genbank/Bacteria/Escherichia_coli_K_12_substr__MG1655_uid225/U00096.ffn

Align the reads with bwa

Align the reads with minimap2

--------------------------------

In order to run bwa we first need to index the reference genome: ::

    bwa index ref.fa

**Output files**: ``ref.fa.sa``, ``ref.fa.amb``, ``ref.fa.ann``, ``ref.fa.pac``, and ``ref.fa.bwt``.

We will need to index the reads as well: ::

We first need to index the reads: ::

    nanopolish index -d fast5_files/ reads.fasta

Then we can align the reads to the reference: ::

    bwa mem -x ont2d ref.fa reads.fasta | samtools sort -o reads-ref.sorted.bam -T reads.tmp

    minimap2 -ax map-ont -t 8 ref.fa reads.fasta | samtools sort -o reads-ref.sorted.bam -T reads.tmp

    samtools index reads-ref.sorted.bam

**Output files**: ``reads-ref.sorted.bam`` and ``reads-ref.sorted.bam.bai``.