Commit 3a17cb1f authored by Steffen Möller's avatar Steffen Möller
Browse files

New upstream version 983

parent 173bf5b1

ChangeLog.txt

+122 −1
Original line number Diff line number Diff line 
2019-07-16  Martin C. Frith  <Martin C. Frith>



	* doc/last-split.txt, doc/last-tutorial.txt, scripts/last-train,

	src/split/last-split-main.cc, test/last-split-test.out, test/last-

	split-test.sh:

	Change last-split -m default to 1

	[2a722dbe737a] [tip]



2019-06-27  Martin C. Frith  <Martin C. Frith>



	* src/split/last-split.cc:

	Fix rare last-split spliced alignment crash

	[4f77b3fc0720]



2019-06-21  Martin C. Frith  <Martin C. Frith>



	* scripts/last-dotplot:

	dotplot: change the undocumented input format

	[0e184d90d5da]



2019-04-22  Martin C. Frith  <Martin C. Frith>



	* doc/last-dotplot.txt, scripts/last-dotplot:

	Add last-dotplot --join option

	[27c26371226b]



2019-04-01  Martin C. Frith  <Martin C. Frith>



	* src/LambdaCalculator.cc, src/LambdaCalculator.hh,

	src/TantanMasker.cc, src/lastal.cc, src/makefile,

	src/mcf_substitution_matrix_stats.cc,

	src/mcf_substitution_matrix_stats.hh:

	Make lastal startup even faster

	[9b416e560dc3]



	* src/LambdaCalculator.cc, src/LambdaCalculator.hh, src/makefile:

	Make lastal startup faster

	[d429589d040e]



	* src/Alignment.cc, src/Centroid.cc, src/Centroid.hh:

	Simplify some probability calculations a bit

	[9eeaeea88f2c]



2019-03-29  Martin C. Frith  <Martin C. Frith>



	* src/Alignment.cc, src/Centroid.cc, src/Centroid.hh:

	Refactor

	[f3f69ccff967]



2019-03-28  Martin C. Frith  <Martin C. Frith>



	* src/Alignment.cc, src/Alignment.hh, src/Centroid.cc,

	src/Centroid.hh, src/GeneralizedAffineGapCosts.cc,

	src/GeneralizedAffineGapCosts.hh, src/LastalArguments.cc,

	src/LastalArguments.hh, src/lastal.cc, src/makefile,

	src/mcf_gap_costs.cc, src/mcf_gap_costs.hh:

	Add piecewise linear gap costs (unfinished)

	[3701b1e2019b]



	* doc/last-train.txt, doc/lastal.txt, scripts/last-train,

	src/LastalArguments.cc, src/LastalArguments.hh, src/ScoreMatrix.cc,

	src/ScoreMatrix.hh, src/lastal.cc, test/dfam3-LTR22B1.fa,

	test/dfam3-LTR22C.fa, test/last-test.out, test/last-test.sh:

	Add lastal/last-train -X option for ambiguous N/X

	[2b0e54aee679]



2019-02-27  Martin C. Frith  <Martin C. Frith>



	* src/lastal.cc:

	Refactor

	[1ed9db7f46ca]



	* src/lastal.cc:

	Refactor

	[ae880ec00591]



2019-02-26  Martin C. Frith  <Martin C. Frith>



	* src/lastal.cc:

	Refactor

	[6189dce68b46]



	* doc/lastal.txt, src/LastalArguments.cc, src/lastal.cc, src/makefile,

	src/split/cbrc_split_aligner.cc:

	Change the effect of setting lastal option -t

	[2073177a7920]



	* src/lastal.cc, test/last-train-test.out:

	Fix fastq-aware last-train (lastal -j7)

	[7dcde7432ed7]



2019-02-25  Martin C. Frith  <Martin C. Frith>



	* src/OneQualityScoreMatrix.cc, src/OneQualityScoreMatrix.hh,

	src/TwoQualityScoreMatrix.cc, src/TwoQualityScoreMatrix.hh,

	src/lastal.cc, src/qualityScoreUtil.hh:

	Refactor

	[1bbe50060a6f]



	* src/lastal.cc, src/mcf_substitution_matrix_stats.cc,

	src/mcf_substitution_matrix_stats.hh:

	Make lastal code more robust for non-DNA

	[a75e610c943c]



	* src/lastal.cc, src/makefile, src/mcf_substitution_matrix_stats.cc,

	src/mcf_substitution_matrix_stats.hh:

	Refactor (prepare new score-matrix analysis)

	[685f6de69cc8]



	* src/LambdaCalculator.cc, src/LambdaCalculator.hh,

	src/TantanMasker.cc, src/lastal.cc:

	Refactor

	[92c7b8f19291]



	* scripts/last-train, src/LastEvaluer.hh, src/LastalArguments.cc,

	src/LastalArguments.hh, src/cbrc_linalg.cc, src/cbrc_linalg.hh,

	src/lastal.cc, src/makefile, src/split/cbrc_linalg.cc,

	src/split/cbrc_linalg.hh:

	Refactor

	[a5c1e7e987ff]



2018-12-26  Martin C. Frith  <Martin C. Frith>



	* src/tantan.cc:

	Make tantan repeat-finding faster

	[44199bff9cef] [tip]

	[44199bff9cef]



	* doc/Makefile, doc/last-parallel.txt, doc/last-split.txt, doc/last-

	tutorial.txt:

doc/last-dotplot.html

+7 −0
Original line number Diff line number Diff line
@@ -422,6 +422,13 @@ for primary and secondary alignments.</td></tr> 
<kbd><span class="option">--pad=<var>FRAC</var></span></kbd></td>

<td>Length of pad to leave when cutting unaligned gaps.</td></tr>

<tr><td class="option-group">

<kbd><span class="option">-j <var>N</var></span>, <span class="option">--join=<var>N</var></span></kbd></td>

<td>Draw horizontal or vertical lines joining adjacent alignments.

0 means don't join, 1 means draw vertical lines joining

alignments that are adjacent in the 1st (horizontal) genome, 2

means draw horizontal lines joining alignments that are adjacent

in the 2nd (vertical) genome.</td></tr>

<tr><td class="option-group">

<kbd><span class="option">--border-pixels=<var>INT</var></span></kbd></td>

<td>Number of pixels between sequences.</td></tr>

<tr><td class="option-group">

doc/last-dotplot.txt

+6 −0
Original line number Diff line number Diff line
@@ -84,6 +84,12 @@ Options 
      Maximum unaligned gap in the 2nd genome.

  --pad=FRAC

      Length of pad to leave when cutting unaligned gaps.

  -j N, --join=N

      Draw horizontal or vertical lines joining adjacent alignments.

      0 means don't join, 1 means draw vertical lines joining

      alignments that are adjacent in the 1st (horizontal) genome, 2

      means draw horizontal lines joining alignments that are adjacent

      in the 2nd (vertical) genome.

  --border-pixels=INT

      Number of pixels between sequences.

  --border-color=COLOR

doc/last-split.html

+11 −32
Original line number Diff line number Diff line
@@ -318,8 +318,8 @@ table.field-list { border: thin solid green } 
<div class="document" id="last-split">

<h1 class="title">last-split</h1>



<p>This program estimates &quot;split alignments&quot; (typically for DNA) or

&quot;spliced alignments&quot; (typically for RNA).</p>

<p>This program finds &quot;split alignments&quot; (typically for DNA) or &quot;spliced

alignments&quot; (typically for RNA).</p>

<p>It reads candidate alignments of query sequences to a genome, and

looks for a unique best alignment for each part of each query.  It

allows different parts of one query to match different parts of the
@@ -334,7 +334,8 @@ format), and the genome is in &quot;genome.fasta&quot; (in fasta format). We 
can do the alignment like this:</p>

<pre class="literal-block">

lastdb -uNEAR -R01 db genome.fasta

lastal -Q1 db q.fastq | last-split &gt; out.maf

last-train -Q1 db q.fastq &gt; train.out

lastal -p train.out db q.fastq | last-split &gt; out.maf

</pre>

</div>

<div class="section" id="spliced-alignment-of-rna-reads-to-a-genome">
@@ -345,7 +346,8 @@ which causes it to do spliced instead of split alignment, and also 
tells it where the splice signals are (GT, AG, etc):</p>

<pre class="literal-block">

lastdb -uNEAR -R01 db genome.fasta

lastal -Q1 -D10 db q.fastq | last-split -g db &gt; out.maf

last-train -Q1 db q.fastq &gt; train.out

lastal -p train.out -D10 db q.fastq | last-split -g db &gt; out.maf

</pre>

<p>This will favour splices starting at GT (and to a lesser extent GC and

AT), and ending at AG (and to a lesser extent AC).  However, it allows
@@ -362,17 +364,7 @@ last-split options.</p> 
</div>

<div class="section" id="alignment-of-two-whole-genomes">

<h3>Alignment of two whole genomes</h3>

<p>We can align the cat and rat genomes like this:</p>

<pre class="literal-block">

lastdb -uMAM8 -cR11 catdb cat.fasta

lastal -m100 -E0.05 catdb rat.fasta | last-split -m1 &gt; out.maf

</pre>

<p>This will align each rat base-pair to at most one cat base-pair, but

not necessarily vice-versa.  We can get 1-to-1 alignments by swapping

the sequences and running last-split again:</p>

<pre class="literal-block">

maf-swap out.maf | last-split -m1 &gt; out2.maf

</pre>

<p>See <a class="reference external" href="https://github.com/mcfrith/last-genome-alignments">here</a>.</p>

</div>

</div>

<div class="section" id="faq">
@@ -393,15 +385,6 @@ matches.</p> 
</tbody>

</table>

</div>

<div class="section" id="going-faster-by-parallelization">

<h2>Going faster by parallelization</h2>

<p>For example, split alignment of DNA reads to a genome:</p>

<pre class="literal-block">

parallel-fastq &quot;lastal -Q1 db | last-split&quot; &lt; q.fastq &gt; out.maf

</pre>

<p>This requires GNU parallel to be installed

(<a class="reference external" href="http://www.gnu.org/software/parallel/">http://www.gnu.org/software/parallel/</a>).</p>

</div>

<div class="section" id="output">

<h2>Output</h2>

<p>The output is in MAF(-like) format:</p>
@@ -514,7 +497,7 @@ lowest possible. In general:</p> 
Error probability &lt;= 10 ^ -((ASCII value - 33) / 10)

</pre>

<p>The &quot;mismap&quot; is simply the lowest probability from the &quot;p&quot; line.  (If

you run last-split twice, as in the genome alignment example, the

you run last-split twice, as in the genome alignment recipes, the

mismap is the lowest combined error probability from both &quot;p&quot; lines.)</p>

</div>

<div class="section" id="split-versus-spliced-alignment">
@@ -545,8 +528,7 @@ of the query.)</p> 
<p>Spliced alignment can be slow.  It can be sped up, at a small cost in

accuracy, by not favouring cis-splices:</p>

<pre class="literal-block">

lastdb -uNEAR -R01 db genome.fasta

lastal -Q1 -D10 db q.fastq | last-split -c0 -t0.004 -g db &gt; out.maf

lastal -p train.out -D10 db q.fastq | last-split -c0 -t0.004 -g db &gt; out.maf

</pre>

<p>The -c0 turns off cis-splicing, and the -t0.004 specifies a higher

probability of trans-splicing.</p>
@@ -557,8 +539,7 @@ probability of trans-splicing.</p> 
middle of the query, we can do &quot;spliced&quot; alignment without considering

splice signals or favouring cis-splices:</p>

<pre class="literal-block">

lastdb -uNEAR -R01 db genome.fasta

lastal -Q1 db q.fastq | last-split -c0 &gt; out.maf

lastal -p train.out db q.fastq | last-split -c0 &gt; out.maf

</pre>

</div>

</div>
@@ -613,9 +594,7 @@ The default value fits human RNA.</td></tr> 
ln(intron length).  The default value fits human RNA.</td></tr>

<tr><td class="option-group">

<kbd><span class="option">-m</span>, <span class="option">--mismap=<var>PROB</var></span></kbd></td>

<td>Don't write alignments with mismap probability &gt; PROB.

Low-confidence alignments will be discarded unless you

increase this value!</td></tr>

<td>Don't write alignments with mismap probability &gt; PROB.</td></tr>

<tr><td class="option-group">

<kbd><span class="option">-s</span>, <span class="option">--score=<var>INT</var></span></kbd></td>

<td><p class="first">Don't write alignments with score &lt; INT.</p>

doc/last-split.txt

+10 −31
Original line number Diff line number Diff line 
last-split

==========



This program estimates "split alignments" (typically for DNA) or

"spliced alignments" (typically for RNA).

This program finds "split alignments" (typically for DNA) or "spliced

alignments" (typically for RNA).



It reads candidate alignments of query sequences to a genome, and

looks for a unique best alignment for each part of each query.  It
@@ -21,7 +21,8 @@ format), and the genome is in "genome.fasta" (in fasta format). We 
can do the alignment like this::



  lastdb -uNEAR -R01 db genome.fasta

  lastal -Q1 db q.fastq | last-split > out.maf

  last-train -Q1 db q.fastq > train.out

  lastal -p train.out db q.fastq | last-split > out.maf



Spliced alignment of RNA reads to a genome

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -32,7 +33,8 @@ which causes it to do spliced instead of split alignment, and also 
tells it where the splice signals are (GT, AG, etc)::



  lastdb -uNEAR -R01 db genome.fasta

  lastal -Q1 -D10 db q.fastq | last-split -g db > out.maf

  last-train -Q1 db q.fastq > train.out

  lastal -p train.out -D10 db q.fastq | last-split -g db > out.maf



This will favour splices starting at GT (and to a lesser extent GC and

AT), and ending at AG (and to a lesser extent AC).  However, it allows
@@ -52,16 +54,7 @@ last-split options. 
Alignment of two whole genomes

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



We can align the cat and rat genomes like this::



  lastdb -uMAM8 -cR11 catdb cat.fasta

  lastal -m100 -E0.05 catdb rat.fasta | last-split -m1 > out.maf



This will align each rat base-pair to at most one cat base-pair, but

not necessarily vice-versa.  We can get 1-to-1 alignments by swapping

the sequences and running last-split again::



  maf-swap out.maf | last-split -m1 > out2.maf

See `here <https://github.com/mcfrith/last-genome-alignments>`_.



FAQ

---
@@ -73,16 +66,6 @@ FAQ 
    can prevent lastal and last-split from wasting time on such

    matches.



Going faster by parallelization

-------------------------------



For example, split alignment of DNA reads to a genome::



  parallel-fastq "lastal -Q1 db | last-split" < q.fastq > out.maf



This requires GNU parallel to be installed

(http://www.gnu.org/software/parallel/).



Output

------
@@ -125,7 +108,7 @@ lowest possible. In general:: 
  Error probability <= 10 ^ -((ASCII value - 33) / 10)



The "mismap" is simply the lowest probability from the "p" line.  (If

you run last-split twice, as in the genome alignment example, the

you run last-split twice, as in the genome alignment recipes, the

mismap is the lowest combined error probability from both "p" lines.)



Split versus spliced alignment
@@ -159,8 +142,7 @@ Faster spliced alignment 
Spliced alignment can be slow.  It can be sped up, at a small cost in

accuracy, by not favouring cis-splices::



  lastdb -uNEAR -R01 db genome.fasta

  lastal -Q1 -D10 db q.fastq | last-split -c0 -t0.004 -g db > out.maf

  lastal -p train.out -D10 db q.fastq | last-split -c0 -t0.004 -g db > out.maf



The -c0 turns off cis-splicing, and the -t0.004 specifies a higher

probability of trans-splicing.
@@ -172,8 +154,7 @@ If we do not wish to allow arbitrarily large unaligned parts in the 
middle of the query, we can do "spliced" alignment without considering

splice signals or favouring cis-splices::



  lastdb -uNEAR -R01 db genome.fasta

  lastal -Q1 db q.fastq | last-split -c0 > out.maf

  lastal -p train.out db q.fastq | last-split -c0 > out.maf



Options

-------
@@ -221,8 +202,6 @@ Options 


  -m, --mismap=PROB

         Don't write alignments with mismap probability > PROB.

         Low-confidence alignments will be discarded unless you

         increase this value!



  -s, --score=INT

         Don't write alignments with score < INT.