README.md

 ## mapDamage
 
-[![install with bioconda](https://img.shields.io/badge/install%20with-bioconda-brightgreen.svg?style=flat)](http://bioconda.github.io/recipes/mapdamage2/README.html) [![Conda](https://img.shields.io/conda/dn/bioconda/mapdamage2.svg)](https://anaconda.org/bioconda/mapdamage2/files)
+[![install with bioconda](https://img.shields.io/badge/install%20with-bioconda-brightgreen.svg?style=flat)](http://bioconda.github.io/recipes/mapdamage2/README.html) [![Conda](https://img.shields.io/conda/dn/bioconda/mapdamage2.svg)](https://anaconda.org/bioconda/mapdamage2/files) ![Conda](https://anaconda.org/bioconda/mapdamage2/badges/latest_release_date.svg) ![Conda](https://anaconda.org/bioconda/mapdamage2/badges/version.svg) [![Project Status: Inactive – The project has reached a stable, usable state but is no longer being actively developed; support/maintenance will be provided as time allows.](https://www.repostatus.org/badges/latest/inactive.svg)](https://www.repostatus.org/#inactive)
+
+
+#### `bioconda` installation
+
+- python3 version **2.2.0**
+```
+conda install -c bioconda mapdamage2=2.2.0
+```
+
+- python3 version **2.2.0** **with** R and 4 mandatory packages for the Bayesian inference:
+```
+conda install -c bioconda mapdamage2=2.2.0=pyr36_1
+```
 
 ---
 
 ### Important
-Users with versions dating prior to June 12 2013 please update. A nasty bug that caused the statistical part of `mapDamage` to use half of the data for estimation of the damage parameters, sorry for the inconvenience.
+
+- From version `2.2.0` the `master` branch is requiring **python3** as `python2` is not supported from 2020-01-01.
+
+- Users with versions dating prior to June 12 2013 please update. A nasty bug that caused the statistical part of `mapDamage` to use half of the data for estimation of the damage parameters, sorry for the inconvenience.
 
 ### Introduction
 Complete documentation, instructions, examples, screenshots and FAQ are available at [this address](http://ginolhac.github.io/mapDamage/).
 
-[mapDamage2](http://geogenetics.ku.dk/publications/mapdamage2.0/) is a computational framework written in **Python3** and **R**, which tracks and quantifies DNA damage patterns
+[mapDamage2](https://geogenetics.ku.dk/publications/mapdamage2.0/) is a computational framework written in **Python3** and **R**, which tracks and quantifies DNA damage patterns
 among ancient DNA sequencing reads generated by Next-Generation Sequencing platforms.
 
-`mapDamage` was developed at the [Centre for GeoGenetics](http://geogenetics.ku.dk/) by the [Orlando Group ](http://geogenetics.ku.dk/research/research_groups/palaeomix_group/).
+`mapDamage` was developed at the [Centre for GeoGenetics](https://geogenetics.ku.dk/) by the [Orlando Group ](https://geogenetics.ku.dk/research_groups/palaeomix_group/).
 
 
 ### Citation
@@ -29,15 +45,30 @@ Ginolhac A, Rasmussen M, Gilbert MT, Willerslev E, Orlando L.
 http://bioinformatics.oxfordjournals.org/content/27/15/2153](http://bioinformatics.oxfordjournals.org/content/27/15/2153)
 
 
-### Test
+### Test the no-stats part and rescaling
 
-in the package, you can test `mapDamage` by running:
+you can test `mapDamage` by running:
 
 ```
 cd mapDamage/mapdamage/
 python3 mp_test.py
 ```
 
+should return
+```
+Started with the command: /usr/local/bin/mapDamage -i tests/test.bam -r tests/fake1.fasta -d tests/results --no-stats
+	Reading from 'tests/test.bam'
+	Writing results to 'tests/results/'
+pdf tests/results/Fragmisincorporation_plot.pdf generated
+additional tests/results/Length_plot.pdf generated
+Successful run
+.
+----------------------------------------------------------------------
+Ran 2 tests in 3.357s
+
+OK
+```
+
 
 ### Contact
 Please report bugs and suggest possible improvements to Aurélien Ginolhac, Mikkel Schubert or Hákon Jónsson by email:

debian/changelog

+mapdamage (2.2.0+dfsg-1) unstable; urgency=medium
+
+  * New upstream version
+  * Remove trailing whitespace in debian/changelog
+  * autopkgtest: s/ADTTMP/AUTOPKGTEST_TMP/g
+  * Set upstream metadata fields: Bug-Database.
+  * Fix and enhance build time and autopkgtest
+
+ -- Andreas Tille <tille@debian.org>  Fri, 13 Dec 2019 14:01:31 +0100
+
 mapdamage (2.1.1+dfsg-1) unstable; urgency=medium
 
   * Team upload.

debian/control

@@ -4,10 +4,15 @@ Uploaders: Andreas Tille <tille@debian.org>
 Section: science
 Priority: optional
 Build-Depends: debhelper-compat (= 12),
-               python3-all,
+               python3,
                dh-python,
                python3-setuptools,
-               python3-pysam <!nocheck>
+               python3-pysam <!nocheck>,
+               seqtk <!nocheck>,
+               r-base-core <!nocheck>,
+               r-cran-gam <!nocheck>,
+               r-cran-ggplot2 <!nocheck>,
+               r-cran-inline <!nocheck>
 Standards-Version: 4.4.1
 Vcs-Browser: https://salsa.debian.org/med-team/mapdamage
 Vcs-Git: https://salsa.debian.org/med-team/mapdamage.git

debian/docs

 debian/README.test
 debian/tests/run-unit-test
 debian/tests/run-r-packages-check
-debian/tests/ref_T.fa

debian/install

-#No longer available?
-#mapdamage/rescale_test	usr/share/doc/mapdamage/tests
+mapdamage/tests/probs	usr/lib/python3/dist-packages/mapdamage/tests

debian/patches/use_debian_packaged_seqtk.patch

@@ -3,13 +3,11 @@ Last-Update: Thu, 28 Jul 2016 15:13:14 +0200
 Bug-Debian: https://bugs.debian.org/859090
 Description: Use Debian packaged seqtk
 
-Index: mapdamage/setup.py
-===================================================================
---- mapdamage.orig/setup.py
-+++ mapdamage/setup.py
-@@ -6,21 +6,6 @@ from distutils.command.install import in
- import os
- import subprocess
+--- a/setup.py
++++ b/setup.py
+@@ -11,21 +11,6 @@ if sys.version_info < (3, 5):
+     print("At least Python 3.5 is required.\n", file=sys.stderr)
+     exit(1)
  
 -def compile_seqtk():
 -    """Compiling the seqtk toolkit"""
@@ -29,7 +27,7 @@ Index: mapdamage/setup.py
  def setup_version():
      if not os.path.exists(".git"):
          # Release version, no .git folder
-@@ -40,17 +25,8 @@ class compileInstall(DistutilsInstall):
+@@ -45,17 +30,8 @@ class compileInstall(DistutilsInstall):
      def run(self):
          self.record=""
          setup_version()
@@ -47,19 +45,17 @@ Index: mapdamage/setup.py
  
  setup(
      cmdclass={'install': compileInstall},
-@@ -59,7 +35,7 @@ setup(
-     author='Aurélien Ginolhac, Mikkel Schubert, Hákon Jónsson',
+@@ -64,7 +40,7 @@ setup(
+     author='Aurélien Ginolhac, Mikkel Schubert, Ãkon Jónsson',
      author_email='MSchubert@snm.ku.dk, jonsson.hakon@gmail.com',
      packages=['mapdamage'],
--    package_data={'mapdamage': ['Rscripts/*.R','Rscripts/stats/*.R','seqtk/seqtk']},
-+    package_data={'mapdamage': ['Rscripts/*.R','Rscripts/stats/*.R']},
+-    package_data={'mapdamage': ['Rscripts/*.R','Rscripts/stats/*.R','tests/*','seqtk/seqtk']},
++    package_data={'mapdamage': ['Rscripts/*.R','Rscripts/stats/*.R','tests/*']},
      scripts=['bin/mapDamage'],
      url='https://github.com/ginolhac/mapDamage',
      license='LICENSE.txt',
-Index: mapdamage/bin/mapDamage
-===================================================================
---- mapdamage.orig/bin/mapDamage
-+++ mapdamage/bin/mapDamage
+--- a/bin/mapDamage
++++ b/bin/mapDamage
 @@ -163,7 +163,7 @@ def main():
          sys.path.insert(0,path_to_mm)
      import mapdamage
@@ -69,10 +65,8 @@ Index: mapdamage/bin/mapDamage
      if not (os.path.isfile(fpath_seqtk) and os.access(fpath_seqtk, os.X_OK)):
          sys.stderr.write("Seqtk executable not accessible; mapDamage has not\n"
                           "been intalled properly or current user does not\n"
-Index: mapdamage/mapdamage/composition.py
-===================================================================
---- mapdamage.orig/mapdamage/composition.py
-+++ mapdamage/mapdamage/composition.py
+--- a/mapdamage/composition.py
++++ b/mapdamage/composition.py
 @@ -33,7 +33,7 @@ def get_base_comp(filename,destination=F
      Gets the basecomposition of all the sequences in filename
      and returns the value to destination if given.

debian/rules

@@ -13,11 +13,13 @@ override_dh_fixperms:
 	find debian -name checkLibraries.R -exec chmod +x \{\} \;
 	find debian -name runGeneral.R     -exec chmod +x \{\} \;
 
-# rescale_test.py seems to have left the building
-#override_dh_install:
-#	dh_install
-#	mv `find debian -name rescale_test.py` $(TESTDIR)
-#	sed -i 's/^import rescale/from mapdamage &/' $(TESTDIR)/rescale_test.py
+override_dh_auto_test:
+ifeq (,$(filter nocheck,$(DEB_BUILD_OPTIONS)))
+	cd mapdamage && \
+	PATH=$(PATH):$(CURDIR)/bin \
+	PYTHONPATH=$(shell pybuild --print build_dir --interpreter python3) \
+	python3 mp_test.py
+endif
 
 override_dh_python3:
 	dh_python3 --no-ext-rename

debian/tests/run-unit-test

@@ -2,20 +2,23 @@
 
 pkg=mapdamage
 
-if [ "$ADTTMP" = "" ] ; then
-  ADTTMP=$(mktemp -d /tmp/${pkg}-test.XXXXXX)
-  trap "rm -rf $ADTTMP" 0 INT QUIT ABRT PIPE TERM
+if [ "$AUTOPKGTEST_TMP" = "" ] ; then
+  AUTOPKGTEST_TMP=$(mktemp -d /tmp/${pkg}-test.XXXXXX)
+  trap "rm -rf $AUTOPKGTEST_TMP" 0 INT QUIT ABRT PIPE TERM
 fi
 
-cd $ADTTMP
+cd $AUTOPKGTEST_TMP
+
+cp -a /usr/lib/python3/dist-packages/mapdamage/tests .
+cp /usr/lib/python3/dist-packages/mapdamage/mp_test.py .
+python3 mp_test.py
+
+exit 0
 
-cp -a /usr/share/doc/${pkg}/tests/* .
 cp -a /usr/share/doc/${pkg}/ref_T.fa .
 gunzip -r *
 
-# FIXME: rescale_test.py throws errors
-#        will discuss with upstream
-# python3 rescale_test.py
+# these old test so not work any more since rescale_test/ was removed upstream
 
 mapDamage -i rescale_test/pe_test/pe.sam -r ref_T.fa
 pdf_result="results_pe/Fragmisincorporation_plot.pdf"

debian/upstream/metadata

@@ -25,3 +25,4 @@ Registry:
   Entry: OMICS_02099
 - Name: conda:bioconda
   Entry: mapdamage2
+Bug-Database: https://github.com/ginolhac/mapDamage/issues

mapdamage/mp_test.py

 import unittest
 import subprocess
-
+import pysam
+import mapdamage
+import optparse
+import filecmp
 
 def read_nocomment(dnacomp):
     with open(dnacomp, 'r') as f:
         a = f.readlines()
         return [x for x in a if not x.startswith('#')]
 
+def mock_options(filename,rescale_out,folder):
+    """Make the options object with nice values for testing"""
+    return optparse.Values({
+        "filename":filename,
+        "rescale_out":rescale_out,
+        "verbose":True,
+        "folder":folder,
+        "rescale_length_5p":12, # default values as in --seq-length
+        "rescale_length_3p":12, # default values as in --seq-length
+        "quiet":True
+        })
+
+
 class testCases(unittest.TestCase):
 
     def test_no_stats(self):
@@ -14,5 +30,31 @@ class testCases(unittest.TestCase):
         subprocess.run(["mapDamage",  "-i",  "tests/test.bam", "-r", "tests/fake1.fasta", "-d", "tests/results", "--no-stats"], check = True)
         self.assertTrue(read_nocomment("tests/dnacomp.txt") == read_nocomment("tests/results/dnacomp.txt"))
 
+class testRescaling(unittest.TestCase):
+    def test_single_end_file(self):
+        """Test, rescaling BAM file"""
+        #
+	    # The expected substition frequencies before and after scaling using the scaled qualities as probalities:
+	    # CT	0.06226411977920493		0.04163524443356556
+	    # TC	0.020395286584806528		0.020395286584806528
+	    # GA	0.04400459948304954		0.03794905109091021
+	    # AG	0.05355350777642983		0.05355350777642983
+	    # Quality metrics before and after scaling
+	    # CT-Q0 	5		5
+	    # CT-Q10 	5		2
+	    # CT-Q20 	3		2
+	    # CT-Q30 	3		2
+	    # CT-Q40 	0		0
+	    # GA-Q0 	5		5
+	    # GA-Q10 	5		4
+	    # GA-Q20 	1		0
+	    # GA-Q30 	1		0
+	    # GA-Q40 	0		0
+        options = mock_options("tests/test.bam","tests/test.rescaled.sam","tests/probs/")
+        ref = pysam.Fastafile("tests/fake1.fasta")
+        mapdamage.rescale.rescale_qual(ref,options,debug=True)
+        self.assertTrue(filecmp.cmp("tests/test.rescaled.sam","tests/test.rescaled.correct.sam"))
+
+
 if  __name__=='__main__':
     unittest.main()

mapdamage/rescale.py

@@ -28,7 +28,8 @@ def get_corr_prob(folder, rescale_length_5p, rescale_length_3p):
     if not os.path.isfile(full_path):
         sys.exit("Missing file, the file \n\tStats_out_MCMC_correct_prob.csv\nshould be in the folder\n\t"+folder+"\nDid you run the MCMC estimates of the parameters?")
     try:
-        fi_handle = csv.DictReader(open(full_path))
+        with open(full_path) as fi:
+            fi_handle = csv.DictReader(fi)
             corr_prob = {}
             for line in fi_handle:
                 if (line["Position"] in corr_prob):
@@ -41,7 +42,6 @@ def get_corr_prob(folder, rescale_length_5p, rescale_length_3p):
             for key in list(corr_prob.keys()):
                 if key < -rescale_length_3p or key > rescale_length_5p:
                     corr_prob.pop(key)
-
             return corr_prob
     except csv.Error as e:
         sys.exit('File %s, line %d: %s' % (os.path.join(folder,"Stats_out_MCMC_correct_prob.csv"), \

mapdamage/tests/probs/Stats_out_MCMC_correct_prob.csv

+"","Position","C.T","G.A"
+"1",1,0.653616777018436,0
+"2",2,0.574821404505089,0
+"3",3,0.524660717195785,0
+"4",4,0.491676371805506,0
+"5",5,0.46919312975022,0
+"6",6,0.453361082263795,0
+"7",7,0.441882878430246,0
+"8",8,0.433345603590333,0
+"9",9,0.426853067214069,0
+"10",10,0.4218187429273,0
+"11",11,0.417847465539491,0
+"12",12,0.414666179579865,0
+"13",-12,0,0.427459648772127
+"14",-11,0,0.430678304009036
+"15",-10,0,0.434694598982698
+"16",-9,0,0.439783285536968
+"17",-8,0,0.446341431282045
+"18",-7,0,0.454957458647015
+"19",-6,0,0.466528745659899
+"20",-5,0,0.482466321045583
+"21",-4,0,0.505055143009322
+"22",-3,0,0.538101706832429
+"23",-2,0,0.588153916474448
+"24",-1,0,0.666277684522864

mapdamage/tests/test.rescaled.correct.sam

+@SQ	SN:fake1	LN:201
+@PG	ID:bwa	PN:bwa	VN:0.7.17-r1188	CL:bwa samse -f input.sam fake1.fasta - input.fastq
+fake1:50-100	0	fake1	69	37	31M	*	0	0	CCATGTCGGGCAGGCTGGTCTCGAACTCCTG	////////E6/EE/E/A////E//AAAE/EE	XT:A:U	NM:i:0	X0:i:1	X1:i:0	XM:i:0	XO:i:0	XG:i:0	MD:Z:31	MR:f:0
+fake1:50-100b	0	fake1	69	37	31M	*	0	0	CTATGTCGGGCAGGCTGGTCTCGAACTCCTG	/#//////E6/EE/E/A////E//AAAE/EE	XT:A:U	NM:i:1	X0:i:1	X1:i:0	XM:i:1	XO:i:0	XG:i:0	MD:Z:1C29	MR:f:0.57482
+fake1:50-100c	4	*	0	0	*	*	0	0	CGGTAGAGATGGAGTTTCACCATGTCGGGCAAG	//////////////A////////////E6/EE/
+fake1:50-100d	4	*	0	0	*	*	0	0	TAATAGAGATGGAGTTTCACCATGTCGTGCAAG	//////////////A////////////E6/EE/
+fake1:70-120	0	fake1	70	37	38M	*	0	0	TATGTCGGGCAGGCTGGTCTCGAACTCCTGACCTCAGA	#////6EE//AA6E//E66E/EAEEEEEAEEEAAEEA#	XT:A:U	NM:i:2	X0:i:1	X1:i:0	XM:i:2	XO:i:0	XG:i:0	MD:Z:0C36G0	MR:f:1.31989
+fake1:80-130	16	fake1	80	25	41M	*	0	0	GAGCTGGTCTCGAACTCCTGACCTCAGGCGATCTGCCTGTC	AAABBCCDDFEEEEEAEEEEA/EE/E/AAE//A//E/A///	XT:A:U	NM:i:3	X0:i:1	X1:i:0	XM:i:3	XO:i:0	XG:i:0	MD:Z:0A0G37C1	MR:f:0
+fake1:80-130b	16	fake1	80	25	51M	*	0	0	AGGCCGGTCTCGAACTCCTGACCTCAGGCGATCTGCCTGCCTTAACCTCCC	AAABBCCDDFEEEEEAEEEEA/EE/E/AAE//A//E/A///E/A$/EE66/	XT:A:U	NM:i:3	X0:i:1	X1:i:0	XM:i:3	XO:i:0	XG:i:0	MD:Z:4T37C1G6	MR:f:0.44188
+fake1:80-130c	16	fake1	80	37	51M	*	0	0	AGGCTGGTCTCGAACTCCTGACCTCAGGCGATCTGCCTGCCTTAGCCCCCC	AAABBCCDDFEEEEEAEEEEA/EE/E/AAE//A//E/A///E/A//EE66/	XT:A:U	NM:i:2	X0:i:1	X1:i:0	XM:i:2	XO:i:0	XG:i:0	MD:Z:42C4T3	MR:f:0
+fake1:80-130d	16	fake1	80	25	51M	*	0	0	AGGCTGGTCTCGAACTCCTGACCTCAGACGATCTGCCTGCCTTAGCCCCCC	AAABBCCDDFEEEEEAEEEEA/EE/E/AAE//A//E/A///E/A//EE66/	XT:A:U	NM:i:3	X0:i:1	X1:i:0	XM:i:3	XO:i:0	XG:i:0	MD:Z:27G14C4T3	MR:f:0

mapdamage/version.py

@@ -2,4 +2,4 @@
 try:
     from mapdamage._version import __version__
 except ImportError:
-    __version__ = "2.1.0"
+    __version__ = "2.2.0"

setup.py

@@ -4,8 +4,13 @@
 from distutils.core import setup
 from distutils.command.install import install as DistutilsInstall
 import os
+import sys
 import subprocess
 
+if sys.version_info < (3, 5):
+    print("At least Python 3.5 is required.\n", file=sys.stderr)
+    exit(1)
+
 def compile_seqtk():
     """Compiling the seqtk toolkit"""
     old_wd = os.getcwd()
@@ -55,11 +60,11 @@ class compileInstall(DistutilsInstall):
 setup(
     cmdclass={'install': compileInstall},
     name='mapdamage',
-    version='2.1.0',
-    author='Aurélien Ginolhac, Mikkel Schubert, Hákon Jónsson',
+    version='2.2.0',
+    author='Aurélien Ginolhac, Mikkel Schubert, Ãkon Jónsson',
     author_email='MSchubert@snm.ku.dk, jonsson.hakon@gmail.com',
     packages=['mapdamage'],
-    package_data={'mapdamage': ['Rscripts/*.R','Rscripts/stats/*.R','seqtk/seqtk']},
+    package_data={'mapdamage': ['Rscripts/*.R','Rscripts/stats/*.R','tests/*','seqtk/seqtk']},
     scripts=['bin/mapDamage'],
     url='https://github.com/ginolhac/mapDamage',
     license='LICENSE.txt',