.github/ISSUE_TEMPLATE/bug_report.md

+---
+name: Bug report
+about: Create a report to help us improve
+title: 'Bug: '
+labels: 'Bug Report'
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior. If possible, could you share the data with us?
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**System (please complete the following information):**
+ - OS: [e.g. Linux, Mac OSX]
+ - Python version [e.g. 3.7.1]
+ - Numpy version [e.g. 1.17.0]
+ - MACS Version [e.g. 2.2.4]
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

+---
+name: Feature request
+about: Suggest an idea for this project
+title: 'Feat: '
+labels: 'Feature Request'
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/ISSUE_TEMPLATE/general_question.md

+---
+name: General question
+about: Ask general questions on usage
+title: 'Q: '
+labels: 'General Question'
+assignees: ''
+
+---
+
+**Use case**
+A clear and concise description of what the use case is. For example: my data is from [...]
+
+**Describe the problem**
+A clear and concise description of what happend.
+
+**Describe the solution you tried**
+A clear and concise description of any solutions you've tried.
+
+**Additional context**
+Add any other context or screenshots about your use case here.

.gitignore

+MANIFEST
+build/*
+dist/*
+*.pyc
+.pydevproject
+.project
+*~
+*.log
+.coverage
+coverage.xml
+\#*\#
+
+test/*.wig
+test/*.bed
+test/log
+test/test*_run_*
+test/subcommandstest_run_*
+test/__pycache__
+test/*.prof
+test/*.tottime
+test/*.calls
+test/*.cumulative
+
+.DS_Store
+.cache
+
+test/NOSPMR.ppois.bdg_ppois.bdg
+test/*_run_*
+
+MACS2.egg-info
+
+.pytest_cache
+.cache

.travis.yml

+language: python
+python:
+  - "3.7"
+  - "3.6"
+  - "3.5"
+install:
+  - pip install --upgrade pip
+  - pip install --upgrade -r requirements.txt
+  - python setup.py install
+script:
+  - pytest --cov-report term --cov=test/
+  - cd test
+  - ./cmdlinetest macs2
+  - cd ..
+deploy:
+  provider: pypi
+  user:
+    secure: lguBZWd+H2u4jqWgq4xFrlDnTniJtGm4eGddGpXGWFrhX4qno2YdJMd5S6L+FVUezrwTXLptyWcKSrJRcddAzR5vvXIX0ibpMThiVjXRrQGtlGkuffAXjiCleIk0QxsOafTElHW4zzGXf3us8ECLcdHwcJdJOjJPUSVLjeWJsTVPJqxySq1eWT9I7E8Wh0yeaZ/WiA3Val5Brc7B6P0ZpDnS5Y0bpSDgWb4uCvbkCJhf74Owj7D2yrbrsfM6H/QZRR2zr97d4w8/mserFQSIdM5GnhNyxbWrtgVUDIoNB4o7X+2og+jfvZUEaZu0xEtST39W2Z89++5NxRXYrn7t3Zsr8+V2+mnspQD1ktdeCJtzrgIcuGtLzVIEVS0vu+358qGXaCnjjxxlGymG8sodsUc0c51GfcPxeqla9aYEgasUATRA43NGFBQNZrusN2Ov+bPKST37lVVRJXEVUtYRVcrJz8AJN1Gdim12luQDHBfhjBq2w4Rl2aBBUDWkL5KuE5TQffdzpIqp/47QKVgxqACUG7CJElNwhHoyHP6GhyMnGHMC4hl4FidqXdq9qJOJtZbKJU/kGYPpclnMa82GU9nYb6JEesItVWAtudlbU7Df4X/JHxUfi0E3VgkrZG6XDHFAkU4LBBUBaeMhP4UVlBMyjv6TNhd0LWBqA17Y1zk=
+  password:
+    secure: NBq2RSTUjhQf4GlxSGg1e9P8vIQrtVskhOc1j6mIYE1zRGEIQFlrujE11+KDrGBoD7KAtWS4HjTFZEctI5nI+YYrZOg0Fu5XgDHD7rRD10qp3/Ipmack3i4pl6JDyyOiIJVmt0+IcN4+fxfIsTrs0XRLSGDw1GaO91V05RpY5GJZ5+crXaUQKsFzvwFgznnqQPK2dY/NsqC63TkYvpiiI4Sn5Eb78XW8kjtEzM84bYFcwMWofib75v8qqRGEabHxUBlXBfWOoDS78T58MdZK+DQ/HJT9XFU0yF1yJ2IhMOHUxnvH2E/nKuLwA3fUBXsPpHFmsbRhgKlCyPUhqjKAhsRyZHqfU6BaF7yJOejeCwbcY6EzQzwQvGr4UQUG6EX49YskKYrwb7IXqC8Y3OjD8maUMdpWU249dPwqDvkwjASa8FlN6LIxJ28f0qd7y1EDFZwM12kS0pdNimxTmxEMYBy76jBxOKDG3Hr349E6Q67trVktbNfp0PdUFVSWqxeJrdujl38w4h07hPKTESugqasPVO/gaY+8+f+xrB7F3TXLDSN+5Dcr9srVX4V2Z72rJtoxrggMbY/n8RUXNC1+45QnRAv/6iLoingtlOwo3p0xA25J7kl7akWqPaP2MwQ+eoPvMgZwrS/S+5Q+7t/rwZn0SkjjSJZIIwB++Ccjbfc=
+  on:
+    tags: true
+  skip_existing: true
+  distributions: sdist
+after_success:
+  - codecov
+  
\ No newline at end of file

ChangeLog

+2019-10-31  Tao Liu  <vladimir.liu@gmail.com>
+	MACS version 2.2.5 (Py3 speed up)
+
+	* Features added
+
+	1) *Github code only and Not included in MACS2 release* New
+	testing data for performance test. An subsampled ENCODE2 CTCF
+	ChIP-seq dataset, including 5million ChIP reads and 5 million
+	control reads, has been included in the test folder for testing
+	CPU and memory usage (i.e. 5M test). Several related scripts ,
+	including `prockreport` for output cpu memory usage, `pyprofile`
+	and `pyprofile_stat` for debuging and profiling MACS2 codes, have
+	been included.
+
+	2) Speed up pvalue-qvalue checkup (pqtable checkup) #335 #338.
+	The old hashtable.pyx implementation copied from Pandas (very old
+	version) doesn't work well in Python3+Cython. It slows down the
+	pqtable checkup using the identical Cython codes as in
+	v2.1.4. While running 5M test, the `__getitem__` function in the
+	hashtable.pyx took 3.5s with 37,382,037 calls in MACS2 v2.1.4, but
+	148.6s with the same number of calls in MACS2 v2.2.4. As a
+	consequence, the standard python dictionary implementation has
+	replaced hashtable.pyx for pqtable checkup. Now MACS2 runs a bit
+	faster than py2 version, but uses a bit more memory. In general,
+	v2.2.5 can finish 5M reads test in 20% less time than MACS2
+	v2.1.4, but use 15% more memory.
+
+	* Bug fixed
+
+	1) More Python3 related fixes, e.g. the return value of keys from
+	py3 dict. #333 #337
+
+
 2019-10-01  Tao Liu  <vladimir.liu@gmail.com>
 	MACS version 2.2.4 (Python3)
 

DOCKER/Dockerfile

+# Use an official Python runtime as a parent image
+FROM python:3.7
+
+# install numpy and MACS2
+RUN git clone https://github.com/taoliu/MACS.git /MACS
+RUN pip install --trusted-host pypi.python.org --upgrade pip && pip install --trusted-host pypi.python.org -r /MACS/requirements.txt
+RUN cd /MACS && python setup.py install
+
+# Define environment variable
+ENTRYPOINT ["macs2"]

DOCKER/Dockerfile-slim

+# Use an official Python runtime as a parent image
+FROM python:3.7 AS build
+
+# install numpy and MACS2
+RUN git clone https://github.com/taoliu/MACS.git /MACS
+RUN pip install --trusted-host pypi.python.org --upgrade pip && pip install --trusted-host pypi.python.org -r /MACS/requirements.txt
+RUN cd /MACS && python setup.py install
+
+FROM python:3.7-slim
+
+# copy compiled files over to python-slim
+COPY --from=build /usr/local/bin/macs2 /usr/local/bin/
+COPY --from=build /usr/local/lib/python3.7/site-packages/ /usr/local/lib/python3.7/site-packages/
+
+# Define environment variable
+ENTRYPOINT ["macs2"]

DOCKER/README.md

+# Official MACS2 v2.2.5 docker
+
+MACS2 is a bioinformatics algorithm to analyze ChIP-seq datasets. 
+
+# Get Started
+
+## Pull the image
+
+Currently, there are two types of MACS2 images, one based on official Python3.7 docker image, and the other based on official Python3.7-slim docker image. Only if space is a concern, and you keep using Python3.7-slim docker for all the Python 3.7 applications, pull the slim one.
+
+To pull the regular MACS2 image:
+
+```docker pull fooliu/macs2```
+
+To pull the MACS2 image based on Python3.7-slim
+
+```docker pull fooliu/macs2:py37-slim```
+
+## Analyze your data
+
+### Example of analyzing ChIP-seq data in the CURRENT directory
+
+Let's assume you pulled the regular MACS2 image. If not, change the image name accordingly. First, ```cd``` to the working directory containing ChIP-seq alignment files such as ```chip-seq-file.bam``` and ```control-seq-file.bam```. Then
+
+```docker run -v $PWD:/data/ fooliu/macs2 callpeak -t /data/chip-seq-file.bam -c /data/control-seq-file.bam -n test-run --outdir /data/```
+
+The first part ```-v $PWD:/data/``` will mount the CURRENT directory ```$PWD``` to ```/data/``` in the container, so please don't forget to add ```/data/``` to the path of input files with ```-t``` and/or ```-c```, and don't forget to set the ```--outdir``` option to ```/data/```. The final outputs will be directly written into the CURRENT directory. Extra MACS2 options can be modified or added after ```docker run -v $PWD:/data/ fooliu/macs2```. The ENTRYPOINT (or the default command when run the container) has been set as ```macs2```.
+
+# Built with
+
+* Python3.7 docker image 3.7
+* pip install numpy version 1.17, cython 0.29, and pytest 4.6 
+* git clone MACS codes from github master branch then run ```setup.py```
+
+# Author
+
+This Docker image is maintained and uploaded by Tao Liu <vladimir.liu@gmail.com>. MACS2 is actively maintained on Github, many users contribute codes and suggestions. 

LICENSE

 BSD 3-Clause License
 
-Copyright (c) 2019, Tao Liu lab at RPCCC and Xiaole Shirley Liu lab at DFCI
-All rights reserved.
+Copyright (c) 2019, Tao Liu lab at Roswell Park Comprehensive Cancer Center and Xiaole Shirley Liu lab at Dana-Farber Cancer Institute, All rights reserved.
 
 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions are met:

MACS2.egg-info/SOURCES.txt

-ChangeLog
-INSTALL.md
-LICENSE
-MANIFEST.in
-README.md
-setup.py
-MACS2/Constants.py
-MACS2/OptValidator.py
-MACS2/OutputWriter.py
-MACS2/PeakDetect.c
-MACS2/PeakDetect.pyx
-MACS2/PeakModel.c
-MACS2/PeakModel.pyx
-MACS2/Pileup.c
-MACS2/Pileup.pyx
-MACS2/Prob.c
-MACS2/Prob.pyx
-MACS2/Signal.c
-MACS2/Signal.pyx
-MACS2/Statistics.c
-MACS2/Statistics.pyx
-MACS2/__init__.py
-MACS2/bdgbroadcall_cmd.py
-MACS2/bdgcmp_cmd.py
-MACS2/bdgdiff_cmd.py
-MACS2/bdgopt_cmd.py
-MACS2/bdgpeakcall_cmd.py
-MACS2/cPosValCalculation.c
-MACS2/cPosValCalculation.h
-MACS2/cPosValCalculation.pxd
-MACS2/cStatistics.c
-MACS2/cStatistics.h
-MACS2/callpeak_cmd.py
-MACS2/cmbreps_cmd.py
-MACS2/diffpeak_cmd.py
-MACS2/filterdup_cmd.py
-MACS2/hashtable.c
-MACS2/hashtable.pyx
-MACS2/khash.h
-MACS2/khash.pxd
-MACS2/pileup_cmd.py
-MACS2/predictd_cmd.py
-MACS2/randsample_cmd.py
-MACS2/refinepeak_cmd.py
-MACS2.egg-info/PKG-INFO
-MACS2.egg-info/SOURCES.txt
-MACS2.egg-info/dependency_links.txt
-MACS2.egg-info/requires.txt
-MACS2.egg-info/top_level.txt
-MACS2/IO/BedGraph.c
-MACS2/IO/BedGraph.pyx
-MACS2/IO/BedGraphIO.c
-MACS2/IO/BedGraphIO.pyx
-MACS2/IO/CallPeakUnit.c
-MACS2/IO/CallPeakUnit.pyx
-MACS2/IO/CallPeakUnitPrecompiled.c
-MACS2/IO/FixWidthTrack.c
-MACS2/IO/FixWidthTrack.pyx
-MACS2/IO/PairedEndTrack.c
-MACS2/IO/PairedEndTrack.pyx
-MACS2/IO/Parser.c
-MACS2/IO/Parser.pyx
-MACS2/IO/PeakIO.c
-MACS2/IO/PeakIO.pyx
-MACS2/IO/ScoreTrack.c
-MACS2/IO/ScoreTrack.pyx
-MACS2/IO/__init__.py
-bin/macs2
\ No newline at end of file

MACS2.egg-info/dependency_links.txt

-

MACS2.egg-info/requires.txt

-numpy>=1.17

MACS2.egg-info/top_level.txt

-MACS2

MACS2/Constants.py

-MACS_VERSION = "2.2.4"
+MACS_VERSION = "2.2.5"
 FILTERDUP_VERSION = "1.0.0 20140616"
 RANDSAMPLE_VERSION = "1.0.0 20120703"
 MAX_PAIRNUM = 1000

MACS2/IO/BedGraph.pyx

 # cython: language_level=3
-# Time-stamp: <2019-10-02 11:05:35 taoliu>
+# cython: profile=True
+# Time-stamp: <2019-10-30 17:48:55 taoliu>
 
 """Module for BedGraph data class.
 
@@ -221,12 +222,11 @@ cdef class bedGraphTrackI:
         else:
             return None
 
-    def get_chr_names (self):
+    cpdef set get_chr_names (self):
         """Return all the chromosome names stored.
         
         """
-        l = set(self.__data.keys())
-        return l
+        return set(sorted(self.__data.keys()))
 
     def write_bedGraph (self, fhd, str name, str description, bool trackline=True):
         """Write all data to fhd in Wiggle Format.
@@ -240,6 +240,7 @@ cdef class bedGraphTrackI:
             int pre, pos, i
             double value
             bytes chrom
+            set chrs
         
         if trackline:
             trackcontents = (name.replace("\"", "\\\""), description.replace("\"", "\\\""))
@@ -278,8 +279,9 @@ cdef class bedGraphTrackI:
             int new_pre_pos, pos, i
             double new_pre_value, value
             bytes chrom
+            set chrs
         
-        chrs = set(self.__data.keys())
+        chrs = self.get_chr_names()
         for chrom in chrs:
             (p,v) = self.__data[chrom]
             pnext = iter(p).__next__
@@ -319,8 +321,9 @@ cdef class bedGraphTrackI:
             int new_pre_pos, pos, i
             double new_pre_value, value
             bytes chrom
+            set chrs
         
-        chrs = set(self.__data.keys())
+        chrs = self.get_chr_names()
         for chrom in chrs:
             (p,v) = self.__data[chrom]
             pnext = iter(p).__next__
@@ -414,6 +417,7 @@ cdef class bedGraphTrackI:
             int peak_length, x, pre_p, p, i, summit, tstart, tend
             double v, summit_value, tvalue
             bytes chrom
+            set chrs
         
         #if call_summits: close_peak = self.__close_peak2
         #else: close_peak = self.__close_peak
@@ -512,6 +516,7 @@ cdef class bedGraphTrackI:
         """
         cdef bytes chrom
         cdef int i, j
+        cdef set chrs
         #cdef int tmp_n
         
         assert lvl1_cutoff > lvl2_cutoff, "level 1 cutoff should be larger than level 2."
@@ -763,7 +768,7 @@ cdef class bedGraphTrackI:
             double t0, t1, t 
 
         # calculate frequencies of each p-score
-        for chrom in self.__data.keys():
+        for chrom in self.get_chr_names():
             pre_p = 0
 
             [pos_array, pscore_array] = self.__data[ chrom ]
@@ -807,7 +812,7 @@ cdef class bedGraphTrackI:
             nhcal += 1
 
         # convert pscore to qscore
-        for chrom in self.__data.keys():
+        for chrom in self.get_chr_names():
             [pos_array, pscore_array] = self.__data[ chrom ]
 
             for i in range( len( pos_array ) ):
@@ -982,7 +987,8 @@ cdef class bedGraphTrackI:
 
     cpdef str cutoff_analysis ( self, int max_gap, int min_length, int steps = 100 ):
         cdef:
-            list chrs, tmplist, peak_content
+            set chrs
+            list tmplist, peak_content
             bytes  chrom
             str ret
             float cutoff
@@ -990,7 +996,7 @@ cdef class bedGraphTrackI:
             dict cutoff_npeaks, cutoff_lpeaks
             float s, midvalue
 
-        chrs = self.__data.keys()
+        chrs = self.get_chr_names()
 
         midvalue = self.minvalue/2 + self.maxvalue/2
         s = float(self.minvalue - midvalue)/steps
@@ -1065,6 +1071,7 @@ def scoreTracktoBedGraph (scoretrack, str colname):
     cdef:
         int pre, i
         bytes chrom
+        set chrs
     
     bdgtrack = bedGraphTrackI( baseline_value = 0 )
     if colname not in ['sample','control','-100logp','-100logq']:

MACS2/IO/BedGraphIO.pyx

 # cython: language_level=3
-# Time-stamp: <2019-10-02 11:05:42 taoliu>
+# cython: profile=True
+# Time-stamp: <2019-10-30 17:49:01 taoliu>
 
 """Module Description:  IO Module for bedGraph file