.editorconfig

+[*.py]
+charset=utf-8
+end_of_line=lf
+insert_final_newline=true
+indent_style=space
+indent_size=4

.travis.yml

-sudo: false
 language: python
+
+dist: xenial
+
 cache:
   directories:
     - $HOME/.cache/pip
+
 python:
   - "2.7"
-  - "3.3"
   - "3.4"
   - "3.5"
   - "3.6"
+  - "3.7"
 
 install:
   - pip install .
 
 script:
-  - nosetests -P tests
+  - python setup.py --version  # Detect encoding problems
+  - python -m pytest
+
+env:
+  global:
+    - TWINE_USERNAME=marcelm
+
+jobs:
+  include:
+    - stage: deploy
+      services:
+        - docker
+      python: "3.6"
+      install: python3 -m pip install twine 'requests-toolbelt!=0.9.0'
+      if: tag IS present
+      script:
+        - |
+          python3 setup.py sdist
+          ls -l dist/
+          python3 -m twine upload dist/*

LICENSE

-Copyright (c) 2010-2016 Marcel Martin <mail@marcelm.net>
+Copyright (c) 2010-2019 Marcel Martin <mail@marcelm.net>
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

PKG-INFO

+Metadata-Version: 2.1
+Name: xopen
+Version: 0.5.0
+Summary: Open compressed files transparently
+Home-page: https://github.com/marcelm/xopen/
+Author: Marcel Martin
+Author-email: mail@marcelm.net
+License: MIT
+Description: .. image:: https://travis-ci.org/marcelm/xopen.svg?branch=master
+            :target: https://travis-ci.org/marcelm/xopen
+        
+        .. image:: https://img.shields.io/pypi/v/xopen.svg?branch=master
+            :target: https://pypi.python.org/pypi/xopen
+        
+        =====
+        xopen
+        =====
+        
+        This small Python module provides an ``xopen`` function that works like the
+        built-in ``open`` function, but can also deal with compressed files.
+        Supported compression formats are gzip, bzip2 and xz. They are automatically
+        recognized by their file extensions `.gz`, `.bz2` or `.xz`.
+        
+        The focus is on being as efficient as possible on all supported Python versions.
+        For example, simply using ``gzip.open`` is very slow in older Pythons, and
+        it is a lot faster to use a ``gzip`` subprocess. For writing to gzip files,
+        ``xopen`` uses ``pigz`` when available.
+        
+        This module has originally been developed as part of the `cutadapt
+        tool <https://cutadapt.readthedocs.io/>`_ that is used in bioinformatics to
+        manipulate sequencing data. It has been in successful use within that software
+        for a few years.
+        
+        ``xopen`` is compatible with Python versions 2.7 and 3.4 to 3.7.
+        
+        
+        Usage
+        -----
+        
+        Open a file for reading::
+        
+            from xopen import xopen
+        
+            with xopen('file.txt.xz') as f:
+                content = f.read()
+        
+        Or without context manager::
+        
+            from xopen import xopen
+        
+            f = xopen('file.txt.xz')
+            content = f.read()
+            f.close()
+        
+        Open a file for writing::
+        
+            from xopen import xopen
+        
+            with xopen('file.txt.gz', mode='w') as f:
+                f.write('Hello')
+        
+        
+        Credits
+        -------
+        
+        The name ``xopen`` was taken from the C function of the same name in the
+        `utils.h file which is part of BWA <https://github.com/lh3/bwa/blob/83662032a2192d5712996f36069ab02db82acf67/utils.h>`_.
+        
+        Kyle Beauchamp <https://github.com/kyleabeauchamp/> has contributed support for appending to files.
+        
+        Some ideas were taken from the `canopener project <https://github.com/selassid/canopener>`_.
+        If you also want to open S3 files, you may want to use that module instead.
+        
+        
+        Changes
+        -------
+        
+        v0.5.0
+        ~~~~~~
+        * By default, pigz is now only allowed to use at most four threads. This hopefully reduces
+          problems some users had with too many threads when opening many files at the same time.
+        * xopen now accepts pathlib.Path objects.
+        
+        
+        Author
+        ------
+        
+        Marcel Martin <mail@marcelm.net> (`@marcelm_ on Twitter <https://twitter.com/marcelm_>`_)
+        
+        Links
+        -----
+        
+        * `Source code <https://github.com/marcelm/xopen/>`_
+        * `Report an issue <https://github.com/marcelm/xopen/issues>`_
+        * `Project page on PyPI (Python package index) <https://pypi.python.org/pypi/xopen/>`_
+        
+Platform: UNKNOWN
+Classifier: Development Status :: 4 - Beta
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 2.7
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.4
+Classifier: Programming Language :: Python :: 3.5
+Classifier: Programming Language :: Python :: 3.6
+Classifier: Programming Language :: Python :: 3.7
+Requires-Python: >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4
+Provides-Extra: dev

README.rst

@@ -23,7 +23,7 @@ tool <https://cutadapt.readthedocs.io/>`_ that is used in bioinformatics to
 manipulate sequencing data. It has been in successful use within that software
 for a few years.
 
-``xopen`` is compatible with Python 2.7, 3.3, 3.4, 3.5 and 3.6.
+``xopen`` is compatible with Python versions 2.7 and 3.4 to 3.7.
 
 
 Usage
@@ -64,6 +64,16 @@ Some ideas were taken from the `canopener project <https://github.com/selassid/c
 If you also want to open S3 files, you may want to use that module instead.
 
 
+Changes
+-------
+
+v0.5.0
+~~~~~~
+* By default, pigz is now only allowed to use at most four threads. This hopefully reduces
+  problems some users had with too many threads when opening many files at the same time.
+* xopen now accepts pathlib.Path objects.
+
+
 Author
 ------
 

debian/changelog

+python-xopen (0.5.0-1) unstable; urgency=medium
+
+  * New upstream version
+  * debhelper 12
+  * Standards-Version: 4.3.0
+  * Drop ancient X-Python-Version field
+  * Point watch file to pypi since download from Github does not lead to
+    a working source tarball
+  * Build-Depends: python-setuptools-scm
+  * Upstream switched to pytest
+
+ -- Andreas Tille <tille@debian.org>  Wed, 30 Jan 2019 15:47:06 +0100
+
 python-xopen (0.3.3-1) unstable; urgency=medium
 
   * New upstream version

debian/compat

-11
+12

debian/control

@@ -4,22 +4,24 @@ Uploaders: Andreas Tille <tille@debian.org>
 Section: python
 Testsuite: autopkgtest-pkg-python
 Priority: optional
-Build-Depends: debhelper (>= 11~),
+Build-Depends: debhelper (>= 12~),
                dh-python,
                python,
                python-setuptools,
                python-nose,
                python-bz2file,
+               python-setuptools-scm,
+               python-pytest,
                python3,
                python3-setuptools,
                python3-nose,
-               python3-bz2file
-Standards-Version: 4.1.4
+               python3-bz2file,
+               python3-setuptools-scm,
+               python3-pytest
+Standards-Version: 4.3.0
 Vcs-Browser: https://salsa.debian.org/med-team/python-xopen
 Vcs-Git: https://salsa.debian.org/med-team/python-xopen.git
 Homepage: https://github.com/marcelm/xopen
-X-Python-Version: >= 2.6
-X-Python3-Version: >= 3.2
 
 Package: python-xopen
 Architecture: all

debian/rules

@@ -3,18 +3,5 @@
 DH_VERBOSE := 1
 export PYBUILD_NAME=xopen
 
-PYTHON2 := $(shell pyversions -r)
-PYTHON3 := $(shell py3versions -r)
-
 %:
 	dh $@ --with python2,python3 --buildsystem=pybuild
-
-override_dh_auto_test:
-ifeq ($(filter nocheck,$(DEB_BUILD_OPTIONS)),)
-	for py in $(PYTHON2) ; do\
-	    $$py setup.py nosetests -v ; \
-	done
-	for py in $(PYTHON3) ; do\
-	    $$py setup.py nosetests -v ; \
-	done
-endif

debian/watch

 version=4
 
-https://github.com/marcelm/xopen/releases .*/archive/v(\d[\d.-]+)\.(?:tar(?:\.gz|\.bz2)?|tgz)
+https://pypi.python.org/simple/xopen .*/xopen-(\d[\d.]+)@ARCHIVE_EXT@#sha256=.*
+
+# This leads to tarball which does not work
+# https://github.com/marcelm/xopen/releases .*/archive/v(\d[\d.-]+)\.(?:tar(?:\.gz|\.bz2)?|tgz)

pyproject.toml

+[build-system]
+requires = ["setuptools", "wheel", "setuptools_scm"]

setup.cfg

+[bdist_wheel]
+universal = 1
+
+[egg_info]
+tag_build = 
+tag_date = 0
+

setup.py

@@ -8,14 +8,10 @@ if sys.version_info < (2, 7):
 with open('README.rst') as f:
     long_description = f.read()
 
-if sys.version_info < (3, ):
-	requires = ['bz2file']
-else:
-	requires = []
-
 setup(
     name='xopen',
-	version='0.3.3',
+    use_scm_version=True,
+    setup_requires=['setuptools_scm'],  # Support pip versions that don't know about pyproject.toml
     author='Marcel Martin',
     author_email='mail@marcelm.net',
     url='https://github.com/marcelm/xopen/',
@@ -23,15 +19,21 @@ setup(
     long_description=long_description,
     license='MIT',
     py_modules=['xopen'],
-	install_requires=requires,
+    install_requires=[
+        'bz2file; python_version=="2.7"',
+    ],
+    extras_require={
+        'dev': ['pytest'],
+    },
+    python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4',
     classifiers=[
         "Development Status :: 4 - Beta",
         "License :: OSI Approved :: MIT License",
         "Programming Language :: Python :: 2.7",
         "Programming Language :: Python :: 3",
-		"Programming Language :: Python :: 3.3",
         "Programming Language :: Python :: 3.4",
         "Programming Language :: Python :: 3.5",
         "Programming Language :: Python :: 3.6",
+        "Programming Language :: Python :: 3.7",
     ]
 )

tests/test_xopen.py

 # coding: utf-8
 from __future__ import print_function, division, absolute_import
-import gzip
+
 import os
 import random
 import sys
 import signal
 from contextlib import contextmanager
-from nose.tools import raises
+import pytest
 from xopen import xopen, PipedGzipReader
 
 
-base = "tests/file.txt"
-files = [ base + ext for ext in ['', '.gz', '.bz2' ] ]
+extensions = ["", ".gz", ".bz2"]
+
 try:
     import lzma
-	files.append(base + '.xz')
+    extensions.append(".xz")
 except ImportError:
     lzma = None
 
-try:
-	import bz2
-except ImportError:
-	bz2 = None
+base = "tests/file.txt"
+files = [base + ext for ext in extensions]
+CONTENT = 'Testing, testing ...\nThe second line.\n'
 
-major, minor = sys.version_info[0:2]
+# File extensions for which appending is supported
+append_extensions = extensions[:]
+if sys.version_info[0] == 2:
+    append_extensions.remove(".bz2")
 
 
 @contextmanager
@@ -36,24 +38,24 @@ def temporary_path(name):
     os.remove(path)
 
 
-def test_xopen_text():
-	for name in files:
+@pytest.mark.parametrize("name", files)
+def test_xopen_text(name):
     with xopen(name, 'rt') as f:
         lines = list(f)
         assert len(lines) == 2
         assert lines[1] == 'The second line.\n', name
 
 
-def test_xopen_binary():
-	for name in files:
+@pytest.mark.parametrize("name", files)
+def test_xopen_binary(name):
     with xopen(name, 'rb') as f:
         lines = list(f)
         assert len(lines) == 2
         assert lines[1] == b'The second line.\n', name
 
 
-def test_no_context_manager_text():
-	for name in files:
+@pytest.mark.parametrize("name", files)
+def test_no_context_manager_text(name):
     f = xopen(name, 'rt')
     lines = list(f)
     assert len(lines) == 2
@@ -62,8 +64,8 @@ def test_no_context_manager_text():
     assert f.closed
 
 
-def test_no_context_manager_binary():
-	for name in files:
+@pytest.mark.parametrize("name", files)
+def test_no_context_manager_binary(name):
     f = xopen(name, 'rb')
     lines = list(f)
     assert len(lines) == 2
@@ -72,65 +74,22 @@ def test_no_context_manager_binary():
     assert f.closed
 
 
-@raises(IOError)
-def test_nonexisting_file():
-	with xopen('this-file-does-not-exist') as f:
-		pass
-
-
-@raises(IOError)
-def test_nonexisting_file_gz():
-	with xopen('this-file-does-not-exist.gz') as f:
-		pass
-
-
-@raises(IOError)
-def test_nonexisting_file_bz2():
-	with xopen('this-file-does-not-exist.bz2') as f:
-		pass
-
-
-if lzma:
-	@raises(IOError)
-	def test_nonexisting_file_xz():
-		with xopen('this-file-does-not-exist.xz') as f:
-			pass
-
-
-@raises(IOError)
-def test_write_to_nonexisting_dir():
-	with xopen('this/path/does/not/exist/file.txt', 'w') as f:
-		pass
-
-
-@raises(IOError)
-def test_write_to_nonexisting_dir_gz():
-	with xopen('this/path/does/not/exist/file.gz', 'w') as f:
-		pass
-
-
-@raises(IOError)
-def test_write_to_nonexisting_dir_bz2():
-	with xopen('this/path/does/not/exist/file.bz2', 'w') as f:
+@pytest.mark.parametrize("ext", extensions)
+def test_nonexisting_file(ext):
+    with pytest.raises(IOError):
+        with xopen('this-file-does-not-exist' + ext) as f:
             pass
 
 
-if lzma:
-	@raises(IOError)
-	def test_write_to_nonexisting_dir():
-		with xopen('this/path/does/not/exist/file.xz', 'w') as f:
+@pytest.mark.parametrize("ext", extensions)
+def test_write_to_nonexisting_dir(ext):
+    with pytest.raises(IOError):
+        with xopen('this/path/does/not/exist/file.txt' + ext, 'w') as f:
             pass
 
 
-def test_append():
-	cases = ["", ".gz"]
-	if bz2 and sys.version_info > (3,):
-		# BZ2 does NOT support append in Py 2.
-		cases.append(".bz2")
-	if lzma:
-		cases.append(".xz")
-	for ext in cases:
-		# On Py3, need to send BYTES, not unicode. Let's do it for all.
+@pytest.mark.parametrize("ext", append_extensions)
+def test_append(ext):
     text = "AB".encode("utf-8")
     reference = text + text
     with temporary_path('truncated.fastq' + ext) as path:
@@ -152,14 +111,8 @@ def test_append():
             assert appended == reference
 
 
-def test_append_text():
-	cases = ["", ".gz"]
-	if bz2 and sys.version_info > (3,):
-		# BZ2 does NOT support append in Py 2.
-		cases.append(".bz2")
-	if lzma:
-		cases.append(".xz")
-	for ext in cases:  # BZ2 does NOT support append
+@pytest.mark.parametrize("ext", append_extensions)
+def test_append_text(ext):
     text = "AB"
     reference = text + text
     with temporary_path('truncated.fastq' + ext) as path:
@@ -210,21 +163,21 @@ class timeout:
 
 
 if sys.version_info[:2] != (3, 3):
-	@raises(EOFError, IOError)
     def test_truncated_gz():
         with temporary_path('truncated.gz') as path:
             create_truncated_file(path)
             with timeout(seconds=2):
+                with pytest.raises((EOFError, IOError)):
                     f = xopen(path, 'r')
                     f.read()
                     f.close()
 
 
-	@raises(EOFError, IOError)
     def test_truncated_gz_iter():
         with temporary_path('truncated.gz') as path:
             create_truncated_file(path)
             with timeout(seconds=2):
+                with pytest.raises((EOFError, IOError)):
                     f = xopen(path, 'r')
                     for line in f:
                         pass
@@ -239,3 +192,60 @@ def test_bare_read_from_gz():
 def test_read_piped_gzip():
     with PipedGzipReader('tests/hello.gz', 'rt') as f:
         assert f.read() == 'hello'
+
+
+def test_write_pigz_threads(tmpdir):
+    path = str(tmpdir.join('out.gz'))
+    with xopen(path, mode='w', threads=3) as f:
+        f.write('hello')
+    with xopen(path) as f:
+        assert f.read() == 'hello'
+
+
+def test_write_stdout():
+    f = xopen('-', mode='w')
+    print("Hello", file=f)
+    f.close()
+    # ensure stdout is not closed
+    print("Still there?")
+
+
+def test_write_stdout_contextmanager():
+    # Do not close stdout
+    with xopen('-', mode='w') as f:
+        print("Hello", file=f)
+    # ensure stdout is not closed
+    print("Still there?")
+
+
+if sys.version_info[:2] >= (3, 4):
+    # pathlib was added in Python 3.4
+    from pathlib import Path
+
+    @pytest.mark.parametrize("file", files)
+    def test_read_pathlib(file):
+        path = Path(file)
+        with xopen(path, mode='rt') as f:
+            assert f.read() == CONTENT
+
+    @pytest.mark.parametrize("file", files)
+    def test_read_pathlib_binary(file):
+        path = Path(file)
+        with xopen(path, mode='rb') as f:
+            assert f.read() == bytes(CONTENT, 'ascii')
+
+    @pytest.mark.parametrize("ext", extensions)
+    def test_write_pathlib(ext, tmpdir):
+        path = Path(str(tmpdir)) / ('hello.txt' + ext)
+        with xopen(path, mode='wt') as f:
+            f.write('hello')
+        with xopen(path, mode='rt') as f:
+            assert f.read() == 'hello'
+
+    @pytest.mark.parametrize("ext", extensions)
+    def test_write_pathlib_binary(ext, tmpdir):
+        path = Path(str(tmpdir)) / ('hello.txt' + ext)
+        with xopen(path, mode='wb') as f:
+            f.write(b'hello')
+        with xopen(path, mode='rb') as f:
+            assert f.read() == b'hello'

tox.ini

 [tox]
-envlist = py27,py33,py34,py35,py36
+envlist = py27,py34,py35,py36,py37
 
 [testenv]
-deps = nose
-commands = nosetests -P tests
+deps = pytest
+commands = pytest

xopen.egg-info/PKG-INFO

+Metadata-Version: 2.1
+Name: xopen
+Version: 0.5.0
+Summary: Open compressed files transparently
+Home-page: https://github.com/marcelm/xopen/
+Author: Marcel Martin
+Author-email: mail@marcelm.net
+License: MIT
+Description: .. image:: https://travis-ci.org/marcelm/xopen.svg?branch=master
+            :target: https://travis-ci.org/marcelm/xopen
+        
+        .. image:: https://img.shields.io/pypi/v/xopen.svg?branch=master
+            :target: https://pypi.python.org/pypi/xopen
+        
+        =====
+        xopen
+        =====
+        
+        This small Python module provides an ``xopen`` function that works like the
+        built-in ``open`` function, but can also deal with compressed files.
+        Supported compression formats are gzip, bzip2 and xz. They are automatically
+        recognized by their file extensions `.gz`, `.bz2` or `.xz`.
+        
+        The focus is on being as efficient as possible on all supported Python versions.
+        For example, simply using ``gzip.open`` is very slow in older Pythons, and
+        it is a lot faster to use a ``gzip`` subprocess. For writing to gzip files,
+        ``xopen`` uses ``pigz`` when available.
+        
+        This module has originally been developed as part of the `cutadapt
+        tool <https://cutadapt.readthedocs.io/>`_ that is used in bioinformatics to
+        manipulate sequencing data. It has been in successful use within that software
+        for a few years.
+        
+        ``xopen`` is compatible with Python versions 2.7 and 3.4 to 3.7.
+        
+        
+        Usage
+        -----
+        
+        Open a file for reading::
+        
+            from xopen import xopen
+        
+            with xopen('file.txt.xz') as f:
+                content = f.read()
+        
+        Or without context manager::
+        
+            from xopen import xopen
+        
+            f = xopen('file.txt.xz')
+            content = f.read()
+            f.close()
+        
+        Open a file for writing::
+        
+            from xopen import xopen
+        
+            with xopen('file.txt.gz', mode='w') as f:
+                f.write('Hello')
+        
+        
+        Credits
+        -------
+        
+        The name ``xopen`` was taken from the C function of the same name in the
+        `utils.h file which is part of BWA <https://github.com/lh3/bwa/blob/83662032a2192d5712996f36069ab02db82acf67/utils.h>`_.
+        
+        Kyle Beauchamp <https://github.com/kyleabeauchamp/> has contributed support for appending to files.
+        
+        Some ideas were taken from the `canopener project <https://github.com/selassid/canopener>`_.
+        If you also want to open S3 files, you may want to use that module instead.
+        
+        
+        Changes
+        -------
+        
+        v0.5.0
+        ~~~~~~
+        * By default, pigz is now only allowed to use at most four threads. This hopefully reduces
+          problems some users had with too many threads when opening many files at the same time.
+        * xopen now accepts pathlib.Path objects.
+        
+        
+        Author
+        ------
+        
+        Marcel Martin <mail@marcelm.net> (`@marcelm_ on Twitter <https://twitter.com/marcelm_>`_)
+        
+        Links
+        -----
+        
+        * `Source code <https://github.com/marcelm/xopen/>`_
+        * `Report an issue <https://github.com/marcelm/xopen/issues>`_
+        * `Project page on PyPI (Python package index) <https://pypi.python.org/pypi/xopen/>`_
+        
+Platform: UNKNOWN
+Classifier: Development Status :: 4 - Beta
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 2.7
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.4
+Classifier: Programming Language :: Python :: 3.5
+Classifier: Programming Language :: Python :: 3.6
+Classifier: Programming Language :: Python :: 3.7
+Requires-Python: >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4
+Provides-Extra: dev

xopen.egg-info/SOURCES.txt

+.editorconfig
+.gitignore
+.travis.yml
+LICENSE
+README.rst
+pyproject.toml
+setup.cfg
+setup.py
+tox.ini
+xopen.py
+tests/file.txt
+tests/file.txt.bz2
+tests/file.txt.gz
+tests/file.txt.xz
+tests/hello.gz
+tests/test_xopen.py
+xopen.egg-info/PKG-INFO
+xopen.egg-info/SOURCES.txt
+xopen.egg-info/dependency_links.txt
+xopen.egg-info/requires.txt
+xopen.egg-info/top_level.txt
\ No newline at end of file

xopen.egg-info/dependency_links.txt

+

xopen.egg-info/requires.txt

+
+[:python_version == "2.7"]
+bz2file
+
+[dev]
+pytest

xopen.egg-info/top_level.txt

+xopen