Commit 3413d1b6 authored by SVN-Git Migration's avatar SVN-Git Migration

Imported Upstream version 0.5.3

parent 86fc4428
------------------------------------------------------------
revno: 485
committer: Per Øyvind Karlsen <peroyvind@mandriva.org>
branch nick: pyliblzma
timestamp: Tue 2010-04-13 23:24:57 +0200
message:
update NEWS for 0.5.3
------------------------------------------------------------
revno: 484
committer: Per Øyvind Karlsen <peroyvind@mandriva.org>
branch nick: pyliblzma
timestamp: Tue 2010-04-13 23:24:44 +0200
message:
update path to headers to include
------------------------------------------------------------
revno: 483
committer: Per Øyvind Karlsen <peroyvind@mandriva.org>
branch nick: pyliblzma
timestamp: Tue 2010-04-13 23:06:47 +0200
message:
bump version
------------------------------------------------------------
revno: 482
committer: Per Øyvind Karlsen <peroyvind@mandriva.org>
branch nick: pyliblzma
timestamp: Tue 2010-04-13 23:05:20 +0200
message:
loop untill there's no data left in LZMA_compress() (fixes #537199)
------------------------------------------------------------
revno: 481
committer: Per Øyvind Karlsen <peroyvind@mandriva.org>
branch nick: pyliblzma
timestamp: Tue 2010-04-13 22:29:41 +0200
message:
check the uncompressed stream with lzma_alone format in test_compress_large_stream
------------------------------------------------------------
revno: 480
committer: Per Øyvind Karlsen <peroyvind@mandriva.org>
branch nick: pyliblzma
timestamp: Tue 2010-04-13 22:26:35 +0200
message:
fix flushing not running all the way till it's done (#521975)
------------------------------------------------------------
revno: 479
committer: Per Øyvind Karlsen <peroyvind@mandriva.org>
branch nick: pyliblzma
timestamp: Tue 2010-04-13 22:23:44 +0200
message:
just use generate_Random for data_large
------------------------------------------------------------
revno: 478
committer: Per Øyvind Karlsen <peroyvind@mandriva.org>
branch nick: pyliblzma
timestamp: Fri 2010-04-09 03:59:35 +0200
message:
get random data for testing compression og large block
------------------------------------------------------------
revno: 477
committer: Per Øyvind Karlsen <peroyvind@localhost.localdomain>
branch nick: pyliblzma
timestamp: Sun 2010-02-21 01:53:50 +0100
message:
make code a bit more consistent with bz2module.c for easier maintenance...
------------------------------------------------------------
revno: 476
committer: Per Øyvind Karlsen <peroyvind@localhost.localdomain>
branch nick: pyliblzma
timestamp: Sun 2010-02-21 00:45:54 +0100
message:
- fix segfault with LZMACompressor (#521975) (and sync layout of the related code with bz2module.c a bit too while at it)
------------------------------------------------------------
revno: 475
committer: Per Øyvind Karlsen <peroyvind@localhost.localdomain>
branch nick: pyliblzma
timestamp: Sun 2010-02-21 00:08:09 +0100
message:
from /python/trunk/Modules/bz2module.c:
Issue #7205: Fix a possible deadlock when using a LZMAFile object from several threads at once.
------------------------------------------------------------
revno: 474
committer: Per Øyvind Karlsen <peroyvind@localhost.localdomain>
branch nick: pyliblzma
timestamp: Sat 2010-02-20 21:57:53 +0100
message:
fix a typo
------------------------------------------------------------
revno: 473
committer: Per Øyvind Karlsen <proyvind@gmail.com>
branch nick: pyliblzma
timestamp: Wed 2010-01-13 11:30:00 +0100
message:
update default options string to be consistent with renamed option names
------------------------------------------------------------
revno: 472
committer: Per Øyvind Karlsen <proyvind@gmail.com>
branch nick: pyliblzma
timestamp: Wed 2010-01-13 09:28:17 +0100
message:
move source and headers into a separate sub-directory
------------------------------------------------------------
revno: 471
committer: Per Øyvind Karlsen <proyvind@gmail.com>
branch nick: pyliblzma
timestamp: Sat 2009-11-28 09:07:27 +0100
message:
move 'module' out of header and make it local to initlzma() which is the only place it's being used...
------------------------------------------------------------
revno: 470
committer: Per Øyvind Karlsen <proyvind@gmail.com>
branch nick: pyliblzma
timestamp: Sat 2009-11-28 09:06:07 +0100
message:
make 'module' extern
------------------------------------------------------------
revno: 469
committer: Per Øyvind Karlsen <proyvind@gmail.com>
branch nick: pyliblzma
timestamp: Sat 2009-07-25 00:40:32 +0200
message:
do actually execute test_main()…
------------------------------------------------------------
revno: 468
committer: Per Øyvind Karlsen <proyvind@gmail.com>
branch nick: pyliblzma
timestamp: Fri 2009-07-24 22:24:12 +0200
message:
don't comment out tests in test_main()
------------------------------------------------------------
revno: 467
committer: Per Øyvind Karlsen <proyvind@gmail.com>
branch nick: pyliblzma
timestamp: Fri 2009-07-24 16:43:49 +0200
message:
remove this one as it were never used…
------------------------------------------------------------
revno: 466
committer: Per Øyvind Karlsen <proyvind@gmail.com>
branch nick: trunk
......
include MANIFEST.in ChangeLog COPYING THANKS README NEWS liblzma*.h python-liblzma.spec setup.cfg
include MANIFEST.in ChangeLog COPYING THANKS README NEWS src/liblzma*.h python-liblzma.spec setup.cfg
recursive-include tests *.py
recursive-include tests/data *
Version 0.5.3 - 13 Apr 2010, by Per Øyvind Karlsen <peroyvind@mandriva.org>
- update compression option names in API documentation.
- fix segfault with LZMACompressor (#521975).
- fix compress() & LZMACompressor.flush() finishing when there's still data left.
(#521975 & #537199)
- Synchronize code with bz2module.c from python upstream:
o Issue #7205: Fix a possible deadlock when using a LZMAFile object from
several threads at once.
Version 0.5.2 - 26 Feb 2009, by Per Øyvind Karlsen <peroyvind@mandriva.org>
- Synchronize code with bz2module.c from python upstream.
- Synchronize code with bz2module.c from python upstream:
o fixes various minor issues and bugs.
o support the context manager protocol (adds __enter__() & __enter__()).
o use Py_buffer.
......
Metadata-Version: 1.0
Name: pyliblzma
Version: 0.5.2
Version: 0.5.3
Summary: Python bindings for liblzma
Home-page: https://launchpad.net/pyliblzma
Author: Per Øyvind Karlsen
......
Metadata-Version: 1.0
Name: pyliblzma
Version: 0.5.2
Version: 0.5.3
Summary: Python bindings for liblzma
Home-page: https://launchpad.net/pyliblzma
Author: Per Øyvind Karlsen
......
......@@ -4,21 +4,7 @@ MANIFEST.in
NEWS
README
THANKS
liblzma.c
liblzma.h
liblzma.py
liblzma_compressobj.c
liblzma_compressobj.h
liblzma_decompressobj.c
liblzma_decompressobj.h
liblzma_file.c
liblzma_file.h
liblzma_fileobj.c
liblzma_fileobj.h
liblzma_options.c
liblzma_options.h
liblzma_util.c
liblzma_util.h
python-liblzma.spec
setup.cfg
setup.py
......@@ -26,8 +12,21 @@ pyliblzma.egg-info/PKG-INFO
pyliblzma.egg-info/SOURCES.txt
pyliblzma.egg-info/dependency_links.txt
pyliblzma.egg-info/top_level.txt
src/liblzma.c
src/liblzma.h
src/liblzma_compressobj.c
src/liblzma_compressobj.h
src/liblzma_decompressobj.c
src/liblzma_decompressobj.h
src/liblzma_file.c
src/liblzma_file.h
src/liblzma_fileobj.c
src/liblzma_fileobj.h
src/liblzma_options.c
src/liblzma_options.h
src/liblzma_util.c
src/liblzma_util.h
tests/__init__.py
tests/regrtest.py
tests/test_liblzma.py
tests/data/teststring.lzma
tests/data/teststring.xz
\ No newline at end of file
......@@ -29,7 +29,7 @@ descr = "Python bindings for liblzma"
long_descr = """PylibLZMA provides a python interface for the liblzma library
to read and write data that has been compressed or can be decompressed
by Lasse Collin's xz / lzma utils."""
version = '0.5.2'
version = '0.5.3'
version_define = [('VERSION', '"%s"' % version)]
# FIXME: Probably some nicer way to do this
......@@ -37,6 +37,8 @@ if 'sdist' in sys.argv:
os.system('bzr log . > ChangeLog')
modules = ['liblzma']
c_files = ['liblzma.c', 'liblzma_compressobj.c', 'liblzma_decompressobj.c', 'liblzma_file.c', 'liblzma_fileobj.c', 'liblzma_options.c', 'liblzma_util.c']
for i in xrange(len(c_files)):
c_files[i] = os.path.join('src', c_files[i])
compile_args = []
warnflags = ['-Wall', '-Wextra', '-pedantic', '-Wswitch-enum', '-Wswitch-default']
......
......@@ -64,22 +64,32 @@ LZMA_compress(__attribute__((unused)) PyObject *self, PyObject *args, PyObject *
else if(filters[0].id == LZMA_FILTER_LZMA1)
{
lzuerror = lzma_alone_encoder(lzus, filters[0].options);
if(!Util_CatchLZMAError(lzuerror, lzus, true))
goto error;
lzus->avail_in = (size_t)datasize;
lzus->next_in = data;
lzus->next_out = (uint8_t *)PyString_AS_STRING(ret);
lzus->avail_out = (size_t)bufsize;
Py_BEGIN_ALLOW_THREADS
lzuerror = lzma_code(lzus, LZMA_FINISH);
Py_END_ALLOW_THREADS
for (;;) {
Py_BEGIN_ALLOW_THREADS
lzuerror = lzma_code(lzus, LZMA_FINISH);
Py_END_ALLOW_THREADS
if (!Util_CatchLZMAError(lzuerror, lzus, true))
goto error;
if(lzuerror == LZMA_STREAM_END)
break; /* no more input data */
if (lzus->avail_out == 0) {
bufsize = Util_NewBufferSize(bufsize);
if (_PyString_Resize(&ret, bufsize) < 0)
goto error;
lzus->next_out = (uint8_t *)PyString_AS_STRING(ret) + lzus->total_out;
lzus->avail_out = (size_t)bufsize - (lzus->next_out - (uint8_t *)PyString_AS_STRING(ret));
}
}
if(!Util_CatchLZMAError(lzuerror, lzus, true))
goto error;
lzma_end(lzus);
if (lzuerror == LZMA_STREAM_END)
_PyString_Resize(&ret, (Py_ssize_t)lzus->total_out);
......@@ -235,7 +245,7 @@ static PyMethodDef lzma_methods[] = {
PyDoc_STRVAR(lzma_module_documentation,
"The python lzma module provides a comprehensive interface for\n\
the lzma compression library. It implements one shot (de)compression\n\
functions, CRC-32 & CRC-64 checksum copmutations, types for sequential\n\
functions, CRC-32 & CRC-64 checksum computations, types for sequential\n\
(de)compression, and advanced options for lzma compression.\n\
");
......@@ -244,7 +254,7 @@ PyMODINIT_FUNC initlzma(void);
PyMODINIT_FUNC
initlzma(void)
{
PyObject *ver, *optionsSingleton;
PyObject *ver, *optionsSingleton, *module;
char verstring[10], major, minor[5], revision[5], s[8];
Py_TYPE(&LZMAComp_Type) = &PyType_Type;
Py_TYPE(&LZMADecomp_Type) = &PyType_Type;
......
......@@ -25,7 +25,12 @@ typedef unsigned long ulong;
#ifdef WITH_THREAD
#include <pythread.h>
#define ACQUIRE_LOCK(obj) PyThread_acquire_lock(obj->lock, 1)
#define ACQUIRE_LOCK(obj) do { \
if (!PyThread_acquire_lock(obj->lock, 0)) { \
Py_BEGIN_ALLOW_THREADS \
PyThread_acquire_lock(obj->lock, 1); \
Py_END_ALLOW_THREADS \
} } while(0)
#define RELEASE_LOCK(obj) PyThread_release_lock(obj->lock)
#else
#define ACQUIRE_LOCK(obj)
......@@ -49,5 +54,4 @@ typedef unsigned long ulong;
#define INITCHECK if (!self->is_initialised) { PyErr_Format(PyExc_RuntimeError, "%s object not initialised!", self->ob_type->tp_name); return NULL; }
PyObject *module;
#endif /* LIBLZMA_H */
......@@ -17,7 +17,7 @@ LZMAComp_compress(LZMACompObject *self, PyObject *args)
Py_buffer pdata;
Py_ssize_t datasize, bufsize = SMALLCHUNK;
uint8_t *data;
uint64_t start_total_out;
uint64_t totalout;
PyObject *ret = NULL;
lzma_stream *lzus = &self->lzus;
lzma_ret lzuerror;
......@@ -38,28 +38,31 @@ LZMAComp_compress(LZMACompObject *self, PyObject *args)
if (!(ret = PyString_FromStringAndSize(NULL, bufsize)))
goto error;
start_total_out = lzus->total_out;
lzus->avail_in = (size_t)datasize;
lzus->next_in = data;
lzus->avail_out = (size_t)bufsize;
lzus->next_out = (uint8_t *)PyString_AS_STRING(ret);
totalout = lzus->total_out;
for (;;) {
Py_BEGIN_ALLOW_THREADS
lzuerror = lzma_code(lzus, LZMA_RUN);
Py_END_ALLOW_THREADS
if (lzus->avail_in == 0 || lzus->avail_out != 0)
break;
if (_PyString_Resize(&ret, bufsize << 1) < 0)
goto error;
lzus->next_out = (uint8_t *)PyString_AS_STRING(ret) + datasize;
lzus->avail_out = (size_t)bufsize;
bufsize = bufsize << 1;
if(!Util_CatchLZMAError(lzuerror, lzus, true))
if (!Util_CatchLZMAError(lzuerror, lzus, true))
goto error;
if (lzus->avail_in == 0)
break; /* no more input data */
if (lzus->avail_out == 0) {
bufsize = Util_NewBufferSize(bufsize);
if (_PyString_Resize(&ret, bufsize) < 0)
goto error;
lzus->next_out = (uint8_t *)PyString_AS_STRING(ret) + (lzus->total_out - totalout);
lzus->avail_out = (size_t)bufsize - (lzus->next_out - (uint8_t *)PyString_AS_STRING(ret));
}
}
_PyString_Resize(&ret, (Py_ssize_t)lzus->total_out - (Py_ssize_t)start_total_out);
_PyString_Resize(&ret, (Py_ssize_t)lzus->total_out - (Py_ssize_t)totalout);
RELEASE_LOCK(self);
PyBuffer_Release(&pdata);
......@@ -88,7 +91,7 @@ LZMAComp_flush(LZMACompObject *self, PyObject *args)
Py_ssize_t bufsize = SMALLCHUNK;
PyObject *ret = NULL;
lzma_action flushmode = LZMA_FINISH;
uint64_t start_total_out;
uint64_t totalout;
lzma_stream *lzus = &self->lzus;
lzma_ret lzuerror;
......@@ -126,27 +129,30 @@ LZMAComp_flush(LZMACompObject *self, PyObject *args)
if (!(ret = PyString_FromStringAndSize(NULL, bufsize)))
goto error;
start_total_out = lzus->total_out;
lzus->avail_in = 0;
lzus->avail_out = (size_t)bufsize;
lzus->next_out = (uint8_t *)PyString_AS_STRING(ret);
totalout = lzus->total_out;
for (;;) {
Py_BEGIN_ALLOW_THREADS
lzuerror = lzma_code(lzus, flushmode);
Py_END_ALLOW_THREADS
if (lzus->avail_in == 0 || lzus->avail_out != 0)
break;
if (_PyString_Resize(&ret, bufsize << 1) < 0)
goto error;
lzus->next_out = (uint8_t *)PyString_AS_STRING(ret) + bufsize;
lzus->avail_out = (size_t)bufsize;
bufsize = bufsize << 1;
if(!Util_CatchLZMAError(lzuerror, lzus, true))
if (!Util_CatchLZMAError(lzuerror, lzus, true))
goto error;
if(lzuerror == LZMA_STREAM_END)
break; /* no more input data */
if (lzus->avail_out == 0) {
bufsize = Util_NewBufferSize(bufsize);
if (_PyString_Resize(&ret, bufsize) < 0)
goto error;
lzus->next_out = (uint8_t *)PyString_AS_STRING(ret) + (lzus->total_out - totalout);;
lzus->avail_out = (size_t)bufsize - (lzus->next_out - (uint8_t *)PyString_AS_STRING(ret));
}
}
_PyString_Resize(&ret, (Py_ssize_t)self->lzus.total_out - (Py_ssize_t)start_total_out);
_PyString_Resize(&ret, (Py_ssize_t)lzus->total_out - (Py_ssize_t)totalout);
RELEASE_LOCK(self);
return ret;
......
......@@ -47,6 +47,8 @@ bool init_lzma_options(const char *funcName, PyObject *kwargs, lzma_filter *filt
PyObject *LZMA_options_get(lzma_filter filter);
#define DEFAULT_OPTIONS_STRING "options={'format':'xz', 'check':'crc32', 'level':6, 'extreme':False,\n\
'dictionary_size':23, 'literal_contextbits':3 'literal_pos_bits':0, 'pos_bits':2, 'mode':2,\n\
'fast_bytes':128, 'match_finder':'bt4', 'match_finder_cycles':0"
'dict_size':23, 'lc':3 'lp':0, 'pb':2, 'mode':2,\n\
'nice_len':128, 'mf':'bt4', 'depth':0"
#endif /* LIBLZMA_OPTIONS_H */
This diff is collapsed.
......@@ -55,8 +55,7 @@ class TestLZMA(unittest.TestCase):
self.compressed_stream_xz = unhexlify('fd377a585a0000016922de360200210116000000742fe5a3e00044003f5d0024194a4202f3d35297250824d0d20f4082855bf62811d85c5b316e30b927eafc568162d33f18013144b99ae29e2067e09d60c69fac0b5caaf44a0a0718bc0000008617931700015745a7cbcad39042990d010000000001595a')
self.compressed_alone = unhexlify('5d00008000ffffffffffffffff0024194a4202f3d35297250824d0d20f4082855bf62811d85c5b316e30b927eafc568162d33f18013144b99ae29e2067e09d60c69fac0b5caaf44a0a0fbe1563d7fb115800')
self.compressed_stream_alone = unhexlify('5d00008000ffffffffffffffff0024194a4202f3d35297250824d0d20f4082855bf62811d85c5b316e30b927eafc568162d33f18013144b99ae29e2067e09d60c69fac0b5caaf44a0a0fbe1563d7fb115800')
self.data_large = "kosogklem"*(1<<18)
self.data_large = generate_random(1<<20)
def test_compression(self):
compressed = lzma.compress(self.plain, options={'format':'xz'})
......@@ -249,7 +248,7 @@ class TestLZMA(unittest.TestCase):
if not tmp: break
outfile.write(compress.compress(tmp))
outfile.write(compress.flush())
self.failUnless(lzma.compress(self.data_large, options={'format':'alone'}) == outfile.getvalue())
self.failUnless(self.data_large == lzma.decompress(outfile.getvalue()))
compress.reset(options={'format':'xz'})
infile = StringIO(self.data_large)
outfile = StringIO()
......@@ -647,11 +646,9 @@ class ChecksumTestCase(unittest.TestCase):
def test_main():
from test import test_support
test_support.run_unittest(TestLZMA)
"""
test_support.run_unittest(TestLZMAOptions)
test_support.run_unittest(TestLZMAFile)
test_support.run_unittest(ChecksumTestCase)
"""
if __name__ == "__main__":
unittest.main()
test_main()
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment