Commit a42b8e07 authored by SVN-Git Migration's avatar SVN-Git Migration

Imported Upstream version 0.3+debian

parents
$Id: CHANGES,v 1.2 2004/02/18 23:05:05 cvs Exp $
version: 0.1
- Initial release.
version: 0.2
- Included patch from Michael Muller <mmuller@enduden.com> that fixes some
possible buffer overflows, and some yEnc incompatibilities.
- A few enanchments in C code, mainly cosmetic (it should be a little faster
in encoding).
- Files opened in O_RDWR mode are now handled properly.
- yenc.i interface adapted to latest SWIG releases.
- Passing a file opened in a wrong mode now raises a ValueError (rather than
an AssertionError).
- encode()/decode() now accepts filenames and stdin/stdout as arguments.
- Reaching EOF while encoding/decoding no more raises an exception (it was
redundant since the number of encoded/decoded bytes were already returned),
this could break some implementations.
- Passing 0 to the encode o decode functions causes file to be encoded/decoded
up to EOF (this is made possible by the previous change).
version: 0.3
- Encoded lines now are correctly terminated with CRLF.
- Encoded lines size now is 128 chars.
- using fread() instead of fgets in decode() function (much faster).
- SWIG isn't used anymore for wrapper generation.
This diff is collapsed.
CHANGES
COPYING
MANIFEST
MANIFEST.in
README
TODO
setup.cfg
setup.py
debian/changelog
debian/control
debian/rules
debian/DEBIAN/control
doc/yenc-draft.1.3.txt
examples/logo.ync
examples/ydecode.py
examples/ydecode_Decoder.py
examples/yencode.py
examples/yencode_Encoder.py
lib/yenc.py
src/_yenc.c
src/_yenc.h
test/test.py
include *.py MANIFEST* README TODO COPYING CHANGES
recursive-include src *.h *.c
recursive-include test *.py
recursive-include examples *
recursive-include doc *txt *html
recursive-include debian *
Metadata-Version: 1.0
Name: yenc
Version: 0.3
Summary: yEnc Module for Python
Home-page: http://golug.cc.uniud.it/yenc.html
Author: Alessandro Duca
Author-email: dual@golug.cc.uniud.it
License: GPL
Description: UNKNOWN
Platform: UNKNOWN
$Id: README,v 1.2 2004/02/18 23:05:05 cvs Exp $
Description:
-----------
This a fairly simple module, it provide only raw yEnc encoding/decoding with
builitin crc32 calculation.
Header parsing, checkings and yenc formatting are left to you (see examples
directory for possible implementations). The interface was originally intended
to be similar to the uu module from Python standard library.
Version 0.2 changed a bit the previous (0.1) behaviour, but it should be
more usable and flexible (now you can encode/decode from and to standard
input and output and use filenames instead of file objects as arguments
for encode() and decode() ). See CHANGES file for details.
Version 0.3 doesn't introduce anything new, just a bugfix a some internals
changes.
Requirements:
------------
A C developement environment, Python>=2.1 and python developement libs (look for
python-dev or something like that if you're using .rpm or .deb packages, if you
installed from sources you already have everything you need).
The module is known to work with Python2.1 and 2.2, maybe it will work also
with 1.5.2 (as long as you have distutils I think).
Installation:
------------
To install:
tar xzfv yenc-0.3.tar.gz
cd yenc-0.3
python setup.py build
su
python setup.py install
To uninstall:
Simply remove _yenc.so, yenc.py and yenc.pyc from your PYTHONPATH.
On my Debian GNU/Linux 3.0:
/usr/local/lib/python2.2/site-packages/{_yenc.so,yenc.py,yenc.pyc}
Usage:
-----
As usual:
import yenc
in your modules. The 'yenc' module defines 2 functions (encode() and decode())
and an error class, yenc.Error(Exception).
encode(in_file, out_file, bytes=0):
Accepts both filenames or file objects as arguments, if "in_file" is a file
type object it must be opened for reading ("r","rw"...), if "out_file" is a
file type object it must be opened for writing ("w","rw"..), when files are
specified as filenames the files are (if possible) automatically opened in the
correct mode.
The "bytes" argument is an optional numeric value, if set to 0 or omitted it
causes the input file to be read and encoded until EOF, otherwise it specifies
the maximum number of bytes to read and encode.
When reading from stdin "bytes" argument can't be 0 (or omitted).
If arguments don't match such criteria an exception is raised. encode() reads
data from "in_file" and writes the encoded data on "out_file", it returns a
tuple containing the number of encoded bytes and a crc32 sum of the original
data.
Specifing "-" as "in_file" or "out_file" arguments causes the input/output to
be read/written on standard input/output.
decode(in_file, out_file, size=0, crc_in=""):
Same as encode (of course it does the inverse job). Exceptions are raised when
output can't be written or calculated crc doesn't match the optional crc_in
argument (useful for writing decoding tools).
CRC32 sums are always represented as lowercase strings, whithout any
preceeding simbol ( like '0x').
Performances:
------------
Fast enough.
Author:
------
Alessandro Duca <dual@golug.cc.uniud.it>
Thanks:
------
Michael Muller <mmuller@endunden.com> for code reviewing and fixing.
Greets, Sandro.
$Id: TODO,v 1.2 2004/02/18 23:05:05 cvs Exp $
version 0.3:
- Provide *real* documentation.
- More examples (especially: multipart encoding/decoding).
- More testing.
This diff is collapsed.
This diff is collapsed.
#!/usr/bin/env python
##=============================================================================
#
# Copyright (C) 2003, 2004 Alessandro Duca
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#
# See the GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
#=============================================================================
#
# $Id: ydecode.py,v 1.2 2004/02/18 23:10:11 cvs Exp $
#
##=============================================================================
import yenc
import os
import sys
import re
NAME_RE = re.compile(r"^.*? name=(.+?)\r\n$")
LINE_RE = re.compile(r"^.*? line=(\d{3}) .*$")
SIZE_RE = re.compile(r"^.*? size=(\d+) .*$")
CRC32_RE = re.compile(r"^.*? crc32=(\w+)")
def main():
if len(sys.argv) > 1:
file_in = open(sys.argv[1],"r")
else:
file_in = sys.stdin
while 1:
line = file_in.readline()
if line.startswith("=ybegin "):
try:
name, size = NAME_RE.match(line).group(1), int(SIZE_RE.match(line).group(1))
m_obj = CRC32_RE.match(line)
if m_obj:
head_crc = m_obj.group(1)
except re.error, e:
sys.stderr.write("err-critical: malformed =ybegin header\n")
sys.exit(1)
break
elif not line:
sys.stderr.write("err-critical: no valid =ybegin header found\n")
sys.exit(1)
file_out = open(name,"w")
try:
dec, dec_crc = yenc.decode(file_in, file_out, size)
except yenc.Error, e:
sys.stderr.write(str(e) + '\n')
sys.exit(1)
head_crc = trail_crc = tmp_crc = ""
garbage = 0
for line in file_in.read().split("\r\n"):
if line.startswith("=yend "):
try:
size = int( SIZE_RE.match(line).group(1) )
m_obj = CRC32_RE.match(line)
if m_obj:
trail_crc = m_obj.group(1)
except re.error, e:
sys.stderr.write("err: malformed =yend trailer\n")
break
elif not line:
continue
else:
garbage = 1
else:
sys.stderr.write("warning: couldn't find =yend trailer\n")
if garbage:
sys.stderr.write("warning: garbage before =yend trailer\n")
if head_crc:
tmp_crc = head_crc.lower()
elif trail_crc:
tmp_crc = trail_crc.lower()
else:
sys.exit(0)
if cmp(tmp_crc,dec_crc):
sys.stderr.write("err: header: %s dec: %s CRC32 mismatch\n" % (tmp_crc,dec_crc) )
sys.exit(1)
else:
sys.exit(0)
if __name__ == "__main__":
main()
#!/usr/bin/env python
##=============================================================================
#
# Copyright (C) 2003, 2004 Alessandro Duca
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#
# See the GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
#=============================================================================
#
# $Id: ydecode_Decoder.py,v 1.1 2004/02/18 11:33:08 cvs Exp $
#
##=============================================================================
import yenc
import os
import sys
import re
NAME_RE = re.compile(r"^.*? name=(.+?)\r\n$")
LINE_RE = re.compile(r"^.*? line=(\d{3}) .*$")
SIZE_RE = re.compile(r"^.*? size=(\d+) .*$")
CRC32_RE = re.compile(r"^.*? crc32=(\w+)")
def main():
head_crc = trail_crc = ""
if len(sys.argv) > 1:
file_in = open(sys.argv[1],"r")
else:
file_in = sys.stdin
while 1:
line = file_in.readline()
if line.startswith("=ybegin "):
try:
name, size = NAME_RE.match(line).group(1), int(SIZE_RE.match(line).group(1))
m_obj = CRC32_RE.match(line)
if m_obj:
head_crc = m_obj.group(1)
except re.error, e:
sys.stderr.write("err-critical: malformed =ybegin header\n")
sys.exit(1)
break
elif not line:
sys.stderr.write("err-critical: no valid =ybegin header found\n")
sys.exit(1)
file_out = open(name,"w")
dec = yenc.Decoder(file_out)
trailer = ""
garbage = 0
while True:
data = file_in.readline()
if data.startswith("=yend"):
trailer = data
break
elif dec.getSize() >= size:
garbage = 1
else:
dec.feed(data)
dec.flush()
#file_out.write(dec.getDecoded())
if trailer:
try:
size = int( SIZE_RE.match(line).group(1) )
m_obj = CRC32_RE.match(line)
if m_obj:
trail_crc = m_obj.group(1)
except re.error, e:
sys.stderr.write("err: malformed =yend trailer\n")
else:
sys.stderr.write("warning: couldn't find =yend trailer\n")
if garbage:
sys.stderr.write("warning: garbage before =yend trailer\n")
if head_crc:
tmp_crc = head_crc.lower()
elif trail_crc:
tmp_crc = trail_crc.lower()
else:
sys.exit(0)
# print "comparing"
if cmp(tmp_crc,dec.getCrc32()):
sys.stderr.write("err: header: %s dec: %s CRC32 mismatch\n" % (tmp_crc,dec.getCrc32()) )
sys.exit(1)
else:
sys.exit(0)
if __name__ == "__main__":
main()
##!/usr/bin/env python
##=============================================================================
#
# Copyright (C) 2003, 2004 Alessandro Duca
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#
# See the GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
#=============================================================================
#
# $Id: yencode.py,v 1.2 2004/02/18 23:10:11 cvs Exp $
#
##=============================================================================
import sys
import os
import os.path
import yenc
import getopt
from stat import *
from binascii import crc32
def main():
try:
opts, args = getopt.getopt(sys.argv[1:], "o:")
except getopt.GetoptError:
usage()
file_out = sys.stdout
for o,a in opts:
if o == '-o':
file_out = open(a,"w")
if args:
filename = args[0]
if os.access( filename, os.F_OK | os.R_OK ):
file_in = open(filename,"r")
else:
print "couldn't access %s" % filename
sys.exit(2)
else:
usage()
crc = hex( crc32( open(filename,"r").read() ) )[2:]
name = os.path.split(filename)[1]
size = os.stat(filename)[ST_SIZE]
file_out.write("=ybegin line=128 size=%d crc32=%s name=%s\r\n" % (size, crc, name) )
try:
encoded, crc = yenc.encode(file_in, file_out, size)
except Exception, e:
print e
sys.exit(3)
file_out.write("=yend size=%d crc32=%s\r\n" % (encoded, crc) )
def usage():
print "Usage: yencode.py <-o outfile> filename"
sys.exit(1)
if __name__ == "__main__":
main()
#!/usr/bin/env python
##=============================================================================
#
# Copyright (C) 2003, 2004 Alessandro Duca
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#
# See the GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
#=============================================================================
#
# $Id: yencode_Encoder.py,v 1.2 2004/02/18 23:09:30 cvs Exp $
#
##=============================================================================
import sys
import os
import os.path
import yenc
import getopt
from stat import *
from binascii import crc32
def main():
try:
opts, args = getopt.getopt(sys.argv[1:], "o:")
except getopt.GetoptError:
usage()
file_out = sys.stdout
for o,a in opts:
if o == '-o':
file_out = open(a,"w")
if args:
filename = args[0]
if os.access( filename, os.F_OK | os.R_OK ):
file_in = open(filename,"r")
else:
print "couldn't access %s" % filename
sys.exit(2)
else:
usage()
crc = hex( crc32( open(filename,"r").read() ) )[2:]
name = os.path.split(filename)[1]
size = os.stat(filename)[ST_SIZE]
file_out.write("=ybegin line=128 size=%d crc32=%s name=%s\r\n" % (size, crc, name) )
file_in = open(filename, "r")
encoder = yenc.Encoder(file_out)
while True:
data_in = file_in.read(1024)
encoder.feed(data_in)
encoder.flush()
if len(data_in) < 1024: break
encoder.terminate()
encoder.flush()
file_out.write("=yend size=%d crc32=%s\r\n" % (size, encoder.getCrc32()) )
def usage():
print "Usage: yencode2.py <-o outfile> filename"
sys.exit(1)
if __name__ == "__main__":
main()
##=============================================================================
#
# Copyright (C) 2003, 2004 Alessandro Duca
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#
# See the GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
#=============================================================================
#
# $Id: yenc.py,v 1.2 2004/02/18 23:08:15 cvs Exp $
#
##=============================================================================
import sys
from cStringIO import StringIO
import _yenc
E_ERROR = 64
E_CRC32 = 65
E_PARMS = 66
class Error(Exception):
""" Class for specific yenc errors
"""
def __init__(self, value="", code=E_ERROR):
self.value = value
def __str__(self):
return "yenc.Error: %s\n" % self.value, self.value
def _checkArgsType(file_in, file_out, bytes):
""" Internal checkings, not to be used from outside this module
"""
if bytes < 0: raise Error("No. of bytes can't be negative", E_PARMS)
if type(file_in) == str:
if file_in == "-":
if bytes == 0: raise Error("No. of bytes is 0 or not \
specified while reading from stdin", E_PARMS)
file_in = sys.stdin
else: file_in = open(file_in,"rb")
if type(file_out) == str:
if file_out == "-": file_out = sys.stdout
else: file_out = open(file_out,"wb")
return file_in, file_out, bytes
def encode(file_in, file_out, bytes=0):
""" encode(file_in, file_out, bytes=0): write "bytes" encoded bytes from
file_in to file_out, if "bytes" is 0 encodes bytes until EOF
"""
file_in, file_out, bytes = _checkArgsType(file_in, file_out, bytes)
encoded, crc32 = _yenc.encode(file_in, file_out, bytes)
crc_hex = hex(crc32)[2:]
return encoded, crc_hex
def decode(file_in, file_out, bytes=0, crc_in=""):
""" decode(file_in, file_out, bytes=0): write "bytes" decoded bytes from
file_in to file_out, if "bytes" is 0 decodes bytes until EOF
"""
file_in, file_out, bytes = _checkArgsType(file_in, file_out, bytes)
decoded, crc32 = _yenc.decode(file_in, file_out, bytes)
crc_hex = hex(crc32)[2:]
if crc_in and not cmp(crc_hex,crc_in.lower()):
raise Error("crc32 error", E_CRC32)
else:
return decoded, crc_hex
class Encoder:
""" class Encoder: facility class for encoding one string at time
"""
def __init__(self, output_file = None):
self._buffer = StringIO()
self._column = 0
self._output_file = output_file
self._crc = -1
self._encoded = 0
self._terminated = 0
def __del__(self):
if(self._output_file):
self._output_file.flush()
self._output_file.close()
def feed(self, data):
""" Encode some data and write the encoded data
into the internal buffer
"""
if self._terminated:
raise IOError("Encoding already terminated")
encoded, self._crc, self._column = _yenc.encode_string(data,
self._crc, self._column)
self._encoded = self._encoded + len(encoded)
self._buffer.write(encoded)
return len(encoded)
def terminate(self):
""" Appends the terminal CRLF sequence to the encoded data
"""
self._terminated = 1
self._buffer.write("\r\n")
def flush(self):
""" Writes the content of the internal buffer on the file
passed as argument to the constructor
"""
if self._output_file:
self._output_file.write(self._buffer.getvalue())
self._buffer = StringIO()
else:
raise ValueError("Output file is 'None'")
def getEncoded(self):
""" Returns the data in the internal buffer
"""
if not self._output_file:
return self._buffer.getvalue()
else:
raise ValueError("Output file is not 'None'")
def getSize(self):
""" Returns the total number of encoded bytes (not the size of
the buffer)
"""
return self._encoded
def getCrc32(self):
""" Returns the calculated crc32 string for the clear
encoded data
"""
return "%08x" % (self._crc ^ -1)
class Decoder:
""" class Decoder: facility class for decoding one string at time
"""
def __init__(self, output_file = None):
self._buffer = StringIO()
self._escape = 0
self._output_file = output_file
self._crc = -1
self._decoded = 0
def __del__(self):
if(self._output_file):
self._output_file.flush()
self._output_file.close()
def feed(self, data):
""" Encode some data and write the encoded data
into the internal buffer
"""
decoded, self._crc, self._escape = _yenc.decode_string(data, self._crc, self._escape)
self._decoded = self._decoded + len(decoded)
self._buffer.write(decoded)
return len(decoded)
def flush(self):
""" Writes the content of the internal buffer on the file
passed as argument to the constructor
"""
if self._output_file:
self._output_file.write(self._buffer.getvalue())
self._buffer = StringIO()
else:
raise ValueError("Output file is 'None'")
def getDecoded(self):
""" Returns the data in the internal buffer
"""
if not self._output_file:
return self._buffer.getvalue()
else:
raise ValueError("Output file is not 'None'")
def getSize(self):
""" Returns the total number of decoded bytes (not the size of
the buffer)
"""
return self._decoded
def getCrc32(self):
""" Returns the calculated crc32 string for the decoded data
"""
return "%08x" % (self._crc ^ -1)
#!/usr/bin/env python
##=============================================================================
#
# Copyright (C) 2003, 2004 Alessandro Duca
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,