Commits · 1.1.0 · Debian / piuparts

08 Jan, 2020 3 commits
- release as 1.1.0 · 0a86aa2f
  Holger Levsen authored Jan 08, 2020
```
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
  0a86aa2f
- mark this a team upload to appease lintian. · 2294d417
  Holger Levsen authored Jan 08, 2020
```
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
  2294d417
- htocs/news.tpl: add farewell note. · 7c22ebb5
  Holger Levsen authored Jan 08, 2020
```
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
  7c22ebb5
29 Dec, 2019 5 commits

instances/*, docs/README_pejacevic.txt and htdocs/index: stop mentioning piu-slave-bm-a.debian.org · c88b177f
Holger Levsen authored Dec 29, 2019
```
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
c88b177f
remove myself from uploaders · 6347166f
Holger Levsen authored Dec 29, 2019
```
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
6347166f

piuparts-report: fix pickle.loading str from python2 · fda3b882

Nis Martensen authored Dec 29, 2019 and

Holger Levsen committed Dec 29, 2019



piuparts-report may encounter old md5cache files pickled by python2 in
text mode. Loading such files may fail if there are non-ascii
characters. The existing cache files involved in generating the reports
on piuparts.debian.org most likely use utf-8 encoding, so let's use that
instead of the default assumption 'ascii'.

Should fix:

Traceback (most recent call last):
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1911, in <module>
    main()
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1851, in main
    section.generate_output(output_directory, section_names, problem_list, web_host)
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1660, in generate_output
    self.generate_html()
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1563, in generate_html
    self._md5cache['old'] = pickle.load(f)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 71: ordinal not in range(128)

Signed-off-by: Holger Levsen <holger@layer-acht.org>

fda3b882

add changelog entry for James · b1738ebd
Holger Levsen authored Dec 29, 2019
```
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
b1738ebd

Fix diff_meta_data reporting every file as new and removed · 14c81dc7

Jessica Clarke authored Dec 29, 2019 and

Holger Levsen committed Dec 29, 2019

This regressed in 0b47bf88

, which
erroneously used the original trees rather than the modified copies.

Signed-off-by: Holger Levsen <holger@layer-acht.org>

14c81dc7

28 Dec, 2019 11 commits

Bump debhelper-compat to 12. · 2f15d124

Holger Levsen authored Dec 28, 2019



after confirming with diffoscope that the changes are sane

Signed-off-by: Holger Levsen <holger@layer-acht.org>

2f15d124

d/rules: set GOCACHE to a safely writable directory during build as the cache... · f1b72b2e

Holger Levsen authored Dec 28, 2019


d/rules: set GOCACHE to a safely writable directory during build as the cache is mandatory with Go 1.12.

Signed-off-by: Holger Levsen <holger@layer-acht.org>

f1b72b2e

Bump standards version to 4.4.1, no changes needed. · b5e2f2e2
Holger Levsen authored Dec 28, 2019
```
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
b5e2f2e2
news 2019-12-27: name all people who contributed to switch to python3 · a43e4458
Holger Levsen authored Dec 28, 2019
```
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
a43e4458
extend changelog for Nis · 837d794e
Holger Levsen authored Dec 28, 2019
```
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
837d794e

piuparts-report: handle EOFError with pickle.load() · f70767fe

Nis Martensen authored Dec 28, 2019 and

Holger Levsen committed Dec 28, 2019



Do not error out if the cache file is empty.

Fixes:

Traceback (most recent call last):
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1911, in <module>
    main()
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1851, in main
    section.generate_output(output_directory, section_names, problem_list, web_host)
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1660, in generate_output
    self.generate_html()
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1563, in generate_html
    self._md5cache['old'] = pickle.load(f)
EOFError: Ran out of input

Signed-off-by: Holger Levsen <holger@layer-acht.org>

f70767fe

piuparts-report, piuparts: pickle needs binary file streams · 08f9a354

Nis Martensen authored Dec 28, 2019 and

Holger Levsen committed Dec 28, 2019



Fixes:

Traceback (most recent call last):
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1911, in <module>
    main()
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1851, in main
    section.generate_output(output_directory, section_names, problem_list, web_host)
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1660, in generate_output
    self.generate_html()
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1600, in generate_html
    pickle.dump(self._md5cache['new'], f)
TypeError: write() argument must be str, not bytes

Signed-off-by: Holger Levsen <holger@layer-acht.org>

08f9a354

piuparts-report: give hashlib the bytes it needs · 484bc88e

Nis Martensen authored Dec 28, 2019 and

Holger Levsen committed Dec 28, 2019

Fixes:

Traceback (most recent call last):
File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1911, in <module>
main()
File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1851, in main
section.generate_output(output_directory, section_names, problem_list, web_host)
File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1660, in generate_output
self.generate_html()
File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1580, in generate_html
self.print_by_dir(self._output_directory, logs_by_dir)
File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 943, in print_by_dir
desc_by_dir[vdir], vlist)
File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 930, in write_log_list_page
"logrows": "".join(lines),
File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 890, in _write_template_html
write_template_html(filename, body, mapping, defer_if_unmodified=defer_if_unmodified, md5cache=self._md5cache)
File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 714, in write_template_html
content_md5 = hashlib.md5(htmlpage.safe_substitute(mapping)).hexdigest()
TypeError: Unicode-objects must be encoded before hashing

Signed-off-by: Holger Levsen <holger@layer-acht.org>

484bc88e

piuparts-report: fix 'dict_keys has no attribute remove' · cc30509e

Nis Martensen authored Dec 28, 2019 and

Holger Levsen committed Dec 28, 2019



In python3 dict().keys() returns a dict_keys() view and not a list as in
python2. Replace the now unavailable remove() by a set subtraction.

Fixes:

Traceback (most recent call last):
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1911, in <module>
    main()
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1851, in main
    section.generate_output(output_directory, section_names, problem_list, web_host)
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1660, in generate_output
    self.generate_html()
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1574, in generate_html
    self.cleanup_removed_packages(logs_by_dir)
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1536, in cleanup_removed_packages
    vdirs.remove("reserved")
AttributeError: 'dict_keys' object has no attribute 'remove'

Signed-off-by: Holger Levsen <holger@layer-acht.org>

cc30509e

fix piupartslib imports · dd4cd1a8

Nis Martensen authored Dec 28, 2019 and

Holger Levsen committed Dec 28, 2019



During the port to python3 piupartslib was changed to no longer
auto-import some of its submodules. Import the required submodules
directly where now needed.

Fixes:

Traceback (most recent call last):
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1910, in <module>
    main()
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 1849, in main
    section = Section(section_name, master_directory, doc_root, packagedb_cache=packagedb_cache)
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 827, in __init__
    self._load_package_database(section, master_directory)
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-report", line 859, in _load_package_database
    db = piupartslib.packagesdb.PackagesDB(prefix=sectiondir)
AttributeError: module 'piupartslib' has no attribute 'packagesdb'

Signed-off-by: Holger Levsen <holger@layer-acht.org>

dd4cd1a8

piuparts-master-backend: handle BrokenPipeError · 9171821b

Nis Martensen authored Dec 27, 2019 and

Holger Levsen committed Dec 28, 2019



No need to crash with a stacktrace when the slave is gone and the pipe
to it is broken. Just log this and exit 1.

Fixes:

Traceback (most recent call last):
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-master-backend", line 433, in <module>
    main()
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-master-backend", line 423, in main
    m = Master(sys.stdin, sys.stdout)
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-master-backend", line 154, in __init__
    self._writeline("hello")
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-master-backend", line 111, in _writeline
    self._output.flush()
BrokenPipeError: [Errno 32] Broken pipe
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
BrokenPipeError: [Errno 32] Broken pipe

Signed-off-by: Holger Levsen <holger@layer-acht.org>

9171821b

27 Dec, 2019 4 commits

htdocs/news.tpl: mention piuparts.debian.org is running the python3 branch. · 45242102
Holger Levsen authored Dec 27, 2019
```
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
45242102
instances/Makefile: set default mirror to deb.debian.org. · 0d44938b
Holger Levsen authored Dec 27, 2019
```
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
0d44938b

piupartslib/__init__: fix byte stream decoding · e526f8a1

Nis Martensen authored Dec 26, 2019 and

Holger Levsen committed Dec 27, 2019



When trying to decode a byte sequence into a string, it is possible that
the sequence ends in the middle of a multibyte character. In this case
the decoding fails.

This hack assumes that the input byte sequence is always encoded in
UTF-8, and that the original data stream never ends in the middle of a
multibyte character.

Signed-off-by: Holger Levsen <holger@layer-acht.org>

e526f8a1

avoid "input" as variable name · 82709f47

Nis Martensen authored Dec 26, 2019 and

Holger Levsen committed Dec 27, 2019



It is better to avoid using names of builtin functions as variable
names.

Signed-off-by: Holger Levsen <holger@layer-acht.org>

82709f47

26 Dec, 2019 4 commits

add changelog entry for Nis - many thanks! · a065bef3
Holger Levsen authored Dec 26, 2019
```
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
a065bef3

piupartslib/packagesdb: encode contents for os.write · 8993738a

Nis Martensen authored Dec 26, 2019 and

Holger Levsen committed Dec 26, 2019



Fixes:

  File "/srv/piuparts.debian.org/lib/python3/dist-packages/piupartslib/packagesdb.py", line 260, in create
    if os.write(fd, contents) != len(contents):
TypeError: a bytes-like object is required, not 'str'

Signed-off-by: Holger Levsen <holger@layer-acht.org>

8993738a

piupartslib/packagesdb: concatenate only lists · 09916537

Nis Martensen authored Dec 26, 2019 and

Holger Levsen committed Dec 26, 2019

While one set of dict_keys() was already wrapped in a list() during the
initial porting work, the second one was still missing. Fixes:

File "/srv/piuparts.debian.org/lib/python3/dist-packages/piupartslib/packagesdb.py", line 818, in get_best_package_state
for state in self._good_states + list(self._propagate_waiting_state.keys()) + self._propagate_error_state.keys():
TypeError: can only concatenate list (not "dict_keys") to list

Signed-off-by: Holger Levsen <holger@layer-acht.org>

09916537

piupartslib/dependencyparser: mynext() does not take args · 640284d2

Nis Martensen authored Dec 26, 2019 and

Holger Levsen committed Dec 26, 2019

Arguments were accidentally introduced when porting the code to python3.
Remove them again. Fixes:

File "/srv/piuparts.debian.org/lib/python3/dist-packages/piupartslib/dependencyparser.py", line 266, in _parse_version_dependency
self._cursor.mynext(self._cursor)
TypeError: mynext() takes 1 positional argument but 2 were given

Signed-off-by: Holger Levsen <holger@layer-acht.org>

640284d2

25 Dec, 2019 1 commit

piupartslib/conf.py: decode subprocess.Popen().stdout · fecdaba6

Nis Martensen authored Dec 25, 2019

subprocess.Popen().stdout is a byte string in python3. We need to decode
it since we want a unicode string here.

fecdaba6

03 Nov, 2019 1 commit
- add changelog entry for Bastian · e0924c08
  Holger Levsen authored Nov 03, 2019
```
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
  e0924c08
02 Nov, 2019 1 commit
- python-debianbts 2.10.0 deprecated some methodcalls · 5583ce64
  Bastian Venthur authored Nov 02, 2019
  
  5583ce64
10 Sep, 2019 7 commits

Revert "enable debug output" · 25130ee5
Holger Levsen authored Sep 10, 2019
```
This reverts commit 030f1e97.
```
25130ee5
enable debug output · 030f1e97
Holger Levsen authored Sep 10, 2019
```
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
030f1e97

decode the file after reading it · d892e48e

Mattia Rizzolo authored Sep 10, 2019 and

Holger Levsen committed Sep 10, 2019



At this point it would probably be best to completely rewrite that thing
using StringIO and some other nicer way to do buffering, but this should
do the job for now.

should fix:

09:46:14 Running section testing2sid (precedence=2)
Traceback (most recent call last):
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-slave", line 1064, in <module>
    main()
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-slave", line 1029, in main
    test_count += section.run(do_processing=(test_count == 0))
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-slave", line 575, in run
    processed = self._process()
  File "/srv/piuparts.debian.org/share/piuparts/piuparts-slave", line 667, in _process
    packagenames)
  File "/srv/piuparts.debian.org/lib/python3/dist-packages/piupartslib/packagesdb.py", line 183, in load_packages_urls
    self._read_file(stream, restrict_packages=restrict_packages)
  File "/srv/piuparts.debian.org/lib/python3/dist-packages/piupartslib/packagesdb.py", line 190, in _read_file
    headers = rfc822_like_header_parse(input)
  File "/srv/piuparts.debian.org/lib/python3/dist-packages/piupartslib/packagesdb.py", line 53, in rfc822_like_header_parse
    line = input.readline()
  File "/srv/piuparts.debian.org/lib/python3/dist-packages/piupartslib/__init__.py", line 60, in readline
    empty = not self._refill()
  File "/srv/piuparts.debian.org/lib/python3/dist-packages/piupartslib/__init__.py", line 52, in _refill
    self._buffer = self._buffer + chunk
TypeError: can only concatenate str (not "bytes") to str

Signed-off-by: Mattia Rizzolo <mattia@debian.org>
Signed-off-by: Holger Levsen <holger@layer-acht.org>

d892e48e

in py3 you can't raise a string, needs to wrap it in the Exception() class · 9838ed84
Mattia Rizzolo authored Sep 10, 2019 and Holger Levsen committed Sep 10, 2019
```
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
9838ed84

fix error handling, passing through the exception object correctly · 46efe15c

Mattia Rizzolo authored Sep 10, 2019 and

Holger Levsen committed Sep 10, 2019



before, httperror was already out of scope, so it couldn't be raised
again.

Signed-off-by: Mattia Rizzolo <mattia@debian.org>
Signed-off-by: Holger Levsen <holger@layer-acht.org>

46efe15c

drop unused imports · dc410dd6

Mattia Rizzolo authored Sep 10, 2019 and

Holger Levsen committed Sep 10, 2019



Signed-off-by: Mattia Rizzolo <mattia@debian.org>
Signed-off-by: Holger Levsen <holger@layer-acht.org>

dc410dd6

piuparts-slave.py: port to python3. (WIP) · 65fa5d92
Holger Levsen authored Sep 10, 2019
```
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
65fa5d92

04 Sep, 2019 3 commits
- use open correctly for python3, thanks deki · 223c3acb
  Holger Levsen authored Sep 04, 2019
```
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
  223c3acb
- adjust PYTHONPATH as needed. thanks olasd and #debconf · 8e822266
  Holger Levsen authored Sep 04, 2019
```
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
  8e822266
- Use /usr/bin/python3 everywhere. · 762d7d31
  Holger Levsen authored Sep 04, 2019
```
Signed-off-by: Holger Levsen <holger@layer-acht.org>
```
  762d7d31