Holger Levsen · Holger Levsen · Holger Levsen · Holger Levsen · Holger Levsen · Holger Levsen
--- a/bug-templates/copyright_file_missing_after_upgrade.mail
+++ b/bug-templates/copyright_file_missing_after_upgrade.mail
@@ -33,7 +33,6 @@ https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-un
 It is recommended to use the dpkg-maintscript-helper commands
 'dir_to_symlink' and 'symlink_to_dir' (available since dpkg 1.17.14)
 to perform the conversion, ideally using d/$PACKAGE.maintscript.
-Do not forget to add 'Pre-Depends: ${misc:Pre-Depends}' in d/control.
 See dpkg-maintscript-helper(1) and dh_installdeb(1) for details.



--- a/bug-templates/logrotate_exits_with_error_after_package_removal.mail
+++ b/bug-templates/logrotate_exits_with_error_after_package_removal.mail
@@ -9,9 +9,14 @@ Usertags: piuparts

 Hi,

-during a test with piuparts I noticed your package's logrotate
-configuration causes logrotate to exit with an error after
-the package has been removed.
+during a test with piuparts I noticed your package's logrotate configuration
+causes logrotate to exit with an error after the package has been removed (*) or
+when logrote is run but no logfile exists.
+
+Usually the solution is to specify 'missingok' in the logrotate configuration.
+
+*) logrotate configuration files remain installed and executed after a package
+has been removed, they only get removed when the package is purged.

 From the attached log (scroll to the bottom...):


--- a/bug-templates/unhandled_symlink_to_directory_conversion.mail
+++ b/bug-templates/unhandled_symlink_to_directory_conversion.mail
@@ -36,7 +36,6 @@ https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-un
 It is recommended to use the dpkg-maintscript-helper commands
 'dir_to_symlink' and 'symlink_to_dir' (available since dpkg 1.17.14)
 to perform the conversion, ideally using d/$PACKAGE.maintscript.
-Do not forget to add 'Pre-Depends: ${misc:Pre-Depends}' in d/control.
 See dpkg-maintscript-helper(1) and dh_installdeb(1) for details.



--- a/conf/crontab-master.in
+++ b/conf/crontab-master.in
@@ -13,9 +13,9 @@ PYTHONPATH=@libdir@/python2.7/dist-packages
 # generate reports twice or three times a day
 #  (dinstall runs 1|7|13|19:52, so this is long after mirror pushes...)
 #
-0  4-23/12 * * * piuparts-analyze
-0  2-23/3 * * * detect_well_known_errors
-0  0-23/3 * * * piuparts-report
+0  4-23/12 * * * master_shell_runner piuparts-analyze
+0  2-23/3 * * * master_shell_runner detect_well_known_errors
+0  0-23/3 * * * master_shell_runner piuparts-report
 15 23 * * * generate_daily_report

 #

--- a/debian/changelog
+++ b/debian/changelog
+piuparts (0.95) unstable; urgency=medium
+
+  [ Holger Levsen ]
+  * piuparts.py: add /var/lib/debian-security-support/ to self.ignored_files.
+    (Closes: #749317)
+  * Reduce number of mails per day, from 34 down to 1. (Closes: #912591)
+    - lib/write_log.sh: new helper to publish logs in htdocs/logs.
+    - master-bin/master_shell_runner: new script to run python scripts to
+      redirect logs to htdocs/logs.
+    - master-bin/generate_daily_report:
+      - redirect logs of called scripts to htdocs/logs.
+      - include listing of htdocs/logs/$TODAY into daily report.
+      - publish dailyreport.txt in htdocs/logs/$TODAY instead of htdocs/.
+    - master-bin/detect_archive_issues, detect_piuparts_issues,
+      detect_network_issues, reschedule_oldest_logs, rotate_master_logs,
+      report_untestable_packages and report_stale_reserved_packages: redirect
+      logs to htdocs/logs.
+    - conf/crontab-master.in: run detect_well_known_errors, piuparts-report
+      and piuparts-analyze through the new master_shell_runner script.
+    - master-bin/rotate_master_logs: compress htdocs/logs older than a week.
+    - piuparts-report.py: add navigation link to https://piuparts.d.o/logs/
+    - htdocs/news.tpl: explain the result of these changes.
+  * d/control: use canonical URL for Vcs-Git, thanks lintian.
+  * Add d/piuparts.lintian-overrides to override the incorrect
+    uses-dpkg-database-directly warnings, after confirming we are not doing
+    that.
+  * bug-templates/logrotate_exits_with_error_after_package_removal: clarify
+    possibles causes of this problem as well as the solution.
+
+  [ Andreas Beckmann ]
+  * Clarify errors from installing objects over existing symlinks.
+  * bug-templates/copyright_file_missing_after_upgrade and
+    bug-templates/unhandled_symlink_to_directory_conversion: adjust advice as
+    symlink_to_dir/dir_to_symlink are supported by dpkg in oldstable.
+
+ -- Holger Levsen <holger@debian.org>  Sun, 09 Dec 2018 22:27:41 +0100
+
 piuparts (0.94) unstable; urgency=medium

  [ Holger Levsen ]

--- a/debian/control
+++ b/debian/control
@@ -24,7 +24,7 @@ Build-Depends:
 Build-Depends-Indep:
 git,
 Homepage: https://piuparts.debian.org
-Vcs-Git: https://salsa.debian.org/debian/piuparts
+Vcs-Git: https://salsa.debian.org/debian/piuparts.git
 Vcs-Browser: https://salsa.debian.org/debian/piuparts
 Xs-Go-Import-Path: salsa.debian.org/debian/piuparts


--- a/debian/piuparts.lintian-overrides
+++ b/debian/piuparts.lintian-overrides
+piuparts: uses-dpkg-database-directly etc/piuparts/scripts-debug-purge/post_remove_postrm_set-x
+piuparts: uses-dpkg-database-directly etc/piuparts/scripts-debug-remove/pre_remove_prerm_postrm_set-x
+piuparts: uses-dpkg-database-directly etc/piuparts/scripts/post_remove_exceptions
+piuparts: uses-dpkg-database-directly etc/piuparts/scripts/pre_remove_40_find_missing_md5sums
+piuparts: uses-dpkg-database-directly usr/sbin/piuparts
--- a/htdocs/news.tpl
+++ b/htdocs/news.tpl
@@ -6,9 +6,13 @@
    </tr>
    <tr class="normalrow">
     <td class="contentcell2">
-      <b>2018-10-19</b>: Three and a half years after stopping linking to packages.qa.debian.org(which was called <em>PTS</em>) we finally drop all references to the PTS and use <em>tracker.d.o</em> instead.
+      <b>2018-11-28</b>: The output of various maintenance scripts is now available at <a href="https://piuparts.debian.org/logs/" target="_blank">https://piuparts.debian.org/logs/</a> instead of sending it via mails to the piuparts.d.o maintainers only.
     </td>
    </tr>
+    <tr class="normalrow">
+     <td class="contentcell2">
+      <b>2018-10-19</b>: Three and a half years after stopping linking to packages.qa.debian.org (which was called <em>PTS</em>) we finally drop all references to the PTS and use <em>tracker.d.o</em> instead.
+     </td>
    </tr>
    <tr class="normalrow">
     <td class="contentcell2">

--- a/known_problems/installs_over_symlink_error.conf
+++ b/known_problems/installs_over_symlink_error.conf
 #
 # detect packages which have the string "dirname part contains a symlink" in their logs
 #
-PATTERN='silently overwrites files via directory symlinks|dirname part contains a symlink'
+PATTERN='silently overwrites files via directory symlinks|installs objects over existing directory symlinks|dirname part contains a symlink'
 WHERE='fail bugged affected'
 ISSUE=0
 HEADER='Packages that install something over existing symlinks'

--- a/known_problems/installs_over_symlink_issue.conf
+++ b/known_problems/installs_over_symlink_issue.conf
 #
 # detect packages which have the string "dirname part contains a symlink" in their logs
 #
-PATTERN='silently overwrites files via directory symlinks|dirname part contains a symlink'
+PATTERN='silently overwrites files via directory symlinks|installs objects over existing directory symlinks|dirname part contains a symlink'
 WHERE='pass'
 ISSUE=1
 HEADER='Packages that install something over existing symlinks'

--- a/lib/write_log.sh
+++ b/lib/write_log.sh
+# Copyright © 2018 Holger Levsen (holger@debian.org)
+#
+# This program is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by the
+# Free Software Foundation; either version 2 of the License, or (at your
+# option) any later version.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
+# Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program. If not, see <https://www.gnu.org/licenses/>
+
+#
+# Helper function for publishing logfiles
+#
+
+publish_logs() {
+	local LOG_OUTPUT=$1
+	local LOG_PREFIX=$2
+	local HTDOCS="$3"
+	local LOG=$4
+	local YEAR=$(date -u +%Y)
+	local MONTH=$(date -u +%m)
+	local DAY=$(date -u +%d)
+	local DIR="$HTDOCS/logs/$YEAR/$MONTH/$DAY"
+	mkdir -p "$DIR"
+	if [ -n "$LOG_PREFIX" ] && [ ! -s "$DIR/$LOG.txt" ] ; then
+		cat $LOG_PREFIX >> "$DIR/$LOG.txt"
+		rm -f $LOG_PREFIX >/dev/null
+	fi
+	cat $LOG_OUTPUT >> "$DIR/$LOG.txt"
+	rm -f $LOG_OUTPUT >/dev/null
+}
--- a/master-bin/detect_archive_issues.in
+++ b/master-bin/detect_archive_issues.in
@@ -18,13 +18,17 @@


 . @sharedir@/piuparts/lib/read_config.sh
+. @sharedir@/piuparts/lib/write_log.sh

 get_config_value MASTER global master-directory
+get_config_value HTDOCS global output-directory
 get_config_value SECTIONS global sections
 get_config_value URLBASE global urlbase https://piuparts.debian.org
 get_config_value DAYS global reschedule-untestable-days 7

 ISSUE_LOG="$MASTER/archive_issues.txt"
+LOG_OUTPUT=`mktemp`
+LOG_PREFIX=$(mktemp)

 STARTDATE=$(date -u +%s)

@@ -59,21 +63,28 @@ if [ -s $URLS ]; then
 	cat $URLS >> $ISSUE_LOG
 	FINALDATE=$(date -u +%s)
 	RUNTIME=$(date -u -d "0 $FINALDATE seconds - $STARTDATE seconds" +%T)
-	echo "Runtime: $RUNTIME"
-	echo
-	echo "Broken packages detected! The following packages have been moved to"
-	echo "$section/untestable and will be tested again in $DAYS days."
-	echo
-	echo "Broken packages are usually a temporary problem in the archive and are"
-	echo "caught by other tools like britney or https://qa.debian.org/dose/debcheck.html"
-	echo "If it is always the same package failing, it's likely to be an issue in the"
-	echo "package."
-	echo
-	grep -f $URLS $ISSUE_LOG | sort | uniq -c | sort -rn
-	echo
-	echo "Those problems were found in failed logs by grep'ing for these patterns:"
-	echo "        'E: Broken packages'"
-	echo "        'E: Unable to correct problems, you have held broken packages'"
-	echo "        'E: Error, pkgProblemResolver::Resolve generated breaks'"
+	(
+		echo "Broken packages detected! The following packages have been moved to"
+		echo "$section/untestable and will be tested again in $DAYS days."
+		echo
+		echo "Broken packages are usually a temporary problem in the archive and are"
+		echo "caught by other tools like britney or https://qa.debian.org/dose/debcheck.html"
+		echo "If it is always the same package failing, it's likely to be an issue in the"
+		echo "package."
+		echo
+		echo "Those problems were found in failed logs by grep'ing for these patterns:"
+		echo "        'E: Broken packages'"
+		echo "        'E: Unable to correct problems, you have held broken packages'"
+		echo "        'E: Error, pkgProblemResolver::Resolve generated breaks'"
+		echo
+	) > $LOG_PREFIX
+	(
+		echo "$(date -u)"
+		echo "Runtime: $RUNTIME"
+		echo
+		grep -f $URLS $ISSUE_LOG | sort | uniq -c | sort -rn
+		echo
+	) > $LOG_OUTPUT
+	publish_logs $LOG_OUTPUT $LOG_PREFIX $HTDOCS archive_issues
 fi
 rm $LOGS $URLS
--- a/master-bin/detect_network_issues.in
+++ b/master-bin/detect_network_issues.in
@@ -18,11 +18,15 @@


 . @sharedir@/piuparts/lib/read_config.sh
+. @sharedir@/piuparts/lib/write_log.sh

 get_config_value MASTER global master-directory
 get_config_value SECTIONS global sections
+get_config_value HTDOCS global output-directory

 STARTDATE=$(date -u +%s)
+LOG_OUTPUT=$(mktemp)
+LOG_PREFIX=$(mktemp)

 #
 # detect network/mirror problems
@@ -72,18 +76,25 @@ done
 if [ -s $FILE ] ; then
 	FINALDATE=$(date -u +%s)
 	RUNTIME=$(date -u -d "0 $FINALDATE seconds - $STARTDATE seconds" +%T)
-	echo "Runtime: $RUNTIME"
-	echo
-	echo "Network problems on detected! The following logfiles have been deleted:"
-	echo
-	for log in $(sort -u $FILE)
-	do
-		echo "$log" | cut -d "/" -f5-
-		grep -E "$PATTERN" "$log" | perl -e 'print grep !$seen{$_}++, <>;' | head -n 10 | sed "s/^/${tab}/"
-		rm -f "$log"
-	done
-	echo
-	echo "Those problems were found in failed logs by grep'ing for these patterns:"
-	echo "$DISPLAY_PATTERN"
+	(
+		echo "Network problems detected! The following logfiles have been deleted:"
+		echo
+		echo "Those problems were found in failed logs by grep'ing for these patterns:"
+		echo "$DISPLAY_PATTERN"
+		echo
+	) > $LOG_PREFIX
+	(
+		echo "$(date -u)"
+		echo "Runtime: $RUNTIME"
+		echo
+		for log in $(sort -u $FILE)
+		do
+			echo "$log" | cut -d "/" -f5-
+			grep -E "$PATTERN" "$log" 2>/dev/null | perl -e 'print grep !$seen{$_}++, <>;' | head -n 10 | sed "s/^/${tab}/"
+			rm -f "$log"
+		done
+		echo
+	) > $LOG_OUTPUT
+	publish_logs $LOG_OUTPUT $LOG_PREFIX $HTDOCS network_issues
 fi
 rm $FILE
--- a/master-bin/detect_piuparts_issues.in
+++ b/master-bin/detect_piuparts_issues.in
@@ -18,11 +18,15 @@


 . @sharedir@/piuparts/lib/read_config.sh
+. @sharedir@/piuparts/lib/write_log.sh

 get_config_value MASTER global master-directory
 get_config_value SECTIONS global sections
+get_config_value HTDOCS global output-directory

 STARTDATE=$(date -u +%s)
+LOG_OUTPUT=$(mktemp)
+LOG_PREFIX=$(mktemp)

 #
 # detect piuparts problems
@@ -95,24 +99,30 @@ done
 if [ -s $FILE ] ; then
 	FINALDATE=$(date -u +%s)
 	RUNTIME=$(date -u -d "0 $FINALDATE seconds - $STARTDATE seconds" +%T)
-	echo "Runtime: $RUNTIME"
-	echo
-	echo "piuparts problems detected! Please review the following logfiles/packages."
-	echo "If it is always the same package failing, it's likely to be an issue in the"
-	echo "package."
-	echo
-	echo "The following logfiles have been deleted:"
-	echo
-	for log in $(sort -u $FILE)
-	do
-		echo "$log" | cut -d "/" -f5-
-		test -s "$log" || echo "${tab}Empty logfile."
-		grep -E "$PATTERN" "$log" | perl -e 'print grep !$seen{$_}++, <>;' | head -n 10 | sed "s/^/${tab}/"
-		rm -f "$log"
-	done
-	echo
-	echo "Those problems were found in failed logs by grep'ing for these patterns:"
-	echo "$DISPLAY_PATTERN"
-	echo
+	(
+		echo "piuparts problems detected! Please review the following logfiles/packages."
+		echo "If it is always the same package failing, it's likely to be an issue in the"
+		echo "package."
+		echo
+		echo "Problems were found in failed logs by grep'ing for these patterns:"
+		echo "$DISPLAY_PATTERN"
+		echo
+	) > $LOG_PREFIX
+	(
+		echo "$(date -u)"
+		echo "Runtime: $RUNTIME"
+		echo
+		echo "The following logfiles have been deleted:"
+		echo
+		for log in $(sort -u $FILE)
+		do
+			echo "$log" | cut -d "/" -f5-
+			test -s "$log" || echo "${tab}Empty logfile."
+			grep -E "$PATTERN" "$log" 2>/dev/null | perl -e 'print grep !$seen{$_}++, <>;' | head -n 10 | sed "s/^/${tab}/"
+			rm -f "$log"
+		done
+		echo
+	) > $LOG_OUTPUT
+	publish_logs $LOG_OUTPUT $LOG_PREFIX $HTDOCS piuparts_issues
 fi
 rm $FILE
--- a/master-bin/detect_well_known_errors.py
+++ b/master-bin/detect_well_known_errors.py
@@ -3,6 +3,7 @@

 # Copyright 2013 David Steele (dsteele@gmail.com)
 # Copyright © 2014 Andreas Beckmann (anbe@debian.org)
+# Copyright © 2017-2018 Holger Levsen (holger@layer-acht.org)
 #
 # This file is part of Piuparts
 #
@@ -132,9 +133,9 @@ def detect_well_known_errors(sections, config, problem_list, recheck, recheck_fa
        except MissingSection:
            pass

+    current_time=time.strftime("%a %b %2d %H:%M:%S %Z %Y", time.localtime())
+    logging.info("%s - total parsed logfiles: %d removed, %d added" % (current_time, total_del, total_add))
    logging.info("")
-    logging.info(time.strftime("%a %b %2d %H:%M:%S %Z %Y", time.localtime()))
-    logging.info("total parsed logfiles: %d removed, %d added" % (total_del, total_add))


 if __name__ == '__main__':

--- a/master-bin/generate_daily_report.in
+++ b/master-bin/generate_daily_report.in
 #!/bin/sh

 # Copyright © 2011-2013 Andreas Beckmann (anbe@debian.org)
+# Copyright © 2018 Holger Levsen (holger@layer-acht.org)
 #
 # This program is free software; you can redistribute it and/or modify it
 # under the terms of the GNU General Public License as published by the
@@ -15,8 +16,8 @@
 # You should have received a copy of the GNU General Public License along
 # with this program. If not, see <https://www.gnu.org/licenses/>

-
 . @sharedir@/piuparts/lib/read_config.sh
+. @sharedir@/piuparts/lib/write_log.sh

 get_config_value MASTER global master-directory
 get_config_value SECTIONS global sections
@@ -24,7 +25,6 @@ get_config_value HTDOCS global output-directory
 get_config_value URLBASE global urlbase https://piuparts.debian.org
 get_config_value PYTHONPATH global PYTHONPATH ''

-
 export PYTHONPATH

 LOCKFILE=$HTDOCS/daily.lock
@@ -36,22 +36,26 @@ fi
 date > $LOCKFILE

 R_STARTDATE=$(date -u +%s)
+R_LOGDATEDIR="$HTDOCS/logs/$(date -u '+%Y/%m/%d')"

 #
-# various scripts to detect issues and cleanup
+# various scripts to detect issues and cleanup to be run right before we generate the daily report
+# (this code block contains cruft and could be refactored)
 #
 OUTPUT=$(mktemp)
+LOG_OUTPUT=$(mktemp)
 for SCRIPT in archive_old_logs ; do
  STARTDATE=$(date -u +%s)
  @sharedir@/piuparts/master/$SCRIPT 2>&1 > $OUTPUT
  FINALDATE=$(date -u +%s)
  if [ -s $OUTPUT ] ; then
 	if grep -q "Runtime: " $OUTPUT ; then
-		cat $OUTPUT | mail -s $SCRIPT $LOGNAME
+		cat $OUTPUT > $LOG_OUTPUT
 	else
 		RUNTIME=$(date -u -d "0 $FINALDATE seconds - $STARTDATE seconds" +%T)
-		( echo "Runtime: $RUNTIME" ; echo ; cat $OUTPUT ) | mail -s $SCRIPT $LOGNAME
+		( echo "Runtime: $RUNTIME" ; echo ; cat $OUTPUT ) > $LOG_OUTPUT
 	fi
+	publish_logs $LOG_OUTPUT "" $HTDOCS $SCRIPT
  fi
 done
 rm $OUTPUT
@@ -96,7 +100,15 @@ rm $expire
 FINALDATE=$(date -u +%s)
 RUNTIME=$(date -u -d "0 $FINALDATE seconds - $R_STARTDATE seconds" +%T)

-( echo "Runtime: $RUNTIME" ; echo ; cat $DAILYREPORT ) | mail -s generate_daily_report $LOGNAME
+(
+	echo "Runtime: $RUNTIME"
+	echo
+	echo $R_LOGDATEDIR
+	ls -lt $R_LOGDATEDIR
+	echo
+	cat $DAILYREPORT
+) | mail -s generate_daily_report $LOGNAME
+mv $DAILYREPORT $R_LOGDATEDIR

 mv $FAILURESTAMP.new $FAILURESTAMP
 rm -f $LOCKFILE
--- a/master-bin/master_shell_runner.in
+++ b/master-bin/master_shell_runner.in
+#!/bin/sh
+
+# Copyright © 2018 Holger Levsen (holger@layer-acht.org)
+#
+# This program is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by the
+# Free Software Foundation; either version 2 of the License, or (at your
+# option) any later version.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
+# Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program. If not, see <https://www.gnu.org/licenses/>
+
+. @sharedir@/piuparts/lib/read_config.sh
+. @sharedir@/piuparts/lib/write_log.sh
+
+get_config_value HTDOCS global output-directory
+
+LOG_OUTPUT=$(mktemp)
+
+$1 >$LOG_OUTPUT 2>&1
+publish_logs $LOG_OUTPUT "" $HTDOCS $1
--- a/master-bin/report_stale_reserved_packages.in
+++ b/master-bin/report_stale_reserved_packages.in
 #!/bin/sh

-# Copyright 2009,2010 Holger Levsen (holger@layer-acht.org)
+# Copyright 2009-2018 Holger Levsen (holger@layer-acht.org)
 # Copyright © 2011-2012 Andreas Beckmann (anbe@debian.org)
 #
 # This program is free software; you can redistribute it and/or modify it
@@ -16,12 +16,12 @@
 # You should have received a copy of the GNU General Public License along
 # with this program. If not, see <https://www.gnu.org/licenses/>

-
 . @sharedir@/piuparts/lib/read_config.sh
+. @sharedir@/piuparts/lib/write_log.sh

 get_config_value MASTER global master-directory
 get_config_value SECTIONS global sections
-
+get_config_value HTDOCS global output-directory

 #
 # find packages which have been reserved for more than one day and unschedule them
@@ -34,11 +34,13 @@ for SECTION in $SECTIONS ; do
 	find $MASTER/$SECTION/reserved/ -mtime +$DAYS -name "*.log" 2>/dev/null >> $LOGS
 done
 if [ -s $LOGS ] ; then
-	echo "Stale reserved packages detected, which have been reserved more then $DAYS days ago!"
-	echo "These packages have been unscheduled."
-	echo
+	LOG_OUTPUT=$(mktemp)
+	echo "Stale reserved packages detected, which have been reserved more then $DAYS days ago!" > $LOG_OUTPUT
+	echo "These packages have been unscheduled." >> $LOG_OUTPUT
+	echo >> $LOG_OUTPUT
 	for package_log in $(cat $LOGS) ; do
-		rm -fv $package_log
+		rm -fv $package_log >> $LOG_OUTPUT
 	done
+	publish_logs $LOG_OUTPUT "" $HTDOCS report_stale_reserved_packages
 fi
 rm $LOGS
--- a/master-bin/report_untestable_packages.in
+++ b/master-bin/report_untestable_packages.in
@@ -18,12 +18,16 @@


 . @sharedir@/piuparts/lib/read_config.sh
+. @sharedir@/piuparts/lib/write_log.sh

 get_config_value MASTER global master-directory
 get_config_value SECTIONS global sections
+get_config_value HTDOCS global output-directory
 get_config_value DAYS global reschedule-untestable-days 7

 STARTDATE=$(date -u +%s)
+LOG_OUTPUT=$(mktemp)
+LOG_PREFIX=$(mktemp)

 #
 # find packages which have been in untestable for more than $DAYS days and reschedule them for testing
@@ -41,13 +45,20 @@ done
 if [ -s $LOGS ] ; then
 	FINALDATE=$(date -u +%s)
 	RUNTIME=$(date -u -d "0 $FINALDATE seconds - $STARTDATE seconds" +%T)
-	echo "Runtime: $RUNTIME"
-	echo
-	echo "Untestable packages detected, which have been tested more than $DAYS days ago!"
-	echo "These packages have been rescheduled for piuparts testing."
-	echo
-	for package_log in $(cat $LOGS) ; do
-		rm -fv $package_log | sed "s#$MASTER/##g" | tr -d "'"
-	done
+	(
+		echo "Untestable packages detected, which have been tested more than $DAYS days ago!"
+		echo "These packages have been rescheduled for piuparts testing."
+		echo
+	) > $LOG_PREFIX
+	(
+		echo "$(date -u)"
+		echo "Runtime: $RUNTIME"
+		echo
+		for package_log in $(cat $LOGS) ; do
+			rm -fv $package_log | sed "s#$MASTER/##g" | tr -d "'"
+		done
+		echo
+	) > $LOG_OUTPUT
+	publish_logs $LOG_OUTPUT $LOG_PREFIX $HTDOCS report_untestable_packages
 fi
 rm $LOGS
--- a/master-bin/reschedule_oldest_logs.in
+++ b/master-bin/reschedule_oldest_logs.in
 #!/bin/sh

-# Copyright © 2009-2017 Holger Levsen (holger@layer-acht.org)
+# Copyright © 2009-2018 Holger Levsen (holger@layer-acht.org)
 # Copyright © 2011-2012 Andreas Beckmann (anbe@debian.org)
 #
 # This program is free software; you can redistribute it and/or modify it
@@ -18,12 +18,15 @@


 . @sharedir@/piuparts/lib/read_config.sh
+. @sharedir@/piuparts/lib/write_log.sh

 get_config_value MASTER global master-directory
 get_config_value SECTIONS global sections
+get_config_value HTDOCS global output-directory

 STARTDATE=$(date -u +%s)
 OUTPUT=$(mktemp)
+LOG_OUTPUT=$(mktemp)

 #
 # reschedule 200 oldest log files, if they are older than 180 days
@@ -38,7 +41,6 @@ get_config_value COUNT      global reschedule-old-count 200
 get_config_value EXPIRE_FAIL_AGE global expire-fail-days 0
 get_config_value FAIL_AGE   global reschedule-fail-days 30
 get_config_value FAIL_COUNT global reschedule-fail-count 25
-
 get_config_value AUTO_RESCHEDULE	global auto-reschedule yes


@@ -187,22 +189,27 @@ rm $QUEUED

 FINALDATE=$(date -u +%s)
 RUNTIME=$(date -u -d "0 $FINALDATE seconds - $STARTDATE seconds" +%T)
-echo "Runtime: $RUNTIME"
-echo
-
-if [ "$TOTAL" -gt "0" ]; then
-	echo "Rescheduled $TOTAL logs."
-fi
-if [ "$TOTAL_EXPIRED" -gt "0" ]; then
-	echo "Deleted $TOTAL_EXPIRED expired logs."
-fi
-if [ "$UNSCHEDULE" -gt "0" ]; then
-	echo "Cancelled $UNSCHEDULE outdated rescheduling requests."
-fi
-if [ "$TOTAL_QUEUED" -gt "0" ]; then
-	echo "Queued logs: $TOTAL_QUEUED"
-fi
-echo
-
-cat $OUTPUT
+(
+	echo "$(date -u)"
+	echo "Runtime: $RUNTIME"
+	echo
+	if [ "$TOTAL" -gt "0" ]; then
+		echo "Rescheduled $TOTAL logs."
+	fi
+	if [ "$TOTAL_EXPIRED" -gt "0" ]; then
+		echo "Deleted $TOTAL_EXPIRED expired logs."
+	fi
+	if [ "$UNSCHEDULE" -gt "0" ]; then
+		echo "Cancelled $UNSCHEDULE outdated rescheduling requests."
+	fi
+	if [ "$TOTAL_QUEUED" -gt "0" ]; then
+		echo "Queued logs: $TOTAL_QUEUED"
+	fi
+	echo
+	cat $OUTPUT
+	echo "##################################################################################################################" >> $OUTPUT
+	echo "##################################################################################################################" >> $OUTPUT
+	echo
+) > $LOG_OUTPUT
+publish_logs $LOG_OUTPUT "" $HTDOCS reschedule_oldest_logs
 rm -f $OUTPUT