Commit b2bb87e7 authored by Thaddeus Black's avatar Thaddeus Black Committed by Colin Watson

Import 1:2.18

* New maintainer, on a trial basis. We'll see how it goes. Thanks,
  Joey, for four years recently filling the gap; and earlier, for
  giving us the software in the first place. Joerg, Goswin, your work
  is appreciated (see? it continues to be used). Frans, we still
  miss you.
  Closes: #768532
* Somewhat refactored the code that distinguishes between
  download methods.
* Implemented the option --method=file, which -- besides being useful
  in its own right -- may also make testing and debugging easier.
* In the unpacked source, implemented "make -C test", which builds
  a small, mock Debian archive against which one can test and debug
  the program.
  Closes: #787760
* Applied Kees Cook's patch to add a retry cycle for rsync
  connection failures, implementing the cycle optionally via the
  new, experimental --retry-rsync-packages option.
  Closes: #576577
* Per Joey's advice, did not apply patch 619363 regarding
  the --ignore-missing-release option.
  Closes: #619363
* Appreciated patch 628779 to work around an out-of-sync parent
  mirror, but did not apply it.  Joey explains why in the bug log.
  Closes: #628779
* Noted per Goswin's advice that the Debian archive ceased to ship
  uncompressed Packages files about 2006.  Whatever trouble debmirror
  might once have had with the 2006 transition is ancient history
  now, so if a bug remains thereto, it probably affects no one and
  won't be fixed.
  Closes: #375381
* Did not yet fix bug #652138.  However, refactored the code that
  downloads Contents files, with the intent to make this bug
  easier later to fix.
* In the debmirror script, as a matter of the new maintainer's
  preference, expanded all leading tabs as spaces.
parent 22aba07d
.PHONY: clean
all: debmirror.1
debmirror.1:
......@@ -5,3 +7,5 @@ debmirror.1:
clean:
rm -f debmirror.1
$(MAKE) -C test $@
debmirror (1:2.18) unstable; urgency=low
* New maintainer, on a trial basis. We'll see how it goes. Thanks,
Joey, for four years recently filling the gap; and earlier, for
giving us the software in the first place. Joerg, Goswin, your work
is appreciated (see? it continues to be used). Frans, we still
miss you.
Closes: #768532
* Somewhat refactored the code that distinguishes between
download methods.
* Implemented the option --method=file, which -- besides being useful
in its own right -- may also make testing and debugging easier.
* In the unpacked source, implemented "make -C test", which builds
a small, mock Debian archive against which one can test and debug
the program.
Closes: #787760
* Applied Kees Cook's patch to add a retry cycle for rsync
connection failures, implementing the cycle optionally via the
new, experimental --retry-rsync-packages option.
Closes: #576577
* Per Joey's advice, did not apply patch 619363 regarding
the --ignore-missing-release option.
Closes: #619363
* Appreciated patch 628779 to work around an out-of-sync parent
mirror, but did not apply it. Joey explains why in the bug log.
Closes: #628779
* Noted per Goswin's advice that the Debian archive ceased to ship
uncompressed Packages files about 2006. Whatever trouble debmirror
might once have had with the 2006 transition is ancient history
now, so if a bug remains thereto, it probably affects no one and
won't be fixed.
Closes: #375381
* Did not yet fix bug #652138. However, refactored the code that
downloads Contents files, with the intent to make this bug
easier later to fix.
* In the debmirror script, as a matter of the new maintainer's
preference, expanded all leading tabs as spaces.
-- Thaddeus H. Black <thb@debian.org> Sat, 13 Jun 2015 00:00:00 -0000
debmirror (1:2.17) unstable; urgency=medium
* Debmirror needs a new maintainer, both in Debian and upstream.
......
Source: debmirror
Section: net
Priority: extra
Maintainer: Debian QA Group <packages@qa.debian.org>
Maintainer: Thaddeus H. Black <thb@debian.org>
Build-Depends: debhelper (>= 7.0.50)
Standards-Version: 3.9.3
Standards-Version: 3.9.6
Vcs-Git: git://git.debian.org/collab-maint/debmirror.git
Vcs-Browser: http://git.debian.org/?p=collab-maint/debmirror.git
......
......@@ -6,6 +6,8 @@ Copyright:
2001-2002 Joerg Wendland <joergland@debian.org>
2003-2007 Goswin von Brederlow <goswin-v-b@web.de>
2009-2010 Frans Pop <fjp@debian.org>
2010-2014 Joey Hess <joeyh@debian.org>
2015 Thaddeus H. Black <thb@debian.org>
License: GPL-2+
The full text of the GPL is distributed as in
/usr/share/common-licenses/GPL-2 on Debian systems.
This diff is collapsed.
#!/usr/bin/make -f
SHELL:=/bin/bash
.PHONY: FORCE clean
debian: FORCE; @script/build-mock-archive
clean:; rm -rf debian
Run "make debian" to make the mock archive ./debian/, or to restore
it to its newly built state. Run "make clean" to delete it.
The mock archive ./debian/ is not to be confused with the
directory ../debian/. The ./debian/ is a small, mock Debian
archive. You can use this archive to test debmirror.
The mock archive's packages are not real packages. You cannot build
or install them. You can however *mirror* them, which is the point.
The mock archive is not a very good mock archive, nor are the scripts
that manage it very good scripts. They merely suffice for some tests,
and may be improved or extended when more tests of thought of. If you
are developing or maintaining debmirror, then feel free to improve or
extend the mock archive and its scripts as you need or wish.
(When these words were written, the mock archive had no non-free
section, but by the time these words are read it might have one. If
the mock archive does have a non-free section, then that section is
mock-non-free -- which is free, insofar as the whole mock archive is
free. If this makes sense to you, then you can stop reading; but if it
confuses then consider the following. It's like an actor in
Shakespeare's Julius Caesar, who is not Caesar, but is an actor who in
the play is named "Caesar". Naming an actor "Caesar" in a play does
not make him Caesar! Likewise, naming software "non-free" in a mock
archive does not make it non-free. As Shakespeare's Juliet almost
said, "What's in a name? That which we call a rose by any other name
would smell as free...." Unfortunately, this rose is named "non-rose,"
more or less, which is especially confusing, but there it is.)
Here is a sample test session:
cd "$DEBMIRROR_SOURCE"
rm -r /tmp/debian
make -C test
./debmirror -r "${PWD}/test/debian" --method=file -d sid -a amd64\
--getcontents --no-check-gpg /tmp/debian
stable-proposed-updates
\ No newline at end of file
Origin: Debian
Label: Debian
Suite: unstable
Codename: sid
Date: Sat, 25 Apr 2015 11:11:17 UTC
Architectures: amd64 i386
Components: main
Description: Debian x.y Unstable - Not Released
This file pretends to be a Contents file.
FILE LOCATION
usr/bin/bar test/bar
usr/bin/foo test/foo
usr/share/foo-data test/foo-data
usr/share/foo-data/data.dat test/foo-data
This file pretends to be a Contents file for i386.
FILE LOCATION
usr/bin/bar test/bar
usr/bin/foo test/foo
usr/share/foo-data test/foo-data
usr/share/foo-data/data.dat test/foo-data
There should be some Contents-source data here.
Package: bar
Version: 1.0-7
Installed-Size: 0
Maintainer: James Bar <jamesbar@debian.org>
Architecture: all
Description: a package that exists for test purposes
Description-md5: 00000000000000000000000000000000
Section: test
Priority: optional
Filename: pool/main/b/bar/bar_1.0-7_all.deb
Size: 0
MD5sum: 00000000000000000000000000000000
SHA1: 0000000000000000000000000000000000000000
SHA256: 0000000000000000000000000000000000000000000000000000000000000000
Package: foo
Version: 1.23-4
Installed-Size: 0
Maintainer: John Foo <johnfoo@debian.org>
Architecture: amd64
Depends: foo-data
Description: an imaginary software suite
Description-md5: 00000000000000000000000000000000
Section: test
Priority: optional
Filename: pool/main/f/foo/foo_1.23-4_amd64.deb
Size: 0
MD5sum: 00000000000000000000000000000000
SHA1: 0000000000000000000000000000000000000000
SHA256: 0000000000000000000000000000000000000000000000000000000000000000
Package: foo-data
Source: foo
Version: 1.23-4
Installed-Size: 0
Maintainer: John Foo <johnfoo@debian.org>
Architecture: all
Depends: bar (>= 1.0), foo-data
Description: data for an imaginary software suite
Description-md5: 00000000000000000000000000000000
Section: test
Priority: optional
Filename: pool/main/f/foo/foo-data_1.23-4_all.deb
Size: 0
MD5sum: 00000000000000000000000000000000
SHA1: 0000000000000000000000000000000000000000
SHA256: 0000000000000000000000000000000000000000000000000000000000000000
Archive: unstable
Origin: Debian
Label: Debian
Component: main
Architecture: amd64
Package: foo-udeb
Source: foo
Version: 1.23-4
Installed-Size: 0
Maintainer: John Foo <johnfoo@debian.org>
Architecture: amd64
Description: an imaginary software suite
Description-md5: 00000000000000000000000000000000
Section: debian-installer
Priority: optional
Filename: pool/main/f/foo/foo-udeb_1.23-4_amd64.udeb
Size: 0
MD5sum: 00000000000000000000000000000000
SHA1: 0000000000000000000000000000000000000000
SHA256: 0000000000000000000000000000000000000000000000000000000000000000
Archive: unstable
Origin: Debian
Label: Debian
Component: main
Architecture: amd64
Package: bar
Description-md5: 00000000000000000000000000000000
Description-en: ein Paket, das zu Testzwecken besteht
Bar ist unexistierenden Software. Du kannst nicht bauen. Du kannst
nicht installieren. Du kannst jedoch *spiegeln* sie, was
der Punkt ist.
Package: foo
Description-md5: 00000000000000000000000000000000
Description-en: eine imaginäre Software-Suie
Was kannst du dich vorstellen? Oh, die Gedanken kannst du denken!
Kannst du dich vorstellen mit der Installation des Pakets? In der
Zwischenheit, versuch es zum spiegeln.
.
Hier ist eine andere Linie der Beschreibungstext, warscheinlich
nicht gut auf Deutch geschriebt.
Package: foo-data
Description-md5: 00000000000000000000000000000000
Description-en: data for an imaginary software suite (German)
Maybe this should be written in German, too.
Package: bar
Description-md5: 00000000000000000000000000000000
Description-en: a package that exists for test purposes
Bar is nonexistent software. You cannot build it. You cannot install it.
You can however *mirror* it, which is the point.
Package: foo
Description-md5: 00000000000000000000000000000000
Description-en: an imaginary software suite
What can you imagine? Oh, the thoughts you can think! Can you imagine
installing and using this package? In the meantime, try mirroring it.
.
Here is another line of descriptive text.
Package: foo-data
Description-md5: 00000000000000000000000000000000
Description-en: data for an imaginary software suite
Imagination apparently wants data. Here it is (isn't).
Package: bar
Description-md5: 00000000000000000000000000000000
Description-en: un paquet qui existe à des fins de test.
Bar est un logiciel inexistant. Vous ne pouvez pas construire.
Vous ne pouvez pas l'installer. Vous pouvez cependant en *miroir*,
ce qui est le point.
Package: foo
Description-md5: 00000000000000000000000000000000
Description-en: une suite logicielle imaginaire
Que pouvez-vous imaginer? Oh, les pensées que vous pouvez penser
Pouvez-vous imaginer installer et d'utiliser ce paquet? En attendant,
essayez de copie miroir.
.
Voici une autre ligne de texte descriptif. (At l'ordinateur traduit
droite? Probablement pas.)
Package: foo-data
Description-md5: 00000000000000000000000000000000
Description-en: data for an imaginary software suite (French)
Maybe this should be written in French, too.
Archive: unstable
Origin: Debian
Label: Debian
Component: main
Architecture: source
Package: bar
Binary: bar
Version: 1.0-7
Maintainer: James Bar <jamesbar@debian.org>
Architecture: all
Standards-Version: 3.9.6
Format: 3.0 (quilt)
Files:
00000000000000000000000000000000 0 bar_1.0-7.dsc
00000000000000000000000000000000 0 bar_1.0.orig.tar.gz
00000000000000000000000000000000 0 bar_1.0-7.debian.tar.gz
Package-List:
bar deb test optional
Directory: pool/main/b/bar
Priority: source
Section: test
Package: foo
Binary: foo, foo-data, foo-udeb
Version: 1.23-4
Maintainer: John Foo <johnfoo@debian.org>
Uploaders: James Bar <jamesbar@debian.org>
Build-Depends: baz (>= 2.0)
Build-Depends-Indep: qux (>= 1.0)
Architecture: any all
Standards-Version: 3.9.6
Format: 3.0 (quilt)
Files:
00000000000000000000000000000000 0 foo_1.23-4.dsc
00000000000000000000000000000000 0 foo_1.23.orig.tar.gz
00000000000000000000000000000000 0 foo_1.23-4.debian.tar.gz
Package-List:
foo deb test optional
foo-data deb test optional
Directory: pool/main/f/foo
Priority: source
Section: test
Origin: Debian
Label: Debian
Suite: proposed-updates
Date: Sat, 25 Apr 2015 11:11:17 UTC
Architectures: amd64
Components: main
Description: Proposed Updates for Debian 8 - Not Released
Package: foo
Version: 1.23+deb8u1-1
Installed-Size: 0
Maintainer: John Foo <johnfoo@debian.org>
Architecture: amd64
Depends: foo-data
Description: an imaginary software suite
Description-md5: 00000000000000000000000000000000
Section: test
Priority: optional
Filename: pool/main/f/foo/foo_1.23-4_amd64.deb
Size: 0
MD5sum: 00000000000000000000000000000000
SHA1: 0000000000000000000000000000000000000000
SHA256: 0000000000000000000000000000000000000000000000000000000000000000
Archive: proposed-updates
Origin: Debian
Label: Debian
Version: 8-updates
Component: main
Architecture: amd64
Package: foo-udeb
Source: foo
Version: 1.23+deb8u1-1
Installed-Size: 0
Maintainer: John Foo <johnfoo@debian.org>
Architecture: amd64
Description: an imaginary software suite
Description-md5: 00000000000000000000000000000000
Section: debian-installer
Priority: optional
Filename: pool/main/f/foo/foo-udeb_1.23+deb8u1-1_amd64.udeb
Size: 0
MD5sum: 00000000000000000000000000000000
SHA1: 0000000000000000000000000000000000000000
SHA256: 0000000000000000000000000000000000000000000000000000000000000000
Archive: proposed-updates
Origin: Debian
Label: Debian
Version: 8-updates
Component: main
Architecture: amd64
Format: 3.0 (quilt)
Source: bar
Binary: bar
Architecture: all
Version: 1.0-7
Maintainer: James Bar <jamesbar@debian.org>
Standards-Version: 3.9.6
Package-List:
bar deb test optional
Files:
00000000000000000000000000000000 0 bar_1.0.orig.tar.gz
00000000000000000000000000000000 0 bar_1.0-7.debian.tar.gz
Davey, Davey Crockett.
Killed him a bar, when he was only three.
Format: 3.0 (quilt)
Source: foo
Binary: foo
Architecture: amd64
Version: 1.23-4
Maintainer: John Foo <johnfoo@debian.org>
Uploaders: James Bar <jamesbar@debian.org>
Standards-Version: 3.9.6
Build-Depends: baz (>= 2.0)
Package-List:
foo deb test optional
foo-data deb test optional
Files:
00000000000000000000000000000000 0 foo_1.23.orig.tar.gz
00000000000000000000000000000000 0 foo_1.23-4.debian.tar.gz
Mon Jun 1 15:43:59 UTC 2015
Using dak v1
Running on host: franck.debian.org
Archive serial: 2015060103
Mon Jun 1 15:59:08 UTC 2015
Used ftpsync version: 20130605
Running on host: rietz.debian.org
Architectures: GUESSED:{ source amd64 armel armhf hurd-i386 i386 ia64 kfreebsd-amd64 kfreebsd-i386 mips mipsel powerpc s390 s390x sparc}
Upstream-mirror: ftp-master.debian.org
dists/sid/main/i18n/Translation-en
dists/sid/main/i18n/Translation-fr
dists/sid/main/i18n/Translation-de
dists/sid/main/Contents-source
dists/sid/main/Contents-amd64
dists/sid/main/Contents-i386
dists/sid/main/Contents-udeb-amd64
dists/stable-proposed-updates/main/Contents-amd64
dists/sid/main/source/Sources
dists/sid/main/binary-amd64/Packages
dists/sid/main/binary-i386/Packages
dists/sid/main/debian-installer/binary-amd64/Packages
dists/stable-proposed-updates/main/binary-amd64/Packages
dists/stable-proposed-updates/main/debian-installer/binary-amd64/Packages
#!/bin/bash -e
cd $(dirname $(readlink -f $0))/..
# Copy the template archive.
rm -rf debian
cp -r script-data/debian-template debian
# Compress some files.
for A in $(find debian/pool -name '*.tar' | sort); do gzip $A; done
for A in $(<script-data/files-to-gzip-early.dat); do
gzip debian/$A
done
for A in $(<script-data/files-to-bzip2-early.dat); do
bzip2 debian/$A
done
for A in $(<script-data/files-to-xz-early.dat); do
xz debian/$A
done
# Finish each source's DSC.
for A in $(find debian/pool -name '*.dsc' | sort); do
TEMP=$(mktemp)
script/sum-dsc $A >$TEMP
cat >$A $TEMP
rm $TEMP
done
# Start scheduling an i386 distribution.
mkdir debian/dists/sid/main/binary-i386
# Finish Packages and Sources files.
TEMP=$(mktemp)
sed -re 's/([ ._-])amd64([ ._-]|$)/\1i386\2/'\
script-data/debian-template/dists/sid/main/binary-amd64/Packages >$TEMP
script/sum-packages\
$TEMP\
script-data/debian-template/dists/sid/main/i18n\
debian/dists/sid/main/binary-i386/Packages\
debian/dists/sid/main/i18n\
debian
rm $TEMP
script/sum-packages\
script-data/debian-template/dists/sid/main/binary-amd64/Packages\
script-data/debian-template/dists/sid/main/i18n\
debian/dists/sid/main/binary-amd64/Packages\
debian/dists/sid/main/i18n\
debian
script/sum-packages\
script-data/debian-template/dists/sid/main/debian-installer/binary-amd64/Packages\
script-data/debian-template/dists/sid/main/i18n\
debian/dists/sid/main/debian-installer/binary-amd64/Packages\
debian/dists/sid/main/i18n\
debian
mkdir debian/dists/stable-proposed-updates/main/i18n
script/sum-packages\
script-data/debian-template/dists/stable-proposed-updates/main/binary-amd64/Packages\
script-data/debian-template/dists/sid/main/i18n\
debian/dists/stable-proposed-updates/main/binary-amd64/Packages\
debian/dists/stable-proposed-updates/main/i18n\
debian
script/sum-packages\
script-data/debian-template/dists/stable-proposed-updates/main/debian-installer/binary-amd64/Packages\
script-data/debian-template/dists/sid/main/i18n\
debian/dists/stable-proposed-updates/main/debian-installer/binary-amd64/Packages\
debian/dists/stable-proposed-updates/main/i18n\
debian
rm -rf debian/dists/stable-proposed-updates/main/i18n
script/sum-sources\
<script-data/debian-template/dists/sid/main/source/Sources\
>debian/dists/sid/main/source/Sources
# Finish scheduling an i386 distribution.
sed -re 's/([ ._-])amd64([ ._-]|$)/\1i386\2/'\
debian/dists/sid/main/binary-amd64/Release\
>debian/dists/sid/main/binary-i386/Release
# Compress other files.
for A in $(<script-data/files-to-gzip.dat); do
gzip -k debian/$A
done
for A in $(<script-data/files-to-bzip2.dat); do
bzip2 -k debian/$A
done
for A in $(<script-data/files-to-xz.dat); do
xz -k debian/$A
done
# Finish the dist's top-level release file.
for DIST in sid stable-proposed-updates; do
(cd debian/dists/$DIST && ../../../script/sum-release >>Release;)
done
# For the latter group of compressed files, remove the originals.
for A in\
$(<script-data/files-to-gzip.dat)\
$(<script-data/files-to-bzip2.dat)\
$(<script-data/files-to-xz.dat);\
do rm debian/$A; done
#!/usr/bin/perl
use warnings;
use strict;
use integer;
# This script prints a checksum in Debian-Release or -Sources format.
our $field_width = 8;
die "usage: $0 [Release|Sources] [SUMMING-COMMAND] [FILE]\n" unless @ARGV == 3;
my ($style, $summing_command, $infile) = @ARGV;
$infile =~ s%^\./%%; # optional
my $sum_output = `$summing_command $infile`;
my ($sum, $filename) = $sum_output =~ /(\S+)\s+(\S+)/;
my @stat = stat $infile;
my $size0 = $stat[7];
my $size = $style eq 'Sources' ? $size0 : sprintf "%${field_width}d", $size0;
my ($basename) = $filename =~ m%/([^/]+)$%;
print " $sum $size " . ($style eq 'Sources' ? $basename : $filename) . "\n";
#!/usr/bin/perl
use warnings;
use strict;
use integer;
our @kind = (
{ cmd => 'sha1sum' , name => 'Checksums-Sha1' },
{ cmd => 'sha256sum', name => 'Checksums-Sha256' },
{ cmd => 'md5sum' , name => 'Files' },
);
@ARGV == 1 or die "usage: $0 [DSC-FILE]\n";
my $dsc = shift @ARGV;
open DSC, '<', $dsc;
chdir $1 if $dsc =~ m%^(.*/).*?$%;
while (<DSC>) {
last if /^Files:/;
print;
}
my @filename;
while (<DSC>) {
/\S/ or last;
my ($sum, $size, $filename) = /(\S+)\s*(\S+)\s*(\S+)/;
push @filename, $filename;
}
close DSC;
for my $kind (@kind) {
print "$kind->{name}:\n";
for my $filename (@filename) {
my $sumout = `$kind->{cmd} $filename`;
my ($sum, $fn) = $sumout =~ /(\S+)\s*(\S+)/;
my @stat = stat $fn;
print " $sum $stat[7] $fn\n";
}
}
print "\n";
#!/usr/bin/perl
use warnings;
use strict;
use integer;
use FindBin;
our @kind = (
{ cmd => 'md5sum' , name => 'MD5sum' },
{ cmd => 'sha1sum' , name => 'SHA1' },
{ cmd => 'sha256sum', name => 'SHA256' },
);
@ARGV == 5 or die "usage: $0"
. " [OLD-PACKAGES-FILE] [OLD-I18N-DIRECTORY]"
. " [NEW-PACKAGES-FILE] [NEW-I18N-DIRECTORY] [NEW-ARCHIVE-DIRECTORY]\n";
my ($file_pkg1, $dir_i18n1, $file_pkg2, $dir_i18n2, $dir_arc) = @ARGV;
my @file_i18n;
opendir DIR, $dir_i18n1 or die "$0: cannot open i18n directory\n";
while(readdir DIR) {
push @file_i18n, $_
unless $_ eq "Translation-en" || $_ eq '.' || $_ eq '..';
}
closedir DIR;
local $/ = '';
my %md; # md5sums of package descriptions
for my $basename ('Translation-en', @file_i18n) {
open F1, '<', "$dir_i18n1/$basename"
or die "$0: failed to open i18n source directory";