Upgrading to GitLab 11.10.0. Expect errors and see debian-infrastructure-announce@lists.debian.org for further information.

Import Upstream version 1.0a

* Axel -- A lighter download accelerator for Linux and other Unices. *
* *
* Copyright 2001 Wilmer van der Gaast *
/* Short API description */
Until version 0.97, a lot of Axel downloading code was 'stuck' in main().
This made the development of alternate (ie graphical) interfaces to the
program quite difficult. That's why Axel 0.97 is a major redesign: All the
downloading code is out of main() now. Writing your own downloader which
uses Axel should not be too difficult now.
This document contains basic instructions on how to write a program which
uses the Axel >=0.97 code to download data.
Some work needs to be done before I can convert axel into a library. I don't
know whether I'll do it at all.. So this API description is only useful if
you want to create an alternate interface for the program, at the moment.
Later on, I might change this. A Perl port of Axel would be nice too. :-)
/* The structures */
If you want to use Axel, you should have all the *.[ch] files in your
program's directory (or subdir, whatever you want...) and include the axel.h
file into your program. Then, the following structures and functions will be
typedef struct
conn_t *conn;
conf_t conf[1];
char filename[MAX_STRING];
double start_time;
int next_state, finish_time;
int bytes_done, start_byte, size;
int bytes_per_second;
int delay_time;
int outfd;
int ready;
message_t *message;
url_t *url;
} axel_t;
This is probably the most important structure.. Each axel structure can
handle a separate download, each with a variable amount of connections.
There is no maximum amount of connections hard-coded into the program
anymore, by the way. The way conn_t and conf_t structures work is not very
important for most people, it's mainly important for internal use. You /can/
use those structures, if you want, they're not that complex...
The filename string is set correctly by axel_new(). If you want data to
be put into a different file, you can change the variable /after/ calling
axel_new(), and /before/ calling axel_open(). The string can also include
a full pathname.
start_time contains the time at which the download started. Not very
interesting for you, probably. Neither should next_state be very important,
it just contains the time at which the next state file should be saved.
(State files are important for resuming support, as described in the README
file..) finish_time might be interesting, though. It contains the estimated
time at which the download should be finished.
bytes_done contains the number of bytes downloaded for this file, size
contains the total file size. start_byte should be zero, usually, unless
you're resuming a download.
The code also calculates the average speed. This speed is put in the
bytes_per_second variable.
delay_time is not interesting at all. It's just used for the code which
tries to slow down the download. You shouldn't really touch outfd either,
it contains the file descriptor of the local file.
ready is set to non-zero as soon as all data is downloaded, or as soon as
something goes wrong. You shouldn't call axel_do() anymore, when ready is
Last but not least, message. This is a linked list of messages to the user.
You, as the programmer, may decide what to do with them. You can just
destroy them (don't just ignore them, the messages do eat memory!) or you
can log/display them. The structure is very simple, and I hope this is clear
typedef struct {
void *next;
char text[MAX_STRING];
} message_t;
Just don't forget to free() the message structures after printing them, and
set axel->message to NULL to prevent crashes. See the print_messages()
function in text.c for an example.
message used to be the last, but I added url. It's a linked list as well,
and in fact url_t == message_t. Not really of any importance, though. This
element contains a number of URL's that'll be used by Axel for the download.
The program can use multiple mirrors at the same time. This structure is
filled in by axel_new, you shouldn't touch it yourself.
/* The functions */
int conf_init( conf_t *conf );
Axel needs some settings. Your program has to allocate a conf_t structure
and initialize it using this function. It sets some defaults, and then it
scans your environment variables for some settings, and it tries to read
a system-wide and personal user configuration file.
axel_t *axel_new( conf_t *conf, char count, char *url );
axel_t *axel_new( conf_t *conf, char count, search_t *urls );
axel_new() allocates a new axel_t structure. You should pass a configuration
structure and an URL. A pointer to a new axel_t structure will be returned.
axel->filename is set now. You can change it, if you want data to be stored
to a different file. Changing axel->filename after calling axel_open does
not make sense, so be quick. :-)
If you want axel to download from more than one mirror at once, you can use
the second syntax. A search_t structure can be generated by the search_*
functions. If you use the second syntax, count should contain the number of
mirrors to be used from the structure. If you just want to pass a string
with one URL (first syntax), count should be zero. Please note that all the
mirrors passed to axel_new() should support acceleration. The support check
should be done before downloading, which isn't much of a problem because
search_getspeeds does it automatically.
The ready element of the returned structure is set to one if nothing goes
wrong. If it's zero, you shouldn't use the returned structure for anything
else than displaying the error message(s) and closing it.
int axel_open( axel_t *axel );
axel_open() opens a local file to store downloaded data. Returns non-zero if
nothing goes wrong. If anything goes wrong, you should still call
axel_close() to clean things up. This is not done automatically, so that you
can read any message still left in the structure.
void axel_start( axel_t *axel );
axel_start() starts the actual downloading. Normally, nothing should go
wrong during this call, so it does not return anything.
void axel_do( axel_t *axel );
axel_do() should be called regularly (ie as often as possible...) to handle
any incoming data. You don't have to do anything else, all data is stored in
the local file automatically. You should stop calling this one as soon as
axel->ready is set. Or you can stop calling it yourself, that's possible.
Just don't forget to call axel_close()!
void axel_close( axel_t *axel );
If you want to stop downloading (ie if the download is complete) you should
deallocate the axel_t structure using this function. Any connection still
open will be closed and deallocated, all messages in the structure are
deleted. You should always call this one when you're ready, if you don't
want to waste memory.
double gettime();
This one is just a 'bonus'... I use it myself in text.c and axel.c, so I
decided to make it global. It just returns the actual time, but with more
/* filesearcher.com interface */
If you want to search for a faster mirror to download your file, or if you
want to download from more than one server at once, you can use this
interface. It's quite simple. You should create an array of this type:
typedef struct
char url[MAX_STRING];
double speed_start_time;
int speed, size;
pthread_t speed_thread[1];
conf_t *conf;
} search_t;
And it's wise to memset() it to zero before you start, btw. You also have to
set the conf pointer for the first index of the array. Other fields will
be filled in by these functions:
int search_makelist( search_t *results, char *url );
This function checks your URL, fetches the file size (needed for the search)
and queries the ftpsearcher.com server for any mirror of this file. This one
is finished in a few seconds on my system. It returns the number of mirrors
found. Please note that, after calling this function, the first index of
your search_t array contains the URL you used as an argument to this
function. The speed field is filled in already for that one.
int search_getspeeds( search_t *results, int count );
This is quite time consuming. It tries all the URL's from the list, and
checks the speed. URL's which do not exist, or URL's on non-supported
servers are marked as bad, they can't be used. This is more time-consuming
than a simple ping (it takes about twenty seconds on my system, but it
heavily depends on the connection and your settings), but it makes sure only
usable URL's are passed to the downloader. The function returns the number
of not-bad servers.
void search_sortlist( search_t *results, int count );
It's very wise to sort the list of mirrors using this function before
passing it to axel_new(). The fastest URL will be put on top of the list,
bad URL's will be put at the bottom. Please note that count has to be the
total number of servers, returned by search_makelist(), and not just the
number of not-bad servers returned by search_getspeed().
Version 1.0a:
- gstrip on Solaris/SPARC breaks the binary, so stripping is made optional
(but enabled by default!) and /usr/ccs/bin/strip is used instead of
- Fixed a small (not harmful) errorcode interpretation bug in the ftp_size
- Downloading from unsupported sites works better now.
- Added support for downloading using more than one local network interface.
- Fixed a potential SIGSEGV bug. Would only happen with about more than 64
connections usually. Still strange things can happen with too many
connections when using Linux..
- Hopefully fixed the problem with downloading from forwarding URL's.
- HTTP %-escapes are handled now.
- Alternate progress indicator with estimation of remaining download time
- Changed the return-code a bit, see man-page for details.
Finished Feb 19 2002
Version 1.0:
- Fixed a reconnect problem: Check for stalled connections was skipped by
previous versions if there is no active connection.
- Solaris does not have www in /etc/services which confused Axel. Fixed.
- Created a new build system.
- install utility not used anymore: Solaris' install does weird things.
- Added support for Cygwin and Solaris.
- Corrected a little problem in de.po.
- Thrown out the packaging stuff.
Finished Dec 6 2001
Version 0.99b brings you:
- Debian package bugfix.
- Restored i18n support.
Finished Nov 16 2001
Version 0.99a brings you:
- Small bugfix for dumb HTTP bug.
Finished Nov 10 2001
Version 0.99 brings you:
- Improved speed limiter. (For low speeds a smaller buffer works better,
so the buffer is resized automatically for low speeds.)
- Some FTP servers don't ask for a password which confused ftp.c. Fixed.
- Some HTTP servers send chunked data when using HTTP/1.1. So I went back
to HTTP/1.0.. (This also fixes the occasional filesearching.com problem)
- Even more problems with FTP server reply codes, but they must be fixed
now, Axel's RFC compliant.
- For HTTP downloads a Range: header is not sent when downloading starting
at byte 0. This is just a work-around for a problem with a weird webserver
which sends corrupted data when a Range: header is sent. It's just a
work-around for a rare problem, it does not really fix anything.
- RPM package for axel-kapt added.
Finished Nov 9 2001
Version 0.98 brings you:
- Fixed the weird percentage indicator bug. (Was buggy for large files,
did not affect the downloaded data.)
- For single connection downloads from Apache: Apache returns a 200 result
code when the specified file range is the complete file. Axel handles
this correctly now.
- Roxen FTP servers return the address and port number without brackets
after a passive command. This is RFC-compliant but quite unique. But now
handled correctly.
- Fixed some things to make it work on Darwin again.
- 'Upgraded' HTTP requests to HTTP/1.1. Tried this to fix a download
corruption bug for downloads from archive.progeny.com, but it did not
work. wget's resume has exactly the same problem. But using HTTP/1.1 is
more compliant (because in fact HTTP/1.0 does not support Ranges) so I'll
keep it this way..
- Previous version used to delete the statefile in some cases. Fixed.
Finished Nov 3 2001
Version 0.97 (Bitcrusher) brings you:
- Major redesign: Moved a lot of code from main() to separate functions,
it should make it easier to create different interfaces for the program.
- All those ifdefs were a mess, they don't exist anymore. Separate threads
for setting up connections are used by default now. Version 0.96 will not
disappear, by the way.
- HTTP HEAD request not used anymore: Squid's headers are incorrect when
using the HEAD request.
- conn_disconnect did not work for FTP connections through HTTP proxies.
- Documentation fix, sort of: The example configuration file still said
proxies are unsupported. But they are supported for quite some time
- Wrote a small (Small? Larger than any other doc.. :-) description of the
Axel API.
- Added finish_time code. (Credits to sjoerd@huiswerkservice.nl)
- Calling conn_setup without calling conn_init first also works when using
a proxy now.
- A client for filesearching.com. You can use it to search for mirrors and
download from more than mirror at once.
- Fixed another segfault bug which did not show up on my own system...
Also fixed by 0.96a.
- Global error string is gone. Unusable in threaded programs.
- The -V switch stopped working some time ago because I forgot to put it
in the getopt string. Now it's back, alive and kickin'...
- TYPE I should be done quite early. Some servers return weird file sizes
when still in ASCII mode.
- ftp.c (ftp_wait) sometimes resized conn->message to below MAX_STRING
which is Not Good(tm).
- I18N support is a bit broken at this time, it'll be fixed later.
- Tidied up the man-page a bit.
- Removed config.h file, -D flags used again.
- Added axel-kapt interface.
- Changed syntax: Local file must be specified using the -o option now.
This allows the user to specify more than one URL and all of them will
be used for the download. A local directory can be passed as well, the
program will append the correct filename.
- Fixed a bug which caused the program not to be able to download files
for which only one byte has to be downloaded for one of the connections.
- Why bitcrusher? Just because I liked to have a code name for this
release... Bit crusher is a name of a musical group here in the
Netherlands, and it's a nice name for a downloader as well, I hope...
Finished Oct 25 2001
Version 0.96 brings you:
- Fixed a terrible bug which caused any FTP download to corrupt. I promise
I will test the program before any next release. :-(( HTTP did work in
Finished Aug 14 2001 (Why is this fix so late? Because the actual release
of version 0.95 was only last Friday/Saturday...)
And yes, I tested it now. It works now. HTTP and FTP.
Version 0.95 brings you:
- An important bugfix: When bringing up the connection failed, the program
used to be unable to reconnect. :(
- Small changes to make the program compile on FreeBSD and Darwin.
- Support check for FTP servers is done only once now.
- SIZE command is really used now.
- Fixed a SIGINT-does-not-abort problem. Btw: Ctrl-\ (SIGQUIT) always works!
- Connection status messages are not displayed by default. You can enable
them with the verbose-option.
Finished Aug 7 2001
Version 0.94 brings you:
- 'make install' uses install instead of mkdir/cp now.
- Added 'make uninstall' option.
- Added more explanations to axelrc.example.
- It uses the HTTP HEAD request now. Didn't know that one before. :)
- Debian packaging stuff and RPM .spec file included by default.
- select() problem now really understood... The real point was, that
sometimes select() was called with an empty fd set. Now I solved the
problem in a more 'useful' way.
Finished Jun 26 2001
Version 0.93 brings you:
- A compile-time option to remove all the multi-connection stuff. Program
works without state files then, and it's a bit smaller, just in case you
need a very small program and if you don't believe in acceleration. :)
- The SIZE command is now used as long as the URL does not contain wildcards.
Because FTP servers are just too different. :(
- You can do FTP downloads through HTTP proxies now.
- The weird initial 1-second delay which happened sometimes does not exist
anymore: It was because of select(), which does not return immediately
if there's data on a socket before the call starts. My first solution
is using a lower timeout, I hope there's a better solution available...
- Local file existence check.
- Small bug fixed in conf.c.
Finished May 22 2001
Version 0.92 brings you:
- A credits file!! ;)
- A German translation. Herrman J. Beckers: Thanks a lot!
- ftp.c should understand weird Macintosh FTP servers too, now.
- Connections are initialized in a different thread because then the program
can go on downloading data from other connections while setting up. Quick
hack, but it works.
- config.h contains the configuration definitions now.
- A URL - can be specified. The program will read a URL from stdin. Might
be useful if you don't want other people to see the URL in the 'ps aux'
output. (Think about passwords in URLs...)
Finished May 11 2001
Version 0.91 brings you:
- A man page.
- A quiet mode.
- A Debian package. (0.9 .deb exists too, but that was after the 'official'
0.9 release..)
- Made the sizes/times displayed after downloading more human-readable.
- Corrected some stupid things in the nl.po file.
- No bug in ftp_wait anymore, (or at least one bug less ;) the program was a
bit too late with the realloc() sometimes. I just hate those multi-line
replies.. :(
- HTTP proxy support. no_proxy configuration flag also in use.
- Support for empty configuration strings.
- URL parser understands wrongly formatted URLs like slashdot.org. (instead
of the correct http://slashdot.org/)
Finished Apr 30 2001
Version 0.9 brings you:
- See the README for all the old features.
- Internationalization support.
- Clearer error messages.
- Probably some bug fixes too.
- A highly sophisticated Makefile.. ;)
Finished Apr 22 2001
This diff is collapsed.
An not-quite-sorted list of people who helped somehow:
- Hermann J. Beckers <hjb-rheine@t-online.de>
German translation.
- Martin Herrman <m.herrman@student.tue.nl>
Requested the human-readable time/size values.
Advertising in the Dutch Linux User's Manual. (http://2mypage.cjb.net/)
- Marten Klencke <mklencke@gmx.net>
Early tester.
- Danny Oude Bos <broilie@mac.com>
For having an iMac with a very badly-behaving FTP server...
- Ralph Slooten <ralph.slooten@quicknet.nl>
For creating the initial native RPM packages instead of my alienated
- Robert <robert@allyourbass.org>
For patching the program to make it work on FreeBSD and Darwin.
For finding some very stupid bugs.
- Sjoerd Hemminga <sjoerd@huiswerkservice.nl>
For adding the finish_time feature. It's not yet in the user interface,
For writing axelq.
- Paul Evans <pevans@users.sourceforge.net>
For being a very good beta tester.
For creating axel-kapt.
- Justin A <justin@bouncybouncy.net>
For some testing and for the new (multi-URL) syntax idea.
- Sebastian Ritterbusch <Sebastian@Ritterbusch.de>
For some interesting ideas for the new versions.
For writing the alternate progress indicator.
## Makefile for Axel ##
## ##
## Copyright 2001 Lintux ##
include Makefile.settings
.SUFFIXES: .po .mo
# Add your translation here..
MOFILES = nl.mo de.mo
all: $(OUTFILE)
install: install-bin install-etc install-man
uninstall: uninstall-bin uninstall-etc uninstall-man
ifdef I18N
all: $(MOFILES)
install: install-i18n
uninstall: uninstall-i18n
rm -f *.o $(OUTFILE) search core *.mo
distclean: clean
rm -f Makefile.settings config.h
mkdir -p $(DESTDIR)$(MANDIR)/man1/
cp axel.1 $(DESTDIR)$(MANDIR)/man1/axel.1
rm -f $(MANDIR)/man1/axel.1
mkdir -p $(DESTDIR)$(ETCDIR)/
cp axelrc.example $(DESTDIR)$(ETCDIR)/axelrc
rm -f $(ETCDIR)/axelrc
$(OUTFILE): axel.o conf.o conn.o ftp.o http.o search.o tcp.o text.o
$(CC) axel.o conf.o conn.o ftp.o http.o search.o tcp.o text.o -o $(OUTFILE) $(LFLAGS)
ifndef DEBUG
$(CC) -c $*.c -o $*.o -Wall $(CFLAGS)
mkdir -p $(DESTDIR)$(BINDIR)/
rm -f $(BINDIR)/$(OUTFILE)
tar: distclean
x=`pwd | sed -e 's/\/.*\///'`; \
cd ..; \
tar czf $$x.tar.gz $$x
### I18N FILES
-@mv $@ $@.bak
xgettext -k_ -o$@ *.[ch]
@if [ -e $@.bak ]; then \
echo -n Merging files...; \
msgmerge -vo $@.combo $@.bak $@; \
rm -f $@ $@.bak; \
mv $@.combo $@; \
.po.mo: $@.po
msgfmt -vo $@ $*.po
@echo Installing locale files...
@for i in $(MOFILES); do \
mkdir -p $(DESTDIR)$(LOCALE)/`echo $$i | cut -d. -f1`/LC_MESSAGES/; \
cp $$i $(DESTDIR)$(LOCALE)/`echo $$i | cut -d. -f1`/LC_MESSAGES/axel.mo; \
cd $(LOCALE); find . -name axel.mo -exec 'rm' '{}' ';'
* Supported architectures *
My primary development platform is Debian Potato at the moment, the program
should compile on any decent Linux system. Additionaly, it should compile
(and run) on BSD, Solaris, Darwin (Mac OS X) and Win32 (Cygwin) systems. If
you want to run Axel on AtheOS, please try an older Axel (0.96b), since that
version can be compiled without pthreads. AtheOS does implement threads, but
not (yet) pthreads.
If the configure script does weird things on your system, please do warn me!
I test it on as many machines and OS'es as possible, but still anything can
go wrong.
* How to install/use *
Run the configure script (you can supply some options if you want, try
'./configure --help' for more info) and then run make. The program should
compile then. There are no special requirements for Axel. You can install
the program using 'make install' or you can just run it from the current
directory. You can copy the axelrc.example file to ~/.axelrc then, if you
want to change some of the settings.
Run the program like this:
axel ftp://ftp.nl.kernel.org/pub/linux/kernel/v2.2/linux-2.2.20.tar.bz2
For a simple single-server-multiple-connection download, or:
axel ftp://ftp.{nl,be,de}.kernel.org/pub/linux/kernel/v2.2/linux-2.2.20.tar.bz2
If you want to use those three servers for the download. The program can do
an automatic search for FTP mirrors as well (filesearching.com), but that's
not yet perfect... Just try the -S option. (The line above should at least
work with the Bash shell. You can type all the mirrors by hand as well, if
you really want to, and/or of necessary...)
Just one other thing you should keep in mind when using this program: Some
FTP operators don't like people who use download accelerators. To quote an
administrator at Progeny.Com in a mail to me:
<quote> Additionally, I should mention that accelerated downloads are
discouraged as I consider them abusive. </quote>
And he certainly has a point.. Using more than one server at once is a fine
solution IMHO, so please use this feature if possible!
Wilmer van der Gaast. <lintux@lintux.cx>
.\"man-page for Axel
.\"Derived from the man-page example in the wonderful book called Beginning
.\"Linux Programming, written by Richard Stone and Neil Matthew.
\fBAxel\fP \- A light download accelerator for Linux.
.B axel
[\fIOPTIONS\fP] \fIurl1\fP [\fIurl2\fP] [\fIurl...\fP]
Axel is a program that downloads a file from a FTP or HTTP server through
multiple connection, each connection downloads its own part of the file.
Unlike most other programs, Axel downloads all the data directly to the
destination file, using one single thread. It just saves some time at the
end because the program doesn't have to concatenate all the downloaded
One argument is required, the URL to the file you want to download. When
downloading from FTP, the filename may contain wildcards and the program
will try to resolve the full filename. Multiple URL's can be specified
as well and the program will use all those URL's for the download. Please
note that the program does not check whether the files are equal.
Other options:
\fB\-\-max\-speed=x\fP, \fB\-s\ x\fP
You can specify a speed (bytes per second) here and Axel will try
to keep the average speed around this speed. Useful if you don't want
the program to suck up all of your bandwidth.
\fB\-\-num\-connections=x\fP, \fB\-n\ x\fP
You can specify an alternative number of connections here.
\fB\-\-output=x\fP, \fB\-o\ x\fP
Downloaded data will be put in a local file with the same name,
unless you specify a different name using this option. You can
specify a directory as well, the program will append the filename.
\fB\-\-search[=x]\fP, \fB-S [x]\fP
Axel can do a search for mirrors using the filesearching.com search
engine. This search will be done if you use this option. You can specify how
many different mirrors should be used for the download as well.
The search for mirrors can be time\-consuming because the program tests
every server's speed, and it checks whether the file's still available.
\fB\-\-no\-proxy\fP, \fB\-N\fP
Don't use any proxy server to download the file. Not possible when a
transparent proxy is active somewhere, of course.
If you want to see more status messages, you can use this option. Use it
more than once if you want to see more.
\fB\-\-quiet\fP, \fB-q\fP
No output to stdout.
\fB\-\-alternate\fP, \fB-a\fP
This will show an alternate progress indicator. A bar displays the progress
and status of the different threads, along with current speed and an
estimate for the remaining download time.
\fB\-\-help\fP, \fB\-h\fP
A brief summary of all the options.
\fB\-\-version\fP, \fB\-V\fP
Get version information.
Long (double dash) options are supported only if your platform knows about
the getopt_long call. If it does not (like *BSD), only the short options can
be used.
The program returns 0 when the download was succesful, 1 if something really
went wrong and 2 if the download was interrupted. If something else comes back,
it must be a bug..
axel ftp://ftp.{be,nl,uk,de}.kernel.org/pub/linux/kernel/
This will use the Belgian, Dutch, English and German kernel.org mirrors to
download a Linux 2.4.17 kernel image.
axel -S 4 ftp://ftp.kernel.org/pub/linux/kernel/v2.4/
This will do a search for the linux-2.4.17.tar.bz2 file on filesearching.com
and it'll use the four (if possible) fastest mirrors for the download.
(Possibly including ftp.kernel.org)
(Of course, the commands are a single line, but they're too long to fit on
one line in this page.)
\fI/etc/axelrc\fP System-wide configuration file
\fI~/.axelrc\fP Personal configuration file
These files are not documented in a man-page, but the example file which
comes with the program contains enough information, I hope. The position
of the system-wide configuration file might be different.
Axel is Copyright 2001-2002 Wilmer van der Gaast.
I'm sure there are some bugs left somewhere, please tell me about them and
I will try to fix them.
A known bug is that the program does weird things when downloading using
hundreds of connections. Well, just don't do that.
Wilmer van der Gaast. (lintux@lintux.cx)
This diff is collapsed.
* Axel -- A lighter download accelerator for Linux and other Unices. *
* *
* Copyright 2001 Wilmer van der Gaast *
/* Main include file */