README 90.6 KB
Newer Older
1
NAME
2
    IO::AIO - Asynchronous/Advanced Input/Output
3 4 5 6

SYNOPSIS
     use IO::AIO;

7
     aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
        my $fh = shift
           or die "/etc/passwd: $!";
        ...
     };

     aio_unlink "/tmp/file", sub { };

     aio_read $fh, 30000, 1024, $buffer, 0, sub {
        $_[0] > 0 or die "read error: $!";
     };

     # version 2+ has request and group objects
     use IO::AIO 2;

     aioreq_pri 4; # give next request a very high priority
     my $req = aio_unlink "/tmp/file", sub { };
     $req->cancel; # cancel request if still in queue

     my $grp = aio_group sub { print "all stats done\n" };
     add $grp aio_stat "..." for ...;

DESCRIPTION
    This module implements asynchronous I/O using whatever means your
31 32
    operating system supports. It is implemented as an interface to "libeio"
    (<http://software.schmorp.de/pkg/libeio.html>).
33 34 35 36 37 38 39 40 41 42 43 44 45 46

    Asynchronous means that operations that can normally block your program
    (e.g. reading from disk) will be done asynchronously: the operation will
    still block, but you can do something else in the meantime. This is
    extremely useful for programs that need to stay interactive even when
    doing heavy I/O (GUI programs, high performance network servers etc.),
    but can also be used to easily do operations in parallel that are
    normally done sequentially, e.g. stat'ing many files, which is much
    faster on a RAID volume or over NFS when you do a number of stat
    operations concurrently.

    While most of this works on all types of file descriptors (for example
    sockets), using these functions on file descriptors that support
    nonblocking operation (again, sockets, pipes etc.) is very inefficient.
47
    Use an event loop for that (such as the EV module): IO::AIO will
48 49 50 51 52 53 54 55 56 57 58 59
    naturally fit into such an event loop itself.

    In this version, a number of threads are started that execute your
    requests and signal their completion. You don't need thread support in
    perl, and the threads created by this module will not be visible to
    perl. In the future, this module might make use of the native aio
    functions available on many operating systems. However, they are often
    not well-supported or restricted (GNU/Linux doesn't allow them on normal
    files currently, for example), and they would only support aio_read and
    aio_write, so the remaining functionality would have to be implemented
    using threads anyway.

60 61 62 63
    In addition to asynchronous I/O, this module also exports some rather
    arcane interfaces, such as "madvise" or linux's "splice" system call,
    which is why the "A" in "AIO" can also mean *advanced*.

64 65 66 67 68 69
    Although the module will work in the presence of other (Perl-) threads,
    it is currently not reentrant in any way, so use appropriate locking
    yourself, always call "poll_cb" from within the same thread, or never
    call "poll_cb" (or other "aio_" functions) recursively.

  EXAMPLE
70 71
    This is a simple example that uses the EV module and loads /etc/passwd
    asynchronously:
72

73
       use EV;
74 75
       use IO::AIO;

76 77
       # register the IO::AIO callback with EV
       my $aio_w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb;
78 79

       # queue the request to open /etc/passwd
80
       aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98
          my $fh = shift
             or die "error while opening: $!";

          # stat'ing filehandles is generally non-blocking
          my $size = -s $fh;

          # queue a request to read the file
          my $contents;
          aio_read $fh, 0, $size, $contents, 0, sub {
             $_[0] == $size
                or die "short read: $!";

             close $fh;

             # file contents now in $contents
             print $contents;

             # exit event loop and program
99
             EV::break;
100 101 102 103 104 105 106
          };
       };

       # possibly queue up other requests, or open GUI windows,
       # check for sockets etc. etc.

       # process events as long as there are some:
107
       EV::run;
108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152

REQUEST ANATOMY AND LIFETIME
    Every "aio_*" function creates a request. which is a C data structure
    not directly visible to Perl.

    If called in non-void context, every request function returns a Perl
    object representing the request. In void context, nothing is returned,
    which saves a bit of memory.

    The perl object is a fairly standard ref-to-hash object. The hash
    contents are not used by IO::AIO so you are free to store anything you
    like in it.

    During their existance, aio requests travel through the following
    states, in order:

    ready
        Immediately after a request is created it is put into the ready
        state, waiting for a thread to execute it.

    execute
        A thread has accepted the request for processing and is currently
        executing it (e.g. blocking in read).

    pending
        The request has been executed and is waiting for result processing.

        While request submission and execution is fully asynchronous, result
        processing is not and relies on the perl interpreter calling
        "poll_cb" (or another function with the same effect).

    result
        The request results are processed synchronously by "poll_cb".

        The "poll_cb" function will process all outstanding aio requests by
        calling their callbacks, freeing memory associated with them and
        managing any groups they are contained in.

    done
        Request has reached the end of its lifetime and holds no resources
        anymore (except possibly for the Perl object, but its connection to
        the actual aio request is severed and calling its methods will
        either do nothing or result in a runtime error).

FUNCTIONS
153
  QUICK OVERVIEW
154 155
    This section simply lists the prototypes most of the functions for quick
    reference. See the following sections for function-by-function
156 157
    documentation.

158
       aio_wd $pathname, $callback->($wd)
159 160
       aio_open $pathname, $flags, $mode, $callback->($fh)
       aio_close $fh, $callback->($status)
161
       aio_seek  $fh,$offset,$whence, $callback->($offs)
162 163 164 165 166 167 168 169 170 171
       aio_read  $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
       aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
       aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
       aio_readahead $fh,$offset,$length, $callback->($retval)
       aio_stat  $fh_or_path, $callback->($status)
       aio_lstat $fh, $callback->($status)
       aio_statvfs $fh_or_path, $callback->($statvfs)
       aio_utime $fh_or_path, $atime, $mtime, $callback->($status)
       aio_chown $fh_or_path, $uid, $gid, $callback->($status)
       aio_chmod $fh_or_path, $mode, $callback->($status)
172
       aio_truncate $fh_or_path, $offset, $callback->($status)
173 174
       aio_allocate $fh, $mode, $offset, $len, $callback->($status)
       aio_fiemap $fh, $start, $length, $flags, $count, $cb->(\@extents)
175
       aio_unlink $pathname, $callback->($status)
176
       aio_mknod $pathname, $mode, $dev, $callback->($status)
177 178
       aio_link $srcpath, $dstpath, $callback->($status)
       aio_symlink $srcpath, $dstpath, $callback->($status)
179
       aio_readlink $pathname, $callback->($link)
180
       aio_realpath $pathname, $callback->($path)
181
       aio_rename $srcpath, $dstpath, $callback->($status)
182
       aio_rename2 $srcpath, $dstpath, $flags, $callback->($status)
183 184 185 186 187 188
       aio_mkdir $pathname, $mode, $callback->($status)
       aio_rmdir $pathname, $callback->($status)
       aio_readdir $pathname, $callback->($entries)
       aio_readdirx $pathname, $flags, $callback->($entries, $flags)
          IO::AIO::READDIR_DENTS IO::AIO::READDIR_DIRS_FIRST
          IO::AIO::READDIR_STAT_ORDER IO::AIO::READDIR_FOUND_UNKNOWN
189 190
       aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs)
       aio_load $pathname, $data, $callback->($status)
191 192
       aio_copy $srcpath, $dstpath, $callback->($status)
       aio_move $srcpath, $dstpath, $callback->($status)
193
       aio_rmtree $pathname, $callback->($status)
194 195
       aio_fcntl $fh, $cmd, $arg, $callback->($status)
       aio_ioctl $fh, $request, $buf, $callback->($status)
196
       aio_sync $callback->($status)
197
       aio_syncfs $fh, $callback->($status)
198 199 200
       aio_fsync $fh, $callback->($status)
       aio_fdatasync $fh, $callback->($status)
       aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
201
       aio_pathsync $pathname, $callback->($status)
202
       aio_msync $scalar, $offset = 0, $length = undef, flags = MS_SYNC, $callback->($status)
203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220
       aio_mtouch $scalar, $offset = 0, $length = undef, flags = 0, $callback->($status)
       aio_mlock $scalar, $offset = 0, $length = undef, $callback->($status)
       aio_mlockall $flags, $callback->($status)
       aio_group $callback->(...)
       aio_nop $callback->()

       $prev_pri = aioreq_pri [$pri]
       aioreq_nice $pri_adjust

       IO::AIO::poll_wait
       IO::AIO::poll_cb
       IO::AIO::poll
       IO::AIO::flush
       IO::AIO::max_poll_reqs $nreqs
       IO::AIO::max_poll_time $seconds
       IO::AIO::min_parallel $nthreads
       IO::AIO::max_parallel $nthreads
       IO::AIO::max_idle $nthreads
221
       IO::AIO::idle_timeout $seconds
222 223 224 225
       IO::AIO::max_outstanding $maxreqs
       IO::AIO::nreqs
       IO::AIO::nready
       IO::AIO::npending
226 227
       $nfd = IO::AIO::get_fdlimit [EXPERIMENTAL]
       IO::AIO::min_fdlimit $nfd [EXPERIMENTAL]
228 229 230

       IO::AIO::sendfile $ofh, $ifh, $offset, $count
       IO::AIO::fadvise $fh, $offset, $len, $advice
231 232
       IO::AIO::mmap $scalar, $length, $prot, $flags[, $fh[, $offset]]
       IO::AIO::munmap $scalar
233
       IO::AIO::mremap $scalar, $new_length, $flags[, $new_address]
234 235 236 237 238
       IO::AIO::madvise $scalar, $offset, $length, $advice
       IO::AIO::mprotect $scalar, $offset, $length, $protect
       IO::AIO::munlock $scalar, $offset = 0, $length = undef
       IO::AIO::munlockall

239
  API NOTES
240 241 242
    All the "aio_*" calls are more or less thin wrappers around the syscall
    with the same name (sans "aio_"). The arguments are similar or
    identical, and they all accept an additional (and optional) $callback
243 244 245 246 247 248 249 250 251
    argument which must be a code reference. This code reference will be
    called after the syscall has been executed in an asynchronous fashion.
    The results of the request will be passed as arguments to the callback
    (and, if an error occured, in $!) - for most requests the syscall return
    code (e.g. most syscalls return -1 on error, unlike perl, which usually
    delivers "false").

    Some requests (such as "aio_readdir") pass the actual results and
    communicate failures by passing "undef".
252 253 254 255 256 257 258

    All functions expecting a filehandle keep a copy of the filehandle
    internally until the request has finished.

    All functions return request objects of type IO::AIO::REQ that allow
    further manipulation of those requests while they are in-flight.

259 260 261 262 263 264 265 266
    The pathnames you pass to these routines *should* be absolute. The
    reason for this is that at the time the request is being executed, the
    current working directory could have changed. Alternatively, you can
    make sure that you never change the current working directory anywhere
    in the program and then use relative paths. You can also take advantage
    of IO::AIOs working directory abstraction, that lets you specify paths
    relative to some previously-opened "working directory object" - see the
    description of the "IO::AIO::WD" class later in this document.
267 268 269

    To encode pathnames as octets, either make sure you either: a) always
    pass in filenames you got from outside (command line, readdir etc.)
270 271 272 273 274
    without tinkering, b) are in your native filesystem encoding, c) use the
    Encode module and encode your pathnames to the locale (or other)
    encoding in effect in the user environment, d) use
    Glib::filename_from_unicode on unicode filenames or e) use something
    else to ensure your scalar has the correct contents.
275 276

    This works, btw. independent of the internal UTF-8 bit, which IO::AIO
277
    handles correctly whether it is set or not.
278

279
  AIO REQUEST FUNCTIONS
280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310
    $prev_pri = aioreq_pri [$pri]
        Returns the priority value that would be used for the next request
        and, if $pri is given, sets the priority for the next aio request.

        The default priority is 0, the minimum and maximum priorities are -4
        and 4, respectively. Requests with higher priority will be serviced
        first.

        The priority will be reset to 0 after each call to one of the
        "aio_*" functions.

        Example: open a file with low priority, then read something from it
        with higher priority so the read request is serviced before other
        low priority open requests (potentially spamming the cache):

           aioreq_pri -3;
           aio_open ..., sub {
              return unless $_[0];

              aioreq_pri -2;
              aio_read $_[0], ..., sub {
                 ...
              };
           };

    aioreq_nice $pri_adjust
        Similar to "aioreq_pri", but subtracts the given value from the
        current priority, so the effect is cumulative.

    aio_open $pathname, $flags, $mode, $callback->($fh)
        Asynchronously open or create a file and call the callback with a
311 312
        newly created filehandle for the file (or "undef" in case of an
        error).
313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328

        The pathname passed to "aio_open" must be absolute. See API NOTES,
        above, for an explanation.

        The $flags argument is a bitmask. See the "Fcntl" module for a list.
        They are the same as used by "sysopen".

        Likewise, $mode specifies the mode of the newly created file, if it
        didn't exist and "O_CREAT" has been given, just like perl's
        "sysopen", except that it is mandatory (i.e. use 0 if you don't
        create new files, and 0666 or 0777 if you do). Note that the $mode
        will be modified by the umask in effect then the request is being
        executed, so better never change the umask.

        Example:

329
           aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
330 331 332 333 334 335 336 337
              if ($_[0]) {
                 print "open successful, fh is $_[0]\n";
                 ...
              } else {
                 die "open failed: $!\n";
              }
           };

338 339 340 341 342 343 344
        In addition to all the common open modes/flags ("O_RDONLY",
        "O_WRONLY", "O_RDWR", "O_CREAT", "O_TRUNC", "O_EXCL" and
        "O_APPEND"), the following POSIX and non-POSIX constants are
        available (missing ones on your system are, as usual, 0):

        "O_ASYNC", "O_DIRECT", "O_NOATIME", "O_CLOEXEC", "O_NOCTTY",
        "O_NOFOLLOW", "O_NONBLOCK", "O_EXEC", "O_SEARCH", "O_DIRECTORY",
345 346
        "O_DSYNC", "O_RSYNC", "O_SYNC", "O_PATH", "O_TMPFILE", "O_TTY_INIT"
        and "O_ACCMODE".
347

348 349
    aio_close $fh, $callback->($status)
        Asynchronously close a file and call the callback with the result
350 351 352 353 354
        code.

        Unfortunately, you can't do this to perl. Perl *insists* very
        strongly on closing the file descriptor associated with the
        filehandle itself.
355

356 357 358 359 360 361
        Therefore, "aio_close" will not close the filehandle - instead it
        will use dup2 to overwrite the file descriptor with the write-end of
        a pipe (the pipe fd will be created on demand and will be cached).

        Or in other words: the file descriptor will be closed, but it will
        not be free for reuse until the perl filehandle is closed.
362

363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381
    aio_seek $fh, $offset, $whence, $callback->($offs)
        Seeks the filehandle to the new $offset, similarly to perl's
        "sysseek". The $whence can use the traditional values (0 for
        "IO::AIO::SEEK_SET", 1 for "IO::AIO::SEEK_CUR" or 2 for
        "IO::AIO::SEEK_END").

        The resulting absolute offset will be passed to the callback, or -1
        in case of an error.

        In theory, the $whence constants could be different than the
        corresponding values from Fcntl, but perl guarantees they are the
        same, so don't panic.

        As a GNU/Linux (and maybe Solaris) extension, also the constants
        "IO::AIO::SEEK_DATA" and "IO::AIO::SEEK_HOLE" are available, if they
        could be found. No guarantees about suitability for use in
        "aio_seek" or Perl's "sysseek" can be made though, although I would
        naively assume they "just work".

382 383
    aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
    aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
384 385
        Reads or writes $length bytes from or to the specified $fh and
        $offset into the scalar given by $data and offset $dataoffset and
386 387
        calls the callback with the actual number of bytes transferred (or
        -1 on error, just like the syscall).
388 389 390

        "aio_read" will, like "sysread", shrink or grow the $data scalar to
        offset plus the actual number of bytes read.
391

392 393 394
        If $offset is undefined, then the current file descriptor offset
        will be used (and updated), otherwise the file descriptor offset
        will not be changed by these calls.
395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418

        If $length is undefined in "aio_write", use the remaining length of
        $data.

        If $dataoffset is less than zero, it will be counted from the end of
        $data.

        The $data scalar *MUST NOT* be modified in any way while the request
        is outstanding. Modifying it can result in segfaults or World War
        III (if the necessary/optional hardware is installed).

        Example: Read 15 bytes at offset 7 into scalar $buffer, starting at
        offset 0 within the scalar:

           aio_read $fh, 7, 15, $buffer, 0, sub {
              $_[0] > 0 or die "read error: $!";
              print "read $_[0] bytes: <$buffer>\n";
           };

    aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
        Tries to copy $length bytes from $in_fh to $out_fh. It starts
        reading at byte offset $in_offset, and starts writing at the current
        file offset of $out_fh. Because of that, it is not safe to issue
        more than one "aio_sendfile" per $out_fh, as they will interfere
419 420
        with each other. The same $in_fh works fine though, as this function
        does not move or use the file offset of $in_fh.
421

422
        Please note that "aio_sendfile" can read more bytes from $in_fh than
423 424 425 426 427
        are written, and there is no way to find out how many more bytes
        have been read from "aio_sendfile" alone, as "aio_sendfile" only
        provides the number of bytes written to $out_fh. Only if the result
        value equals $length one can assume that $length bytes have been
        read.
428 429 430 431 432 433 434 435 436

        Unlike with other "aio_" functions, it makes a lot of sense to use
        "aio_sendfile" on non-blocking sockets, as long as one end
        (typically the $in_fh) is a file - the file I/O will then be
        asynchronous, while the socket I/O will be non-blocking. Note,
        however, that you can run into a trap where "aio_sendfile" reads
        some data with readahead, then fails to write all data, and when the
        socket is ready the next time, the data in the cache is already
        lost, forcing "aio_sendfile" to again hit the disk. Explicit
437
        "aio_read" + "aio_write" let's you better control resource usage.
438

439
        This call tries to make use of a native "sendfile"-like syscall to
440
        provide zero-copy operation. For this to work, $out_fh should refer
441
        to a socket, and $in_fh should refer to an mmap'able file.
442

443
        If a native sendfile cannot be found or it fails with "ENOSYS",
444 445 446 447 448 449 450 451 452 453
        "EINVAL", "ENOTSUP", "EOPNOTSUPP", "EAFNOSUPPORT", "EPROTOTYPE" or
        "ENOTSOCK", it will be emulated, so you can call "aio_sendfile" on
        any type of filehandle regardless of the limitations of the
        operating system.

        As native sendfile syscalls (as practically any non-POSIX interface
        hacked together in a hurry to improve benchmark numbers) tend to be
        rather buggy on many systems, this implementation tries to work
        around some known bugs in Linux and FreeBSD kernels (probably
        others, too), but that might fail, so you really really should check
454
        the return value of "aio_sendfile" - fewer bytes than expected might
455
        have been transferred.
456 457 458 459 460 461 462 463 464 465 466 467

    aio_readahead $fh,$offset,$length, $callback->($retval)
        "aio_readahead" populates the page cache with data from a file so
        that subsequent reads from that file will not block on disk I/O. The
        $offset argument specifies the starting point from which data is to
        be read and $length specifies the number of bytes to be read. I/O is
        performed in whole pages, so that offset is effectively rounded down
        to a page boundary and bytes are read up to the next page boundary
        greater than or equal to (off-set+length). "aio_readahead" does not
        read beyond the end of the file. The current file offset of the file
        is left unchanged.

468
        If that syscall doesn't exist (likely if your kernel isn't Linux) it
469 470 471 472 473
        will be emulated by simply reading the data, which would have a
        similar effect.

    aio_stat $fh_or_path, $callback->($status)
    aio_lstat $fh, $callback->($status)
474 475 476 477
        Works almost exactly like perl's "stat" or "lstat" in void context.
        The callback will be called after the stat and the results will be
        available using "stat _" or "-s _" and other tests (with the
        exception of "-B" and "-T").
478 479 480 481 482 483 484 485 486

        The pathname passed to "aio_stat" must be absolute. See API NOTES,
        above, for an explanation.

        Currently, the stats are always 64-bit-stats, i.e. instead of
        returning an error when stat'ing a large file, the results will be
        silently truncated unless perl itself is compiled with large file
        support.

487 488 489 490 491 492 493 494 495
        To help interpret the mode and dev/rdev stat values, IO::AIO offers
        the following constants and functions (if not implemented, the
        constants will be 0 and the functions will either "croak" or fall
        back on traditional behaviour).

        "S_IFMT", "S_IFIFO", "S_IFCHR", "S_IFBLK", "S_IFLNK", "S_IFREG",
        "S_IFDIR", "S_IFWHT", "S_IFSOCK", "IO::AIO::major $dev_t",
        "IO::AIO::minor $dev_t", "IO::AIO::makedev $major, $minor".

496 497 498
        To access higher resolution stat timestamps, see "SUBSECOND STAT
        TIME ACCESS".

499 500 501 502 503 504 505
        Example: Print the length of /etc/passwd:

           aio_stat "/etc/passwd", sub {
              $_[0] and die "stat failed: $!";
              print "size is ", -s _, "\n";
           };

506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548
    aio_statvfs $fh_or_path, $callback->($statvfs)
        Works like the POSIX "statvfs" or "fstatvfs" syscalls, depending on
        whether a file handle or path was passed.

        On success, the callback is passed a hash reference with the
        following members: "bsize", "frsize", "blocks", "bfree", "bavail",
        "files", "ffree", "favail", "fsid", "flag" and "namemax". On
        failure, "undef" is passed.

        The following POSIX IO::AIO::ST_* constants are defined: "ST_RDONLY"
        and "ST_NOSUID".

        The following non-POSIX IO::AIO::ST_* flag masks are defined to
        their correct value when available, or to 0 on systems that do not
        support them: "ST_NODEV", "ST_NOEXEC", "ST_SYNCHRONOUS",
        "ST_MANDLOCK", "ST_WRITE", "ST_APPEND", "ST_IMMUTABLE",
        "ST_NOATIME", "ST_NODIRATIME" and "ST_RELATIME".

        Example: stat "/wd" and dump out the data if successful.

           aio_statvfs "/wd", sub {
              my $f = $_[0]
                 or die "statvfs: $!";

              use Data::Dumper;
              say Dumper $f;
           };

           # result:
           {
              bsize   => 1024,
              bfree   => 4333064312,
              blocks  => 10253828096,
              files   => 2050765568,
              flag    => 4096,
              favail  => 2042092649,
              bavail  => 4333064312,
              ffree   => 2042092649,
              namemax => 255,
              frsize  => 1024,
              fsid    => 1810
           }

549 550 551 552 553
    aio_utime $fh_or_path, $atime, $mtime, $callback->($status)
        Works like perl's "utime" function (including the special case of
        $atime and $mtime being undef). Fractional times are supported if
        the underlying syscalls support them.

554 555 556 557
        When called with a pathname, uses utimensat(2) or utimes(2) if
        available, otherwise utime(2). If called on a file descriptor, uses
        futimens(2) or futimes(2) if available, otherwise returns ENOSYS, so
        this is not portable.
558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580

        Examples:

           # set atime and mtime to current time (basically touch(1)):
           aio_utime "path", undef, undef;
           # set atime to current time and mtime to beginning of the epoch:
           aio_utime "path", time, undef; # undef==0

    aio_chown $fh_or_path, $uid, $gid, $callback->($status)
        Works like perl's "chown" function, except that "undef" for either
        $uid or $gid is being interpreted as "do not change" (but -1 can
        also be used).

        Examples:

           # same as "chown root path" in the shell:
           aio_chown "path", 0, -1;
           # same as above:
           aio_chown "path", 0, undef;

    aio_truncate $fh_or_path, $offset, $callback->($status)
        Works like truncate(2) or ftruncate(2).

581
    aio_allocate $fh, $mode, $offset, $len, $callback->($status)
582 583
        Allocates or frees disk space according to the $mode argument. See
        the linux "fallocate" documentation for details.
584

585 586
        $mode is usually 0 or "IO::AIO::FALLOC_FL_KEEP_SIZE" to allocate
        space, or "IO::AIO::FALLOC_FL_PUNCH_HOLE |
587 588
        IO::AIO::FALLOC_FL_KEEP_SIZE", to deallocate a file range.

589
        IO::AIO also supports "FALLOC_FL_COLLAPSE_RANGE", to remove a range
590 591 592 593
        (without leaving a hole), "FALLOC_FL_ZERO_RANGE", to zero a range,
        "FALLOC_FL_INSERT_RANGE" to insert a range and
        "FALLOC_FL_UNSHARE_RANGE" to unshare shared blocks (see your
        fallocate(2) manpage).
594

595
        The file system block size used by "fallocate" is presumably the
596 597
        "f_bsize" returned by "statvfs", but different filesystems and
        filetypes can dictate other limitations.
598 599 600 601

        If "fallocate" isn't available or cannot be emulated (currently no
        emulation will be attempted), passes -1 and sets $! to "ENOSYS".

602 603 604 605 606 607 608
    aio_chmod $fh_or_path, $mode, $callback->($status)
        Works like perl's "chmod" function.

    aio_unlink $pathname, $callback->($status)
        Asynchronously unlink (delete) a file and call the callback with the
        result code.

609
    aio_mknod $pathname, $mode, $dev, $callback->($status)
610 611 612 613 614 615
        [EXPERIMENTAL]

        Asynchronously create a device node (or fifo). See mknod(2).

        The only (POSIX-) portable way of calling this function is:

616
           aio_mknod $pathname, IO::AIO::S_IFIFO | $mode, 0, sub { ...
617

618 619 620
        See "aio_stat" for info about some potentially helpful extra
        constants and functions.

621 622 623 624 625 626 627 628 629
    aio_link $srcpath, $dstpath, $callback->($status)
        Asynchronously create a new link to the existing object at $srcpath
        at the path $dstpath and call the callback with the result code.

    aio_symlink $srcpath, $dstpath, $callback->($status)
        Asynchronously create a new symbolic link to the existing object at
        $srcpath at the path $dstpath and call the callback with the result
        code.

630
    aio_readlink $pathname, $callback->($link)
631 632 633 634
        Asynchronously read the symlink specified by $path and pass it to
        the callback. If an error occurs, nothing or undef gets passed to
        the callback.

635
    aio_realpath $pathname, $callback->($path)
636
        Asynchronously make the path absolute and resolve any symlinks in
637
        $path. The resulting path only consists of directories (same as
638 639 640 641 642
        Cwd::realpath).

        This request can be used to get the absolute path of the current
        working directory by passing it a path of . (a single dot).

643 644 645 646
    aio_rename $srcpath, $dstpath, $callback->($status)
        Asynchronously rename the object at $srcpath to $dstpath, just as
        rename(2) and call the callback with the result code.

647 648 649 650
        On systems that support the AIO::WD working directory abstraction
        natively, the case "[$wd, "."]" as $srcpath is specialcased -
        instead of failing, "rename" is called on the absolute path of $wd.

651 652 653 654 655 656 657 658 659 660 661 662 663 664 665
    aio_rename2 $srcpath, $dstpath, $flags, $callback->($status)
        Basically a version of "aio_rename" with an additional $flags
        argument. Calling this with "$flags=0" is the same as calling
        "aio_rename".

        Non-zero flags are currently only supported on GNU/Linux systems
        that support renameat2. Other systems fail with "ENOSYS" in this
        case.

        The following constants are available (missing ones are, as usual
        0), see renameat2(2) for details:

        "IO::AIO::RENAME_NOREPLACE", "IO::AIO::RENAME_EXCHANGE" and
        "IO::AIO::RENAME_WHITEOUT".

666 667 668 669 670 671 672 673 674
    aio_mkdir $pathname, $mode, $callback->($status)
        Asynchronously mkdir (create) a directory and call the callback with
        the result code. $mode will be modified by the umask at the time the
        request is executed, so do not change your umask.

    aio_rmdir $pathname, $callback->($status)
        Asynchronously rmdir (delete) a directory and call the callback with
        the result code.

675 676 677 678
        On systems that support the AIO::WD working directory abstraction
        natively, the case "[$wd, "."]" is specialcased - instead of
        failing, "rmdir" is called on the absolute path of $wd.

679 680 681 682 683
    aio_readdir $pathname, $callback->($entries)
        Unlike the POSIX call of the same name, "aio_readdir" reads an
        entire directory (i.e. opendir + readdir + closedir). The entries
        will not be sorted, and will NOT include the "." and ".." entries.

684 685 686 687
        The callback is passed a single argument which is either "undef" or
        an array-ref with the filenames.

    aio_readdirx $pathname, $flags, $callback->($entries, $flags)
688 689 690
        Quite similar to "aio_readdir", but the $flags argument allows one
        to tune behaviour and output format. In case of an error, $entries
        will be "undef".
691 692 693 694 695 696

        The flags are a combination of the following constants, ORed
        together (the flags will also be passed to the callback, possibly
        modified):

        IO::AIO::READDIR_DENTS
697 698
            Normally the callback gets an arrayref consisting of names only
            (as with "aio_readdir"). If this flag is set, then the callback
699
            gets an arrayref with "[$name, $type, $inode]" arrayrefs, each
700
            describing a single directory entry in more detail:
701 702 703 704 705 706 707 708 709 710

            $name is the name of the entry.

            $type is one of the "IO::AIO::DT_xxx" constants:

            "IO::AIO::DT_UNKNOWN", "IO::AIO::DT_FIFO", "IO::AIO::DT_CHR",
            "IO::AIO::DT_DIR", "IO::AIO::DT_BLK", "IO::AIO::DT_REG",
            "IO::AIO::DT_LNK", "IO::AIO::DT_SOCK", "IO::AIO::DT_WHT".

            "IO::AIO::DT_UNKNOWN" means just that: readdir does not know. If
711 712 713
            you need to know, you have to run stat yourself. Also, for
            speed/memory reasons, the $type scalars are read-only: you must
            not modify them.
714 715 716 717 718 719 720 721

            $inode is the inode number (which might not be exact on systems
            with 64 bit inode numbers and 32 bit perls). This field has
            unspecified content on systems that do not deliver the inode
            information.

        IO::AIO::READDIR_DIRS_FIRST
            When this flag is set, then the names will be returned in an
722 723 724 725
            order where likely directories come first, in optimal stat
            order. This is useful when you need to quickly find directories,
            or you want to find all directories while avoiding to stat()
            each entry.
726 727 728

            If the system returns type information in readdir, then this is
            used to find directories directly. Otherwise, likely directories
729 730
            are names beginning with ".", or otherwise names with no dots,
            of which names with short names are tried first.
731 732 733 734

        IO::AIO::READDIR_STAT_ORDER
            When this flag is set, then the names will be returned in an
            order suitable for stat()'ing each one. That is, when you plan
735 736
            to stat() most or all files in the given directory, then the
            returned order will likely be faster.
737 738 739

            If both this flag and "IO::AIO::READDIR_DIRS_FIRST" are
            specified, then the likely dirs come first, resulting in a less
740 741
            optimal stat order for stat'ing all entries, but likely a more
            optimal order for finding subdirectories.
742 743 744 745

        IO::AIO::READDIR_FOUND_UNKNOWN
            This flag should not be set when calling "aio_readdirx".
            Instead, it is being set by "aio_readdirx", when any of the
746
            $type's found were "IO::AIO::DT_UNKNOWN". The absence of this
747 748
            flag therefore indicates that all $type's are known, which can
            be used to speed up some algorithms.
749

750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776
    aio_slurp $pathname, $offset, $length, $data, $callback->($status)
        Opens, reads and closes the given file. The data is put into $data,
        which is resized as required.

        If $offset is negative, then it is counted from the end of the file.

        If $length is zero, then the remaining length of the file is used.
        Also, in this case, the same limitations to modifying $data apply as
        when IO::AIO::mmap is used, i.e. it must only be modified in-place
        with "substr". If the size of the file is known, specifying a
        non-zero $length results in a performance advantage.

        This request is similar to the older "aio_load" request, but since
        it is a single request, it might be more efficient to use.

        Example: load /etc/passwd into $passwd.

           my $passwd;
           aio_slurp "/etc/passwd", 0, 0, $passwd, sub {
              $_[0] >= 0
                 or die "/etc/passwd: $!\n";

              printf "/etc/passwd is %d bytes long, and contains:\n", length $passwd;
              print $passwd;
           };
           IO::AIO::flush;

777
    aio_load $pathname, $data, $callback->($status)
778 779 780
        This is a composite request that tries to fully load the given file
        into memory. Status is the same as with aio_read.

781 782 783
        Using "aio_slurp" might be more efficient, as it is a single
        request.

784 785 786
    aio_copy $srcpath, $dstpath, $callback->($status)
        Try to copy the *file* (directories not supported as either source
        or destination) from $srcpath to $dstpath and call the callback with
787
        a status of 0 (ok) or -1 (error, see $!).
788

789 790
        Existing destination files will be truncated.

791 792 793 794
        This is a composite request that creates the destination file with
        mode 0200 and copies the contents of the source file into it using
        "aio_sendfile", followed by restoring atime, mtime, access mode and
        uid/gid, in that order.
795 796 797 798 799 800 801 802

        If an error occurs, the partial destination file will be unlinked,
        if possible, except when setting atime, mtime, access mode and
        uid/gid, where errors are being ignored.

    aio_move $srcpath, $dstpath, $callback->($status)
        Try to move the *file* (directories not supported as either source
        or destination) from $srcpath to $dstpath and call the callback with
803
        a status of 0 (ok) or -1 (error, see $!).
804

805 806 807
        This is a composite request that tries to rename(2) the file first;
        if rename fails with "EXDEV", it copies the file with "aio_copy"
        and, if that is successful, unlinks the $srcpath.
808

809
    aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs)
810 811 812 813 814 815
        Scans a directory (similar to "aio_readdir") but additionally tries
        to efficiently separate the entries of directory $path into two sets
        of names, directories you can recurse into (directories), and ones
        you cannot recurse into (everything else, including symlinks to
        directories).

816 817
        "aio_scandir" is a composite request that generates many sub
        requests. $maxreq specifies the maximum number of outstanding aio
818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836
        requests that this function generates. If it is "<= 0", then a
        suitable default will be chosen (currently 4).

        On error, the callback is called without arguments, otherwise it
        receives two array-refs with path-relative entry names.

        Example:

           aio_scandir $dir, 0, sub {
              my ($dirs, $nondirs) = @_;
              print "real directories: @$dirs\n";
              print "everything else: @$nondirs\n";
           };

        Implementation notes.

        The "aio_readdir" cannot be avoided, but "stat()"'ing every entry
        can.

837 838 839 840 841 842 843 844 845 846 847 848 849 850 851
        If readdir returns file type information, then this is used directly
        to find directories.

        Otherwise, after reading the directory, the modification time, size
        etc. of the directory before and after the readdir is checked, and
        if they match (and isn't the current time), the link count will be
        used to decide how many entries are directories (if >= 2).
        Otherwise, no knowledge of the number of subdirectories will be
        assumed.

        Then entries will be sorted into likely directories a non-initial
        dot currently) and likely non-directories (see "aio_readdirx"). Then
        every entry plus an appended "/." will be "stat"'ed, likely
        directories first, in order of their inode numbers. If that
        succeeds, it assumes that the entry is a directory or a symlink to
852
        directory (which will be checked separately). This is often faster
853 854 855 856
        than stat'ing the entry itself because filesystems might detect the
        type of the entry without reading the inode data (e.g. ext2fs
        filetype feature), even on systems that cannot return the filetype
        information on readdir.
857 858 859 860 861 862 863 864 865 866 867

        If the known number of directories (link count - 2) has been
        reached, the rest of the entries is assumed to be non-directories.

        This only works with certainty on POSIX (= UNIX) filesystems, which
        fortunately are the vast majority of filesystems around.

        It will also likely work on non-POSIX filesystems with reduced
        efficiency as those tend to return 0 or 1 as link counts, which
        disables the directory counting heuristic.

868
    aio_rmtree $pathname, $callback->($status)
869 870 871 872 873
        Delete a directory tree starting (and including) $path, return the
        status of the final "rmdir" only. This is a composite request that
        uses "aio_scandir" to recurse into and rmdir directories, and unlink
        everything else.

874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891
    aio_fcntl $fh, $cmd, $arg, $callback->($status)
    aio_ioctl $fh, $request, $buf, $callback->($status)
        These work just like the "fcntl" and "ioctl" built-in functions,
        except they execute asynchronously and pass the return value to the
        callback.

        Both calls can be used for a lot of things, some of which make more
        sense to run asynchronously in their own thread, while some others
        make less sense. For example, calls that block waiting for external
        events, such as locking, will also lock down an I/O thread while it
        is waiting, which can deadlock the whole I/O system. At the same
        time, there might be no alternative to using a thread to wait.

        So in general, you should only use these calls for things that do
        (filesystem) I/O, not for things that wait for other events
        (network, other processes), although if you are careful and know
        what you are doing, you still can.

892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921
        The following constants are available (missing ones are, as usual
        0):

        "F_DUPFD_CLOEXEC",

        "F_OFD_GETLK", "F_OFD_SETLK", "F_OFD_GETLKW",

        "FIFREEZE", "FITHAW", "FITRIM", "FICLONE", "FICLONERANGE",
        "FIDEDUPERANGE".

        "FS_IOC_GETFLAGS", "FS_IOC_SETFLAGS", "FS_IOC_GETVERSION",
        "FS_IOC_SETVERSION", "FS_IOC_FIEMAP".

        "FS_IOC_FSGETXATTR", "FS_IOC_FSSETXATTR",
        "FS_IOC_SET_ENCRYPTION_POLICY", "FS_IOC_GET_ENCRYPTION_PWSALT",
        "FS_IOC_GET_ENCRYPTION_POLICY", "FS_KEY_DESCRIPTOR_SIZE".

        "FS_SECRM_FL", "FS_UNRM_FL", "FS_COMPR_FL", "FS_SYNC_FL",
        "FS_IMMUTABLE_FL", "FS_APPEND_FL", "FS_NODUMP_FL", "FS_NOATIME_FL",
        "FS_DIRTY_FL", "FS_COMPRBLK_FL", "FS_NOCOMP_FL", "FS_ENCRYPT_FL",
        "FS_BTREE_FL", "FS_INDEX_FL", "FS_JOURNAL_DATA_FL", "FS_NOTAIL_FL",
        "FS_DIRSYNC_FL", "FS_TOPDIR_FL", "FS_FL_USER_MODIFIABLE".

        "FS_XFLAG_REALTIME", "FS_XFLAG_PREALLOC", "FS_XFLAG_IMMUTABLE",
        "FS_XFLAG_APPEND", "FS_XFLAG_SYNC", "FS_XFLAG_NOATIME",
        "FS_XFLAG_NODUMP", "FS_XFLAG_RTINHERIT", "FS_XFLAG_PROJINHERIT",
        "FS_XFLAG_NOSYMLINKS", "FS_XFLAG_EXTSIZE", "FS_XFLAG_EXTSZINHERIT",
        "FS_XFLAG_NODEFRAG", "FS_XFLAG_FILESTREAM", "FS_XFLAG_DAX",
        "FS_XFLAG_HASATTR",

922 923 924
    aio_sync $callback->($status)
        Asynchronously call sync and call the callback when finished.

925 926 927 928 929 930 931 932 933 934 935
    aio_fsync $fh, $callback->($status)
        Asynchronously call fsync on the given filehandle and call the
        callback with the fsync result code.

    aio_fdatasync $fh, $callback->($status)
        Asynchronously call fdatasync on the given filehandle and call the
        callback with the fdatasync result code.

        If this call isn't available because your OS lacks it or it couldn't
        be detected, it will be emulated by calling "fsync" instead.

936 937 938 939 940 941
    aio_syncfs $fh, $callback->($status)
        Asynchronously call the syncfs syscall to sync the filesystem
        associated to the given filehandle and call the callback with the
        syncfs result code. If syncfs is not available, calls sync(), but
        returns -1 and sets errno to "ENOSYS" nevertheless.

942 943 944 945 946 947 948 949 950 951 952 953
    aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
        Sync the data portion of the file specified by $offset and $length
        to disk (but NOT the metadata), by calling the Linux-specific
        sync_file_range call. If sync_file_range is not available or it
        returns ENOSYS, then fdatasync or fsync is being substituted.

        $flags can be a combination of
        "IO::AIO::SYNC_FILE_RANGE_WAIT_BEFORE",
        "IO::AIO::SYNC_FILE_RANGE_WRITE" and
        "IO::AIO::SYNC_FILE_RANGE_WAIT_AFTER": refer to the sync_file_range
        manpage for details.

954
    aio_pathsync $pathname, $callback->($status)
955 956 957 958 959 960 961 962 963 964 965 966
        This request tries to open, fsync and close the given path. This is
        a composite request intended to sync directories after directory
        operations (E.g. rename). This might not work on all operating
        systems or have any specific effect, but usually it makes sure that
        directory changes get written to disc. It works for anything that
        can be opened for read-only, not just directories.

        Future versions of this function might fall back to other methods
        when "fsync" on the directory fails (such as calling "sync").

        Passes 0 when everything went ok, and -1 on error.

967
    aio_msync $scalar, $offset = 0, $length = undef, flags = MS_SYNC,
968 969 970 971 972 973 974 975 976 977 978
    $callback->($status)
        This is a rather advanced IO::AIO call, which only works on
        mmap(2)ed scalars (see the "IO::AIO::mmap" function, although it
        also works on data scalars managed by the Sys::Mmap or Mmap modules,
        note that the scalar must only be modified in-place while an aio
        operation is pending on it).

        It calls the "msync" function of your OS, if available, with the
        memory area starting at $offset in the string and ending $length
        bytes later. If $length is negative, counts from the end, and if
        $length is "undef", then it goes till the end of the string. The
979 980
        flags can be either "IO::AIO::MS_ASYNC" or "IO::AIO::MS_SYNC", plus
        an optional "IO::AIO::MS_INVALIDATE".
981 982 983 984 985 986 987 988 989 990

    aio_mtouch $scalar, $offset = 0, $length = undef, flags = 0,
    $callback->($status)
        This is a rather advanced IO::AIO call, which works best on
        mmap(2)ed scalars.

        It touches (reads or writes) all memory pages in the specified range
        inside the scalar. All caveats and parameters are the same as for
        "aio_msync", above, except for flags, which must be either 0 (which
        reads all pages and ensures they are instantiated) or
991
        "IO::AIO::MT_MODIFY", which modifies the memory pages (by reading
992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033
        and writing an octet from it, which dirties the page).

    aio_mlock $scalar, $offset = 0, $length = undef, $callback->($status)
        This is a rather advanced IO::AIO call, which works best on
        mmap(2)ed scalars.

        It reads in all the pages of the underlying storage into memory (if
        any) and locks them, so they are not getting swapped/paged out or
        removed.

        If $length is undefined, then the scalar will be locked till the
        end.

        On systems that do not implement "mlock", this function returns -1
        and sets errno to "ENOSYS".

        Note that the corresponding "munlock" is synchronous and is
        documented under "MISCELLANEOUS FUNCTIONS".

        Example: open a file, mmap and mlock it - both will be undone when
        $data gets destroyed.

           open my $fh, "<", $path or die "$path: $!";
           my $data;
           IO::AIO::mmap $data, -s $fh, IO::AIO::PROT_READ, IO::AIO::MAP_SHARED, $fh;
           aio_mlock $data; # mlock in background

    aio_mlockall $flags, $callback->($status)
        Calls the "mlockall" function with the given $flags (a combination
        of "IO::AIO::MCL_CURRENT" and "IO::AIO::MCL_FUTURE").

        On systems that do not implement "mlockall", this function returns
        -1 and sets errno to "ENOSYS".

        Note that the corresponding "munlockall" is synchronous and is
        documented under "MISCELLANEOUS FUNCTIONS".

        Example: asynchronously lock all current and future pages into
        memory.

           aio_mlockall IO::AIO::MCL_FUTURE;

1034
    aio_fiemap $fh, $start, $length, $flags, $count, $cb->(\@extents)
1035
        Queries the extents of the given file (by calling the Linux "FIEMAP"
1036
        ioctl, see <http://cvs.schmorp.de/IO-AIO/doc/fiemap.txt> for
1037 1038
        details). If the ioctl is not available on your OS, then this
        request will fail with "ENOSYS".
1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049

        $start is the starting offset to query extents for, $length is the
        size of the range to query - if it is "undef", then the whole file
        will be queried.

        $flags is a combination of flags ("IO::AIO::FIEMAP_FLAG_SYNC" or
        "IO::AIO::FIEMAP_FLAG_XATTR" - "IO::AIO::FIEMAP_FLAGS_COMPAT" is
        also exported), and is normally 0 or "IO::AIO::FIEMAP_FLAG_SYNC" to
        query the data portion.

        $count is the maximum number of extent records to return. If it is
1050
        "undef", then IO::AIO queries all extents of the range. As a very
1051
        special case, if it is 0, then the callback receives the number of
1052 1053
        extents instead of the extents themselves (which is unreliable, see
        below).
1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064

        If an error occurs, the callback receives no arguments. The special
        "errno" value "IO::AIO::EBADR" is available to test for flag errors.

        Otherwise, the callback receives an array reference with extent
        structures. Each extent structure is an array reference itself, with
        the following members:

           [$logical, $physical, $length, $flags]

        Flags is any combination of the following flag values (typically
1065
        either 0 or "IO::AIO::FIEMAP_EXTENT_LAST" (1)):
1066 1067 1068 1069 1070 1071 1072 1073 1074 1075

        "IO::AIO::FIEMAP_EXTENT_LAST", "IO::AIO::FIEMAP_EXTENT_UNKNOWN",
        "IO::AIO::FIEMAP_EXTENT_DELALLOC", "IO::AIO::FIEMAP_EXTENT_ENCODED",
        "IO::AIO::FIEMAP_EXTENT_DATA_ENCRYPTED",
        "IO::AIO::FIEMAP_EXTENT_NOT_ALIGNED",
        "IO::AIO::FIEMAP_EXTENT_DATA_INLINE",
        "IO::AIO::FIEMAP_EXTENT_DATA_TAIL",
        "IO::AIO::FIEMAP_EXTENT_UNWRITTEN", "IO::AIO::FIEMAP_EXTENT_MERGED"
        or "IO::AIO::FIEMAP_EXTENT_SHARED".

1076
        At the time of this writing (Linux 3.2), this request is unreliable
1077
        unless $count is "undef", as the kernel has all sorts of bugs
1078 1079 1080
        preventing it to return all extents of a range for files with a
        large number of extents. The code (only) works around all these
        issues if $count is "undef".
1081

1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124
    aio_group $callback->(...)
        This is a very special aio request: Instead of doing something, it
        is a container for other aio requests, which is useful if you want
        to bundle many requests into a single, composite, request with a
        definite callback and the ability to cancel the whole request with
        its subrequests.

        Returns an object of class IO::AIO::GRP. See its documentation below
        for more info.

        Example:

           my $grp = aio_group sub {
              print "all stats done\n";
           };

           add $grp
              (aio_stat ...),
              (aio_stat ...),
              ...;

    aio_nop $callback->()
        This is a special request - it does nothing in itself and is only
        used for side effects, such as when you want to add a dummy request
        to a group so that finishing the requests in the group depends on
        executing the given code.

        While this request does nothing, it still goes through the execution
        phase and still requires a worker thread. Thus, the callback will
        not be executed immediately but only after other requests in the
        queue have entered their execution phase. This can be used to
        measure request latency.

    IO::AIO::aio_busy $fractional_seconds, $callback->() *NOT EXPORTED*
        Mainly used for debugging and benchmarking, this aio request puts
        one of the request workers to sleep for the given time.

        While it is theoretically handy to have simple I/O scheduling
        requests like sleep and file handle readable/writable, the overhead
        this creates is immense (it blocks a thread for a long time) so do
        not use this function except to put your application under
        artificial I/O pressure.

1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169
  IO::AIO::WD - multiple working directories
    Your process only has one current working directory, which is used by
    all threads. This makes it hard to use relative paths (some other
    component could call "chdir" at any time, and it is hard to control when
    the path will be used by IO::AIO).

    One solution for this is to always use absolute paths. This usually
    works, but can be quite slow (the kernel has to walk the whole path on
    every access), and can also be a hassle to implement.

    Newer POSIX systems have a number of functions (openat, fdopendir,
    futimensat and so on) that make it possible to specify working
    directories per operation.

    For portability, and because the clowns who "designed", or shall I
    write, perpetrated this new interface were obviously half-drunk, this
    abstraction cannot be perfect, though.

    IO::AIO allows you to convert directory paths into a so-called
    IO::AIO::WD object. This object stores the canonicalised, absolute
    version of the path, and on systems that allow it, also a directory file
    descriptor.

    Everywhere where a pathname is accepted by IO::AIO (e.g. in "aio_stat"
    or "aio_unlink"), one can specify an array reference with an IO::AIO::WD
    object and a pathname instead (or the IO::AIO::WD object alone, which
    gets interpreted as "[$wd, "."]"). If the pathname is absolute, the
    IO::AIO::WD object is ignored, otherwise the pathname is resolved
    relative to that IO::AIO::WD object.

    For example, to get a wd object for /etc and then stat passwd inside,
    you would write:

       aio_wd "/etc", sub {
          my $etcdir = shift;

          # although $etcdir can be undef on error, there is generally no reason
          # to check for errors here, as aio_stat will fail with ENOENT
          # when $etcdir is undef.

          aio_stat [$etcdir, "passwd"], sub {
             # yay
          };
       };

1170 1171 1172
    The fact that "aio_wd" is a request and not a normal function shows that
    creating an IO::AIO::WD object is itself a potentially blocking
    operation, which is why it is done asynchronously.
1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198

    To stat the directory obtained with "aio_wd" above, one could write
    either of the following three request calls:

       aio_lstat "/etc"    , sub { ...  # pathname as normal string
       aio_lstat [$wd, "."], sub { ...  # "." relative to $wd (i.e. $wd itself)
       aio_lstat $wd       , sub { ...  # shorthand for the previous

    As with normal pathnames, IO::AIO keeps a copy of the working directory
    object and the pathname string, so you could write the following without
    causing any issues due to $path getting reused:

       my $path = [$wd, undef];

       for my $name (qw(abc def ghi)) {
          $path->[1] = $name;
          aio_stat $path, sub {
             # ...
          };
       }

    There are some caveats: when directories get renamed (or deleted), the
    pathname string doesn't change, so will point to the new directory (or
    nowhere at all), while the directory fd, if available on the system,
    will still point to the original directory. Most functions accepting a
    pathname will use the directory fd on newer systems, and the string on
1199 1200
    older systems. Some functions (such as "aio_realpath") will always rely
    on the string form of the pathname.
1201

1202
    So this functionality is mainly useful to get some protection against
1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227
    "chdir", to easily get an absolute path out of a relative path for
    future reference, and to speed up doing many operations in the same
    directory (e.g. when stat'ing all files in a directory).

    The following functions implement this working directory abstraction:

    aio_wd $pathname, $callback->($wd)
        Asynchonously canonicalise the given pathname and convert it to an
        IO::AIO::WD object representing it. If possible and supported on the
        system, also open a directory fd to speed up pathname resolution
        relative to this working directory.

        If something goes wrong, then "undef" is passwd to the callback
        instead of a working directory object and $! is set appropriately.
        Since passing "undef" as working directory component of a pathname
        fails the request with "ENOENT", there is often no need for error
        checking in the "aio_wd" callback, as future requests using the
        value will fail in the expected way.

    IO::AIO::CWD
        This is a compiletime constant (object) that represents the process
        current working directory.

        Specifying this object as working directory object for a pathname is
        as if the pathname would be specified directly, without a directory
1228
        object. For example, these calls are functionally identical:
1229 1230 1231 1232

           aio_stat "somefile", sub { ... };
           aio_stat [IO::AIO::CWD, "somefile"], sub { ... };

1233 1234 1235 1236 1237 1238 1239 1240 1241 1242
    To recover the path associated with an IO::AIO::WD object, you can use
    "aio_realpath":

       aio_realpath $wd, sub {
          warn "path is $_[0]\n";
       };

    Currently, "aio_statvfs" always, and "aio_rename" and "aio_rmdir"
    sometimes, fall back to using an absolue path.

1243 1244 1245 1246 1247 1248 1249 1250
  IO::AIO::REQ CLASS
    All non-aggregate "aio_*" functions return an object of this class when
    called in non-void context.

    cancel $req
        Cancels the request, if possible. Has the effect of skipping
        execution when entering the execute state and skipping calling the
        callback when entering the the result state, but will leave the
1251 1252 1253
        request otherwise untouched (with the exception of readdir). That
        means that requests that currently execute will not be stopped and
        resources held by the request will not be freed prematurely.
1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289

    cb $req $callback->(...)
        Replace (or simply set) the callback registered to the request.

  IO::AIO::GRP CLASS
    This class is a subclass of IO::AIO::REQ, so all its methods apply to
    objects of this class, too.

    A IO::AIO::GRP object is a special request that can contain multiple
    other aio requests.

    You create one by calling the "aio_group" constructing function with a
    callback that will be called when all contained requests have entered
    the "done" state:

       my $grp = aio_group sub {
          print "all requests are done\n";
       };

    You add requests by calling the "add" method with one or more
    "IO::AIO::REQ" objects:

       $grp->add (aio_unlink "...");

       add $grp aio_stat "...", sub {
          $_[0] or return $grp->result ("error");

          # add another request dynamically, if first succeeded
          add $grp aio_open "...", sub {
             $grp->result ("ok");
          };
       };

    This makes it very easy to create composite requests (see the source of
    "aio_move" for an application) that work and feel like simple requests.

1290 1291 1292 1293 1294 1295 1296 1297 1298 1299
    *   The IO::AIO::GRP objects will be cleaned up during calls to
        "IO::AIO::poll_cb", just like any other request.

    *   They can be canceled like any other request. Canceling will cancel
        not only the request itself, but also all requests it contains.

    *   They can also can also be added to other IO::AIO::GRP objects.

    *   You must not add requests to a group from within the group callback
        (or any later time).
1300 1301 1302 1303 1304 1305

    Their lifetime, simplified, looks like this: when they are empty, they
    will finish very quickly. If they contain only requests that are in the
    "done" state, they will also finish. Otherwise they will continue to
    exist.

1306 1307 1308 1309 1310
    That means after creating a group you have some time to add requests
    (precisely before the callback has been invoked, which is only done
    within the "poll_cb"). And in the callbacks of those requests, you can
    add further requests to the group. And only when all those requests have
    finished will the the group itself finish.
1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324

    add $grp ...
    $grp->add (...)
        Add one or more requests to the group. Any type of IO::AIO::REQ can
        be added, including other groups, as long as you do not create
        circular dependencies.

        Returns all its arguments.

    $grp->cancel_subs
        Cancel all subrequests and clears any feeder, but not the group
        request itself. Useful when you queued a lot of events but got a
        result early.

1325 1326 1327
        The group request will finish normally (you cannot add requests to
        the group).

1328 1329
    $grp->result (...)
        Set the result value(s) that will be passed to the group callback
1330
        when all subrequests have finished and set the groups errno to the
1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350
        current value of errno (just like calling "errno" without an error
        number). By default, no argument will be passed and errno is zero.

    $grp->errno ([$errno])
        Sets the group errno value to $errno, or the current value of errno
        when the argument is missing.

        Every aio request has an associated errno value that is restored
        when the callback is invoked. This method lets you change this value
        from its default (0).

        Calling "result" will also set errno, so make sure you either set $!
        before the call to "result", or call c<errno> after it.

    feed $grp $callback->($grp)
        Sets a feeder/generator on this group: every group can have an
        attached generator that generates requests if idle. The idea behind
        this is that, although you could just queue as many requests as you
        want in a group, this might starve other requests for a potentially
        long time. For example, "aio_scandir" might generate hundreds of
1351
        thousands of "aio_stat" requests, delaying any later requests for a
1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365
        long time.

        To avoid this, and allow incremental generation of requests, you can
        instead a group and set a feeder on it that generates those
        requests. The feed callback will be called whenever there are few
        enough (see "limit", below) requests active in the group itself and
        is expected to queue more requests.

        The feed callback can queue as many requests as it likes (i.e. "add"
        does not impose any limits).

        If the feed does not queue more requests when called, it will be
        automatically removed from the group.

1366 1367
        If the feed limit is 0 when this method is called, it will be set to
        2 automatically.
1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387

        Example:

           # stat all files in @files, but only ever use four aio requests concurrently:

           my $grp = aio_group sub { print "finished\n" };
           limit $grp 4;
           feed $grp sub {
              my $file = pop @files
                 or return;

              add $grp aio_stat $file, sub { ... };
           };

    limit $grp $num
        Sets the feeder limit for the group: The feeder will be called
        whenever the group contains less than this many requests.

        Setting the limit to 0 will pause the feeding process.

1388 1389 1390
        The default value for the limit is 0, but note that setting a feeder
        automatically bumps it up to 2.

1391 1392 1393 1394 1395
  SUPPORT FUNCTIONS
   EVENT PROCESSING AND EVENT LOOP INTEGRATION
    $fileno = IO::AIO::poll_fileno
        Return the *request result pipe file descriptor*. This filehandle
        must be polled for reading by some mechanism outside this module
1396 1397 1398
        (e.g. EV, Glib, select and so on, see below or the SYNOPSIS). If the
        pipe becomes readable you have to call "poll_cb" to check the
        results.
1399 1400 1401 1402

        See "poll_cb" for an example.

    IO::AIO::poll_cb
1403 1404 1405
        Process some requests that have reached the result phase (i.e. they
        have been executed but the results are not yet reported). You have
        to call this "regularly" to finish outstanding requests.
1406

1407 1408 1409 1410 1411 1412 1413 1414 1415 1416
        Returns 0 if all events could be processed (or there were no events
        to process), or -1 if it returned earlier for whatever reason.
        Returns immediately when no events are outstanding. The amount of
        events processed depends on the settings of "IO::AIO::max_poll_req",
        "IO::AIO::max_poll_time" and "IO::AIO::max_outstanding".

        If not all requests were processed for whatever reason, the poll
        file descriptor will still be ready when "poll_cb" returns, so
        normally you don't have to do anything special to have it called
        later.
1417

1418 1419 1420 1421 1422 1423 1424
        Apart from calling "IO::AIO::poll_cb" when the event filehandle
        becomes ready, it can be beneficial to call this function from loops
        which submit a lot of requests, to make sure the results get
        processed when they become available and not just when the loop is
        finished and the event loop takes over again. This function returns
        very fast when there are no outstanding requests.

1425
        Example: Install an Event watcher that automatically calls
1426 1427
        IO::AIO::poll_cb with high priority (more examples can be found in
        the SYNOPSIS section, at the top of this document):
1428 1429 1430 1431 1432

           Event->io (fd => IO::AIO::poll_fileno,
                      poll => 'r', async => 1,
                      cb => \&IO::AIO::poll_cb);

1433
    IO::AIO::poll_wait
1434 1435 1436 1437 1438
        Wait until either at least one request is in the result phase or no
        requests are outstanding anymore.

        This is useful if you want to synchronously wait for some requests
        to become ready, without actually handling them.
1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457

        See "nreqs" for an example.

    IO::AIO::poll
        Waits until some requests have been handled.

        Returns the number of requests processed, but is otherwise strictly
        equivalent to:

           IO::AIO::poll_wait, IO::AIO::poll_cb

    IO::AIO::flush
        Wait till all outstanding AIO requests have been handled.

        Strictly equivalent to:

           IO::AIO::poll_wait, IO::AIO::poll_cb
              while IO::AIO::nreqs;

1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468
        This function can be useful at program aborts, to make sure
        outstanding I/O has been done ("IO::AIO" uses an "END" block which
        already calls this function on normal exits), or when you are merely
        using "IO::AIO" for its more advanced functions, rather than for
        async I/O, e.g.:

           my ($dirs, $nondirs);
           IO::AIO::aio_scandir "/tmp", 0, sub { ($dirs, $nondirs) = @_ };
           IO::AIO::flush;
           # $dirs, $nondirs are now set

1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538
    IO::AIO::max_poll_reqs $nreqs
    IO::AIO::max_poll_time $seconds
        These set the maximum number of requests (default 0, meaning
        infinity) that are being processed by "IO::AIO::poll_cb" in one
        call, respectively the maximum amount of time (default 0, meaning
        infinity) spent in "IO::AIO::poll_cb" to process requests (more
        correctly the mininum amount of time "poll_cb" is allowed to use).

        Setting "max_poll_time" to a non-zero value creates an overhead of
        one syscall per request processed, which is not normally a problem
        unless your callbacks are really really fast or your OS is really
        really slow (I am not mentioning Solaris here). Using
        "max_poll_reqs" incurs no overhead.

        Setting these is useful if you want to ensure some level of
        interactiveness when perl is not fast enough to process all requests
        in time.

        For interactive programs, values such as 0.01 to 0.1 should be fine.

        Example: Install an Event watcher that automatically calls
        IO::AIO::poll_cb with low priority, to ensure that other parts of
        the program get the CPU sometimes even under high AIO load.

           # try not to spend much more than 0.1s in poll_cb
           IO::AIO::max_poll_time 0.1;

           # use a low priority so other tasks have priority
           Event->io (fd => IO::AIO::poll_fileno,
                      poll => 'r', nice => 1,
                      cb => &IO::AIO::poll_cb);

   CONTROLLING THE NUMBER OF THREADS
    IO::AIO::min_parallel $nthreads
        Set the minimum number of AIO threads to $nthreads. The current
        default is 8, which means eight asynchronous operations can execute
        concurrently at any one time (the number of outstanding requests,
        however, is unlimited).

        IO::AIO starts threads only on demand, when an AIO request is queued
        and no free thread exists. Please note that queueing up a hundred
        requests can create demand for a hundred threads, even if it turns
        out that everything is in the cache and could have been processed
        faster by a single thread.

        It is recommended to keep the number of threads relatively low, as
        some Linux kernel versions will scale negatively with the number of
        threads (higher parallelity => MUCH higher latency). With current
        Linux 2.6 versions, 4-32 threads should be fine.

        Under most circumstances you don't need to call this function, as
        the module selects a default that is suitable for low to moderate
        load.

    IO::AIO::max_parallel $nthreads
        Sets the maximum number of AIO threads to $nthreads. If more than
        the specified number of threads are currently running, this function
        kills them. This function blocks until the limit is reached.

        While $nthreads are zero, aio requests get queued but not executed
        until the number of threads has been increased again.

        This module automatically runs "max_parallel 0" at program end, to
        ensure that all threads are killed and that there are no outstanding
        requests.

        Under normal circumstances you don't need to call this function.

    IO::AIO::max_idle $nthreads
        Limit the number of threads (default: 4) that are allowed to idle
1539 1540 1541 1542
        (i.e., threads that did not get a request to process within the idle
        timeout (default: 10 seconds). That means if a thread becomes idle
        while $nthreads other threads are also idle, it will free its
        resources and exit.
1543 1544 1545 1546 1547 1548 1549 1550 1551 1552

        This is useful when you allow a large number of threads (e.g. 100 or
        1000) to allow for extremely high load situations, but want to free
        resources under normal circumstances (1000 threads can easily
        consume 30MB of RAM).

        The default is probably ok in most situations, especially if thread
        creation is fast. If thread creation is very slow on your system you
        might want to use larger values.

1553 1554 1555 1556
    IO::AIO::idle_timeout $seconds
        Sets the minimum idle timeout (default 10) after which worker
        threads are allowed to exit. SEe "IO::AIO::max_idle".

1557
    IO::AIO::max_outstanding $maxreqs
1558 1559 1560 1561 1562 1563 1564 1565 1566
        Sets the maximum number of outstanding requests to $nreqs. If you do
        queue up more than this number of requests, the next call to
        "IO::AIO::poll_cb" (and other functions calling "poll_cb", such as
        "IO::AIO::flush" or "IO::AIO::poll") will block until the limit is
        no longer exceeded.

        In other words, this setting does not enforce a queue limit, but can
        be used to make poll functions block if the limit is exceeded.

1567 1568 1569 1570
        This is a very bad function to use in interactive programs because
        it blocks, and a bad way to reduce concurrency because it is
        inexact: Better use an "aio_group" together with a feed callback.

1571
        Its main use is in scripts without an event loop - when you want to
1572
        stat a lot of files, you can write something like this:
1573 1574 1575 1576 1577 1578 1579 1580 1581

           IO::AIO::max_outstanding 32;

           for my $path (...) {
              aio_stat $path , ...;
              IO::AIO::poll_cb;
           }

           IO::AIO::flush;
1582

1583 1584 1585 1586 1587
        The call to "poll_cb" inside the loop will normally return
        instantly, but as soon as more thna 32 reqeusts are in-flight, it
        will block until some requests have been handled. This keeps the
        loop from pushing a large number of "aio_stat" requests onto the
        queue.
1588

1589 1590
        The default value for "max_outstanding" is very large, so there is
        no practical limit on the number of outstanding requests.
1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610

   STATISTICAL INFORMATION
    IO::AIO::nreqs
        Returns the number of requests currently in the ready, execute or
        pending states (i.e. for which their callback has not been invoked
        yet).

        Example: wait till there are no outstanding requests anymore:

           IO::AIO::poll_wait, IO::AIO::poll_cb
              while IO::AIO::nreqs;

    IO::AIO::nready
        Returns the number of requests currently in the ready state (not yet
        executed).

    IO::AIO::npending
        Returns the number of requests currently in the pending state
        (executed, but not yet processed by poll_cb).

1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695
   SUBSECOND STAT TIME ACCESS
    Both "aio_stat"/"aio_lstat" and perl's "stat"/"lstat" functions can
    generally find access/modification and change times with subsecond time
    accuracy of the system supports it, but perl's built-in functions only
    return the integer part.

    The following functions return the timestamps of the most recent stat
    with subsecond precision on most systems and work both after
    "aio_stat"/"aio_lstat" and perl's "stat"/"lstat" calls. Their return
    value is only meaningful after a successful "stat"/"lstat" call, or
    during/after a successful "aio_stat"/"aio_lstat" callback.

    This is similar to the Time::HiRes "stat" functions, but can return full
    resolution without rounding and work with standard perl "stat",
    alleviating the need to call the special "Time::HiRes" functions, which
    do not act like their perl counterparts.

    On operating systems or file systems where subsecond time resolution is
    not supported or could not be detected, a fractional part of 0 is
    returned, so it is always safe to call these functions.

    $seconds = IO::AIO::st_atime, IO::AIO::st_mtime, IO::AIO::st_ctime,
    IO::AIO::st_btime
        Return the access, modication, change or birth time, respectively,
        including fractional part. Due to the limited precision of floating
        point, the accuracy on most platforms is only a bit better than
        milliseconds for times around now - see the *nsec* function family,
        below, for full accuracy.

        File birth time is only available when the OS and perl support it
        (on FreeBSD and NetBSD at the time of this writing, although support
        is adaptive, so if your OS/perl gains support, IO::AIO can take
        avdantage of it). On systems where it isn't available, 0 is
        currently returned, but this might change to "undef" in a future
        version.

    ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtime
        Returns access, modification, change and birth time all in one go,
        and maybe more times in the future version.

    $nanoseconds = IO::AIO::st_atimensec, IO::AIO::st_mtimensec,
    IO::AIO::st_ctimensec, IO::AIO::st_btimensec
        Return the fractional access, modifcation, change or birth time, in
        nanoseconds, as an integer in the range 0 to 999999999.

        Note that no accessors are provided for access, modification and
        change times - you need to get those from "stat _" if required ("int
        IO::AIO::st_atime" and so on will *not* generally give you the
        correct value).

    $seconds = IO::AIO::st_btimesec
        The (integral) seconds part of the file birth time, if available.

    ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtimensec
        Like the functions above, but returns all four times in one go (and
        maybe more in future versions).

    $counter = IO::AIO::st_gen
        Returns the generation counter of the file. This is only available
        on platforms which have this member in their "struct stat" (most
        BSDs at the time of this writing) and generally only to the root
        usert. If unsupported, 0 is returned, but this might change to
        "undef" in a future version.

    Example: print the high resolution modification time of /etc, using
    "stat", and "IO::AIO::aio_stat".

       if (stat "/etc") {
          printf "stat(/etc) mtime: %f\n", IO::AIO::st_mtime;
       }

       IO::AIO::aio_stat "/etc", sub {
          $_[0]
             and return;

          printf "aio_stat(/etc) mtime: %d.%09d\n", (stat _)[9], IO::AIO::st_mtimensec;
       };

       IO::AIO::flush;

    Output of the awbove on my system, showing reduced and full accuracy:

       stat(/etc) mtime: 1534043702.020808
       aio_stat(/etc) mtime: 1534043702.020807792

1696
   MISCELLANEOUS FUNCTIONS
1697 1698 1699 1700
    IO::AIO implements some functions that are useful when you want to use
    some "Advanced I/O" function not available to in Perl, without going the
    "Asynchronous I/O" route. Many of these have an asynchronous "aio_*"
    counterpart.
1701

1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725
    $numfd = IO::AIO::get_fdlimit
        This function is *EXPERIMENTAL* and subject to change.

        Tries to find the current file descriptor limit and returns it, or
        "undef" and sets $! in case of an error. The limit is one larger
        than the highest valid file descriptor number.

    IO::AIO::min_fdlimit [$numfd]
        This function is *EXPERIMENTAL* and subject to change.

        Try to increase the current file descriptor limit(s) to at least
        $numfd by changing the soft or hard file descriptor resource limit.
        If $numfd is missing, it will try to set a very high limit, although
        this is not recommended when you know the actual minimum that you
        require.

        If the limit cannot be raised enough, the function makes a
        best-effort attempt to increase the limit as much as possible, using
        various tricks, while still failing. You can query the resulting
        limit using "IO::AIO::get_fdlimit".

        If an error occurs, returns "undef" and sets $!, otherwise returns
        true.

1726 1727 1728 1729 1730 1731 1732 1733 1734 1735
    IO::AIO::sendfile $ofh, $ifh, $offset, $count
        Calls the "eio_sendfile_sync" function, which is like
        "aio_sendfile", but is blocking (this makes most sense if you know
        the input data is likely cached already and the output filehandle is
        set to non-blocking operations).

        Returns the number of bytes copied, or -1 on error.

    IO::AIO::fadvise $fh, $offset, $len, $advice
        Simply calls the "posix_fadvise" function (see its manpage for
1736
        details). The following advice constants are available:
1737 1738 1739 1740 1741 1742 1743 1744 1745
        "IO::AIO::FADV_NORMAL", "IO::AIO::FADV_SEQUENTIAL",
        "IO::AIO::FADV_RANDOM", "IO::AIO::FADV_NOREUSE",
        "IO::AIO::FADV_WILLNEED", "IO::AIO::FADV_DONTNEED".

        On systems that do not implement "posix_fadvise", this function
        returns ENOSYS, otherwise the return value of "posix_fadvise".

    IO::AIO::madvise $scalar, $offset, $len, $advice
        Simply calls the "posix_madvise" function (see its manpage for
1746
        details). The following advice constants are available:
1747 1748 1749 1750
        "IO::AIO::MADV_NORMAL", "IO::AIO::MADV_SEQUENTIAL",
        "IO::AIO::MADV_RANDOM", "IO::AIO::MADV_WILLNEED",
        "IO::AIO::MADV_DONTNEED".

1751 1752 1753 1754
        If $offset is negative, counts from the end. If $length is negative,
        the remaining length of the $scalar is used. If possible, $length
        will be reduced to fit into the $scalar.

1755 1756 1757 1758 1759 1760
        On systems that do not implement "posix_madvise", this function
        returns ENOSYS, otherwise the return value of "posix_madvise".

    IO::AIO::mprotect $scalar, $offset, $len, $protect
        Simply calls the "mprotect" function on the preferably AIO::mmap'ed
        $scalar (see its manpage for details). The following protect
1761
        constants are available: "IO::AIO::PROT_NONE", "IO::AIO::PROT_READ",
1762 1763
        "IO::AIO::PROT_WRITE", "IO::AIO::PROT_EXEC".

1764 1765 1766 1767
        If $offset is negative, counts from the end. If $length is negative,
        the remaining length of the $scalar is used. If possible, $length
        will be reduced to fit into the $scalar.

1768 1769 1770 1771 1772
        On systems that do not implement "mprotect", this function returns
        ENOSYS, otherwise the return value of "mprotect".

    IO::AIO::mmap $scalar, $length, $prot, $flags, $fh[, $offset]
        Memory-maps a file (or anonymous memory range) and attaches it to
1773 1774
        the given $scalar, which will act like a string scalar. Returns true
        on success, and false otherwise.
1775

1776 1777 1778 1779 1780 1781 1782 1783
        The scalar must exist, but its contents do not matter - this means
        you cannot use a nonexistant array or hash element. When in doubt,
        "undef" the scalar first.

        The only operations allowed on the mmapped scalar are
        "substr"/"vec", which don't change the string length, and most
        read-only operations such as copying it or searching it with regexes
        and so on.
1784 1785 1786 1787

        Anything else is unsafe and will, at best, result in memory leaks.

        The memory map associated with the $scalar is automatically removed
1788 1789
        when the $scalar is undef'd or destroyed, or when the
        "IO::AIO::mmap" or "IO::AIO::munmap" functions are called on it.
1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802

        This calls the "mmap"(2) function internally. See your system's
        manual page for details on the $length, $prot and $flags parameters.

        The $length must be larger than zero and smaller than the actual
        filesize.

        $prot is a combination of "IO::AIO::PROT_NONE",
        "IO::AIO::PROT_EXEC", "IO::AIO::PROT_READ" and/or
        "IO::AIO::PROT_WRITE",

        $flags can be a combination of "IO::AIO::MAP_SHARED" or
        "IO::AIO::MAP_PRIVATE", or a number of system-specific flags (when
1803 1804
        not available, the are 0): "IO::AIO::MAP_ANONYMOUS" (which is set to
        "MAP_ANON" if your system only provides this constant),
1805 1806 1807 1808 1809
        "IO::AIO::MAP_LOCKED", "IO::AIO::MAP_NORESERVE",
        "IO::AIO::MAP_POPULATE", "IO::AIO::MAP_NONBLOCK",
        "IO::AIO::MAP_FIXED", "IO::AIO::MAP_GROWSDOWN",
        "IO::AIO::MAP_32BIT", "IO::AIO::MAP_HUGETLB" or
        "IO::AIO::MAP_STACK".
1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831

        If $fh is "undef", then a file descriptor of -1 is passed.

        $offset is the offset from the start of the file - it generally must
        be a multiple of "IO::AIO::PAGESIZE" and defaults to 0.

        Example:

           use Digest::MD5;
           use IO::AIO;

           open my $fh, "<verybigfile"
              or die "$!";

           IO::AIO::mmap my $data, -s $fh, IO::AIO::PROT_READ, IO::AIO::MAP_SHARED, $fh
              or die "verybigfile: $!";

           my $fast_md5 = md5 $data;

    IO::AIO::munmap $scalar
        Removes a previous mmap and undefines the $scalar.

1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855
    IO::AIO::mremap $scalar, $new_length, $flags = MREMAP_MAYMOVE[,
    $new_address = 0]
        Calls the Linux-specific mremap(2) system call. The $scalar must
        have been mapped by "IO::AIO::mmap", and $flags must currently
        either be 0 or "IO::AIO::MREMAP_MAYMOVE".

        Returns true if successful, and false otherwise. If the underlying
        mmapped region has changed address, then the true value has the
        numerical value 1, otherwise it has the numerical value 0:

           my $success = IO::AIO::mremap $mmapped, 8192, IO::AIO::MREMAP_MAYMOVE
              or die "mremap: $!";

           if ($success*1) {
              warn "scalar has chanegd address in memory\n";
           }

        "IO::AIO::MREMAP_FIXED" and the $new_address argument are currently
        implemented, but not supported and might go away in a future
        version.

        On systems where this call is not supported or is not emulated, this
        call returns falls and sets $! to "ENOSYS".

1856 1857 1858 1859 1860 1861 1862 1863 1864 1865
    IO::AIO::munlock $scalar, $offset = 0, $length = undef
        Calls the "munlock" function, undoing the effects of a previous
        "aio_mlock" call (see its description for details).

    IO::AIO::munlockall
        Calls the "munlockall" function.

        On systems that do not implement "munlockall", this function returns
        ENOSYS, otherwise the return value of "munlockall".

1866 1867 1868 1869 1870
    IO::AIO::splice $r_fh, $r_off, $w_fh, $w_off, $length, $flags
        Calls the GNU/Linux splice(2) syscall, if available. If $r_off or
        $w_off are "undef", then "NULL" is passed for these, otherwise they
        should be the file offset.

1871 1872 1873
        $r_fh and $w_fh should not refer to the same file, as splice might
        silently corrupt the data in this case.

1874 1875 1876 1877 1878 1879 1880
        The following symbol flag values are available:
        "IO::AIO::SPLICE_F_MOVE", "IO::AIO::SPLICE_F_NONBLOCK",
        "IO::AIO::SPLICE_F_MORE" and "IO::AIO::SPLICE_F_GIFT".

        See the splice(2) manpage for details.

    IO::AIO::tee $r_fh, $w_fh, $length, $flags
1881
        Calls the GNU/Linux tee(2) syscall, see its manpage and the
1882 1883
        description for "IO::AIO::splice" above for details.

1884 1885 1886 1887 1888 1889
    $actual_size = IO::AIO::pipesize $r_fh[, $new_size]
        Attempts to query or change the pipe buffer size. Obviously works
        only on pipes, and currently works only on GNU/Linux systems, and
        fails with -1/"ENOSYS" everywhere else. If anybody knows how to
        influence pipe buffer size on other systems, drop me a note.

1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909
    ($rfh, $wfh) = IO::AIO::pipe2 [$flags]
        This is a direct interface to the Linux pipe2(2) system call. If
        $flags is missing or 0, then this should be the same as a call to
        perl's built-in "pipe" function and create a new pipe, and works on
        systems that lack the pipe2 syscall. On win32, this case invokes
        "_pipe (..., 4096, O_BINARY)".

        If $flags is non-zero, it tries to invoke the pipe2 system call with
        the given flags (Linux 2.6.27, glibc 2.9).

        On success, the read and write file handles are returned.

        On error, nothing will be returned. If the pipe2 syscall is missing
        and $flags is non-zero, fails with "ENOSYS".

        Please refer to pipe2(2) for more info on the $flags, but at the
        time of this writing, "IO::AIO::O_CLOEXEC", "IO::AIO::O_NONBLOCK"
        and "IO::AIO::O_DIRECT" (Linux 3.4, for packet-based pipes) were
        supported.

1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993
        Example: create a pipe race-free w.r.t. threads and fork:

           my ($rfh, $wfh) = IO::AIO::pipe2 IO::AIO::O_CLOEXEC
              or die "pipe2: $!\n";

    $fh = IO::AIO::eventfd [$initval, [$flags]]
        This is a direct interface to the Linux eventfd(2) system call. The
        (unhelpful) defaults for $initval and $flags are 0 for both.

        On success, the new eventfd filehandle is returned, otherwise
        returns "undef". If the eventfd syscall is missing, fails with
        "ENOSYS".

        Please refer to eventfd(2) for more info on this call.

        The following symbol flag values are available:
        "IO::AIO::EFD_CLOEXEC", "IO::AIO::EFD_NONBLOCK" and
        "IO::AIO::EFD_SEMAPHORE" (Linux 2.6.30).

        Example: create a new eventfd filehandle:

           $fh = IO::AIO::eventfd 0, IO::AIO::O_CLOEXEC
              or die "eventfd: $!\n";

    $fh = IO::AIO::timerfd_create $clockid[, $flags]
        This is a direct interface to the Linux timerfd_create(2) system
        call. The (unhelpful) default for $flags is 0.

        On success, the new timerfd filehandle is returned, otherwise
        returns "undef". If the eventfd syscall is missing, fails with
        "ENOSYS".

        Please refer to timerfd_create(2) for more info on this call.

        The following $clockid values are available:
        "IO::AIO::CLOCK_REALTIME", "IO::AIO::CLOCK_MONOTONIC"
        "IO::AIO::CLOCK_CLOCK_BOOTTIME" (Linux 3.15)
        "IO::AIO::CLOCK_CLOCK_REALTIME_ALARM" (Linux 3.11) and
        "IO::AIO::CLOCK_CLOCK_BOOTTIME_ALARM" (Linux 3.11).

        The following $flags values are available (Linux 2.6.27):
        "IO::AIO::TFD_NONBLOCK" and "IO::AIO::TFD_CLOEXEC".

        Example: create a new timerfd and set it to one-second repeated
        alarms, then wait for two alarms:

           my $fh = IO::AIO::timerfd_create IO::AIO::CLOCK_BOOTTIME, IO::AIO::TFD_CLOEXEC
              or die "timerfd_create: $!\n";

           defined IO::AIO::timerfd_settime $fh, 0, 1, 1
              or die "timerfd_settime: $!\n";

           for (1..2) {
              8 == sysread $fh, my $buf, 8
                 or die "timerfd read failure\n";

              printf "number of expirations (likely 1): %d\n",
                 unpack "Q", $buf;
           }

    ($cur_interval, $cur_value) = IO::AIO::timerfd_settime $fh, $flags,
    $new_interval, $nbw_value
        This is a direct interface to the Linux timerfd_settime(2) system
        call. Please refer to its manpage for more info on this call.

        The new itimerspec is specified using two (possibly fractional)
        second values, $new_interval and $new_value).

        On success, the current interval and value are returned (as per
        "timerfd_gettime"). On failure, the empty list is returned.

        The following $flags values are available:
        "IO::AIO::TFD_TIMER_ABSTIME" and "IO::AIO::TFD_TIMER_CANCEL_ON_SET".

        See "IO::AIO::timerfd_create" for a full example.

    ($cur_interval, $cur_value) = IO::AIO::timerfd_gettime $fh
        This is a direct interface to the Linux timerfd_gettime(2) system
        call. Please refer to its manpage for more info on this call.

        On success, returns the current values of interval and value for the
        given timerfd (as potentially fractional second values). On failure,
        the empty list is returned.

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023
EVENT LOOP INTEGRATION
    It is recommended to use AnyEvent::AIO to integrate IO::AIO
    automatically into many event loops:

     # AnyEvent integration (EV, Event, Glib, Tk, POE, urxvt, pureperl...)
     use AnyEvent::AIO;

    You can also integrate IO::AIO manually into many event loops, here are
    some examples of how to do this:

     # EV integration
     my $aio_w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb;

     # Event integration
     Event->io (fd => IO::AIO::poll_fileno,
                poll => 'r',
                cb => \&IO::AIO::poll_cb);

     # Glib/Gtk2 integration
     add_watch Glib::IO IO::AIO::poll_fileno,
               in => sub { IO::AIO::poll_cb; 1 };

     # Tk integration
     Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",
                               readable => \&IO::AIO::poll_cb);

     # Danga::Socket integration
     Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
                                 \&IO::AIO::poll_cb);

2024
  FORK BEHAVIOUR
2025 2026 2027
    Usage of pthreads in a program changes the semantics of fork
    considerably. Specifically, only async-safe functions can be called
    after fork. Perl doesn't know about this, so in general, you cannot call
2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040
    fork with defined behaviour in perl if pthreads are involved. IO::AIO
    uses pthreads, so this applies, but many other extensions and (for
    inexplicable reasons) perl itself often is linked against pthreads, so
    this limitation applies to quite a lot of perls.

    This module no longer tries to fight your OS, or POSIX. That means
    IO::AIO only works in the process that loaded it. Forking is fully
    supported, but using IO::AIO in the child is not.

    You might get around by not *using* IO::AIO before (or after) forking.
    You could also try to call the IO::AIO::reinit function in the child:

    IO::AIO::reinit
2041
        Abandons all current requests and I/O threads and simply
2042
        reinitialises all data structures. This is not an operation
2043
        supported by any standards, but happens to work on GNU/Linux and
2044 2045 2046 2047 2048 2049 2050
        some newer BSD systems.

        The only reasonable use for this function is to call it after
        forking, if "IO::AIO" was used in the parent. Calling it while
        IO::AIO is active in the process will result in undefined behaviour.
        Calling it at any time will also result in any undefined (by POSIX)
        behaviour.
2051

2052 2053 2054 2055 2056 2057 2058 2059
  LINUX-SPECIFIC CALLS
    When a call is documented as "linux-specific" then this means it
    originated on GNU/Linux. "IO::AIO" will usually try to autodetect the
    availability and compatibility of such calls regardless of the platform
    it is compiled on, so platforms such as FreeBSD which often implement
    these calls will work. When in doubt, call them and see if they fail wth
    "ENOSYS".

2060 2061 2062 2063 2064 2065 2066 2067 2068
  MEMORY USAGE
    Per-request usage:

    Each aio request uses - depending on your architecture - around 100-200
    bytes of memory. In addition, stat requests need a stat buffer (possibly
    a few hundred bytes), readdir requires a result buffer and so on. Perl
    scalars and other data passed into aio requests will also be locked and
    will consume memory till the request has entered the done state.

2069
    This is not awfully much, so queuing lots of requests is not usually a
2070 2071 2072 2073 2074 2075 2076 2077 2078
    problem.

    Per-thread usage:

    In the execution phase, some aio requests require more memory for
    temporary buffers, and each thread requires a stack and other data
    structures (usually around 16k-128k, depending on the OS).

KNOWN BUGS
2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089
    Known bugs will be fixed in the next release :)

KNOWN ISSUES
    Calls that try to "import" foreign memory areas (such as "IO::AIO::mmap"
    or "IO::AIO::aio_slurp") do not work with generic lvalues, such as
    non-created hash slots or other scalars I didn't think of. It's best to
    avoid such and either use scalar variables or making sure that the
    scalar exists (e.g. by storing "undef") and isn't "funny" (e.g. tied).

    I am not sure anything can be done about this, so this is considered a
    known issue, rather than a bug.
2090 2091

SEE ALSO
2092 2093
    AnyEvent::AIO for easy integration into event loops, Coro::AIO for a
    more natural syntax.
2094 2095 2096 2097 2098

AUTHOR
     Marc Lehmann <schmorp@schmorp.de>
     http://home.schmorp.de/