New upstream version 0.45.1+dfsg

                    Building and Installing HTSlib

                    ==============================

Requirements

============

Building HTSlib requires a few programs and libraries to be present.

At least the following are required:

    GNU make

    C compiler (e.g. gcc or clang)

In addition, building the configure script requires:

    autoheader

    autoconf

Running the configure script uses awk, along with a number of

standard UNIX tools (cat, cp, grep, mv, rm, sed, among others).  Almost

all installations will have these already.  

Running the test harness (make test) uses:

    bash

    perl

HTSlib uses the following external libraries.  Building requires both the

library itself, and include files needed to compile code that uses functions

from the library.  Note that some Linux distributions put include files in

a development ('-dev' or '-devel') package separate from the main library.

    libz       (required)

    libbz2     (required, unless configured with --disable-bz2)

    liblzma    (required, unless configured with --disable-lzma)

    libcurl    (optional, but strongly recommended)

    libcrypto  (optional for Amazon S3 support; not needed on MacOS)

Disabling libbzip2 and liblzma will make some CRAM files unreadable, so

is not recommended.

Using libcurl provides HTSlib with better network protocol support, for

example it enables the use of https:// URLs.  It is also required if

direct access to Amazon S3 or Google Cloud Storage is enabled.

Amazon S3 support requires an HMAC function to calculate a message

authentication code.  On MacOS, the CCHmac function from the standard

library is used.  Systems that do not have CChmac will get this from

libcrypto.  libcrypto is part of OpenSSL or one of its derivatives (LibreSSL

or BoringSSL).

Building Configure

==================

This step is only needed if configure.ac has been changed, or if configure

does not exist (for example, when building from a git clone).  The

configure script and config.h.in can be built by running:

    autoheader

    autoconf

If you have a full GNU autotools install, you can alternatively run:

    autoreconf

Basic Installation

==================

To build and install HTSlib, 'cd' to the htslib-1.x directory containing

the package's source and type the following commands:

    ./configure

    make

    make install

The './configure' command checks your build environment and allows various

optional functionality to be enabled (see Configuration below).  If you

don't want to select any optional functionality, you may wish to omit

configure and just type 'make; make install' as for previous versions

of HTSlib.  However if the build fails you should run './configure' as

it can diagnose the common reasons for build failures.

The 'make' command builds the HTSlib library and and various useful

utilities: bgzip, htsfile, and tabix.  If compilation fails you should

run './configure' as it can diagnose problems with your build environment

that cause build failures.

The 'make install' command installs the libraries, library header files,

utilities, several manual pages, and a pkgconfig file to /usr/local.

The installation location can be changed by configuring with --prefix=DIR

or via 'make prefix=DIR install' (see Installation Locations below).

Configuration

=============

By default, './configure' examines your build environment, checking for

requirements such as the zlib development files, and arranges for a plain

HTSlib build.  The following configure options can be used to enable

various features and specify further optional external requirements:

--enable-plugins

    Use plugins to implement exotic file access protocols and other

    specialised facilities.  This enables such facilities to be developed

    and packaged outwith HTSlib, and somewhat isolates HTSlib-using programs

    from their library dependencies.  By default (or with --disable-plugins),

    any enabled pluggable facilities (such as libcurl file access) are built

    directly within HTSlib.

    The <https://github.com/samtools/htslib-plugins> repository contains

    several additional plugins, including the iRODS (<http://irods.org/>)

    file access plugin previously distributed with HTSlib.

--with-plugin-dir=DIR

    Specifies the directory into which plugins built while building HTSlib

    should be installed; by default, LIBEXECDIR/htslib.

--with-plugin-path=DIR:DIR:DIR...

    Specifies the list of directories that HTSlib will search for plugins.

    By default, only the directory specified via --with-plugin-dir will be

    searched; you can use --with-plugin-path='DIR:$(plugindir):DIR' and so

    on to cause additional directories to be searched.

--enable-libcurl

    Use libcurl (<http://curl.haxx.se/>) to implement network access to

    remote files via FTP, HTTP, HTTPS, etc.  By default, HTSlib uses its

    own simple networking code to provide access via FTP and HTTP only.

--enable-gcs

    Implement network access to Google Cloud Storage.  By default or with

    --enable-gcs=check, this is enabled when libcurl is enabled.

--enable-s3

    Implement network access to Amazon AWS S3.  By default or with

    --enable-s3=check, this is enabled when libcurl is enabled.

--disable-bz2

    Bzip2 is an optional compression codec format for CRAM, included

    in HTSlib by default.  It can be disabled with --disable-bz2, but

    be aware that not all CRAM files may be possible to decode.

--disable-lzma

    LZMA is an optional compression codec for CRAM, included in HTSlib

    by default.  It can be disabled with --disable-lzma, but be aware

    that not all CRAM files may be possible to decode.

The configure script also accepts the usual options and environment variables

for tuning installation locations and compilers: type './configure --help'

for details.  For example,

    ./configure CC=icc --prefix=/opt/icc-compiled

would specify that HTSlib is to be built with icc and installed into bin,

lib, etc subdirectories under /opt/icc-compiled.

Installation Locations

======================

By default, 'make install' installs HTSlib libraries under /usr/local/lib,

HTSlib header files under /usr/local/include, utility programs under

/usr/local/bin, etc.  (To be precise, the header files are installed within

a fixed 'htslib' subdirectory under the specified .../include location.)

You can specify a different location to install HTSlib by configuring

with --prefix=DIR or specify locations for particular parts of HTSlib by

configuring with --libdir=DIR and so on.  Type './configure --help' for

the full list of such install directory options.

Alternatively you can specify different locations at install time by

typing 'make prefix=DIR install' or 'make libdir=DIR install' and so on.

Consult the list of prefix/exec_prefix/etc variables near the top of the

Makefile for the full list of such variables that can be overridden.

You can also specify a staging area by typing 'make DESTDIR=DIR install',

possibly in conjunction with other --prefix or prefix=DIR settings.

For example,

    make DESTDIR=/tmp/staging prefix=/opt

would install into bin, lib, etc subdirectories under /tmp/staging/opt.

[Files in this distribution outwith the cram/ subdirectory are distributed

according to the terms of the following MIT/Expat license.]

The MIT/Expat License

Copyright (C) 2012-2014 Genome Research Ltd.

Permission is hereby granted, free of charge, to any person obtaining a copy

of this software and associated documentation files (the "Software"), to deal

in the Software without restriction, including without limitation the rights

to use, copy, modify, merge, publish, distribute, sublicense, and/or sell

copies of the Software, and to permit persons to whom the Software is

furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in

all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL

THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER

LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING

FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER

DEALINGS IN THE SOFTWARE.

[Files within the cram/ subdirectory in this distribution are distributed

according to the terms of the following Modified 3-Clause BSD license.]

The Modified-BSD License

Copyright (C) 2012-2014 Genome Research Ltd.

Redistribution and use in source and binary forms, with or without

modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice,

   this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice,

   this list of conditions and the following disclaimer in the documentation

   and/or other materials provided with the distribution.

3. Neither the names Genome Research Ltd and Wellcome Trust Sanger Institute

   nor the names of its contributors may be used to endorse or promote products

   derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY GENOME RESEARCH LTD AND CONTRIBUTORS "AS IS"

AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE

IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE

DISCLAIMED. IN NO EVENT SHALL GENOME RESEARCH LTD OR ITS CONTRIBUTORS BE LIABLE

FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL

DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR

SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER

CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,

OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE

OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

[The use of a range of years within a copyright notice in this distribution

should be interpreted as being equivalent to a list of years including the

first and last year specified and all consecutive years between them.

For example, a copyright notice that reads "Copyright (C) 2005, 2007-2009,

2011-2012" should be interpreted as being identical to a notice that reads

"Copyright (C) 2005, 2007, 2008, 2009, 2011, 2012" and a copyright notice

that reads "Copyright (C) 2005-2012" should be interpreted as being identical

to a notice that reads "Copyright (C) 2005, 2006, 2007, 2008, 2009, 2010,

2011, 2012".]

Noteworthy changes in release 1.4.1  (8th May 2017)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This is primarily a security bug fix update.

* Fixed SECURITY issue with buffer overruns with malicious data. (#514).

* S3 support for non Amazon AWS endpoints. (#506)

* Support for variant breakpoints in bcftools. (#516)

* Improved handling of BCF NaNs. (#485)

* Compilation / portability improvements. (#255, #423, #498, #488)

* Miscellaneous bug fixes (#482, #521, #522, #523, #524).

* Sanitise headers (#509)

Release 1.4 (13 March 2017)

* Incompatible changes: several functions and data types have been changed

  in this release, and the shared library soversion has been bumped to 2.

  - bam_pileup1_t has an additional field (which holds user data)

  - bam1_core_t has been modified to allow for >64K CIGAR operations

    and (along with bam1_t) so that CIGAR entries are aligned in memory

  - hopen() has vararg arguments for setting URL scheme-dependent options

  - the various tbx_conf_* presets are now const

  - auxiliary fields in bam1_t are now always stored in little-endian byte

    order (previously this depended on if you read a bam, sam or cram file)

  - index metadata (accessible via hts_idx_get_meta()) is now always

    stored in little-endian byte order (previously this depended on if

    the index was in tbi or csi format)

  - bam_aux2i() now returns an int64_t value

  - fai_load() will no longer save local copies of remote fasta indexes

  - hts_idx_get_meta() now takes a uint32_t * for l_meta (was int32_t *)

* HTSlib now links against libbz2 and liblzma by default.  To remove these

  dependencies, run configure with options --disable-bz2 and --disable-lzma,

  but note that this may make some CRAM files produced elsewhere unreadable. 

* Added a thread pool interface and replaced the bgzf multi-threading

  code to use this pool.  BAM and CRAM decoding is now multi-threaded

  too, using the pool to automatically balance the number of threads

  between decode, encode and any data processing jobs.

* New errmod_cal(), probaln_glocal(), sam_cap_mapq(), and sam_prob_realn()

  functions, previously internal to SAMtools, have been added to HTSlib.

* Files can now be accessed via Google Cloud Storage using gs: URLs, when

  HTSlib is configured to use libcurl for network file access rather than

  the included basic knetfile networking.

* S3 file access now also supports the "host_base" setting in the

  $HOME/.s3cfg configuration file.

* Data URLs ("data:,text") now follow the standard RFC 2397 format and may

  be base64-encoded (when written as "data:;base64,text") or may include

  percent-encoded characters.  HTSlib's previous over-simplified "data:text"

  format is no longer supported -- you will need to add an initial comma.

* When plugins are enabled, S3 support is now provided by a separate

  hfile_s3 plugin rather than by hfile_libcurl itself as previously.

  When --enable-libcurl is used, by default both GCS and S3 support

  and plugins will also be built; they can be individually disabled

  via --disable-gcs and --disable-s3.

* The iRODS file access plugin has been moved to a separate repository.

  Configure no longer has a --with-irods option; instead build the plugin

  found at <https://github.com/samtools/htslib-plugins>.

* APIs to portably read and write (possibly unaligned) data in little-endian

  byte order have been added.

* New functions bam_auxB_len(), bam_auxB2i() and bam_auxB2f() have been

  added to make accessing array-type auxiliary data easier.  bam_aux2i()

  can now return the full range of values that can be stored in an integer

  tag (including unsigned 32 bit tags).  bam_aux2f() will return the value

  of integer tags (as a double) as well as floating-point ones.  All of

  the bam_aux2 and bam_auxB2 functions will set errno if the requested

  conversion is not valid.

* New functions fai_load3() and fai_build3() allow fasta indexes to be

  stored in a different location to the indexed fasta file.

* New functions bgzf_index_dump_hfile() and bgzf_index_load_hfile()

  allow bgzf index files (.gzi) to be written to / read from an existing

  hFILE handle.

* hts_idx_push() will report when trying to add a range to an index that

  is beyond the limits that the given index can handle.  This means trying

  to index chromosomes longer than 2^29 bases with a .bai or .tbi index

  will report an error instead of apparantly working but creating an invalid

  index entry.

* VCF formatting is now approximately 4x faster.  (Whether this is

  noticable depends on what was creating the VCF.)

* CRAM lossy_names mode now works with TLEN of 0 or TLEN within +/- 1

  of the computed value.  Note in these situations TLEN will be

  generated / fixed during CRAM decode.

* CRAM now supports bzip2 and lzma codecs.  Within htslib these are

  disabled by default, but can be enabled by specifying "use_bzip2" or

  "use_lzma" in an hts_opt_add() call or via the mode string of the

  hts_open_format() function.

Noteworthy changes in release 1.3.2  (13 September 2016)

* Corrected bin calculation when converting directly from CRAM to BAM.

  Previously a small fraction of converted reads would fail Picard's

  validation with "bin field of BAM record does not equal value computed"

  (SAMtools issue #574).

* Plugins can now signal to HTSlib which of RTLD_LOCAL and RTLD_GLOBAL

  they wish to be opened with -- previously they were always RTLD_LOCAL.

Noteworthy changes in release 1.3.1  (22 April 2016)

* Improved error checking and reporting, especially of I/O errors when

  writing output files (#17, #315, PR #271, PR #317).

* Build fixes for 32-bit systems; be sure to run configure to enable

  large file support and access to 2GiB+ files.

* Numerous VCF parsing fixes (#321, #322, #323, #324, #325; PR #370).

  Particular thanks to Kostya Kortchinsky of the Google Security Team

  for testing and numerous input parsing bug reports.

* HTSlib now prints an informational message when initially creating a

  CRAM reference cache in the default location under your $HOME directory.

  (No message is printed if you are using $REF_CACHE to specify a location.)

* Avoided rare race condition when caching downloaded CRAM reference sequence

  files, by using distinctive names for temporary files (in addition to O_EXCL,

  which has always been used).  Occasional corruption would previously occur

  when multiple tools were simultaneously caching the same reference sequences

  on an NFS filesystem that did not support O_EXCL (PR #320).

* Prevented race condition in file access plugin loading (PR #341).

* Fixed mpileup memory leak, so no more "[bam_plp_destroy] memory leak [...]

  Continue anyway" warning messages (#299).

* Various minor CRAM fixes.

* Fixed documentation problems #348 and #358.

Noteworthy changes in release 1.3  (15 December 2015)

* Files can now be accessed via HTTPS and Amazon S3 in addition to HTTP

  and FTP, when HTSlib is configured to use libcurl for network file access

  rather than the included basic knetfile networking.

* HTSlib can be built to use remote access hFILE backends (such as iRODS

  and libcurl) via a plugin mechanism.  This allows other backends to be

  easily added and facilitates building tools that use HTSlib, as they

  don't need to be linked with the backends' various required libraries.

* When writing CRAM output, sam_open() etc now default to writing CRAM v3.0

  rather than v2.1.

* fai_build() and samtools faidx now accept initial whitespace in ">"

  headers (e.g., "> chr1 description" is taken to refer to "chr1").

* tabix --only-header works again (was broken in 1.2.x; #249).

* HTSlib's configure script and Makefile now fully support the standard

  convention of allowing CC/CPPFLAGS/CFLAGS/LDFLAGS/LIBS to be overridden

  as needed.  Previously the Makefile listened to $(LDLIBS) instead; if you

  were overriding that, you should now override LIBS rather than LDLIBS.

* Fixed bugs #168, #172, #176, #197, #206, #225, #245, #265, #295, and #296.

Noteworthy changes in release 1.2.1  (3 February 2015)

* Reinstated hts_file_type() and FT_* macros, which were available until 1.1

  but briefly removed in 1.2.  This function is deprecated and will be removed

  in a future release -- you should use hts_detect_format() etc instead

Noteworthy changes in release 1.2  (2 February 2015)

* HTSlib now has a configure script which checks your build environment

  and allows for selection of optional extras.  See INSTALL for details

* By default, reference sequences are fetched from the EBI CRAM Reference

  Registry and cached in your $HOME cache directory.  This behaviour can

  be controlled by setting REF_PATH and REF_CACHE enviroment variables

  (see the samtools(1) man page for details)

* Numerous CRAM improvements:

  - Support for CRAM v3.0, an upcoming revision to CRAM supporting

    better compression and per-container checksums

  - EOF checking for v2.1 and v3.0 (similar to checking BAM EOF blocks)

  - Non-standard values for PNEXT and TLEN fields are now preserved

  - hts_set_fai_filename() now provides a reference file when encoding

  - Generated read names are now numbered from 1, rather than being

    labelled 'slice:record-in-slice'

  - Multi-threading and speed improvements

* New htsfile command for identifying file formats, and corresponding

  file format detection APIs

* New tabix --regions FILE, --targets FILE options for filtering via BED files

* Optional iRODS file access, disabled by default.  Configure with --with-irods

  to enable accessing iRODS data objects directly via 'irods:DATAOBJ'

* All occurences of 2^29 in the source have been eliminated, so indexing

  and querying against reference sequences larger than 512Mbp works (when

  using CSI indices)

* Support for plain GZIP compression in various places

* VCF header editing speed improvements

* Added seq_nt16_int[] (equivalent to the samtools API's bam_nt16_nt4_table)

* Reinstated faidx_fetch_nseq(), which was accidentally removed from 1.1.

  Now faidx_fetch_nseq() and faidx_nseq() are equivalent; eventually

  faidx_fetch_nseq() will be deprecated and removed [#156]

* Fixed bugs #141, #152, #155, #158, #159, and various memory leaks

HTSlib is an implementation of a unified C library for accessing common file

formats, such as SAM, CRAM, VCF, and BCF, used for high-throughput sequencing

data.  It is the core library used by samtools and bcftools.

See INSTALL for building and installation instructions.