New upstream version 5.2.1

    machine: true

    environment:

      - GCLOUD: /opt/google-cloud-sdk/bin/gcloud

      - GSUTIL: /opt/google-cloud-sdk/bin/gsutil

      - BOTO_CONFIG: /dev/null

    steps:

      - checkout

            # otherwise init cloud

            echo $GCLOUD_SERVICE_KEY | base64 --decode --ignore-garbage > ${HOME}/gcloud-service-key.json

            sudo $GCLOUD components install kubectl

            sudo $GCLOUD auth activate-service-account --key-file=${HOME}/gcloud-service-key.json

            sudo $GCLOUD config set project snakemake-testing

            sudo chown -R $USER:$USER /home/circleci/.config/gcloud

            $GCLOUD auth activate-service-account --key-file=${HOME}/gcloud-service-key.json

            $GCLOUD config set project snakemake-testing

      - run:

          name: Run tests

          no_output_timeout: 30m

          command: |

            export GCLOUD_CLUSTER=t-`uuidgen`

            export GOOGLE_APPLICATION_CREDENTIALS=${HOME}/gcloud-service-key.json

# Change Log

# [5.2.0] - 2018-06-28

## Changed

- Directory outputs have to marked with `directory`. This ensures proper handling of timestamps and cleanup. This is a breaking change. Implemented by Rasmus Ågren. 

- Fixed kubernetes tests, fixed kubernetes volume handling. Implemented by Andrew Schriefer.

- jinja2 and networkx are not optional dependencies when installing via pip.

- When conda or singularity directives are used and the corresponding CLI flags are not specified, the user is notified at the beginning of the log output.

- Fixed numerous small bugs and papercuts and extended documentation.

# [5.1.5] - 2018-06-24

## Changed

- fixed missing version info in docker image.

- Mattias Franberg

- Matt Shirley

- Paul Moore

- percyfal

- Per Unneberg

- Ryan C. Thompson

- Ryan Dale

to your shell command in the problematic rule.

I don't want Snakemake to detect an error if my shell command exits with an exitcode > 1. What can I do?

---------------------------------------------------------------------------------------------------------

Sometimes, tools encode information in exit codes bigger than 1. Snakemake by default treats anything > 0 as an error. Special cases have to be added by yourself. For example, you can write

.. code-block:: python

    shell:

        """

        set +e

        somecommand ...

        exitcode=$?

        if [ $exitcode -eq 1 ]

        then

            exit 1

        else

            exit 0

        fi

        """

This way, Snakemake only treats exit code 1 as an error, and thinks that everything else is fine.

Note that such tools are an excellent use case for contributing a `wrapper <https://snakemake-wrappers.readthedocs.io>`_.

.. _glob-wildcards:

How do I run my rule on all files of a certain directory?

    $ snakemake mytarget --config foo=bar

How do I enforce config values given at the command line to be interpreted as strings?

--------------------------------------------------------------------------------------

When passing config values like this

.. code-block:: console

    $ snakemake --config version=2018_1

Snakemake will first try to interpret the given value as number.

Only if that fails, it will interpret the value as string.

Here, it does not fail, because the underscore `_` is interpreted as thousand separator.

In order to ensure that the value is interpreted as string, you have to pass it in quotes.

Since bash otherwise automatically removes quotes, you have to also wrap the entire entry into quotes, e.g.:

.. code-block:: console

    $ snakemake --config 'version="2018_1"'

How do I make my rule fail if an output file is empty?

------------------------------------------------------

        input:

            S3.remote(expand("bucket-name/{letter}-2.txt", letter=["A", "B", "C"]))

It is possible to use S3-compatible storage by specifying a different endpoint address as the `host` kwarg in the provider, as the kwargs used in instantiating the provider are passed in to `boto <https://boto.readthedocs.org/en/latest/ref/s3.html#boto.s3.connection.S3Connection>`_:

.. code-block:: python

    from snakemake.remote.S3 import RemoteProvider as S3RemoteProvider

    S3 = S3RemoteProvider(access_key_id="MYACCESSKEY", secret_access_key="MYSECRET", host="mystorage.example.com")

    rule all:

        input:

            S3.remote("bucket-name/file.txt")

Only remote files needed to satisfy the DAG build are downloaded for the workflow. By default, remote files are downloaded prior to rule execution and are removed locally as soon as no rules depend on them. Remote files can be explicitly kept by setting the ``keep_local=True`` keyword argument:

.. code-block:: python