README.md 3.92 KB
Newer Older
Iñaki Malerba's avatar
Iñaki Malerba committed
1
# Debian pipeline for Developers
Agustin Henze's avatar
Agustin Henze committed
2 3 4 5 6 7 8 9 10 11

Gitlab integrates very well the continuous integration (CI) with the repository, so since Debian has migrated to Gitlab we can have CI in our repositories easily. Actually, we can have the same services that Debian QA offers today but before uploading a release to the archive. The main idea is having a faster feedback when you are working in a package, it is not intended to replace Debian QA. Debian QA services are the answer if you want to know if your package is reproducible or if piuparts is passing ok, etc.

We started looking for the services that Debian QA is offering today and tried to reach the same point but inside Gitlab CI. The services we got working until now, are the following:

 * Building the package from the source (only gbp is supported for now)
 * Lintian check
 * Reproducible build (Using reprotest)
 * piuparts
 * Autopkgtest
12
 * Buildd Log Scanner
Agustin Henze's avatar
Agustin Henze committed
13

Iñaki Malerba's avatar
Iñaki Malerba committed
14 15 16
Those services are enabled by something we called `salsa-pipeline` and it'll be shared for all Gitlab projects who adopt it. Having the same pipeline on GCI ensure that every package accomplishes the minimum quality to be in the archive and if we improve or add a new service the project will get the benefit instantaneously.


Iñaki Malerba's avatar
Iñaki Malerba committed
17
## Using Salsa-Pipeline
Iñaki Malerba's avatar
Iñaki Malerba committed
18

Iñaki Malerba's avatar
Iñaki Malerba committed
19 20 21 22 23 24
The `salsa-ci.yml` template only delivers the jobs definitions. Including only this file, no job will be added to the pipeline.
On the other hand, `pipeline-jobs.yml` includes all the jobs' implementations.

To use the Salsa Pipeline, simply add a `debian/gitlab-ci.yml` like the following:

```yaml
25
include:
26 27
  - https://salsa.debian.org/salsa-ci-team/pipeline/raw/master/salsa-ci.yml
  - https://salsa.debian.org/salsa-ci-team/pipeline/raw/master/pipeline-jobs.yml
Iñaki Malerba's avatar
Iñaki Malerba committed
28 29

variables:
30
  RELEASE: 'unstable'
Iñaki Malerba's avatar
Iñaki Malerba committed
31 32 33 34 35 36 37 38 39 40 41
```

`RELEASE: 'unstable'` can be replaced with any of the releases provided.

On Debian projects, you would normally want to put this file under the `debian/` folder, so changing the config path on your project will be necessary.
This can be done on `Settings` -> `CI/CD` -> `General Pipelines` -> `Custom CI config path`.

### Including only the jobs' definition

The `salsa-ci.yml` file provides the definition of build and test jobs.
It you want to use this tools to build and test your project, but you want to explicitly define which jobs to run, you can do as follows:
Iñaki Malerba's avatar
Iñaki Malerba committed
42

Iñaki Malerba's avatar
Iñaki Malerba committed
43 44
```yaml
include: https://salsa.debian.org/salsa-ci-team/pipeline/raw/master/salsa-ci.yml
Iñaki Malerba's avatar
Iñaki Malerba committed
45

Iñaki Malerba's avatar
Iñaki Malerba committed
46
variables:
47
  RELEASE: 'unstable'
Iñaki Malerba's avatar
Iñaki Malerba committed
48

Iñaki Malerba's avatar
Iñaki Malerba committed
49
build:
50
  extends: .build-package
Iñaki Malerba's avatar
Iñaki Malerba committed
51

Iñaki Malerba's avatar
Iñaki Malerba committed
52
reprotest:
53
  extends: .test-reprotest
Iñaki Malerba's avatar
Iñaki Malerba committed
54 55

lintian:
56
  extends: .test-lintian
Iñaki Malerba's avatar
Iñaki Malerba committed
57 58

autopkgtest:
59
  extends: .test-autopkgtest
Iñaki Malerba's avatar
Iñaki Malerba committed
60

61
blhc:
62
  extends: .test-blhc
63

Iñaki Malerba's avatar
Iñaki Malerba committed
64
piuparts:
65
  extends: .test-piuparts
Iñaki Malerba's avatar
Iñaki Malerba committed
66 67 68

```

69
On the previous example, the package is built on Debian unstable and tested on all five tests.
70
You can choose to run only some of the jobs.
Iñaki Malerba's avatar
Iñaki Malerba committed
71 72 73
Anyway, we **firmly recommend NOT to do it**.

### Building
Iñaki Malerba's avatar
Iñaki Malerba committed
74
The Debian release can be specified declaring the variable `RELEASE` on any of the images availables.
75
 - experimental
Iñaki Malerba's avatar
Iñaki Malerba committed
76 77 78 79 80
 - unstable
 - stretch-backports
 - stretch
 - jessie

81
This release is also going to be used for some stages like lintian.
Iñaki Malerba's avatar
Iñaki Malerba committed
82 83 84 85 86 87 88 89 90
By default, `unstable` is used.

To change the `RELEASE`, define this after including the yaml:
```yaml
variables:
    RELEASE: 'stretch'
```
Replace `stretch` with any of the releases listed previously.

Iñaki Malerba's avatar
Iñaki Malerba committed
91 92

### Testing
93
5 different tests are available to be used:
94
 - [test-autopkgtest](https://salsa.debian.org/ci-team/autopkgtest/raw/master/doc/README.package-tests.rst)
Iñaki Malerba's avatar
Iñaki Malerba committed
95 96
 - [test-lintian](https://github.com/Debian/lintian)
 - [test-reprotest](https://reproducible-builds.org/tools)
Iñaki Malerba's avatar
Iñaki Malerba committed
97
   - Reprotest stage can be run with diffoscope, which is an useful tool that helps identifying reproducibility issues. Large projects won't pass on low resources runners as the ones available right now. To use it, just extend from `test-reprotest-diffoscope`
Iñaki Malerba's avatar
Iñaki Malerba committed
98
 - [test-piuparts](https://piuparts.debian.org)
99
 - [test-blhc](https://qa.debian.org/bls/)
Iñaki Malerba's avatar
Iñaki Malerba committed
100 101

## Support
102
\#salsaci on OFTC or open an issue here.