Newer Older
nicoo's avatar
nicoo committed
1 2
# Hacking on Dune

This section is for people who want to work on Dune itself.
nicoo's avatar
nicoo committed
4 5 6

## Bootstrap

7 8
In order to build itself, Dune uses an OCaml script
([bootstrap.ml](bootstrap.ml)) that dumps most of the sources of Dune into a
nicoo's avatar
nicoo committed
9 10 11
single `boot.ml` file. This file is built using `ocamlopt` or `ocamlc`
and used to build everything else.

12 13 14 15 16 17 18 19
To get the development version bootstrapped and installed, run the following
while in the project's root directory:

$ make          # create the dune.exe file and bootstrap the dune build
$ make install  # install the newly built dune

nicoo's avatar
nicoo committed
20 21 22 23 24 25 26 27 28 29
Note that we don't include all of the sources in boot.ml. We skip a
few parts to speed up the build. In particular:
- vendored libraries are replaced by simpler implementations taken
  from `vendor/boot`
- a few files in `src` have an alternative version. These alternatives
  versions are named `XXX.boot.EXT`. For instance: `glob_lexer.boot.ml`

## OCaml compatibility test

Install opam switches for all the entries in the
[dune-workspace.dev](dune-workspace.dev) file and run:
nicoo's avatar
nicoo committed
31 32 33 34 35 36 37

$ make all-supported-ocaml-versions

## Repository organization

38 39
- `vendor/` contains dependencies of Dune, that have been vendored
- `plugin/` contains the API given to `dune` files that are OCaml
nicoo's avatar
nicoo committed
- `src/` contains the core of `Dune`
nicoo's avatar
nicoo committed
42 43 44 45 46
- `bin/` contains the command line interface
- `doc/` contains the manual and rules to generate the manual pages

## Design

47 48 49
Dune (nee "JBuilder") was initially designed to sort out the public release of
Jane Street packages which became incredibly complicated over time. It is still
successfully used for this purpose.
nicoo's avatar
nicoo committed
50 51 52 53 54 55 56 57 58

One necessary feature to achieve this is the ability to precisely
report the external dependencies necessary to build a given set of
targets without running any command, just by looking at the source
tree. This is used to automatically generate the `<package>.opam`
files for all Jane Street packages.

To implement this, the build rules are described using a build arrow,
which is defined in [src/build.mli](src/build.mli). In the end it makes the
development of the internal rules of Dune very composable and
nicoo's avatar
nicoo committed
60 61
quite pleasant.

62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
To deal with process multiplexing, Dune uses a simplified
Lwt/Async-like monad, implemented in [src/fiber/fiber.mli](src/fiber/fiber.mli).

## Tests

Dune uses [cram style](https://blog.janestreet.com/testing-with-expectations/)
tests for its test suite. The test suite is contained in
[test/blackbox-tests](test/blackbox-tests). A single test consists of a test
directory in the test-cases/ sub directory which contains a run.t file defining
the test.

An example `run.t` file:

A description of the test. The command running the tests is preceeded by
two spaces and a $. The expected output of the command is also indented by
two spaces and is right below the command (note the absence of a $)

  $ echo "the expected output is below"
  the expected output is below

Running the entire test suite is done with `$ make test`. A particular test can
be executed with `$ dune build @<test-name>`.

Running the test will execute the command after the `$` and its output will be
compared against the expected output right below the command. Any differences
will result in a test failure.

### Adding a Test

Simply add a new directory in test/blackbox-tests/test-cases and then `$ make`
generate the rules for the test , followed by `$ make promote` to accept the new

### Accepting Corrections

A failing expect test will generate a diff between the expected and actual
output. If the new output generated by the command is satisfactory, it can be
*promoted* with the `$ make promote` command.
nicoo's avatar
nicoo committed
102 103 104

## Code flow

105 106
- [src/dune_file.mli](src/dune_file.mli) contains the internal representation
  of `dune` files and the parsing code
nicoo's avatar
nicoo committed
107 108
- [src/jbuild_load.mli](src/jbuild_load.mli) contains the code to scan
  a source tree and build the internal database by reading
  the `dune` files
nicoo's avatar
nicoo committed
- [src/gen_rules.mli](src/gen_rules.mli) contains all the build rules
  of Dune
nicoo's avatar
nicoo committed
112 113 114
- [src/build_system.mli](src/build_system.mli) contains a trivial
  implementation of a Build system. This is what Jenga will provide
  when implementing the bridge