configure.markdown 37.6 KB
Newer Older
Wyatt Alt's avatar
Wyatt Alt committed
1
---
2
title: "PuppetDB 4.4: Configuring PuppetDB"
3
layout: default
4
canonical: "/puppetdb/latest/configure.html"
5 6
---

7
[logback]: http://logback.qos.ch/manual/configuration.html
8 9
[dashboard]: ./maintain_and_tune.html#monitor-the-performance-dashboard
[repl]: ./repl.html
10
[pg_trgm]: http://www.postgresql.org/docs/current/static/pgtrgm.html
11 12
[postgres_ssl]: ./postgres_ssl.html
[module]: ./install_via_module.html
13
[puppetdb.conf]: ./connect_puppet_master.html#edit-puppetdbconf
14 15
[ha]: ./ha.html
[node-ttl]: #node-ttl
16 17 18 19 20 21

Summary
-----

PuppetDB has three main groups of settings:

22 23 24
* The init script's configuration file, which sets the JVM heap size and the location of PuppetDB's main config file.
* Logging settings, which go in the [logback.xml](#logging-config) file and can be changed without restarting PuppetDB.
* All other settings, which go in PuppetDB's configuration file(s) and take effect after the service is restarted.
25 26 27 28

Init Script Config File
-----

29 30
If you installed PuppetDB from packages or used the `rake install`
installation method, an init script was created for PuppetDB. This
31
script has its own configuration file, the location of which varies by
32
platform and by package:
33

34 35 36 37 38 39
OS and Package               | File
-----------------------------|----------------------------
Red Hat-like (open source)   | `/etc/sysconfig/puppetdb`
Red Hat-like (PE)            | `/etc/sysconfig/pe-puppetdb`
Debian/Ubuntu (open source)  | `/etc/default/puppetdb`
Debian/Ubuntu (PE)           | `/etc/default/pe-puppetdb`
40 41 42

 In this file, you can change the following settings:

43 44 45 46 47
- **`JAVA_BIN`**: the location of the Java binary.
- **`JAVA_ARGS`**: command line options for the Java binary, most notably the `-Xmx` (max heap size) flag.
- **`USER`**: the user PuppetDB should be running as.
- **`INSTALL_DIR`**: the directory into which PuppetDB is installed.
- **`CONFIG`**: the location of the PuppetDB config file, which may be a single file or a directory of .ini files.
48

49
### Configuring the Java heap size
50

51 52 53
To change the JVM heap size for PuppetDB, edit the [init script config
file](#init-script-config-file) by setting a new value for the `-Xmx`
flag in the `JAVA_ARGS` variable.
54 55 56 57 58 59 60 61 62

For example, to cap PuppetDB at 192MB of memory:

    JAVA_ARGS="-Xmx192m"

To use 1GB of memory:

    JAVA_ARGS="-Xmx1g"

63
### Configuring JMX access
64 65 66 67 68 69 70 71 72 73 74 75

While all JMX metrics are exposed using the `/metrics` namespace, you can also
expose direct JMX access using standard JVM means as documented
[here](http://docs.oracle.com/javase/6/docs/technotes/guides/management/agent.html).
This can be done using the `JAVA_ARGS` init script setting, similar to configuring the heap size.

For example, adding the following JVM options will open
up a JMX socket on port 1099:

    JAVA_ARGS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=1099"


76
The Logback logging-config file
77 78
-----

79 80 81 82
Logging is configured with a logback.xml file, whose location is
defined with the [`logging-config`](#logging-config) setting. If you
change the log settings while PuppetDB is running, it will apply the
new settings without requiring a restart.
83

84
[See the Logback documentation][logback] for more information about logging options.
85 86


87
The PuppetDB configuration file(s)
88 89
-----

90 91 92 93
PuppetDB is configured using an INI-style config format with several
`[sections]`. This is very similar to the format used by Puppet. All
of the sections and settings described below belong in the PuppetDB
config file(s).
94

95
> **Note:** Whenever you change PuppetDB's configuration settings, you must restart the service for the changes to take effect.
96

97 98 99
You can change the location of the main config file in [the init
script config file](#init-script-config-file). This location can point
to a single configuration file or a directory of .ini files. If you
100
specify a directory (in _conf.d_ style), PuppetDB will merge the .ini
101
files in alphabetical order.
102

103 104
If you've installed PuppetDB from a package, by default it will use
the _conf.d_ config style. The default config directory is
105
`/etc/puppetlabs/puppetdb/conf.d`. If you're running from source, you
106 107
may use the `-c` command-line argument to specify your config file or
directory.
108 109 110 111 112

An example configuration file:

    [global]
    vardir = /var/lib/puppetdb
113
    logging-config = /var/lib/puppetdb/logback.xml
114 115 116 117 118 119

    [database]
    classname = org.postgresql.Driver
    subprotocol = postgresql
    subname = //localhost:5432/puppetdb

120 121 122 123
    [puppetdb]
    certificate-whitelist = /path/to/file/containing/certnames
    disable-update-checking = false

124 125 126
    [jetty]
    port = 8080

127
### Playing nice with the PuppetDB module
128

129 130 131 132 133 134
If you [installed PuppetDB with the puppetlabs-puppetdb
module][module], PuppetDB's settings will be managed by Puppet. Most
of the settings you care about can be configured with the module's
class parameters; see [the module's
documentation](https://forge.puppetlabs.com/puppetlabs/puppetdb) for
details.
135

136
If you _do_ need to change those rare settings that the module doesn't
137
manage, you can do the following:
138

139 140 141 142
Create a new class in a new module (something like
`site::puppetdb::server::extra`), declare any number of `ini_setting`
resources as shown below, set the class to refresh the
`puppetdb::server` class, and assign it to your PuppetDB server.
143

144
~~~ ruby
145 146 147 148 149
    # Site-specific PuppetDB settings. Declare this class on any node that gets the puppetdb::server class.
    class site::puppetdb::server::extra {

      # Restart the PuppetDB service if settings change
      Class[site::puppetdb::server::extra] ~> Class[puppetdb::server]
150

151 152 153 154
      # Get PuppetDB confdir
      include puppetdb::params
      $confdir = $puppetdb::params::confdir

155
      # Set resource defaults
156
      Ini_setting {
157
        ensure  => present,
158 159 160
        require => Class['puppetdb::server::validate_db'],
      }

161
      ini_setting {'puppetdb-extra-setting':
162 163
        path    => "${confdir}/global.ini",
        section => 'global',
164
        setting => <some-extra-setting>,
165
        value   => 'true',
166 167
      }
    }
168
~~~
169

170
`[global]` settings
171 172 173 174 175 176
-----

The `[global]` section is used to configure application-wide behavior.

### `vardir`

177 178 179
This defines the parent directory for the MQ's data directory. The
directory must exist and be writable by the PuppetDB user in order for
the application to run.
180 181 182

### `logging-config`

183 184
This describes the full path to a
[logback.xml](http://logback.qos.ch/manual/configuration.html)
185 186
file. Covering all the options available for configuring Logback is
outside the scope of this guide: see the [Logback documentation][logback] for
187
exhaustive information.
188

189 190
If this setting isn't provided, PuppetDB defaults to logging at INFO
level to standard out.
191

192 193 194 195
If you installed from packages, PuppetDB will use the logback.xml file
in the `/etc/puppetdb/` or `/etc/puppetlabs/puppetdb`
directory. Otherwise, you can find an example file in the `ext`
directory of the source.
196

197 198
You can edit the logging configuration file while PuppetDB is running,
and it will automatically react to changes after a few seconds.
199 200 201

### `update-server`

202
The URL to query when checking for newer versions; defaults to
203
`http://updates.puppetlabs.com/check-for-updates`. Overriding this
204
setting may be useful if your PuppetDB server is firewalled and can't
205
make external HTTP requests. In this case you can configure a proxy
206 207
server to send requests to the `updates.puppetlabs.com` URL and
override this setting to point to your proxy server.
208

209
`[puppetdb]` settings
210 211
-----

212 213
The `[puppetdb]` section is used to configure PuppetDB
application-specific behavior.
214 215 216

### `certificate-whitelist`

217
Optional. This describes the path to a file that contains a list of
218
certificate names, one per line. Incoming HTTPS requests will have
219
their certificates validated against this list of names and only those
220
with an **exact** matching entry will be allowed through. (For a Puppet
221 222
master, this compares against the value of the `certname` setting,
rather than the `dns_alt_names` setting.)
223

224 225 226
If not supplied, PuppetDB uses standard HTTPS without any additional
authorization. All HTTPS clients must still supply valid, verifiable
SSL client certificates.
227

228
### `historical-catalogs-limit` (PE only)
229

230
> **Note**: This setting has no effect and will be retired in a future release.
231

232 233
### `disable-update-checking`

234 235
Optional. Setting this to `true` disables checking for updated
versions of PuppetDB and sending basic analytics data to Puppet. Defaults to `false`.
236

237 238 239 240 241 242 243 244 245 246
If `disable-update-checking` is set to `false`, PuppetDB checks for updates upon start or restart, and every 24 hours thereafter, and sends the following data to Puppet:

* Product name
* Database name
* Database version
* PuppetDB version
* IP address
* Data collection timestamp

The data Puppet collects provides just one of many methods we use for learning about our community of users. The more we know about how you use Puppet, the better we can address your needs. No personally identifiable information is collected, and the data we collect is never used or shared outside Puppet.
247

248
`[database]` settings
249 250 251
-----

The `[database]` section configures PuppetDB's database settings.
252
PuppetDB stores its data in PostgreSQL.
253

254
> **FAQ: Why no MySQL or Oracle support?**
255
>
256
> MySQL lacks several features that PuppetDB relies on, most notably including recursive queries. We have no plans to ever support MySQL.
257 258 259 260 261
>
> Depending on demand, Oracle support may be forthcoming in a future version of PuppetDB. This hasn't been decided yet.

### Using PostgreSQL

262
Before using the PostgreSQL backend, you must set up a PostgreSQL
263
server. Note that users installing PuppetDB via [the module][module]
264 265 266 267 268 269 270
will already have PostgreSQL configured properly and these steps
should not be necessary.

At a minimum, you will need to ensure that you have PostgreSQL 9.4 or
later running that will accept incoming connections, a user and
an empty database for PuppetDB. Information on
connection/authentication configuration in PostgreSQL and be found
271 272 273 274 275
[here](https://www.postgresql.org/docs/9.4/static/auth-pg-hba-conf.html). Docs
on setting up users and databases can be found in the [Getting
Started](https://www.postgresql.org/docs/9.4/static/tutorial-start.html)
section of the [PostgreSQL
manual](https://www.postgresql.org/docs/9.4/static/index.html).
276 277 278 279

Completely configuring PostgreSQL is beyond the scope of this guide,
but a example setup is described below. First, you can create a user
and database as follows:
280 281 282

    $ sudo -u postgres sh
    $ createuser -DRSP puppetdb
283
    $ createdb -E UTF8 -O puppetdb puppetdb
284 285
    $ exit

286 287 288 289
You should install the RegExp-optimized index extension
[`pg_trgm`][pg_trgm]. This may require installing the
`postgresql-contrib` (or equivalent) package, depending on your
distribution:
290 291 292 293 294

    $ sudo -u postgres sh
    $ psql puppetdb -c 'create extension pg_trgm'
    $ exit

295 296
Next, you will most likely need to modify the `pg_hba.conf` file to
allow for MD5 authentication from at least localhost. To locate the
297 298 299
file you can either issue a `locate pg_hba.conf` command (if your
distribution supports it) or consult your distribution's documentation
for the PostgreSQL `confdir`.
300

301
The following example `pg_hba.conf` file allows MD5 authentication
302
from localhost for both IPv4 and IPv6 connections:
303

304
    # TYPE  DATABASE   USER   CIDR-ADDRESS  METHOD
305 306 307
    local   all        all                  md5
    host    all        all    127.0.0.1/32  md5
    host    all        all    ::1/128       md5
308

309
Restart PostgreSQL and ensure you can log in by running:
310

311
    $ sudo service postgresql restart
312 313 314 315 316 317 318 319
    $ psql -h localhost puppetdb puppetdb

To configure PuppetDB to use this database, put the following in the `[database]` section:

    subname = //<HOST>:<PORT>/<DATABASE>
    username = <USERNAME>
    password = <PASSWORD>

320 321 322
Replace `<HOST>` with the DB server's hostname. Replace `<PORT>` with
the port on which PostgreSQL is listening. Replace `<DATABASE>` with
the name of the database you've created for use with PuppetDB.
323 324 325

#### Using SSL With PostgreSQL

326
It's possible to use SSL to protect connections to the database. There
327
are several extra steps and considerations when doing so. See the
328 329
[PostgreSQL SSL setup page][postgres_ssl] for complete details.

330 331
The main difference in the config file is that you must be sure to add
`?ssl=true` to the `subname` setting:
332 333 334 335 336

    subname = //<HOST>:<PORT>/<DATABASE>?ssl=true

### `gc-interval`

337 338 339
This controls how often, in minutes, to compact the database. The compaction
process reclaims space and deletes unnecessary rows. If not supplied, the
default is every 60 minutes.
340

341
If `gc-interval` is set to zero, all database GC processes will be disabled. When
342 343
using this value, you should explicitly set a `dlo-compression-interval` if your
system will receive any commands.
344

345 346
### `node-ttl`

Russell Mull's avatar
Russell Mull committed
347
Mark as 'expired' nodes that haven't seen any activity (no new catalogs,
Wyatt Alt's avatar
Wyatt Alt committed
348
facts, or reports) in the specified amount of time. Expired nodes behave the same
Russell Mull's avatar
Russell Mull committed
349 350 351
as manually-deactivated nodes.

You may specify the time as a string using any of the following suffixes:
352 353 354 355 356 357 358

    `d`  - days
    `h`  - hours
    `m`  - minutes
    `s`  - seconds
    `ms` - milliseconds

359
For example, a value of `30d` would set the time-to-live to 30 days, and a value of
360 361 362 363 364
`48h` would set the time-to-live to 48 hours.

Nodes will be checked for staleness every `gc-interval` minutes. Manual
deactivation will continue to work as always.

365
If unset or set to 0s, auto-expiration of nodes is disabled.
366

367 368
### `node-purge-ttl`

369 370 371
Automatically delete nodes that have been deactivated or expired for the
specified amount of time. This will also delete all facts, catalogs, and reports
for the relevant nodes. This TTL may be specified the same way as `node-ttl` above.
372

373
If unset or set to 0s, auto-deletion of nodes is disabled.
374

375 376 377 378 379 380 381 382 383
### `report-ttl`

Automatically delete reports that are older than the specified amount of time.
You may specify the time as a string using any of the suffixes described in the
`node-ttl` section above.

Outdated reports will be deleted during the database garbage collection, which
runs every `gc-interval` minutes.

384
If unset, the default value is 14 days.
385 386 387

### `subname`

388 389 390
This describes where to find the database. It should be something like
`//<HOST>:<PORT>/<DATABASE>`, replacing `<HOST>` with the DB server's
hostname, `<PORT>` with the port on which PostgreSQL is listening, and
391
`<DATABASE>` with the name of the database. Append `?ssl=true` to
392
this if your PostgreSQL server is using SSL.
393 394 395 396 397 398 399 400 401

### `username`

This is the username to use when connecting. Only used with PostgreSQL.

### `password`

This is the password to use when connecting. Only used with PostgreSQL.

402
### `maximum-pool-size`
403

404
From the [HikariCP documentation](https://github.com/brettwooldridge/HikariCP):
405

406
> "This property controls the maximum size that the pool is allowed to reach,
407 408
> including both idle and in-use connections. Basically this value will
> determine the maximum number of actual connections to the database backend. A
409
> reasonable value for this is best determined by your execution environment."
410 411 412 413 414

When the pool reaches this size, and no idle connections are available, attempts
to get a connection will wait for `connection-timeout` milliseconds before timing
out.

Wyatt Alt's avatar
Wyatt Alt committed
415 416
The default value is 25. Note that PuppetDB will use one pool for writes and another
for reads, so the total number of connections used will be twice this setting.
417 418 419

### `conn-max-age`

420 421
The maximum time (in minutes), for a pooled connection to remain
unused before it is closed off.
422

423
If not supplied, the default value is 60 minutes.
424 425 426

### `conn-lifetime`

427 428 429 430
The maximum time (in minutes) a pooled connection should remain
open. Any connections older than this setting will be closed
off. Connections currently in use will not be affected until they are
returned to the pool.
431

432 433
If not supplied, we won't terminate connections based on their age
alone.
434

435 436
### `connection-timeout`

437 438 439
The maximum time to wait (in milliseconds) to acquire a connection
from the pool of database connections. If not supplied, defaults to
1000.
440

441 442 443 444 445 446 447
## Deprecated settings

### `classname`

**Note**: This setting is deprecated and ignored by PuppetDB. It will be removed
from PuppetDB in a future release.

448 449
This sets the JDBC class to use. It should be
`org.postgresql.Driver`, which is the default. You should not need to
450 451 452 453 454 455 456
change it.

### `subprotocol`

**Note**: This setting is deprecated and ignored by PuppetDB. It will be removed
from PuppetDB in a future release.

457
This should be `postgresql`, which is the default. You should not
458 459 460 461 462 463 464
need to change it.

### `log-slow-statements`

**Note**: This setting is deprecated and ignored by PuppetDB. It will be removed
from PuppetDB in a future release.

465
This sets the number of seconds before an SQL query is considered "slow." Slow SQL queries are logged as warnings, to assist in debugging and tuning. Note that PuppetDB does not interrupt slow queries, but simply reports them after they complete.
466

467
The default value is 10 seconds. A value of zero will disable logging of slow queries.
468 469 470 471 472 473

### `conn-keep-alive`

**Note**: This setting is deprecated and ignored by PuppetDB. It will be removed
from PuppetDB in a future release.

474
This sets the time (in minutes) for a connection to remain idle before sending a test query to the database. This is useful to prevent a database from timing out connections on its end.
475

476
If not supplied, the default value is 45 minutes.
477

Ken Barber's avatar
Ken Barber committed
478
### `statements-cache-size`
479

480 481 482
**Note**: This setting is deprecated and ignored by PuppetDB. It will be removed
from PuppetDB in a future release.

483 484
This setting defines how many prepared statements are cached automatically. For a large amount of dynamic queries this number could be increased to increase performance, at the cost of memory consumption and database resources.

485
If not supplied, the default value is zero.
486

487
`[read-database]` settings
488 489
-----

490 491 492
The `[read-database]` section configures PuppetDB's _read-database_
settings, useful when running a PostgreSQL [Hot
Standby](http://wiki.postgresql.org/wiki/Hot_Standby) cluster.
493 494 495 496
Currently, only configuring a PostgreSQL read-database is supported.  See
the PostgreSQL documentation [here](http://wiki.postgresql.org/wiki/Hot_Standby)
for details on configuring the cluster. The `[read-database]` portion
of the configuration is in addition to the `[database]` settings. If
497
`[read-database]` is specified, `[database]` must also be specified.
498

499
To configure PuppetDB to use a read-only database from the cluster,
500
add the following to the `[read-database]` section:
501 502 503 504 505 506 507

    classname = org.postgresql.Driver
    subprotocol = postgresql
    subname = //<HOST>:<PORT>/<DATABASE>
    username = <USERNAME>
    password = <PASSWORD>

508 509 510
Replace `<HOST>` with the DB server's hostname. Replace `<PORT>` with
the port on which PostgreSQL is listening. Replace `<DATABASE>` with
the name of the database you've created for use with PuppetDB.
511 512 513

#### Using SSL With PostgreSQL

514 515
It's possible to use SSL to protect connections to the database. There
are several extra steps and considerations when doing so; see the
516 517
[PostgreSQL SSL setup page][postgres_ssl] for complete details.

518 519
The main difference in the config file is that you must be sure to add
`?ssl=true` to the `subname` setting:
520 521 522 523 524

    subname = //<HOST>:<PORT>/<DATABASE>?ssl=true

### `subname`

525
This describes where to find the database. Set this to `//<HOST>:<PORT>/<DATABASE>` when using PostgreSQL, replacing `<HOST>` with the DB server's hostname, `<PORT>` with the port on which PostgreSQL is listening, and `<DATABASE>` with the name of the database.
526

527
Append `?ssl=true` to this if your PostgreSQL server is using SSL.
528 529 530 531 532 533 534 535 536

### `username`

This is the username to use when connecting.

### `password`

This is the password to use when connecting.

537
### `maximum-pool-size`
538

539
From the [HikariCP documentation](https://github.com/brettwooldridge/HikariCP):
540

541
> "This property controls the maximum size that the pool is allowed to reach,
542 543
> including both idle and in-use connections. Basically this value will
> determine the maximum number of actual connections to the database backend. A
544
> reasonable value for this is best determined by your execution environment."
545 546 547 548 549 550

When the pool reaches this size, and no idle connections are available, attempts
to get a connection will wait for `connection-timeout` milliseconds before timing
out.

The default value is 10.
551 552 553

### `conn-max-age`

554
The maximum time (in minutes) for a pooled connection to remain
555
unused before it is closed off.
556

557
If not supplied, the default value is 60 minutes.
558 559 560

### `conn-lifetime`

561 562 563 564
The maximum time (in minutes) a pooled connection should remain
open. Any connections older than this setting will be closed
off. Connections currently in use will not be affected until they are
returned to the pool.
565

566 567
If not supplied, we won't terminate connections based on their age
alone.
568

569 570
### `connection-timeout`

571 572 573
The maximum time to wait (in milliseconds) to acquire a connection
from the pool of database connections. If not supplied, defaults to
500.
574

575 576 577 578 579 580 581
## Deprecated settings

### `classname`

**Note**: This setting is deprecated and ignored by PuppetDB. It will be removed
from PuppetDB in a future release.

582 583
This sets the JDBC class to use. It should be
`org.postgresql.Driver`, which is the default. You should not need to
584 585 586 587 588 589 590
change it.

### `subprotocol`

**Note**: This setting is deprecated and ignored by PuppetDB. It will be removed
from PuppetDB in a future release.

591
This should be `postgresql`, which is the default. You should not
592 593 594 595 596 597 598
need to change it.

### `log-slow-statements`

**Note**: This setting is deprecated and ignored by PuppetDB. It will be removed
from PuppetDB in a future release.

599
This sets the number of seconds before an SQL query is considered "slow." Slow SQL queries are logged as warnings, to assist in debugging and tuning. Note PuppetDB does not interrupt slow queries, but simply reports them after they complete.
600

601
The default value is 10 seconds. A value of zero will disable logging of slow queries.
602 603 604 605 606 607

### `conn-keep-alive`

**Note**: This setting is deprecated and ignored by PuppetDB. It will be removed
from PuppetDB in a future release.

608
This sets the time (in minutes) for a connection to remain idle before sending a test query to the database. This is useful to prevent a database from timing out connections on its end.
609

610
If not supplied, the default setting is 45 minutes.
611 612 613 614 615 616 617 618

###`statements-cache-size`

**Note**: This setting is deprecated and ignored by PuppetDB. It will be removed
from PuppetDB in a future release.

This setting defines how many prepared statements are cached automatically. For a large amount of dynamic queries this number could be increased to increase performance, at the cost of memory consumption and database resources.

619
If not supplied, the default setting is zero.
620

621

622 623 624
`[command-processing]` Settings
-----

625 626
The `[command-processing]` section configures the command-processing
subsystem.
627

628 629 630
Every change to PuppetDB's data stores arrives via **commands** that
are inserted into a message queue (MQ). Command processor threads pull
items off of that queue, persisting those changes.
631 632 633

### `threads`

634 635 636
This defines how many command processing threads to use. Each thread
can process a single command at a time. [The number of threads can be
tuned based on what you see in the performance dashboard.][dashboard]
637

638 639
This setting defaults to half the number of cores in your system.

640 641 642 643 644 645 646 647 648 649 650
### `concurrent-writes`

This sets a limit on the number of threads that can write to the disk at
any one time. The default value is the smaller number of half the number
of CPU cores and 4.

If your load is low, your disk is fast (i.e. an SSD), and commands
aren't being processed quickly enough, then you could increasing this
value in order to alleviate that, but this is unlikely to be the
bottleneck for command processing.

651 652 653 654 655 656 657 658 659 660
### `dlo-compression-interval`

**Note**: This setting is deprecated and ignored by PuppetDB. It will be removed
from PuppetDB in a future release.

Any PuppetDB instance which receives commands must perform periodic maintenance
on the message queue. This setting controls the interval at which that process
is performed. By default, it is equal to `gc-interval` (60 minutes by default).
You may wish to set this explicitly if you are using a zero `gc-interval`.

661 662
### `dlo-compression-threshold`

663 664 665
**Note**: This setting is deprecated and ignored by PuppetDB. It will be removed
from PuppetDB in a future release.

666 667 668 669 670
This setting specifies the maximum duration to keep messages in the
dead-letter office before archiving them. This process will check for
compressible messages on startup and after every `gc-interval`, but
will only perform the archive once per
`dlo-compression-threshold`. The same format can be used as for the
671 672
`node-ttl` setting above. If set to zero seconds, this behavior will be
disabled. The default value is one day.
673

674 675
### `store-usage`

676 677 678
**Note**: This setting is deprecated and is ignored by PuppetDB,
except when migrating away from ActiveMQ.  It will be removed from
PuppetDB in a future release.
679

680
Sets the maximum amount of space in megabytes that
681
PuppetDB's ActiveMQ can use for persistent message storage.
682 683 684

### `temp-usage`

685 686 687
**Note**: This setting is deprecated and ignored by PuppetDB, except
when migrating away from ActiveMQ.  It will be removed from PuppetDB
in a future release.
688

689
Sets the maximum amount of space in megabytes that
690
PuppetDB's ActiveMQ can use for temporary message storage.
691

692 693
### `memory-usage`

694 695 696
**Note**: This setting is deprecated and ignored by PuppetDB, except
when migrating away from ActiveMQ.  It will be removed in a future
release.
697

698 699 700 701 702 703 704 705 706
This setting sets the maximum amount of memory in megabytes available for
PuppetDB's ActiveMQ Broker.

**Warning** Setting this value too high (such that memory-usage exceeds the size
of the heap) can cause out of memory (OOM) errors. ActiveMQ does not treat this
as a hard limit. In testing, we've seen it use up to `125%` of the specified
value, and overall memory usage will also be affected by the `max-command-size`
and `threads` parameters.

707 708
### `max-frame-size`

709 710 711
**Note**: This setting is deprecated and ignored by PuppetDB, except
when migrating away from ActiveMQ.  It will be removed from PuppetDB
in a future release.
712

713 714
Sets the maximum frame size for persisted ActiveMQ messages
supplied in bytes. Default value is 209715200 (or 200MB).
715

716 717
### `reject-large-commands`

718 719 720
This is a Boolean that enables rejecting (returning an HTTP 413 error)
commands that are too large to process, such as a
catalog that is too large, causing PuppetDB to run out of
721 722
memory. This setting can be used along with `max-command-size`.

723
This setting is false by default.
724 725 726

### `max-command-size`

727
This is an integer that specifies (in bytes) which commands are "too
728
large" to process with PuppetDB. By default this setting is a fraction
729 730 731 732 733
of the total heap space. It is strongly recommended that users set
this manually as the default is probably too conservative. To help
determine the current size of commands being processed, enable debug
logging for the `puppetlabs.puppetdb.middleware` appender in the
[logback.xml](#logging-config). This setting has no effect when
734
`reject-large-commands` is set to false.
735 736


737
`[jetty]` (HTTP) settings
738 739 740 741
-----

The `[jetty]` section configures HTTP for PuppetDB.

742
> **Note:** If you are using Puppet Enterprise and want to enable the PuppetDB dashboard from the PE console, refer to [our guide to changing PuppetDB's parameters]({{pe}}/maintain_console-db.html#changing-puppetdbs-parameters) for more information. PE users should not edit `jetty.ini`.
743

744 745
### `host`

746
Sets the IP interface to listen on for **unencrypted** HTTP
747 748 749
traffic. If not supplied, we bind to `localhost`, which will reject
connections from anywhere but the PuppetDB server itself. To listen on
all available interfaces, use `0.0.0.0`.
750

751
To avoid DNS resolution confusion, if you wish to set this to something other than `localhost`, we reccomend using an IP address instead of a hostname.
752

753 754 755 756 757 758 759
> **Note:** Unencrypted HTTP is the only way to view the [performance
    dashboard][dashboard], since PuppetDB uses host verification for
    SSL. However, it can also be used to make any call to PuppetDB's
    API, including inserting exported resources and retrieving
    arbitrary data about your Puppet-managed nodes. **If you enable
    cleartext HTTP, you MUST configure your firewall to protect
    unverified access to PuppetDB.**
760 761 762

### `port`

763
Establishes which port to use for **unencrypted** HTTP traffic. If not
764
supplied, we won't listen for unencrypted traffic at all.
765

766 767
### `max-threads`

768
Sets the maximum number of threads assigned to responding to HTTP
769 770
and HTTPS requests, effectively changing how many concurrent requests
can be made at one time. Defaults to 50.
771

772
> **Note:** Due to the behaviour of our web server (Jetty 9), this setting
773
    must be higher then the number of CPUs on your system or it will
774
    stop processing any HTTP requests.
775

776 777
### `ssl-host`

778
Sets which IP interface to listen on for **encrypted** HTTPS traffic. If
779 780
not supplied, we bind to `localhost`. To listen on all available
interfaces, use `0.0.0.0`.
781

782
To avoid DNS resolution confusion, if you wish to set this to something other than `localhost`, we reccomend using an IP address instead of a hostname
783 784 785

### `ssl-port`

786
Establishes which port to use for **encrypted** HTTPS traffic. If not
787
supplied, we won't listen for encrypted traffic at all.
788

789 790
### `ssl-cert`

791 792
Sets the path to the server certificate PEM file used by the
PuppetDB web service for HTTPS. During the SSL handshake for a
793
connection, certificates extracted from this file are presented to the
794
client for the client's use in validating the server. This file may
795
contain a single certificate or a chain of certificates ordered from
796
the end certificate first to the most-root certificate last. For
797
example, a certificate chain could contain:
798

799 800 801
* An end certificate.
* An intermediate CA certificate with which the end certificate was issued.
* A root CA certificate with which the intermediate CA certificate was issued.
802

803 804 805
In the PEM file, the end certificate should appear first, the
intermediate CA certificate should appear second, and the root CA
certificate should appear last.
806

807
If a chain is present, it is not required to be complete. If a path
808 809 810
has been specified for the `ssl-cert-chain` setting, the server will
construct the cert chain starting with the first certificate found in
the `ssl-cert` PEM and followed by any certificates in the
811 812
`ssl-cert-chain` PEM. In the latter case, any certificates in the
`ssl-cert` PEM beyond the first one are ignored.
813 814 815

### `ssl-key`

816
This sets the path to the private key PEM file that corresponds with
817
the `ssl-cert`, if used by the PuppetDB web service for HTTPS.
818 819 820

### `ssl-ca-cert`

821
This sets the path to the CA certificate PEM file used for client
822
authentication. Authorized clients must be signed by the CA that corresponds to this certificate.
823

824 825
### `cipher-suites`

826 827 828
Optional. A comma-separated list of cryptographic ciphers to allow for
incoming SSL connections. Valid names are listed in the [official JDK
cryptographic providers
829
documentation](http://docs.oracle.com/javase/7/docs/technotes/guides/security/SunProviders.html#SupportedCipherSuites). Note that you must use the all-caps cipher suite name.
830

831 832 833
If not supplied, PuppetDB uses the default cipher suites for your
local system on JDK versions older than 1.7.0u6. On newer JDK
versions, PuppetDB will use only non-DHE cipher suites.
834

835 836
### `ssl-protocols`

837 838 839
Optional. A comma-separated list of protocols to allow for incoming
SSL connections. Valid names are listed in the [official JDK
cryptographic protocol
840
documentation](http://docs.oracle.com/javase/7/docs/technotes/guides/security/SunProviders.html#SunJSSEProvider). Note that you must use the names with verbatim capitalization. For
841
example: `TLSv1, TLSv1.1, TLSv1.2`.
842

843
If not supplied, PuppetDB uses a default of `TLSv1, TLSv1.1, TLSv1.2`. By default, SSLv3 is not included in that list due to known vulnerabilities. Users wanting to use SSLv3 need to explicitly specify it in their list.
844

845 846
### `ssl-crl-path`

847 848 849
Optional. This describes a path to a Certificate Revocation List
file. Incoming SSL connections will be rejected if the client
certificate matches a revocation entry in the file.
850 851 852 853

### `ssl-cert-chain`

This sets the path to a PEM with CA certificates for use in presenting a
854
client with the server's chain of trust. Certs found in this PEM file are
855
appended after the first certificate from the `ssl-cert` PEM in the
856
construction of the certificate chain. This is an optional setting. The
857
certificates in the `ssl-cert-chain` PEM file should be ordered from the
858
least-root CA certificate first to the most-root CA certificate last. For
859 860
example, a certificate chain could contain:

861 862 863
* An end certificate.
* An intermediate CA certificate with which the end certificate was issued.
* A root CA certificate with which the intermediate CA certificate was issued.
864

865
The end certificate should appear in the `ssl-cert` PEM file. In the
866 867 868 869 870 871 872 873
`ssl-cert-chain` PEM file, the intermediate CA certificate should appear
first and the root CA certificate should appear last.

The chain is not required to be complete.

> **Note:** This setting overrides the alternate configuration settings
`keystore` and `key-password`.

874 875
### `access-log-config`

876
Optional. This is a path to an XML file containing configuration
877
information for the `logback-access` module. If present, a logger will
878 879 880
be set up to log information about any HTTP requests Jetty receives
according to the logging configuration, as long as the XML file
pointed to exists and is valid. Information on configuring the
881
`logback-access` module is available
882
[here](http://logback.qos.ch/access.html#configuration).
883 884 885 886 887 888 889 890 891 892 893 894 895

A configuration file may resemble the following:

    <configuration debug="false">
      <appender name="FILE" class="ch.qos.logback.core.FileAppender">
        <file>./dev-resources/access.log</file>
          <encoder>
            <pattern>%h %l %u %user %date "%r" %s %b</pattern>
          </encoder>
        </appender>
        <appender-ref ref="FILE" />
    </configuration>

896 897 898 899 900 901
This example configures a `FileAppender` that outputs to a file,
`access.log`, in the `dev-resources` directory. It will log the remote
host making the request, the log name, the remote user making the
request, the date/time of the request, the URL and method of the
request, the status of the response, and the size in bytes of the
response.
902 903

### `graceful-shutdown-timeout`
904

905
After receiving a shutdown, this is the number of milliseconds the
906 907 908
server will wait for in-flight requests to complete before actually
shutting down. New requests will be blocked during this time. Defaults
to 30000.
909 910 911

### `request-header-max-size`

912 913
This sets the maximum size of an HTTP request header. If a header is
sent that exceeds this value, Jetty will return an HTTP 413 error
914
response. This defaults to 8192 bytes, and only needs to be configured
915
if an exceedingly large header is being sent in an HTTP request.
916

917
`[nrepl]` settings
918 919
-----

920 921
The `[nrepl]` section configures remote runtime modification. For
more detailed info, see [our guide to debugging with the remote REPL][repl].
922

923 924 925 926 927
Enabling a remote
[REPL](http://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop)
allows you to manipulate the behavior of PuppetDB at runtime. This
should only be done for debugging purposes, and is thus disabled by
default. An example configuration stanza:
928

929
    [nrepl]
930
    type = nrepl
Wyatt Alt's avatar
Wyatt Alt committed
931
    port = 8082
932
    host = 127.0.0.1
933 934 935

### `enabled`

936
To enable the REPL, set to true. Defaults to false.
937 938 939 940 941

### `port`

The port to use for the REPL.

942 943
### `host`

944 945
Specifies the host or IP address for the REPL service to listen on. By
default this is `127.0.0.1` only. As this is an insecure channel this
946
is the only recommended setting for production environments.
947

948
If you wish to listen on all interfaces, you can specify `0.0.0.0`, for example, although this is generally not recommended for production.
949 950 951 952 953 954 955 956 957 958 959 960 961 962 963

`[developer]` settings
-----

The `[developer]` section contains configuration items that may be useful to
users developing against the PuppetDB API. These settings may impede
performance, and are not recommended for production use.

### `pretty-print`

Enables/disables default pretty-printing of API responses. Defaults to false.
Enabling default pretty-printing is not recommended in production because it
incurs a penalty in data transfer speed and size. Users may override this
setting on a per-query basis by supplying a `?pretty=` parameter in the URL,
valued `true` or `false`.
964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013

`[sync]` settings (Puppet Enterprise only)
-----

The `[sync]` section of the PuppetDB configuration file is used to configure
synchronization for a high-availability system. See
the [HA configuration guide][ha] for complete system configuration instructions.

### `remotes`

The `remotes` configuration key indicates that PuppetDB should poll a remote
PuppetDB server for changes. When it finds changed or updated records on that
server, it will download the records and submit them to the local command queue.

In the configuration file, you specify a `remote` for each server you want to
pull data from. It is perfectly reasonable, and expected, for two servers to
pull data from each other. For each remote, you must provide:

 - The remote server url. This is a root url which should include the protocol
   and port to use (eg. "https://puppetdb.myco.net:8081"). The protocol is
   mandatory and must be either "http" or "https". If the port is not provided,
   it will default to `8080` for http and `8081` for https.

 - The interval at which to poll the remote server for new data. This is
   formatted as a timespan with units (e.g. '2m'). See the
   [node-ttl documentation][node-ttl] for further reference.

You should not configure PuppetDB to sync with itself.

#### HOCON

If you are using HOCON to configure PuppetDB, use the following structure in
your .conf file:

    sync: {
      remotes: [{server_url: "https://remote-puppetdb.myco.net:8081",
                 interval: 2m}]
    }

#### ini

If you are using a .ini file to configure PuppetDB, use the following structure:

    [sync]
    server_urls = https://remote-puppetdb.myco.net:8081
    intervals = 2m

Multiple values may be provided by comma-separating them, with no whitespace.
You must have exactly the same number of entries in the `server_urls` and
`intervals` values.