Commit 51a10d35 authored by Ken Barber's avatar Ken Barber

(PDB-843) Namespace changes for source and test code

Our engineering org has decided to use puppetlabs.<library_name> as a standard
across our Clojure projects, this patch lines us up with that standard.

This also moves the libraries in com.puppetlabs.* into puppetlabs.puppetdb also,
to avoid collisions with any other project. These files were designed to be
pulled out of PuppetDB one day, but for now we've moved them all into the same
namespace.

We also have re-aligned our tests to be more standard, adopting the namespace-test
convention. This aids with editors also, as now we can switch easily between src
and test.

All the various places that care about this change have been changed, including
SysV scripts, systemd definitions and the command line utilies that are responsible
for launching PuppetDB, since we now have puppetlabs.puppetdb.core as the main
entry point. boostrap.cfg, and logback.xml have been changed also.

Since we were changing a lot of the (ns ..) blocks across all files we took the
opportunity to clean them up. :use is now now longer used (excuse the pun).
Comments at the top of libraries are now in the ns doctstring also.

The test-resources directory has also been aligned with the new namespaces and
tests are now pointing at the new layout.

For backwards compatibility, we've kept the old namespace pinned for JMX & ActiveMQ
queues, since these are still required for our stable API commitments. In the future
we'll have to fix this.

The concurrent.clj code has been removed, I found it wasn't used so removed it.
Signed-off-by: default avatarKen Barber <ken@bob.sh>
parent 0af456b7
......@@ -44,7 +44,7 @@
necessary.
* In these rare circumstances where a schema change must be made against more
than one branch of the code, see the notes at the beginning of the file
`src/com/puppetlabs/puppetdb/scf/migrate.clj` on this subject. Exercise extreme
`src/puppetlabs/puppetdb/scf/migrate.clj` on this subject. Exercise extreme
caution to make sure that you are adhering to the guidelines therein.
# Additional Resources
......
......@@ -25,7 +25,7 @@ step "Verify we've been talking to the correct database" do
# in the code, and it's just generally moronic. We should provide a lein task
# or some way of interrogating the latest expected schema version from
# the command line so that we can get rid of this crap.
source_migration_version = on(database, "java -cp /usr/share/puppetdb/puppetdb.jar clojure.main -m com.puppetlabs.puppetdb.core version |grep target_schema_version|cut -f2 -d'='").stdout.strip
source_migration_version = on(database, "java -cp /usr/share/puppetdb/puppetdb.jar clojure.main -m puppetlabs.puppetdb.core version |grep target_schema_version|cut -f2 -d'='").stdout.strip
assert_equal(db_migration_version, source_migration_version,
"Expected migration version from source code '#{source_migration_version}' " +
......
......@@ -40,9 +40,9 @@ Executing Functions
Within the REPL, you can interactively execute PuppetDB's functions. For example, to manually compact the database:
user=> (use 'com.puppetlabs.puppetdb.cli.services)
user=> (use 'puppetlabs.puppetdb.cli.services)
nil
user=> (use 'com.puppetlabs.puppetdb.scf.storage)
user=> (use 'puppetlabs.puppetdb.scf.storage)
nil
user=> (use 'clojure.java.jdbc)
nil
......@@ -55,17 +55,17 @@ Redefining Functions
You can also manipulate the running PuppetDB instance by redefining functions on the fly. Let's say that for debugging purposes, you'd like to log every time a catalog is deleted. You can just redefine the existing `delete-catalog!` function dynamically:
user=> (ns com.puppetlabs.puppetdb.scf.storage)
user=> (ns puppetlabs.puppetdb.scf.storage)
nil
com.puppetlabs.puppetdb.scf.storage=>
puppetlabs.puppetdb.scf.storage=>
(def original-delete-catalog! delete-catalog!)
#'com.puppetlabs.puppetdb.scf.storage/original-delete-catalog!
com.puppetlabs.puppetdb.scf.storage=>
#'puppetlabs.puppetdb.scf.storage/original-delete-catalog!
puppetlabs.puppetdb.scf.storage=>
(defn delete-catalog!
[catalog-hash]
(log/info (str "Deleting catalog " catalog-hash))
(original-delete-catalog! catalog-hash))
#'com.puppetlabs.puppetdb.scf.storage/delete-catalog!
#'puppetlabs.puppetdb.scf.storage/delete-catalog!
Now any time that function is called, you'll see a message logged.
......
......@@ -30,7 +30,7 @@ JARFILE="puppetdb.jar"
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME
JAVA_ARGS="${JAVA_ARGS} -cp ${INSTALL_DIR}/${JARFILE} clojure.main -m com.puppetlabs.puppetdb.core services -c ${CONFIG} "
JAVA_ARGS="${JAVA_ARGS} -cp ${INSTALL_DIR}/${JARFILE} clojure.main -m puppetlabs.puppetdb.core services -c ${CONFIG} "
EXTRA_ARGS="--chuid $USER --background --make-pidfile"
# Exit if the package is not installed
......
#!/bin/sh
daemon=$(/usr/local/bin/javaPathHelper -c <%= @name -%>)
daemon_flags="-cp <% @install_dir -%>/puppetdb.jar clojure.main -m com.puppetlabs.puppetdb.core services -c /etc/puppetdb/conf.d/puppetdb.conf"
daemon_flags="-cp <% @install_dir -%>/puppetdb.jar clojure.main -m puppetlabs.puppetdb.core services -c /etc/puppetdb/conf.d/puppetdb.conf"
. /etc/rc.d/rc.subr
......
......@@ -38,7 +38,7 @@ PATH=/opt/puppet/bin:/opt/puppet/sbin:/sbin:/usr/sbin:/bin:/usr/bin
PATH=/sbin:/usr/sbin:/bin:/usr/bin
<% end -%>
JARFILE="puppetdb.jar"
JAVA_ARGS="${JAVA_ARGS} -cp ${INSTALL_DIR}/${JARFILE} clojure.main -m com.puppetlabs.puppetdb.core services -c ${CONFIG} "
JAVA_ARGS="${JAVA_ARGS} -cp ${INSTALL_DIR}/${JARFILE} clojure.main -m puppetlabs.puppetdb.core services -c ${CONFIG} "
EXTRA_ARGS="--chuid $USER --background --make-pidfile"
lockfile=/var/lock/subsys/$prog
EXEC="$JAVA_BIN -XX:OnOutOfMemoryError=\"kill -9 %p\" $JAVA_ARGS"
......
......@@ -33,7 +33,7 @@ JARFILE="puppetdb.jar"
PIDFILE=/var/run/$NAME/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME
JAVA_ARGS="${JAVA_ARGS} -cp ${INSTALL_DIR}/${JARFILE} clojure.main -m com.puppetlabs.puppetdb.core services -c ${CONFIG} "
JAVA_ARGS="${JAVA_ARGS} -cp ${INSTALL_DIR}/${JARFILE} clojure.main -m puppetlabs.puppetdb.core services -c ${CONFIG} "
# Exit if the package is not installed
[ -x "$JAVA_BIN" ] || exit 0
......
......@@ -25,11 +25,11 @@
<!-- turn on to see verbose storage activity -->
<!--
<logger name="com.puppetlabs.puppetdb.scf.storage" level="debug"/>
<logger name="puppetlabs.puppetdb.scf.storage" level="debug"/>
-->
<!-- Info level provides migration and index creation information -->
<logger name="com.puppetlabs.puppetdb.scf.migrate" level="info"/>
<logger name="puppetlabs.puppetdb.scf.migrate" level="info"/>
<root level="info">
<!-- <appender-ref ref="STDOUT" /> -->
......
......@@ -2,4 +2,4 @@
. <%= @libexec_dir %>/<%= @name %>.env
${JAVA_BIN} <%= @java_args || @default_java_args -%> -cp ${INSTALL_DIR}/puppetdb.jar clojure.main -m com.puppetlabs.puppetdb.core anonymize "$@"
${JAVA_BIN} <%= @java_args || @default_java_args -%> -cp ${INSTALL_DIR}/puppetdb.jar clojure.main -m puppetlabs.puppetdb.core anonymize "$@"
......@@ -2,4 +2,4 @@
. <%= @libexec_dir %>/<%= @name %>.env
${JAVA_BIN} <%= @java_args || @default_java_args -%> -cp ${INSTALL_DIR}/puppetdb.jar clojure.main -m com.puppetlabs.puppetdb.core export "$@"
${JAVA_BIN} <%= @java_args || @default_java_args -%> -cp ${INSTALL_DIR}/puppetdb.jar clojure.main -m puppetlabs.puppetdb.core export "$@"
......@@ -4,4 +4,4 @@
ARGS="$@"
su ${USER} -s /bin/bash -c "${JAVA_BIN} ${JAVA_ARGS} -cp ${INSTALL_DIR}/puppetdb.jar clojure.main -m com.puppetlabs.puppetdb.core services -c ${CONFIG} ${@}"
su ${USER} -s /bin/bash -c "${JAVA_BIN} ${JAVA_ARGS} -cp ${INSTALL_DIR}/puppetdb.jar clojure.main -m puppetlabs.puppetdb.core services -c ${CONFIG} ${@}"
......@@ -2,4 +2,4 @@
. <%= @libexec_dir %>/<%= @name %>.env
${JAVA_BIN} <%= @java_args || @default_java_args -%> -cp ${INSTALL_DIR}/puppetdb.jar clojure.main -m com.puppetlabs.puppetdb.core import "$@"
${JAVA_BIN} <%= @java_args || @default_java_args -%> -cp ${INSTALL_DIR}/puppetdb.jar clojure.main -m puppetlabs.puppetdb.core import "$@"
......@@ -11,13 +11,13 @@ PIDFile=/var/run/<%= @name %>/<%= @name %>.pid
ExecStart=<%= @java_bin %> \
$JAVA_ARGS \
-cp ${INSTALL_DIR}/puppetdb.jar \
clojure.main -m com.puppetlabs.puppetdb.core \
clojure.main -m puppetlabs.puppetdb.core \
services -c ${CONFIG} $@
<%- else -%>
ExecStart=/usr/lib/jvm/jre-1.7.0-openjdk/bin/java \
$JAVA_ARGS \
-cp ${INSTALL_DIR}/puppetdb.jar \
clojure.main -m com.puppetlabs.puppetdb.core \
clojure.main -m puppetlabs.puppetdb.core \
services -c ${CONFIG} $@
<%- end -%>
......
......@@ -96,4 +96,4 @@
:jar-exclusions [#"leiningen/"]
:main ^:skip-aot com.puppetlabs.puppetdb.core)
:main ^:skip-aot puppetlabs.puppetdb.core)
......@@ -7,4 +7,4 @@
puppetlabs.trapperkeeper.services.webserver.jetty9-service/jetty9-service
# Load the PuppetDB service
com.puppetlabs.puppetdb.cli.services/puppetdb-service
puppetlabs.puppetdb.cli.services/puppetdb-service
......@@ -13,9 +13,9 @@
<!-- turn on to see verbose storage activity -->
<!--
<logger name="com.puppetlabs.puppetdb.scf.storage" level="debug"/>
<logger name="puppetlabs.puppetdb.scf.storage" level="debug"/>
-->
<logger name="com.puppetlabs.puppetdb.scf.migrate" level="info"/>
<logger name="puppetlabs.puppetdb.scf.migrate" level="info"/>
<root level="info">
<appender-ref ref="STDOUT" />
......
;; ## Concurrency-related Utility Functions
;;
;; This namespace contains some utility functions for multi-thread operations.
;; In most cases these will simply be thin wrappers around clojure concurrency
;; functions and/or structures from `java.util.concurrent`.
(ns com.puppetlabs.concurrent
(:import [java.util.concurrent Semaphore]))
(defn bound-via-semaphore
"Given a semaphore `sem` function `f`, return a new function which simply
acquires the semaphore, executes `f`, and then releases the semaphore. This
is mostly intended to be a helper function for use by `bounded-pmap`."
[sem f]
{:pre [(instance? Semaphore sem)
(ifn? f)]
:post [(ifn? %)]}
(fn [& args]
(.acquire sem)
(try
(apply f args)
(finally (.release sem)))))
(defn bounded-pmap
"Similar to clojure's built-in `pmap`, but prevents concurrent evaluation of
more than `max-threads` number of items in the resulting sequence."
[max-threads f coll]
{:pre [(integer? max-threads)
(ifn? f)
(coll? coll)]}
(let [sem (Semaphore. max-threads)
bounded-fn (bound-via-semaphore sem f)]
(pmap bounded-fn coll)))
(ns com.puppetlabs.puppetdb.anonymizer
(:require [com.puppetlabs.puppetdb.catalogs :as catalog]
[com.puppetlabs.puppetdb.reports :as report]
[com.puppetlabs.puppetdb.utils :as utils]
[clojure.string :as string])
(:use [com.puppetlabs.concurrent :only (bounded-pmap)]
[puppetlabs.kitchensink.core :only (regexp? boolean? uuid string-contains?)]
[clojure.walk :only (keywordize-keys)]
[com.puppetlabs.random]))
(ns puppetlabs.puppetdb.anonymizer
(:require [puppetlabs.puppetdb.catalogs :as catalog]
[puppetlabs.puppetdb.reports :as report]
[puppetlabs.puppetdb.utils :as utils]
[clojure.string :as str]
[puppetlabs.kitchensink.core :refer [regexp? boolean? uuid string-contains?]]
[clojure.walk :refer [keywordize-keys]]
[puppetlabs.puppetdb.random :refer :all]))
;; Validation functions, for use within pre/post conditions
......@@ -241,7 +240,7 @@
[type]
{:pre [(string? type)]
:post [(string? %)]}
(string/join "::" (map string/capitalize (string/split type #"::"))))
(str/join "::" (map str/capitalize (str/split type #"::"))))
(defn anonymize-tag
"Anonymize a tag"
......@@ -251,7 +250,7 @@
(let [newtag (capitalize-resource-type tag)
newcontext {"node" (get context "node")
"type" newtag}]
(string/lower-case (anonymize-leaf newtag :type newcontext config))))
(str/lower-case (anonymize-leaf newtag :type newcontext config))))
(defn anonymize-tags
"Anonymize a collection of tags"
......
;; ## Compression / Archive Utility library
;;
;; This namespace contains functions for reading and writing compressed
;; archive files. Currently only supports gzipped tar archives.
(ns puppetlabs.puppetdb.archive
"Compression / Archive Utility library
(ns com.puppetlabs.archive
(:import [java.io Closeable File OutputStream FileOutputStream IOException InputStream FileInputStream]
[org.apache.commons.compress.archivers.tar TarArchiveEntry TarArchiveOutputStream TarArchiveInputStream]
[org.apache.commons.compress.compressors.gzip GzipCompressorOutputStream GzipCompressorInputStream])
(:use [clojure.java.io]
[clj-time.core :only [now]]
[clj-time.coerce :only [to-date]]))
This namespace contains functions for reading and writing compressed
archive files. Currently only supports gzipped tar archives."
(:import [java.io
Closeable
File
OutputStream
FileOutputStream
IOException
InputStream
FileInputStream]
[org.apache.commons.compress.archivers.tar
TarArchiveEntry
TarArchiveOutputStream
TarArchiveInputStream]
[org.apache.commons.compress.compressors.gzip
GzipCompressorOutputStream
GzipCompressorInputStream])
(:require [clojure.java.io :refer :all]
[clj-time.core :refer [now]]
[clj-time.coerce :refer [to-date]]))
;; A simple type for writing tar/gz streams
(defrecord TarGzWriter [tar-stream tar-writer gzip-stream]
......
;; ## Catalog generation and manipulation
;;
;; A suite of functions that aid in constructing random catalogs, or
;; randomly modifying an existing catalog (wire format or parsed).
(ns com.puppetlabs.puppetdb.catalog.utils
(:require [clojure.string :as string]
[com.puppetlabs.puppetdb.catalogs :as cat]
(ns puppetlabs.puppetdb.catalog.utils
"Catalog generation and manipulation
A suite of functions that aid in constructing random catalogs, or
randomly modifying an existing catalog (wire format or parsed)."
(:require [puppetlabs.puppetdb.catalogs :as cat]
[clojure.walk :as walk]
[clojure.set :as set]
[clojure.string :as str])
(:use [com.puppetlabs.random :only [random-resource random-kw-resource random-parameters]]))
[clojure.string :as str]
[puppetlabs.puppetdb.random :refer [random-resource random-kw-resource
random-parameters]]))
(defn convert-to-wire
"Converts a catalog in the internal format to the wire format"
......
;; ## Puppet catalog parsing
;;
;; Functions that handle conversion of catalogs from wire format to
;; internal PuppetDB format.
;;
;; The wire format is described in detail in [the
;; spec](../spec/catalog-wire-format.md).
;;
;; There are a number of transformations we apply to wire format
;; catalogs during conversion to our internal format; while wire
;; format catalogs contain complete records of all resources and
;; edges, and most things are properly encoded as lists or maps, there
;; are still a number of places where structure is absent or lacking:
;;
;; 1. Resource specifiers are represented as opaque strings, like
;; `Class[Foobar]`, as opposed to something like
;; `{"type" "Class" "title" "Foobar"}`
;;
;; 2. Tags are represented as lists (and may contain duplicates)
;; instead of sets
;;
;; 3. Resources are represented as a list instead of a map, making
;; operations that need to correlate against specific resources
;; unneccesarily difficult
;;
;; 4. Keys to all maps are strings (to conform with JSON), instead of
;; more convenient Clojure keywords
;;
;; ### Terminology
;;
;; Unless otherwise indicated, all terminology for catalog components
;; matches terms listed in [the spec](../spec/catalog-wire-format.md).
;;
;; ### Transformed constructs
;;
;; ### Resource Specifier (resource-spec)
;;
;; A map of the form `{:type "Class" :title "Foobar"}`. This is a
;; unique identifier for a resource within a catalog.
;;
;; ### Resource
;;
;; A map that represents a single resource in a catalog:
;;
;; {:type "..."
;; :title "..."
;; :... "..."
;; :tags #{"tag1", "tag2", ...}
;; :parameters {:name1 "value1"
;; :name2 "value2"
;; ...}}
;;
;; Certain attributes are treated special:
;;
;; * `:type` and `:title` are used to produce a `resource-spec` for
;; this resource
;;
;; ### Edge
;;
;; A representation of an "edge" (dependency or containment) in the
;; catalog. All edges have the following form:
;;
;; {:source <resource spec>
;; :target <resource spec>
;; :relationship <relationship id>}
;;
;; A relationship identifier can be one of:
;;
;; * `:contains`
;; * `:required-by`
;; * `:notifies`
;; * `:before`
;; * `:subscription-of`
;;
;; ### Catalog
;;
;; A wire-format-neutral representation of a Puppet catalog. It is a
;; map with the following structure:
;;
;; {:certname "..."
;; :version "..."
;; :resources {<resource-spec> <resource>
;; <resource-spec> <resource>
;; ...}
;; :edges #(<dependency-spec>,
;; <dependency-spec>,
;; ...)}
;;
(ns com.puppetlabs.puppetdb.catalogs
(ns puppetlabs.puppetdb.catalogs
"Puppet catalog parsing
Functions that handle conversion of catalogs from wire format to
internal PuppetDB format.
The wire format is described in detail in [the spec](../spec/catalog-wire-format.md).
There are a number of transformations we apply to wire format
catalogs during conversion to our internal format; while wire
format catalogs contain complete records of all resources and
edges, and most things are properly encoded as lists or maps, there
are still a number of places where structure is absent or lacking:
1. Resource specifiers are represented as opaque strings, like
`Class[Foobar]`, as opposed to something like
`{\"type\" \"Class\" \"title\" \"Foobar\"}`
2. Tags are represented as lists (and may contain duplicates)
instead of sets
3. Resources are represented as a list instead of a map, making
operations that need to correlate against specific resources
unneccesarily difficult
4. Keys to all maps are strings (to conform with JSON), instead of
more convenient Clojure keywords
### Terminology
Unless otherwise indicated, all terminology for catalog components
matches terms listed in [the spec](../spec/catalog-wire-format.md).
### Transformed constructs
### Resource Specifier (resource-spec)
A map of the form `{:type \"Class\" :title \"Foobar\"}`. This is a
unique identifier for a resource within a catalog.
### Resource
A map that represents a single resource in a catalog:
{:type \"...\"
:title \"...\"
:... \"...\"
:tags #{\"tag1\", \"tag2\", ...}
:parameters {:name1 \"value1\"
:name2 \"value2\"
...}}
Certain attributes are treated special:
* `:type` and `:title` are used to produce a `resource-spec` for
this resource
### Edge
A representation of an \"edge\" (dependency or containment) in the
catalog. All edges have the following form:
{:source <resource spec>
:target <resource spec>
:relationship <relationship id>}
A relationship identifier can be one of:
* `:contains`
* `:required-by`
* `:notifies`
* `:before`
* `:subscription-of`
### Catalog
A wire-format-neutral representation of a Puppet catalog. It is a
map with the following structure:
{:certname \"...\"
:version \"...\"
:resources {<resource-spec> <resource>
<resource-spec> <resource>
...}
:edges #(<dependency-spec>,
<dependency-spec>,
...)}"
(:require [clojure.string :as string]
[clojure.tools.logging :as log]
[clojure.set :as set]
[com.puppetlabs.cheshire :as json]
[puppetlabs.puppetdb.cheshire :as json]
[digest]
[puppetlabs.kitchensink.core :as kitchensink]
[schema.core :as s]
[com.puppetlabs.puppetdb.schema :as pls]
[puppetlabs.puppetdb.schema :as pls]
[clojure.walk :as walk]
[com.puppetlabs.puppetdb.utils :as utils])
(:use [clojure.core.match :only [match]]))
[puppetlabs.puppetdb.utils :as utils]
[clojure.core.match :refer [match]]))
(def ^:const catalog-version
"Constant representing the version number of the PuppetDB
......
(ns ^{:doc "Cheshire related functions
(ns puppetlabs.puppetdb.cheshire
"Cheshire related functions
This front-ends the common set of core cheshire functions:
This front-ends the common set of core cheshire functions:
* generate-string
* generate-stream
* parse-string
* parse-stream
* generate-string
* generate-stream
* parse-string
* parse-stream
This namespace when 'required' will also setup some common JSON encoders
globally, so you can avoid doing this for each call."}
com.puppetlabs.cheshire
This namespace when 'required' will also setup some common JSON encoders
globally, so you can avoid doing this for each call."
(:require [cheshire.generate :as generate]
[cheshire.core :as core]
[clj-time.coerce :as coerce]
......
(ns com.puppetlabs.puppetdb.cli.anonymize
(:use [com.puppetlabs.puppetdb.cli.export :only [export-metadata-file-name]]
[com.puppetlabs.puppetdb.cli.import :only [parse-metadata]])
(:import [com.puppetlabs.archive TarGzReader TarGzWriter]
[org.apache.commons.compress.archivers.tar TarArchiveEntry])
(ns puppetlabs.puppetdb.cli.anonymize
(:import [puppetlabs.puppetdb.archive TarGzReader TarGzWriter]
[org.apache.commons.compress.archivers.tar TarArchiveEntry])
(:require [puppetlabs.kitchensink.core :as kitchensink]
[com.puppetlabs.cheshire :as json]
[puppetlabs.puppetdb.cheshire :as json]
[clojure.java.io :as io]
[clojure.string :as string]
[com.puppetlabs.archive :as archive]
[com.puppetlabs.puppetdb.anonymizer :as anon]
[puppetlabs.puppetdb.archive :as archive]
[puppetlabs.puppetdb.anonymizer :as anon]
[puppetlabs.puppetdb.schema :as pls]
[schema.core :as s]
[slingshot.slingshot :refer [try+]]
[com.puppetlabs.puppetdb.utils :refer [export-root-dir add-tar-entry]]
[com.puppetlabs.puppetdb.schema :as pls]
[schema.core :as s]))
[puppetlabs.puppetdb.utils :refer [export-root-dir add-tar-entry]]
[puppetlabs.puppetdb.cli.export :refer [export-metadata-file-name]]
[puppetlabs.puppetdb.cli.import :refer [parse-metadata]]))
(def cli-description "Anonymize puppetdb dump files")
......
;; ## Benchmark suite
;;
;; This command-line utility will simulate catalog submission for a
;; population. It requires that a separate, running instance of
;; PuppetDB for it to submit catalogs to.
;;
;; Aspects of a population this tool currently models:
;;
;; * Number of nodes
;; * Run interval
;; * How often a host's catalog changes
;; * A starting catalog
;;
;; We attempt to approximate a number of hosts submitting catalogs at
;; the specified runinterval with the specified rate-of-churn in
;; catalog content.
;;
;; The list of nodes is modeled in the tool as a set of Clojure
;; agents, with one agent per host. Each agent has the following
;; state:
;;
;; {:host ;; the host's name
;; :lastrun ;; when the host last sent a catalog
;; :catalog ;; the last catalog sent}
;;
;; When a host needs to submit a new catalog, we determine if the new
;; catalog should be different than the previous one (based on a
;; user-specified threshold) and send the resulting catalog to
;; PuppetDB.
;;
;; ### Main loop
;;
;; The main loop is written in the form of a wall-clock
;; simulator. Each run through the main loop, we send each agent an
;; `update` message with the current wall-clock. Each agent decides
;; independently whether or not to submit a catalog during that clock
;; tick.
;;
(ns com.puppetlabs.puppetdb.cli.benchmark
(ns puppetlabs.puppetdb.cli.benchmark
"Benchmark suite
This command-line utility will simulate catalog submission for a
population. It requires that a separate, running instance of
PuppetDB for it to submit catalogs to.
Aspects of a population this tool currently models:
* Number of nodes
* Run interval
* How often a host's catalog changes
* A starting catalog
We attempt to approximate a number of hosts submitting catalogs at
the specified runinterval with the specified rate-of-churn in
catalog content.
The list of nodes is modeled in the tool as a set of Clojure
agents, with one agent per host. Each agent has the following
state:
{:host ;; the host's name
:lastrun ;; when the host last sent a catalog
:catalog ;; the last catalog sent}
When a host needs to submit a new catalog, we determine if the new
catalog should be different than the previous one (based on a
user-specified threshold) and send the resulting catalog to
PuppetDB.
### Main loop
The main loop is written in the form of a wall-clock
simulator. Each run through the main loop, we send each agent an
`update` message with the current wall-clock. Each agent decides
independently whether or not to submit a catalog during that clock
tick."
(:import (java.io File))
(:require [clojure.tools.logging :as log]
[com.puppetlabs.puppetdb.catalogs :as cat]
[com.puppetlabs.puppetdb.catalog.utils :as catutils]