Commit 8bd86e21 authored by Ken Barber's avatar Ken Barber

(PDB-1034) Ezbake source based testing

This change switched PuppetDB over to use the ezbake_utils helpers in ezbake
to install PuppetDB via source. This effectively means the old installation
methodology can be retired.

For now, this just pins us to a branch for beaker, until that patch is merged.

The script is now modified to ensure that leiningen
is now installed (since its required by ezbake helpers in beaker).

Some minor code was removed that is no longer needed, but a major retirement
patch will be forthcoming to remove the greater amount of code no longer
required. Since it was deemed such a patch would make this patch hard to read.

Also, retry was added to the acceptance test jobs, to ensure we can avoid
transient issues getting at the git version of beaker.
Signed-off-by: 's avatarKen Barber <>
parent ca1ff210
......@@ -42,7 +42,10 @@ group :test do
group :acceptance do
gem 'beaker', '~> 2.2'
#gem 'beaker', '~> 2.2'
# TODO: unpin when released
gem 'beaker', :git => 'git://',
:branch => 'pdb-1034-ezbake-pr-testing'
# This forces google-api-client to not download retirable 2.0.0 which lacks
# ruby 1.9.x support.
gem 'retriable', '~> 1.4'
......@@ -20,15 +20,13 @@ An example of how to run a package based build, with vagrant/virtualbox on Cento
rake beaker:acceptance
Source based build, from a particular branch:
Source based build:
rake beaker:acceptance
EC2 build from source on Debian 6:
BEAKER_CONFIG="ec2-west-debian6-64mda-64a" \
rake beaker:acceptance
EC2 build with packages on Debian 6:
......@@ -116,10 +114,6 @@ in your hash; the environment variable names are the same but uppercased
from source. This is primarily so we can test against alternate versions of
Puppet, so if the Puppet repo is not specified we fall back to using packages.
* `:puppetdb_repo_puppetdb` (`PUPPETDB_REPO_PUPPETDB`) :
Specify the git repository and reference for where we are to download the
PuppetDB source code.
* `:puppetdb_git_ref` (`REF`) :
Specify the specific git ref that the tests should be run against. This should
almost always be passed in by the Jenkins job, and not overridden by configuration.
......@@ -89,9 +89,6 @@ module PuppetDBExtensions
puppetdb_repo_facter = get_option_value(options[:puppetdb_repo_facter],
nil, "git repo for facter source installs", "PUPPETDB_REPO_FACTER", nil)
puppetdb_repo_puppetdb = get_option_value(options[:puppetdb_repo_puppetdb],
nil, "git repo for puppetdb source installs", "PUPPETDB_REPO_PUPPETDB", nil)
puppetdb_git_ref = get_option_value(options[:puppetdb_git_ref],
nil, "git revision of puppetdb to test against", "REF", nil)
......@@ -114,7 +111,6 @@ module PuppetDBExtensions
:repo_puppet => puppetdb_repo_puppet,
:repo_hiera => puppetdb_repo_hiera,
:repo_facter => puppetdb_repo_facter,
:repo_puppetdb => puppetdb_repo_puppetdb,
:git_ref => puppetdb_git_ref,
......@@ -425,32 +421,7 @@ module PuppetDBExtensions
# @return [void]
# @api public
def install_puppetdb_via_rake(host)
os = PuppetDBExtensions.config[:os_families][]
case os
when :debian
preinst = "debian/puppetdb.preinst install"
postinst = "debian/puppetdb.postinst"
when :redhat, :fedora
preinst = "dev/redhat/redhat_dev_preinst install"
postinst = "dev/redhat/redhat_dev_postinst install"
raise ArgumentError, "Unsupported OS family: '#{os}'"
# We tag here so the build system knows what version to use, first
# we grab the version to use as tag from the project.clj,
# reformatting the output as necessary (since its in CLJ).
result = on host, "#{LeinCommandPrefix} lein with-profile ci pprint :version | tail -n 1"
jar_version = result.stdout.chomp.gsub(/"/, '')
on host, "#{LeinCommandPrefix} git tag -f -a '#{jar_version}' -m 'temporary tag for source build to work'"
on host, "rm -rf /etc/puppetdb/ssl"
on host, "#{LeinCommandPrefix} rake package:bootstrap"
on host, "#{LeinCommandPrefix} rake template"
on host, "bash -x #{GitReposDir}/puppetdb/ext/files/#{preinst}"
on host, "#{LeinCommandPrefix} rake install"
on host, "bash -x #{GitReposDir}/puppetdb/ext/files/#{postinst}"
install_from_ezbake host
step "Configure database.ini file" do
manifest = "
......@@ -466,7 +437,7 @@ module PuppetDBExtensions
def install_puppetdb_termini_via_rake(host, database)
on host, "#{LeinCommandPrefix} rake sourceterminus"
install_termini_from_ezbake host
manifest = <<-EOS
include puppetdb::master::routes
......@@ -972,7 +943,7 @@ EOS
tmp_repositories = []
repos = Hash[* {|k, v| k =~ /^repo_/ and k != 'repo_puppetdb' }.flatten].values
repos = Hash[* {|k, v| k =~ /^repo_/}.flatten].values
repos.each do |uri|
raise(ArgumentError, "#{uri} is not recognized.") unless(uri =~ git_uri)
......@@ -4,21 +4,7 @@ step "Install development build of PuppetDB on the PuppetDB server" do
case test_config[:install_type]
when :git
raise "No PUPPETDB_REPO_PUPPETDB set" unless test_config[:repo_puppetdb]
case os
when :redhat, :fedora
on database, "yum install -y git-core ruby rubygem-rake"
when :debian
on database, "apt-get install -y git-core ruby rake"
raise "OS #{os} not supported"
on database, "rm -rf #{GitReposDir}/puppetdb"
repo = extract_repo_info_from(test_config[:repo_puppetdb].to_s)
install_from_git database, GitReposDir, repo,
:refspec => '+refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*'
Log.notify("Install puppetdb from source")
if (test_config[:database] == :postgres)
......@@ -30,8 +30,18 @@ then
mkdir vendor
bundle install --path vendor/bundle --without test
bundle install --path=vendor/bundle --without=test --retry=10
# Install a copy of leiningen
if [ -d "leiningen" ];
rm -rf leiningen
mkdir leiningen
wget '' -O leiningen/lein
chmod +x leiningen/lein
# Now run our tests
PATH=$PATH:$(pwd)/leiningen \
bundle exec rake test:beaker
......@@ -36,7 +36,7 @@ then
mkdir vendor
bundle install --path vendor/bundle --without test
bundle install --path=vendor/bundle --without=test --retry=10
# Now run our tests
bundle exec rake beaker:acceptance
......@@ -120,7 +120,7 @@
:ezbake {:dependencies ^:replace [[puppetlabs/puppetdb ~pdb-version]
[org.clojure/tools.nrepl "0.2.3"]]
:name "puppetdb"
:plugins [[puppetlabs/lein-ezbake "0.1.0"
:plugins [[puppetlabs/lein-ezbake "0.2.0"
:exclusions [org.clojure/clojure]]]}
:ci {:plugins [[lein-pprint "1.1.1"]]}}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment