1. 09 May, 2017 1 commit
    • Adrien Thebo's avatar
      (maint) Use a real scope object for terminus testing · 4013d322
      Adrien Thebo authored
      Changes coming in Puppet 5.0.0 change how Puppet::Parser::Resource
      instances set default values; in specific they do so upon object
      construction which requires a functioning(ish) scope object. The
      Puppetdb terminus specs were relying on the scope object being
      relatively simple and easily stubbed but this change in defaults makes
      it much harder to treat the scope object as an easily stubbed object.
      
      To simplify stubbing and potentially reduce future breakage this commit
      changes the scope stubbing to partially stub a full scope object instead
      of creating a pure stub object and adding methods as necessary. By using
      a full scope object we can let the internal APIs of Puppet handle
      implementation changes.
      4013d322
  2. 03 Apr, 2017 1 commit
    • Rob Browning's avatar
      (PDB-3322) Redact sensitive parameters in terminus · fdd4ceff
      Rob Browning authored
      A resource parameter marked Sensitive()
      
        notify {'hi':  message => Sensitive('there')}
      
      will show up in the terminus
      like this:
      
              input = {...
                       'environment'=>'production',
                       'resources'=>
                       [...
                        {'type'=>'Notify',
                         'title'=>'hi',
                         'tags'=>Puppet::Util::TagSet.new(['notify', 'hi', 'class']),
                         'file'=> 'site.pp',
                         'line'=>1,
                         'exported'=>false,
                         'parameters'=>{:message=>'there'},
                         'sensitive_parameters'=>[:message]}],
                       'edges'=>
                       [{'source'=>'Stage[main]', 'target'=>'Class[Settings]'},
                        {'source'=>'Stage[main]', 'target'=>'Class[main]'},
                        {'source'=>'Class[main]', 'target'=>'Notify[hi]'}],
                       'classes'=>['settings']}
      
      Remove any sensitive values from 'parameters', and remove
      'sensitive_parameters' before sending data to PuppetDB.
      
      Aside from the fact that we don't want senstitive parameters to be
      stored in (or even make it to) PuppetDB, the 'sensitive_parameters'
      element can cause command processing to fail, e.g. when an existing
      parameter becomes sensitive.  Add an integration test for that.
      fdd4ceff
  3. 29 Mar, 2017 1 commit
    • Wyatt Alt's avatar
      (maint) fix spec test failures against puppet master · f34fcdd7
      Wyatt Alt authored
      A recent patch to Puppet broke several of our spec tests. This corrects
      the ones that are due to a changed error message. There are some other
      failures that are due to what seems like a Puppet bug to me. I'll wait
      to get confirmation before changing those ones.
      f34fcdd7
  4. 21 Mar, 2017 1 commit
    • Russell Mull's avatar
      (PDB-3328) Munge package inventory facts in terminus · d254ea9b
      Russell Mull authored
      This adds terminus support for that package inventory feature. When the
      _puppet_inventory_1 fact is seen it is removed, reformatted to be in the tabular
      format expected by puppetdb, and placed in a top-level 'package_inventory' key
      in the facts command payload.
      d254ea9b
  5. 10 Mar, 2017 1 commit
  6. 03 Mar, 2017 1 commit
    • Ruth Linehan's avatar
      (PE-18347) Handle multiple arities for `get` in http client class · ad0905ab
      Ruth Linehan authored
      Prior to Puppet Server 5, Puppet Server's http client class had an
      implementation of `get` with an arity of 2 (where `path` and `headers` are the
      arguments). However, in Ruby Puppet and in Puppet Server 5, the http client
      class has an implementation of `get` with an arity of 3 (with arguments
      `path`, `headers`, and `options`).
      
      Since we do not know under which version of Puppet Server or the Ruby Puppet
      master this terminus code will be run under, this commit adds a
      `multi_arity_get` helper function that inspects the `http_instance` class to
      determine the arity of `get` before calling it with the appropriate arguments
      for its arity.
      ad0905ab
  7. 01 Mar, 2017 1 commit
    • Ruth Linehan's avatar
      (PE-18347) Add metric_id option to http requests in termini · a98f40a4
      Ruth Linehan authored
      Add the `metric_id` option to the http requests in the facts terminus,
      resource terminus, query function, and for command submission. These
      `metric_id`s will be used by the http client Puppet Server uses to create
      metrics in its registry, which can then be queried to determine how long
      requests Puppet Server makes to PuppetDB are taking.
      
      For each element in a `metric_id` array, a metric is created. So, a `metric_id`
      `[:foo :bar :baz]` will result in metrics `foo`, `foo.bar`, and `foo.bar.baz`.
      The elements in the array can be symbols or strings. For more information, see
      the [documentation on metrics in the clj-http-client
      library](https://github.com/puppetlabs/clj-http-client/blob/master/doc/metrics.md).
      
      For the facts `find` action, the `metric_id` is `[:puppetdb :facts :find
      <node-name>]`.
      
      For the facts `search` action, the `metric_id` is `[:puppetdb :facts
      :search]`. The query is not included as part of the `metric_id` because it
      would not create useful metrics to capture - each search query can be pretty
      much anything, and tracking a single search query over time probably won't be
      very useful.
      
      For the resource `search` action, the `metric_id` is `[:puppetdb :resource
      :search <resource type>]`.
      
      For the http call made by the puppetdb query function, the `metric_id` is
      `[:puppetdb :query]`. This metric-id does not include the actual query, since
      there are so many possible queries that tracking metrics on them does not seem
      particularly useful.
      
      For the http requests for submitting commands, `metric_id`s will be
      `[:puppetdb :command <command name> <certname>]`.
      a98f40a4
  8. 08 Feb, 2017 1 commit
    • Jeremy Barlow's avatar
      (PDB-2640) Add support for gzip-compressing commands via terminus · 33c63ffc
      Jeremy Barlow authored
      This commit adds support for gzip-compressing commands which the
      PuppetDB terminus sends out.  When stockpile persists the commands to
      disk, it stores the gzipped json bytes with a .json.gz file extension.
      Command payloads are decompressed and parsed from JSON when processed
      back out of the queue.
      
      The terminus does not gzip-compress the command content itself.
      Instead, to minimize memory usage somewhat and realize a bit better
      performance, it passes through its intent for the implementation
      upstream from the Puppet::Network::HttpPool.http_instance to perform
      the gzip compression.  Corresponding work in Puppet Server and its
      upstream clj-http-client library enables the payload to be compressed.
      33c63ffc
  9. 07 Feb, 2017 3 commits
  10. 25 Oct, 2016 1 commit
    • Wyatt Alt's avatar
      (PDB-3071) Log a warning if retired server/port config items are in use. · 9be9a8ac
      Wyatt Alt authored
      It's relatively common for users to miss the retirement of server/port
      when upgrading from versions of PDB that supported it. This has the
      effect of defaulting their server/port to puppetdb:8081, producing agent
      run failures that log a puppetdb hostname different from what they
      believe they've configured.
      
      This patch logs warnings when server and port are detected in the config
      file. I think it would be preferable to remove the defaulting behavior
      and throw an error when server_urls is not present, but I don't have a
      sense of whether anyone relies on the defaulting today.
      9be9a8ac
  11. 10 Oct, 2016 1 commit
    • Wyatt Alt's avatar
      (PDB-3071) Log a warning if retired server/port config items are in use. · c95db47b
      Wyatt Alt authored
      It's relatively common for users to miss the retirement of server/port
      when upgrading from versions of PDB that supported it. This has the
      effect of defaulting their server/port to puppetdb:8081, producing agent
      run failures that log a puppetdb hostname different from what they
      believe they've configured.
      
      This patch logs warnings when server and port are detected in the config
      file. I think it would be preferable to remove the defaulting behavior
      and throw an error when server_urls is not present, but I don't have a
      sense of whether anyone relies on the defaulting today.
      c95db47b
  12. 04 Oct, 2016 1 commit
  13. 03 Oct, 2016 1 commit
  14. 11 Aug, 2016 1 commit
  15. 10 Aug, 2016 1 commit
  16. 03 Aug, 2016 1 commit
  17. 07 Jul, 2016 1 commit
  18. 06 Jul, 2016 1 commit
  19. 23 Jun, 2016 1 commit
  20. 19 Jun, 2016 1 commit
  21. 14 Jun, 2016 1 commit
  22. 09 Jun, 2016 1 commit
  23. 13 May, 2016 1 commit
    • Karel Březina's avatar
      (PE-15587) Set noop flag on a report to true only if all its resource events are noop · b6c2243f
      Karel Březina authored
      This commit improves the algorithm which tries to recognize that puppet run was started with --noop param. Originally, any noop resource event caused that the whole report was marked as noop. However, a noop puppet run would have all resource events in noop state. With this commit, all resource events need to be noop to mark the whole report as noop.
      
      The current implementation is not able to recognize an unchanged noop puppet run since it doesn't contain any events. So, this improvement is just a workaround. To be able to recognize noop puppet runs properly, puppet needs to report in which mode was the run started. This problem is logged as PUP-6294.
      b6c2243f
  24. 21 Apr, 2016 1 commit
    • lafont's avatar
      Add a basic checking of the "query" parameter · 3e46965b
      lafont authored
      To be compatible with Puppet 4.0.0 where recursive types
      are not possible, the type checked here is not accurate.
      Ideally, the type should be:
      
        Variant[String[1], StringTree]
      
      where StringTree is defined (recursively) by:
      
        StringTree = Array[Variant[String, StringTree]]
      3e46965b
  25. 31 Mar, 2016 1 commit
  26. 07 Mar, 2016 1 commit
  27. 16 Feb, 2016 3 commits
  28. 26 Jan, 2016 1 commit
  29. 21 Jan, 2016 1 commit
  30. 13 Jan, 2016 1 commit
  31. 18 Dec, 2015 1 commit
    • Ryan Senior's avatar
      (PDB-2256) Fix terminus bug with large binary catalog data · 0b96f28b
      Ryan Senior authored
      The terminus includes code to help users diagnose the source of data
      that cannot be converted to UTF-8. There are several sources of this,
      but one is having an incorrect (or known) character set for portions of
      the catalog. When this invalid character data is large, it would cause
      the terminus to hang trying to include debugging information.
      
      This patch changes the terminus to only look for the first instance of
      bad data. It will also avoid doing the extra calculations needed for the
      error context unless debug mode is enabled. When not in debug mode there
      should be no impact on performance.
      0b96f28b
  32. 17 Dec, 2015 1 commit
    • Ryan Senior's avatar
      (PDB-2256) Fix terminus bug with large binary catalog data · fef7418b
      Ryan Senior authored
      The terminus includes code to help users determine the source of data
      that cannot be converted to UTF-8. There are several sources of this,
      but one is having an incorrect (or known) character set for portions of
      the catalog. When this invalid character data is large, it would cause
      the terminus to hang trying to include debugging information.
      
      This patch changes the terminus to only look for the first instance of
      bad data. It will also avoid doing the extra calculations needed for the
      error context unless debug mode is enabled. When not in debug mode there
      should be no impact on performance.
      fef7418b
  33. 09 Nov, 2015 1 commit
  34. 03 Nov, 2015 2 commits
  35. 30 Oct, 2015 1 commit
    • Russell Mull's avatar
      (PDB-2071) Use shared atom for terminus failover · 953b47d6
      Russell Mull authored
      When running under PuppetServer, there can be many terminii running at
      once in many JRuby instances. For sticky read failover to work properly,
      the index of the last working server must be shared among these. This
      uses the new PuppetDBTerminusState java class in PuppetServer to store
      the shared data. 
      
      When the java class is not available, or when running under MRI, a local
      instances of a Ruby atom class is used. It obeys the same semantics but 
      is not shared across Ruby instances. 
      953b47d6