Commit 8103418e authored by Wyatt Alt's avatar Wyatt Alt

(PDB-2838) server side changes for remediation field

Add support for the new "remediation" field in the /reports and /nodes
responses, along with corresponding changes for the benchmark tool.
parent 7d9da4cc
......@@ -125,8 +125,8 @@ payload of this command.
### "store report", version 8
* The nullable `producer` property has been added.
* The nullable `noop_pending` property has been added.
* The nullable `producer`, `noop_pending`, and `corrective_change` fields have
been added.
The payload is expected to be a report, containing events that occurred on
Puppet resources. It is structured as a JSON object, conforming to the
......
......@@ -67,6 +67,10 @@ The below fields are allowed as filter criteria and are returned in all response
* `latest_report_noop_pending` (boolean): indicates whether the most recent
report for the node contained noop events.
* `latest_report_corrective_change` (boolean): a flag indicating whether the latest
report for the node included events that remediated configuration drift. This
field is only populated in PE.
* `cached_catalog_status` (string): Cached catalog status of the
last puppet run for the node. Possible values are `explicitly_requested`,
`on_failure`, `not_used` or `null`.
......
......@@ -112,6 +112,9 @@ run used a cached catalogs and whether or not the cached catalog was used due to
an error or not. Possible values include `explicitly_requested`, `on_failure`,
`not_used` or `null`.
* `corrective_change`: (Boolean): a flag indicating whether any of the report's
events remediated configuration drift. This field is only populated in PE.
* `latest_report?` (Boolean): return only reports associated with the most
recent Puppet run for each node. **Note:** this field does not appear in the
response.
......
......@@ -28,6 +28,7 @@ noted, `null` is not allowed anywhere in the report.
"cached_catalog_status": <string>,
"status": <string>,
"noop": <boolean>,
"corrective_change": <boolean>,
"noop_pending": <boolean>
}
......@@ -81,6 +82,9 @@ error or not. This field may be `null`.
events. These may result from use of the `--noop` flag, or from resources
tagged with the `noop` parameter. This field may be `null`.
`"corrective_change"` is a flag that indicates whether the report contained changes
to correct configuration drift. This field may be `null`.
`"resources"` is an array of objects of the following form:
......@@ -104,6 +108,7 @@ In each `<resource>` object `"events"` is an array of objects of the following f
"new_value": <new value for resource property>,
"old_value": <old value of resource property>,
"message": <description of what happened during event>,
"corrective_change": <flag indicating whether the event corrected system drift>
}
`"metrics"` is either null or an array of metric objects:
......@@ -127,7 +132,8 @@ In each `<resource>` object `"events"` is an array of objects of the following f
}
>**Note: Fields that allow `NULL` values**
>In the resource_event schema above, `containment_path`, `new_value`, `old_value`, `property`, `file`, `line`, `status`, and `message` may all be null.
>In the resource_event schema above, `containment_path`, `new_value`, `old_value`,
`property`, `file`, `line`, `status`, `corrective_change`, and `message` may all be null.
### Encoding
......
......@@ -142,96 +142,96 @@ msgstr ""
msgid "''not'' takes exactly one argument, but {0} were supplied"
msgstr ""
#: src/puppetlabs/puppetdb/query.clj:375
#: src/puppetlabs/puppetdb/query.clj:376
msgid "The argument to extract must be a select operator, not ''{0}''"
msgstr ""
#: src/puppetlabs/puppetdb/query.clj:379
#: src/puppetlabs/puppetdb/query.clj:380
msgid "Can't extract unknown {0} field ''{1}''. Acceptable fields are: {2}"
msgstr ""
#: src/puppetlabs/puppetdb/query.clj:395
#: src/puppetlabs/puppetdb/query.clj:396
msgid ""
"Can''t match on unknown {0} field ''{1}'' for ''in''. Acceptable fields are: "
"{2}"
msgstr ""
#: src/puppetlabs/puppetdb/query.clj:399
#: src/puppetlabs/puppetdb/query.clj:400
msgid "The subquery argument of ''in'' must be an ''extract'', not ''{0}''"
msgstr ""
#: src/puppetlabs/puppetdb/query.clj:465 src/puppetlabs/puppetdb/query.clj:655
#: src/puppetlabs/puppetdb/query.clj:746
#: src/puppetlabs/puppetdb/query.clj:466 src/puppetlabs/puppetdb/query.clj:656
#: src/puppetlabs/puppetdb/query.clj:747
msgid "= requires exactly two arguments, but {0} were supplied"
msgstr ""
#: src/puppetlabs/puppetdb/query.clj:499
#: src/puppetlabs/puppetdb/query.clj:500
msgid "''{0}'' is not a queryable object for resources in the version {1} API"
msgstr ""
#: src/puppetlabs/puppetdb/query.clj:531
#: src/puppetlabs/puppetdb/query.clj:532
msgid ""
"''{0}'' cannot be the target of a regexp match for version {1} of the "
"resources API"
msgstr ""
#: src/puppetlabs/puppetdb/query.clj:563
#: src/puppetlabs/puppetdb/query.clj:564
msgid "{0} is not a queryable object for version {1} of the facts query api"
msgstr ""
#: src/puppetlabs/puppetdb/query.clj:594
#: src/puppetlabs/puppetdb/query.clj:595
msgid "{0} is not a valid version {1} operand for regexp comparison"
msgstr ""
#: src/puppetlabs/puppetdb/query.clj:614
#: src/puppetlabs/puppetdb/query.clj:615
msgid "{0} is not a queryable object for facts"
msgstr ""
#: src/puppetlabs/puppetdb/query.clj:616
#: src/puppetlabs/puppetdb/query.clj:617
msgid "Value {0} must be a number for {1} comparison."
msgstr ""
#: src/puppetlabs/puppetdb/query.clj:627 src/puppetlabs/puppetdb/query.clj:764
#: src/puppetlabs/puppetdb/query.clj:628 src/puppetlabs/puppetdb/query.clj:765
msgid "{0} requires exactly two arguments, but {1} were supplied"
msgstr ""
#: src/puppetlabs/puppetdb/query.clj:640
#: src/puppetlabs/puppetdb/query.clj:641
msgid "''{0}'' is not a valid timestamp value"
msgstr ""
#: src/puppetlabs/puppetdb/query.clj:643
#: src/puppetlabs/puppetdb/query.clj:644
msgid "{0} operator does not support object ''{1}'' for resource events"
msgstr ""
#: src/puppetlabs/puppetdb/query.clj:698 src/puppetlabs/puppetdb/query.clj:735
#: src/puppetlabs/puppetdb/query.clj:699 src/puppetlabs/puppetdb/query.clj:736
msgid ""
"''{0}'' is not a queryable object for version {1} of the resource events API"
msgstr ""
#: src/puppetlabs/puppetdb/query.clj:710
#: src/puppetlabs/puppetdb/query.clj:711
msgid "~ requires exactly two arguments, but {0} were supplied"
msgstr ""
#: src/puppetlabs/puppetdb/query.clj:754
#: src/puppetlabs/puppetdb/query.clj:755
msgid "{0} is not a queryable object for event counts"
msgstr ""
#: src/puppetlabs/puppetdb/query.clj:772
#: src/puppetlabs/puppetdb/query.clj:773
msgid "{0} operator does not support object ''{1}'' for event counts"
msgstr ""
#: src/puppetlabs/puppetdb/query_eng.clj:66
msgid "Invalid entity '{0}' in query"
#: src/puppetlabs/puppetdb/query_eng.clj:68
msgid "Invalid entity ''{0}'' in query"
msgstr ""
#: src/puppetlabs/puppetdb/query_eng.clj:193
#: src/puppetlabs/puppetdb/query_eng.clj:199
#: src/puppetlabs/puppetdb/query_eng.clj:195
#: src/puppetlabs/puppetdb/query_eng.clj:201
msgid ""
"Error executing query ''{0}'' with query options ''{1}''. Returning a 400 "
"error code."
msgstr ""
#: src/puppetlabs/puppetdb/query_eng.clj:204
#: src/puppetlabs/puppetdb/query_eng.clj:206
msgid "Caught PSQL processing exception"
msgstr ""
......
......@@ -44,6 +44,7 @@ Puppet::Reports.register_report(:puppetdb) do
defaulted_code_id = defined?(code_id) ? code_id : nil
defaulted_cached_catalog_status = defined?(cached_catalog_status) ? cached_catalog_status : nil
defaulted_noop_pending = defined?(noop_pending) ? noop_pending : nil
defaulted_corrective_change = defined?(corrective_change) ? corrective_change : nil
{
"certname" => host,
......@@ -58,6 +59,7 @@ Puppet::Reports.register_report(:puppetdb) do
"status" => status,
"noop" => is_noop,
"noop_pending" => defaulted_noop_pending,
"corrective_change" => defaulted_corrective_change,
"logs" => build_logs_list,
"metrics" => build_metrics_list,
"resources" => resources,
......@@ -154,12 +156,14 @@ Puppet::Reports.register_report(:puppetdb) do
# @return Hash[<String, Object>]
# @api private
def event_to_hash(event)
corrective_change = defined?(event.corrective_change) ? event.corrective_change : nil
{
"status" => event.status,
"timestamp" => Puppet::Util::Puppetdb.to_wire_time(event.time),
"property" => event.property,
"new_value" => event.desired_value,
"old_value" => event.previous_value,
"corrective_change" => corrective_change,
"message" => event.message,
}
end
......@@ -177,15 +181,17 @@ Puppet::Reports.register_report(:puppetdb) do
# @return Hash[<String, Object>]
# @api private
def resource_status_to_hash(resource_status)
defaulted_corrective_change = defined?(resource_status.corrective_change) ? resource_status.corrective_change : nil
{
"skipped" => resource_status.skipped,
"timestamp" => Puppet::Util::Puppetdb.to_wire_time(resource_status.time),
"resource_type" => resource_status.resource_type,
"resource_title" => resource_status.title.to_s,
"file" => resource_status.file,
"line" => resource_status.line,
"containment_path" => resource_status.containment_path,
"events" => build_events_list(resource_status.events),
"skipped" => resource_status.skipped,
"timestamp" => Puppet::Util::Puppetdb.to_wire_time(resource_status.time),
"resource_type" => resource_status.resource_type,
"resource_title" => resource_status.title.to_s,
"file" => resource_status.file,
"line" => resource_status.line,
"containment_path" => resource_status.containment_path,
"corrective_change" => defaulted_corrective_change,
"events" => build_events_list(resource_status.events),
}
end
......
......@@ -116,6 +116,18 @@ describe processor do
end
end
it "should include corrective_change or nil" do
if defined?(subject.corrective_change) then
subject["corrective_change"] = false
end
result = subject.send(:report_to_hash)
if defined?(subject.corrective_change) then
result["corrective_change"].should == false
else
result["corrective_change"].should == nil
end
end
it "should include the cached_catalog_status or nil" do
if defined?(subject.cached_catalog_status) then
subject.cached_catalog_status = 'not_used'
......@@ -204,6 +216,9 @@ describe processor do
event.desired_value = "fooval"
event.previous_value = "oldfooval"
event.message = "foomessage"
if defined?(event.corrective_change) then
event.corrective_change = true
end
status.add_event(event)
result = subject.send(:report_to_hash)
......@@ -216,12 +231,22 @@ describe processor do
res["line"].should == 1
res["containment_path"].should == ["foo", "bar", "baz"]
res["events"].length.should == 1
if defined?(event.corrective_change) then
res["corrective_change"].should == true
else
res["corrective_change"].should == nil
end
res_event = res["events"][0]
res_event["property"].should == "fooprop"
res_event["new_value"].should == "fooval"
res_event["old_value"].should == "oldfooval"
res_event["message"].should == "foomessage"
if defined?(event.corrective_change) then
res_event["corrective_change"].should == true
else
res_event["corrective_change"].should == nil
end
end
end
......
......@@ -4,6 +4,8 @@
"configuration_version": "1423833741",
"end_time": "2015-02-13T13:22:52.014Z",
"environment": "production",
"corrective_change": false,
"noop_pending": false,
"logs": [
{
"file": null,
......@@ -244,18 +246,23 @@
"old_value": "absent",
"property": "message",
"status": "success",
"corrective_change": false,
"timestamp": "2015-02-13T13:22:54.116Z"
}
],
"file": "/etc/puppet/modules/loadtest/manifests/init.pp",
"line": 3,
"resource_title": "foo",
"corrective_change": false,
"resource_type": "Notify",
"skipped": false,
"timestamp": "2015-02-13T13:22:54.116Z"
}
],
"start_time": "2015-02-13T13:22:51.075Z",
"catalog_uuid": "2e59e974-7d40-4e41-bb81-20d91e2c2679",
"producer": "foo.com",
"code_id": "5ce7eda4-a281-49d5-94e9-edb4b3b1d0b5",
"status": "changed",
"transaction_uuid": "6906e9f2-d15a-414d-928c-ec1eac4b5e92"
}
......@@ -4,6 +4,8 @@
"configuration_version": "1423833650",
"end_time": "2015-02-13T13:21:04.382Z",
"environment": "production",
"corrective_change": false,
"noop_pending": false,
"logs": [
{
"file": null,
......@@ -472,10 +474,12 @@
"old_value": "{md5}a3f731187a6aaaa23d4a7b244f223f33",
"property": "content",
"status": "noop",
"corrective_change": false,
"timestamp": "2015-02-13T13:21:06.444Z"
}
],
"file": "/etc/puppet/modules/motd/manifests/init.pp",
"corrective_change": false,
"line": 33,
"resource_title": "/etc/motd",
"resource_type": "File",
......@@ -495,11 +499,13 @@
"old_value": "absent",
"property": "message",
"status": "noop",
"corrective_change": false,
"timestamp": "2015-02-13T13:21:06.488Z"
}
],
"file": "/etc/puppet/modules/loadtest/manifests/init.pp",
"line": 3,
"corrective_change": false,
"resource_title": "foo",
"resource_type": "Notify",
"skipped": false,
......@@ -519,12 +525,14 @@
"old_value": "absent",
"property": "ensure",
"status": "noop",
"corrective_change": false,
"timestamp": "2015-02-13T13:21:06.575Z"
}
],
"file": "/etc/puppet/modules/concat/manifests/fragment.pp",
"line": 113,
"resource_title": "/var/lib/puppet/concat/_tmp_file/fragments/02_tmpfile2",
"corrective_change": false,
"resource_type": "File",
"skipped": false,
"timestamp": "2015-02-13T13:21:06.575Z"
......@@ -543,12 +551,14 @@
"old_value": "{md5}df14e44b311152c34358a675ae34afe0",
"property": "content",
"status": "noop",
"corrective_change": false,
"timestamp": "2015-02-13T13:21:06.589Z"
}
],
"file": "/etc/puppet/modules/concat/manifests/fragment.pp",
"line": 113,
"resource_title": "/var/lib/puppet/concat/_tmp_file/fragments/01_tmpfile",
"corrective_change": false,
"resource_type": "File",
"skipped": false,
"timestamp": "2015-02-13T13:21:06.589Z"
......@@ -556,5 +566,8 @@
],
"start_time": "2015-02-13T13:21:03.326Z",
"status": "unchanged",
"catalog_uuid": "90de2540-d70e-40bb-a264-c84e5a95bbef",
"producer": "foo.com",
"code_id": "6764b61b-6f2a-4187-85e8-cef3e29cda5e",
"transaction_uuid": "d00ab23c-bd14-42a5-b5ef-855ceede8a40"
}
......@@ -4,6 +4,8 @@
"configuration_version": "1423833650",
"end_time": "2015-02-13T13:21:14.240Z",
"environment": "production",
"corrective_change": false,
"noop_pending": false,
"logs": [
{
"file": null,
......@@ -2672,6 +2674,7 @@
"old_value": "{md5}a3f731187a6aaaa23d4a7b244f223f33",
"property": "content",
"status": "noop",
"corrective_change": false,
"timestamp": "2015-02-13T13:21:16.348Z"
}
],
......@@ -2679,6 +2682,7 @@
"line": 33,
"resource_title": "/etc/motd",
"resource_type": "File",
"corrective_change": false,
"skipped": false,
"timestamp": "2015-02-13T13:21:16.348Z"
},
......@@ -2695,6 +2699,7 @@
"old_value": "absent",
"property": "message",
"status": "noop",
"corrective_change": false,
"timestamp": "2015-02-13T13:21:16.393Z"
}
],
......@@ -2703,6 +2708,7 @@
"resource_title": "foo",
"resource_type": "Notify",
"skipped": false,
"corrective_change": false,
"timestamp": "2015-02-13T13:21:16.393Z"
},
{
......@@ -2719,12 +2725,14 @@
"old_value": "absent",
"property": "ensure",
"status": "noop",
"corrective_change": false,
"timestamp": "2015-02-13T13:21:16.508Z"
}
],
"file": "/etc/puppet/modules/concat/manifests/fragment.pp",
"line": 113,
"resource_title": "/var/lib/puppet/concat/_tmp_file/fragments/02_tmpfile2",
"corrective_change": false,
"resource_type": "File",
"skipped": false,
"timestamp": "2015-02-13T13:21:16.508Z"
......@@ -2743,6 +2751,7 @@
"old_value": "{md5}df14e44b311152c34358a675ae34afe0",
"property": "content",
"status": "noop",
"corrective_change": false,
"timestamp": "2015-02-13T13:21:16.523Z"
}
],
......@@ -2750,11 +2759,15 @@
"line": 113,
"resource_title": "/var/lib/puppet/concat/_tmp_file/fragments/01_tmpfile",
"resource_type": "File",
"corrective_change": false,
"skipped": false,
"timestamp": "2015-02-13T13:21:16.523Z"
}
],
"start_time": "2015-02-13T13:21:13.238Z",
"status": "unchanged",
"catalog_uuid": "dcedddbf-25be-43fa-bf6d-a8ffdf765c73",
"producer": "foo.com",
"code_id": "b6187ff7-0e4f-4400-818d-8ee1dfd0fe38",
"transaction_uuid": "ac63fb6f-d191-4d8a-94af-995ccaadb7dd"
}
......@@ -4,6 +4,8 @@
"configuration_version": "1423833650",
"end_time": "2015-02-13T13:21:49.346Z",
"environment": "production",
"corrective_change": true,
"noop_pending": false,
"logs": [
{
"file": null,
......@@ -279,12 +281,14 @@
"old_value": "{md5}02d304aace03a918311acc96cd339cc3",
"property": "content",
"status": "success",
"corrective_change": true,
"timestamp": "2015-02-13T13:21:52.052Z"
}
],
"file": "/etc/puppet/modules/motd/manifests/init.pp",
"line": 33,
"resource_title": "/etc/motd",
"corrective_change": true,
"resource_type": "File",
"skipped": false,
"timestamp": "2015-02-13T13:21:52.052Z"
......@@ -302,12 +306,14 @@
"old_value": "absent",
"property": "message",
"status": "success",
"corrective_change": false,
"timestamp": "2015-02-13T13:21:52.095Z"
}
],
"file": "/etc/puppet/modules/loadtest/manifests/init.pp",
"line": 3,
"resource_title": "foo",
"corrective_change": false,
"resource_type": "Notify",
"skipped": false,
"timestamp": "2015-02-13T13:21:52.095Z"
......@@ -315,5 +321,8 @@
],
"start_time": "2015-02-13T13:21:48.201Z",
"status": "changed",
"catalog_uuid": "a86149f6-e0fd-46e4-9080-f873002837bb",
"producer": "foo.com",
"code_id": "75ffb3e4-ae64-4a58-9017-1d84563852b2",
"transaction_uuid": "af8d1c1c-f1b9-450a-949d-f6990cf24d63"
}
......@@ -4,6 +4,8 @@
"configuration_version": "1423833741",
"end_time": "2015-02-13T13:22:42.965Z",
"environment": "production",
"corrective_change": false,
"noop_pending": false,
"logs": [
{
"file": null,
......@@ -244,11 +246,13 @@
"old_value": "absent",
"property": "message",
"status": "success",
"corrective_change": false,
"timestamp": "2015-02-13T13:22:45.121Z"
}
],
"file": "/etc/puppet/modules/loadtest/manifests/init.pp",
"line": 3,
"corrective_change": false,
"resource_title": "foo",
"resource_type": "Notify",
"skipped": false,
......@@ -257,5 +261,8 @@
],
"start_time": "2015-02-13T13:22:42.019Z",
"status": "changed",
"catalog_uuid": "065ebf9f-8592-4f83-8537-d270a69dd91e",
"producer": "foo.com",
"code_id": "9d1df9cd-99e6-4f8e-8536-5d892aca3600",
"transaction_uuid": "8659eaf5-90a6-41d7-8898-6b0aec08d106"
}
......@@ -4,6 +4,8 @@
"configuration_version": "1423833741",
"end_time": "2015-02-13T13:26:57.691Z",
"environment": "production",
"corrective_change": true,
"noop_pending": false,
"logs": [
{
"file": null,
......@@ -596,11 +598,13 @@
"old_value": "notrun",
"property": "returns",
"status": "success",
"corrective_change": true,
"timestamp": "2015-02-13T13:26:57.409Z"
}
],
"file": "/etc/puppet/modules/postgresql/manifests/server/config_entry.pp",
"line": 83,
"corrective_change": true,
"resource_title": "postgresql_data_directory",
"resource_type": "Exec",
"skipped": false,
......@@ -619,6 +623,7 @@
"old_value": "absent",
"property": "ensure",
"status": "success",
"corrective_change": false,
"timestamp": "2015-02-13T13:26:58.573Z"
}
],
......@@ -626,6 +631,7 @@
"line": 120,
"resource_title": "/etc/sysconfig/pgsql/postgresql-8.4",
"resource_type": "File",
"corrective_change": false,
"skipped": false,
"timestamp": "2015-02-13T13:26:58.573Z"
},
......@@ -643,11 +649,13 @@
"old_value": "need_to_run",
"property": "returns",
"status": "success",
"corrective_change": false,
"timestamp": "2015-02-13T13:26:58.781Z"
}
],
"file": "/etc/puppet/modules/postgresql/manifests/server/config_entry.pp",
"line": 90,
"corrective_change": false,
"resource_title": "override PGDATA in /etc/sysconfig/pgsql/postgresql",
"resource_type": "Augeas",
"skipped": false,
......@@ -667,6 +675,7 @@
"old_value": "absent",
"property": "ensure",
"status": "success",
"corrective_change": false,
"timestamp": "2015-02-13T13:26:58.814Z"
}
],
......@@ -674,6 +683,7 @@
"line": 105,
"resource_title": "data_directory",
"resource_type": "Postgresql_conf",
"corrective_change": false,
"skipped": false,
"timestamp": "2015-02-13T13:26:58.814Z"
},
......@@ -691,12 +701,14 @@
"old_value": "absent",
"property": "message",
"status": "success",
"corrective_change": false,
"timestamp": "2015-02-13T13:26:58.975Z"
}
],
"file": "/etc/puppet/manifests/site.pp",
"line": 6,
"resource_title": "foo",
"corrective_change": false,
"resource_type": "Notify",
"skipped": false,
"timestamp": "2015-02-13T13:26:58.975Z"
......@@ -714,11 +726,13 @@
"old_value": "stopped",
"property": "ensure",
"status": "success",
"corrective_change": false,
"timestamp": "2015-02-13T13:26:59.281Z"
}
],
"file": "/etc/puppet/modules/postgresql/manifests/server/service.pp",
"line": 14,
"corrective_change": false,
"resource_title": "postgresqld",
"resource_type": "Service",
"skipped": false,
......@@ -727,5 +741,8 @@
],
"start_time": "2015-02-13T13:26:52.564Z",
"status": "changed",
"catalog_uuid": "f9e15df3-bbfb-404c-80bd-f29ecdbe5283",
"producer": "foo.com",
"code_id": "8589ce99-5690-4900-a249-a052e044ca11",
"transaction_uuid": "9df4e7ce-5646-4178-8ff8-db6add84e9fc"
}
......@@ -4,6 +4,8 @@
"configuration_version": "1423833741",
"end_time": "2015-02-13T13:27:14.717Z",
"environment": "production",
"corrective_change": false,
"noop_pending": false,
"logs": [
{
"file": null,
......@@ -250,12 +252,14 @@
"old_value": "absent",
"property": "message",
"status": "success",
"corrective_change": false,
"timestamp": "2015-02-13T13:27:17.956Z"
}