1.5.7

This is the 1.5.7 stable release of ara.

It features a new "hosts" page to browse and search playbook reports by host
as well as fixes and improvements.

Instructions for upgrading are included in the upgrade notes.

Changes since 1.5.6:

UI
--

- Added a new "hosts" page to browse and search reports by host name
- Improved page HTML titles to be dynamic based on the context
- Added a note highlighting if a task has been delegated to another host
  (https://github.com/ansible-community/ara/issues/282)
- Improved how long file paths or playbook names are truncated and displayed

API
---

- Added a new read-only API endpoint: /api/v1/latesthosts
  It provides the latest playbook result for each host name.
  Under the hood, it implements the machinery for updating the latest host
  every time a host is created or deleted and includes a SQL migration to
  initially populate a new database table with the latest hosts.
- Added a `delegated_to` field to results in order to record a host id to which
  a task has been delegated.
- Added support for finding results delegated to a specific host:
  /api/v1/results?delegated_to=<host_id>

Callback plugin
---------------

- Fixed tasks and results being recorded out of order when using "strategy: free"
  (https://github.com/ansible-community/ara/issues/260)
- Added support for recording 'delegate_to' on tasks

Documentation
-------------

- Removed an unused sphinx lexer to allow recent versions of sphinx>=4
- Created a new troubleshooting guide with common issues:
  https://ara.readthedocs.io/en/latest/troubleshooting.html
- Added a database relationship graph to the endpoint documentation:
  https://ara.readthedocs.io/en/latest/api-documentation.html#relationship-between-objects

Upgrade notes
-------------

It is always recommended to take a backup of your database before upgrading.

This release includes two database migrations that must be run:
- One for populating the data for the new /api/v1/latesthosts endpoint as well
  as the new 'hosts' page
- One for adding a `delegated_to` field in the results.
  Note that delegated tasks will only be recorded as such from 1.5.7 on.

After upgrading to 1.5.7, database migrations can be run manually with the
`ara-manage migrate` command if they are not taken care of automatically by the
callback plugin.

Known issues
------------

ara will not record task delegation for tasks that are skipped or for
items in a loop that are skipped because Ansible doesn't provide the
necessary information in those cases.

Improve the wording, include the new LatestHost and fill in gaps
about some of the objects that were not explained.

It may not be obvious at first glance what this does.
Add a short tooltip to make it clearer.

The files in these directories are not intended to ship in the release
package tarballs.

Christoph Hille authored Jul 30, 2021 and David Moreau-Simard committed Jul 30, 2021

* this fixes issue #311 and make ara compatible with previous versions again

8df13a99 avoided recording tasks with
delegate_to that had been skipped entirely due to a 'when' condition
because Ansible doesn't template the variable sent to the callback.

It turns out that we also need to avoid recording it for skipped items
while in a loop because the delegate_to also doesn't get templated by
Ansible.

It looks like there are many different cases to test and validate and
what started as a simple enough approach doesn't cut it.

Refactor it a bit to make it a bit more modular.

When delegate_to is a variable and the task is skipped, Ansible doesn't
template the variable before sending it to the callback.

We can afford to work around this for skipped tasks.

We have a custom perform_destroy that runs when deleting a host via the
API (DELETE /api/v1/hosts/<id>) that takes care of updating the
latesthost entry.

However, because hosts get deleted via the database model
(on_delete=models.CASCADE) when deleting a playbook via the API, the
latesthost entry would get deleted when the host was deleted instead of
being updated.

This adds a very similar perform_destroy for playbooks that iterates
over the hosts and if one of the hosts happens to be a LatestHost, it
will update the entry before deleting.

After testing the initial implementation in production, it had two issues:
1) A task could potentially be delegated to more than a single host if
   delegate_to is a variable provided by a loop (ex: {{ item }})
2) Inside the callback, result._task.delegate_to could return a variable
   instead of a value when the task is a loop.
   See ansible/ansible issue #75339 for details.

We're addressing the first issue by changing the model and API to expect
delegated_to to be a list -- even if it's empty or just one item.

For the second issue, we found that the value of delegate_to is properly
templated for a task loop from within the v2_runner_item_on_* callback
hooks. The callback was adjusted to leverage these item hooks to get the
proper list of delegated host names.

commit 982f1a84 made it so we could no
longer filter by created or updated for LatestHost which made it
inconsistent with all the other endpoints.

This adds host__ fields to filter on the nested host fields instead.

We have a similar message elsewhere when no results are found, let's add
one here too.

When querying for created or updated dates for the latesthost endpoint,
filter on the dates of the referenced host instead.

Displaying that a task had been delegated in the details of a result
didn't feel sufficient so we'll highlight it in the result tables as
well.

The intent was to order hosts by their latest update with the most
recent at the top: -updated.

However, it turns out that we were sorting LatestHost objects instead of
Host objects. This commits fixes it so we sort based on the date of the
host, not the latesthost reference.

The API link changes based on the page that is displayed to best fit a
general API call for the resource the user is looking at.

Because the new host index uses latesthosts by default, point to
/api/v1/latesthosts but if all reports are included, provide
/api/v1/hosts instead.

This has no incidence on functionality and is just a name change intended
to clarify and streamline the meaning and vocabulary around the feature.

Depending on the context, the words "latest" and "distinct" could have
different meanings or be used interchangeably.

Instead of having the notion of a DistinctHost which has a "latest"
that in reality is a "host", do away with the potential confusion and
name it LatestHost with a "host" reference instead of "latest".

The index and host_index templates relied on pagination data to provide
context in the page titles. These turned out empty when doing static
generation since there is no pagination there.

This adds a new index page for hosts, analogous to the index page for playbooks.
Instead of browsing and searching through playbooks, it is now possible to do it
for hosts.

By default, it will list only the latest report for each host (made possible by
the recent addition of /api/v1/distincthosts) and includes a toggle to list all
reports for every host.

The page title was always "ARA Records Ansible" which made it harder to
distinguish particular pages across multiple tabs.

The new titles provide a bit of context and provide summary information.
Uncapitalizing ARA because it's another recursive acronym and the
current logo doesn't capitalize the "ara".

ara did not previously record whether or not a task had been delegated
and if so, to which host. Now it does.

This adds a field for it in the result model so we can display it when
browsing a result's details.

From an API perspective, it also adds support for searching results that
may have been delegated to a particular host ID like so:
    /api/v1/results?delegated_to=<id>

This ensures that ara records tasks and results properly, even when
using the free strategy.

When running a playbook with free strategy, multiple hosts could be running
different playbook tasks at the same time.

This caused ARA to associate results to the wrong task id, resuting in task
names not matching the task result in the ARA web interface.

Now, the task is cached by it's uuid. The task result includes the task uuid, so
this is used to ensure the correct task id is associate with a particular result.

This allows to search a task by play id such as:
     /api/v1/tasks?play=<id>

This was always meant to be possible but was apparently forgotten.

Docs were failing to build on sphinx>=4 due to the pygments lexer that
we no longer use for the repository since roles have been moved out to
a collection.

Remove it and, if need be, consider using the following in the future:
https://github.com/ansible-community/ansible-pygments

Most models refer back to base and duration which made the graph a
little noisy when including inheritance arrows.

This is an addition to help flesh out the docs about relationships
between objects and a quick win since it works out of the box with
django_extensions' graph_models.

Since the database model doesn't move very much, it's rendered a single
time manually (instead of, say, automatically after every commit) and
there are instructions included to generate it again, if need be, in order
to avoid adding django_extensions as a dependency.

api: Add a new distincthosts endpoint

Hosts are unique per-playbook in ara: a host with the same name across two playbooks would result in two hosts under /api/v1/hosts.

This adds a new API endpoint at /api/v1/distincthosts that provides the latest playbook result for each host name.
Under the hood, it implements the machinery for updating the latest host every time a host is created or deleted and includes a SQL migration to initially populate the distincthost table with the latest hosts.

Co-authored-by: Christoph Hille <hille721@gmail.com>
Co-authored-by: David Moreau Simard <moi@dmsimard.com>

readme: update irc from freenode to libera.chat

Reviewed-by: https://github.com/apps/ansible-zuul

See: https://github.com/ansible-community/ara/issues/277

docs: First iteration of troubleshooting docs

Reviewed-by: https://github.com/apps/ansible-zuul

Some issues are frequent enough to warrant documenting so we can
refer users to them.

ui: Fix truncatepath calculations

Reviewed-by: https://github.com/apps/ansible-zuul