ironic 18.2.1 release

meta:version: 18.2.1
meta:diff-start: -
meta:series: xena
meta:release-type: release
meta:pypi: no
meta:first: no
meta:release:Author: Riccardo Pittau <elfosardo@gmail.com>
meta:release:Commit: Riccardo Pittau <elfosardo@gmail.com>
meta:release:Change-Id: I2eb663d659d180566b1ad9e54844584d9c75fcb9
meta:release:Code-Review+1: Dmitry Tantsur <dtantsur@protonmail.com>
meta:release:Code-Review+2: Elod Illes <elod.illes@est.tech>
meta:release:Code-Review+2: Thierry Carrez <thierry@openstack.org>
meta:release:Workflow+1: Thierry Carrez <thierry@openstack.org>

Since the default value resource_url is None, make sure the
parameter is set to 'ports' when getting all ports.

Change-Id: Id603ae5a4a802dfc8f866b15c8d327d95eba9310
(cherry picked from commit 2ac740e0)

Since the default value resource_url is None, make sure the
parameter is set to 'nodes' when getting all nodes.

Change-Id: I6cc52eb56c7888a433d24aa79154143d6f35cf83
(cherry picked from commit 69227c66)

Some adapters may have addresses that are not MAC. See for example:
https://github.com/metal3-io/ironic-image/issues/314

Conflicts:
	ironic/drivers/modules/inspect_utils.py
	ironic/tests/unit/drivers/modules/test_inspect_utils.py

Change-Id: I0023e0750e372185747ca28cddd2a8dda110dd7f
(cherry picked from commit 9308c0a0)

Story: 2009772
Task: 44249

Change-Id: I8e559b3c7e833c361e12d01d744510ac5c8d8cf6
(cherry picked from commit b824ea7f)

On neutron routed provider networks IP allocation is
deferred until 'binding:host_id' is set. When ironic
creates neutron ports it first creates the port, then
updates the port setting binding information.

When using IPv6 networking ironic adds additional address
allocations to ensure network chain-booting will succeed.
When address allocation is deferred on port create ironic
cannot detect that IPv6 is used and does not add the
required additional addresses.

This change ensures the 'port' object is updated after the
port update setting the port binding required for neutron
to allocate the address. This allows ironic to correctly
detect IPv6 is used, and it will add the required IP
address allocations.

Story: 2009773
Task: 44254
Change-Id: I863dd4ab9615a9ce3b3dcb8798af674ac9966bf2
(cherry picked from commit 3404dc91)

Currently duplicate DHCP options are added when a port have
multiple address in the same address familiy. The duplicate
entries cause an internal server error in neutron.

This fixes the issue by only adding DHCP options for each
IP-version.

Story: 2009774
Task: 44255
Change-Id: I8bef819dc3101f88f4e04ea986e1039e34d917ed
(cherry picked from commit c4721da2)

    Xena. It's ok because it's very likely only unit test brokeness, and
    Ironic is still working:
    - test_show_makes_datetimes
    - test__commit_raid_config_with_logical_drives

`TaskMonitor.task_monitor` is deprecated and was removed from sushy
at Ie48fab19199c0c2f3455d2212d50460a29685eed. Somehow missed to
update it for `redfish` firmware upgrade steps previously.

Change-Id: Iced8e921335995e1e084d6e632c6e81d3fc9249f
(cherry picked from commit 9076f185)

We validate the boot interface during adoption because of:
a) potential rebuilding,
b) non-local boot.

Rebuild proved a rarely used feature, and local boot is the default
nowadays, so it makes less sense to unconditionally validate the boot
interface during adoption. We will run the validation anyway the next
time we need to do something with booting.

Similarly, do not record is_whole_disk_image None if it cannot be
reliably determined.

Conflicts:
	ironic/conductor/manager.py

Change-Id: I95252aea808c48ea2d94569449c871f0d483caaa
(cherry picked from commit d342b07d)

We're seeing OOM events in CI, hopefully this helps.

Change-Id: Id8c0e4830011ca2fa526df461ed5b9b01f769cf9

Some transient errors can ultimately cause the client to need to be
completely restarted due to cached connection data.

Ironic now explicitly removes the cache entry when a sushy
AccessError or python AttributeError is detected originating
from the library. This will now result in the prior cached connection
object to be discarded, and upon the next attempt to interact with
the same node, a new connection will be launched.

This will result in new sessions being created, but in all likelihood
the prior session had already timed out or had been administratively
removed. Sushy's code, as of
https://review.opendev.org/c/openstack/sushy/+/820076
will raise SessionService lookup access errors as AccessErrors.
Prior to that change, they should have been raised as AttributeError
as the previous call sould have returned None to be used as an object.

*Also* Includes follow-up change Ia59f774c9340e3a6fa63418afedf12098c709052
squashed into this change, and necessary backport friendly mock of
AccessError which had changed since this release of ironic.

Change-Id: Icc6e5dd74d9f15e679a7e764fe49238ed6b8dc1e
Story: 2009719
Task: 44107
(cherry picked from commit 1439af27)
(cherry picked from commit 01997c8418b9e6ade47437b09dd9412310b90eac)
(cherry picked from commit e3e7deaf48a5315fe10350e8a87b2fcc4e189406)

CI is memory intensive, and we realistically don't need 2 or
more API workers running for every single WSGI process which
does not implement it's own specific override value.

This should reduce the memory footprint by an average of six processes
which consume 60-90 MB each.

Change-Id: Ia0a986152c2b9fc9c5ff54cf698a351db452fbbd

Sushy 4.0.0 has breaking changes in it which prevent usage
with this version of ironic. While the global upper requirements
is unlikely ever to attempt to increment to 4.0.0, the explicit
requirements invocation, like what is in place for running
driver unit tests, was trying to pull in 4.0.0 resulting in test
failures.

Change-Id: Ief828f139bac9ba00f3250d8ddf4fd3d84a6fa7a

Change-Id: I02a8ed6802fffee071e94be3c0cab2382b7e60ca
(cherry picked from commit 3e225d2c)

The NoValidDefaultForInterface exception is a little misleading
in that if one doesn't have the base interface enabled, and they
attempt to enable a hardware type which requires or only supports
disabled interfaces, they will also get an exeption. The reality
is we need to suggest for them to look at enabling the interfaces
before looking at the default interface overrides, because logically
the brain jumps to setting a default before checking the interface
settings.

Change-Id: I50d4381e11da96cb7ae0ee8cbda18534380bd471
(cherry picked from commit fdc6424d)

Change-Id: I42a314dd4d338b169fc5fb9be375ef004e153eac
(cherry picked from commit 2ddd8ef2)

There are a few places in unit tests where we rely on Redfish constants
having well-known values. Use constants themselves instead.

An even more problematic place is _set_boot_device: it relies on storing
Redfish constants in driver_internal_info. With the future switch to
enums, it will no longer be possible. Use Ironic constants and provide a
compatibility layer.

Change-Id: I8e075f808e013afd778dd7cdc0927fe339309b70
(cherry picked from commit 93fd147b)

In all of the copying/pasting/repeating, some misalignments between
labels/parameters utilized for the RBAC tests occured.

Nothing serious, everything seems to be just fine.

Change-Id: Ib0ded5a53e586e10006b066241f4b4a92bb2c463
(cherry picked from commit a249c025)

In some cases we handle the same exception twice in a row: in agent_base
and in deployments.do_next_deploy_step. This change avoids it.

Also make deploy step error messages more uniform across the board.

Change-Id: Ic84c04118b1a85b10a761fc58796827583a5b086
(cherry picked from commit 4f089380)

As with WS-Man iDRAC API setting boot device requires creating BIOS
job and there can be only 1 open job per subsystem present in iDRAC,
there is validation to check that the job queue is empty before
continuing setting boot device. This does not work well for cases when
using autoupdatescheduler that creates `Repository Update` job that
stays Scheduled until executed and then followed by new Scheduled
`Repository Update` job.

This patch allows non-BIOS jobs to be present in the queue when setting
boot device. This will still fail for cases when there are BIOS jobs
present. In such cases should consider moving to idrac-redfish that
does not create BIOS or any other job to set boot device.

Story: 2009251
Task: 43437
Change-Id: I91e9ba3024a85897aeead21cede57464294b409b
(cherry picked from commit b1d08ae8)

The kickstart template expects a dictionary with 'ks_options'
as the key. Instead build_kickstart_config_options function
returns a dict with keys 'liveimg_url', 'agent_token' and
'heartbeat_url'.

This change fixes this problem by returning a dictionary of
dict with 'ks_options' as key and the dictionary with
keys 'liveimg_url', 'agent_token' and heartbeat_url' as
value.

Fix a bug where the deploy() method of anaconda deploy
interface where it did not return states.DEPLOYWAIT instead
it returned 'None' which caused the instance to go straight to
'active' instead of 'wait call-back'.

Fix issues in the default kickstart template where heartbeat was
missing 'callback_url' parameter and the HTTP method should be
'POST' not 'PUT'.

Fix issues with automated cleaning when anaconda deploy interface
is used.

Anaconda deploy interface could not deploy tarballs as
the disk image sent to the anaconda interface via liveimg --url
kickstart command does not include any file extension. When
no file extension is present the kickstart command liveimg --url
assumes the disk is a mountable partiton image. We fix this
problem by enabling the user to specify file extensions using
a glance image property named 'disk_file_extension' on the OS
image.

Co-Authored-By: Ruby Loo <opensrloo@gmail.com>
Change-Id: I556f8c9efbc5ab0941513c3ecaa2aa3ca7f346ae

Currently Ironic always adds a filename parameter to virtual media
image URL, for example:
http://localhost/redfish/boot.iso?filename=file.iso. This is useful
on certain BMCs which require a ".iso" suffix in the image URL
especially when using swift as the image source.
This should not be neccessary with file based virtual media images
and adding this parameter can cause issues in BMCs which do not
accept special characters such as "=" and "?" in virtual image URL.
This change alters Ironic's behaviour to only add the filename
parameter if swift backing store is used for the image.

Story: 2009278
Task: 43547

Change-Id: I3338bdb6a24d9695ce32d205221183b9b876bcc8
(cherry picked from commit e05f74c6)

We currently use the last component, which is:
a) a potential information exposure,
b) can hit file name length limits [1]

Use the master path file name instead, which is always a UUID.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=2014630

Change-Id: I2aa1230468132c2b7a8585d691ec180947f89c1e
(cherry picked from commit 7d85694f)

Avoid resume clean before raid is actually configured.
After creating the RAID, set up polling will notify ironic to wait for the
RAID configuration to complete before proceeding to the next step instead of check IPA.

Story: #2009231
Task: #43336

Signed-off-by: Zhou Hao <zhouhao@fujitsu.com>
Change-Id: I7a551199224c24ca270b8b02d76b39386632c94f
(cherry picked from commit b617e434)

Fixed the bug of repeated resume cleaning due to the value of `fgi_status` not being updated correctly when obtaining the
RAID configuration status of the node managed by the `irmc` hardware type.

```
Unexpected error when processing next clean step. TypeError: 'NoneType' object is not subscriptable
```

This `NoneType` error occurs because ironic resumes clean without waiting for IPA to get the clean steps, and then it tries to read the
clean steps which should be a list but actually is none.
During auto clean, resume clean should be triggered by IPA, but in this case, it is triggered by a iRMC periodic task which checks
the progress of running raid config.
This error does not occur every time, sometime raid can be configured, auto clean can complete without error and the node can be deployed successfully.

Story: #2009206
Task: #43265

Signed-off-by: Zhou Hao <zhouhao@fujitsu.com>
Change-Id: I5a1c5708bdc1709e928f0faf7e18396e260dc551
(cherry picked from commit 8a5c672f)