The standalone job changes boot_option in runtime, so local boot
can be used even when the default boot option is netboot.

Change-Id: Ia538907f3662e8cd84d988ea5d862c7f488558e1
(cherry picked from commit 7ac48041)

The left/right single quotation marks cause an ascii decode error in
some python3.6 environments.

Change-Id: I80d1b3e7f9e2e23ddaca640714a342b63ae3a5fa
(cherry picked from commit b8ef35a9)

Cirros partition images are not compatible with local boot since they
don't ship grub (nor a normal root partition). This change adds a script
that builds a partition image with UEFI artifacts present. It still
cannot be booted in legacy mode, but it's a progress.

Set the tempest plugin's partition_netboot option. We need it to inform
the tempest plugin about the ability to do local boot. This option
already exists but is never set.

Also set the new default_boot_option parameter, which will be introduced
and used in Iaba563a2ecbca029889bc6894b2a7f0754d27b88.

Remove netboot from most of the UEFI jobs.

Conflicts:
	devstack/files/bindep.txt
	zuul.d/ironic-jobs.yaml

Change-Id: I15189e7f5928126c6b336b1416ce6408a4950062
(cherry picked from commit bbceca56)

This is a partial cherry-pick of 720b42d5,
we don't need to disable the standalone job since it's not broken here.

On Victoria we also remove the Grenade job because Ussuri is EM now.

Change-Id: I8bd051ea709d328cb5efa2c2cbd5a226bdb4cfd3
(cherry picked from commit 720b42d5)

On neutron routed provider networks IP allocation is
deferred until 'binding:host_id' is set. When ironic
creates neutron ports it first creates the port, then
updates the port setting binding information.

When using IPv6 networking ironic adds additional address
allocations to ensure network chain-booting will succeed.
When address allocation is deferred on port create ironic
cannot detect that IPv6 is used and does not add the
required additional addresses.

This change ensures the 'port' object is updated after the
port update setting the port binding required for neutron
to allocate the address. This allows ironic to correctly
detect IPv6 is used, and it will add the required IP
address allocations.

Story: 2009773
Task: 44254
Change-Id: I863dd4ab9615a9ce3b3dcb8798af674ac9966bf2
(cherry picked from commit 3404dc91)

We're seeing OOM events in CI, hopefully this helps.

Change-Id: Id8c0e4830011ca2fa526df461ed5b9b01f769cf9

Some transient errors can ultimately cause the client to need to be
completely restarted due to cached connection data.

Ironic now explicitly removes the cache entry when a sushy
AccessError or python AttributeError is detected originating
from the library. This will now result in the prior cached connection
object to be discarded, and upon the next attempt to interact with
the same node, a new connection will be launched.

This will result in new sessions being created, but in all likelihood
the prior session had already timed out or had been administratively
removed. Sushy's code, as of
https://review.opendev.org/c/openstack/sushy/+/820076
will raise SessionService lookup access errors as AccessErrors.
Prior to that change, they should have been raised as AttributeError
as the previous call sould have returned None to be used as an object.

*Also* Includes follow-up change Ia59f774c9340e3a6fa63418afedf12098c709052
squashed into this change, and necessary backport friendly mock of
AccessError which had changed since this release of ironic.

Change-Id: Icc6e5dd74d9f15e679a7e764fe49238ed6b8dc1e
Story: 2009719
Task: 44107
(cherry picked from commit 1439af27)
(cherry picked from commit 01997c8418b9e6ade47437b09dd9412310b90eac)
(cherry picked from commit e3e7deaf48a5315fe10350e8a87b2fcc4e189406)
(cherry picked from commit ab5495eecb409b37ad853a679032d6cdb4fefc72)
(cherry picked from commit e57aa7598b220bb2d1fd8fcc0aab6132926d5cce)

As with WS-Man iDRAC API setting boot device requires creating BIOS
job and there can be only 1 open job per subsystem present in iDRAC,
there is validation to check that the job queue is empty before
continuing setting boot device. This does not work well for cases when
using autoupdatescheduler that creates `Repository Update` job that
stays Scheduled until executed and then followed by new Scheduled
`Repository Update` job.

This patch allows non-BIOS jobs to be present in the queue when setting
boot device. This will still fail for cases when there are BIOS jobs
present. In such cases should consider moving to idrac-redfish that
does not create BIOS or any other job to set boot device.

Story: 2009251
Task: 43437
Change-Id: I91e9ba3024a85897aeead21cede57464294b409b
(cherry picked from commit b1d08ae8)

set_power_state has returned to the caller immediately without
confirming the system has reached the requested state. This fixes that
by synchronously waiting until the target state has been read before
returning.

That bug can cause instance workload deployments to fail on Dell EMC
PowerEdge server models on which IPA ramdisk soft power off fails and
ironic employs its OOB fallback strategy. After an otherwise successful
deployment, the node is active, but is powered off. No error is reported
in last_error. If the subsequent instance workflow expects the system to
be powered on into the operating system, it fails.

Story: 2009204
Task: 43261
Change-Id: I3112a22149c07e5508f26c79f33d09aeb905c308
(cherry picked from commit 2a0fd1d1)

Also fix the documentation to use the correct paths and versions.

Change-Id: I7f004d40c1b8c617f9a456216df091e44d69693f
(cherry picked from commit 294046be)

All the tox jobs are based on openstack-tox, we should convert
ironic-tox-unit-with-driver-libs too.

Change-Id: I20836d586edccfb8cd8fed1f3a89f1497ff96943
(cherry picked from commit 475af371)

Actually, it's only available on RHEL/CentOS 8, but I hope the other
distributions will catch up.

Change-Id: I53314b8f16fd7b965c58370e33ab83501e7cb067
(cherry picked from commit 3199d289)

In order to avoid potential cache coherency issues
when using a globally cached AgentClient, e.g. with
TSL certificates from the IPA, cache the AgentClient
on a per task basis.

Co-Authored-By: Arne Wiebalck <arne.wiebalck@cern.ch>

Conflicts:
	ironic/drivers/modules/agent.py
	ironic/drivers/modules/agent_base.py
	ironic/drivers/modules/ansible/deploy.py
	ironic/drivers/modules/iscsi_deploy.py
	ironic/tests/unit/drivers/modules/test_agent.py

Story: #2009004
Task: #42678

Change-Id: I0c458c8d9ae673181beb6d85c2ee68235ccef239
(cherry picked from commit fcb6a109)

Fixes an issue of powering off with the ``idrac-wsman`` management
interface while the execution of a clear job queue cleaning step is
proceeding.
Prior to this fix, the clean step would fail when powering off a node.

Story: 2008988
Task: 42641

Change-Id: Ib4ab755c806f028d97379b80a8c27d6ade63cba1
(cherry picked from commit 741a4d8a)

The instance_uuid handling on the detailed node information
endpoint of the api (/v1/nodes/detail?instance_uuid=<uuid>),
which is used by services such as Nova for explicit node status
lookups, previously had special conditional logic surrounding it
which skipped the inclusion of the API requestor project-id, from
being incorporated into the database query.

Ultimately, this allowed an authenticated user to obtain a partially
redacted node entry where sensitive informational fields were scrubbed
from the response payload.

With this fix, queries for an explicit instance_uuid now follow the
standard path inside the Ironic API to the database which includes
inclusion of a requestor Project-ID if required by configured policy.

Change-Id: I9bfa5a54e02c8a1e9c8cad6b9acdbad6ab62bef3
Story: 2008976
Task: 42620
(cherry picked from commit be3c153d)

- Re-usable helper created to avoid duplication.
- Although there is only one manager for system in known iDRAC systems
still iterate through collection for future changes.
- Restructured exception raising and error logging for better feedback.
- Removed some unit tests to avoid duplication that is covered by
method specific unit tests

Change-Id: I03fdb48e47c9557c207a20ee876eccf3f3459d9f
(cherry picked from commit 39cd751a)

The branch was never updated during the release and we have some jobs
(and developers) which could pull in master branch IPA as a result
in testing and whatnot. In order to prevent this possible case and
confusion, the branch should be set on the job.

Change-Id: I8034bff4068463fe9b28fe721cdb9bb58367728e

Yes, project conundrum is a code-name for our transition to OFTC
as we do not want to put anything into freenode IRC which may abruptly
result in the channel being siezed or shutdown by the new
owners/operators of freenode

Change-Id: I45c07e0b2138f6643f865d58155c64317114fd02
See: http://lists.openstack.org/pipermail/openstack-discuss/2021-May/022718.html
(cherry picked from commit d5971fdf)

The openstack Ussuri and Victoria versions no longer support the
Centos7 and pyrhon2 environment packages. Correct the missing
problems in the latest document

Change-Id: I60787243fdc6ed2741522355ec79970bdb912f41
(cherry picked from commit 35dea078)
(cherry picked from commit 77be4c6c)

Since ironic-python-agent-bulder has stable branches starting from
wallaby, we need to point all the older branches to stable/wallaby,
unless they're already pinned to an older version.

Change-Id: I90a0d4d75fb4581805f11e79ca7185cfdb66f77a

If the agent accepts a command, but is unable to reply to Ironic (which
sporadically happens before of the eventlet's TLS implementation), we
currently retry the request and fail because the command is already
executing. Ironic now detects this situation by checking the list of
executing commands after receiving a connection error. If the requested
command is last one, we assume that the command request succeeded.

Ideally, we should pass a request ID to IPA and then compare it. Such
a change would affect the API contract between the agent and Ironic
and thus would not be backportable.

Change-Id: I2ea21c9ec440fa7ddf8578cf7b34d6d0ebbb5dc8
(cherry picked from commit abfe383c)

InvalidImageRef is a kind of InvalidParameterValue and can happen during
validation, causing a traceback now.

Change-Id: I5f10fe7240e74d337f991bbd1a5220cc4e713de7
(cherry picked from commit 47398edd)

Also make sure the pregenerated flag is always reset.

Change-Id: I73aaa803d3eb84ddac59a778e998836a645217eb
(cherry picked from commit c6e8281f)

Some higher versions of grub2 (e.g. 2.05 or 2.06-rc1) use
grub.cfg-01-MAC, while another lower versions of
grub2 (e.g. 2.04) use MAC.conf, so we generate both paths
in order to be compatible with both.

For more information we can see:
https://docs.oracle.com/en/operating-systems/oracle-linux/7/install/ol7-install-prepare.html#ol7-install-pxe-boot-uefi



Change-Id: Icbdd284de38b8e54c52cdbba709bba0e644c35cd
Signed-off-by: Armstrong Liu <vpbvmw651078@gmail.com>
(cherry picked from commit 8f474bfe)

Calculating the ipmitool `-N` and `-R` arguments from ironic.conf
[ipmi] `command_retry_timeout` and `min_command_interval` now takes
into account the 1 second interval increment that ipmitool adds on
each retry event.

Failure-path ipmitool run duration will now be just less than
`command_retry_timeout` instead of much longer.

Change-Id: Ia3d8d85497651290c62341ac121e2aa438b4ac50
(cherry picked from commit 1de3db3b)

Instead of using process_event('fail') use error_handlers,
otherwise in case of failure node gets stuck and fails
because of timeout, instead of failing earlier due to
step failure.

And improve coverage to test this error handling
and also happy paths.

Story: 2008307
Task: 41197
Change-Id: I1e957c2b526abc37920212b6431b11eedc9f89be
(cherry picked from commit 83ce7c42)

The fix for https://storyboard.openstack.org/#!/story/2008252 synced
the boot mode after changing the boot device, because Supermicro nodes
reset the boot mode if not included in the boot device set. However this
can cause a problem on Dell nodes when changing the mode uefi->bios or
bios->uefi. Restrict the syncing of the boot mode to Supermicro.

Story: 2008712
Task: 42046
Change-Id: I9f305cb3f33766c1c93cf4347368b1ce025fc635
(cherry picked from commit 8bd25a98)

Currently if the baremetal boot mode is unknown and the driver doesn't
support setting the boot mode then the error is logged and deployment
continues.

However if the BMC doesn't support getting or setting the boot mode
then setting the boot mode raises an error which results in the deploy
failing. This is the case for HPE Gen9 baremetal, which doesn't have a
'BootSourceOverrideMode' attribute in its system Boot field, and
raises a 400 iLO.2.14.UnsupportedOperation in response to setting the
boot mode.

This is raised from set_boot_mode as a RedfishError. This change
raises UnsupportedDriverExtension exception when the 'mode' attribute
is missing from the 'boot' field, allowing the deployment to continue.

Change-Id: I360ff8180be252de21f5fcd2208947087e332a39
(cherry picked from commit 9f221a7d)

Tinycore 12 requires some more RAM than its predecessor.

Change-Id: Ibced843f34c78af7780bfe1ade833208b458bb8b
(cherry picked from commit 807a6d2b)

In order to reduce the load on the database backend, only lazy-load
a node's ports, portgroups, volume_connectors, and volume_targets.
With the power-sync as the main user, this change should reduce the
number of DB operations by two thirds roughly.

Change-Id: Id9a9a53156f7fd866d93569347a81e27c6f0673c
(cherry picked from commit 82cab603)

Restore test symmetry.

Change-Id: I54a9fed73e366a30545c3cd1982588d2f544d228
(cherry picked from commit 61c5b3fd)