The idrac hardware type requires python-dracclient versions greater than
or equal to 1.3.0 and less than 2.0.0.

This is not a backport of a similar change which merged on the master
and stable/stein branches [1]. That change specifies python-dracclient
versions greater than or equal to 1.5.0 and less than 3.0.0 are
required.

Subsequently, a change which merged only on master modified the bounds
to greater than or equal to 3.0.0 and less than 4.0.0 [2].

[1] https://review.opendev.org/#/q/I5f6d6be53aec9d72e8ba5cfa436f12e01966f871
[2] https://review.opendev.org/#/c/634903/

Conflicts:
	doc/source/admin/drivers/idrac.rst
	driver-requirements.txt

Change-Id: If5b0c5b69f0f8759136051072c1ff53c6d86217d
(cherry picked from commit f78ccef3)

This is a combination of 2 commits to solve the dependency issue, which
came up after the announced removal of global bindep-fallback.txt

1. Add bindep.txt file

This is used by `bindep` [0] to assess if needed system packages are
installed.
It should allow easier installation of Python packages on a clean
system from source and help packagers too.
For now it is just a copy/merge of `devstack/files/[debs|rpms]/ironic`
files with added platform specifiers.

Packages have been tagged with certain bindep profiles:
- default - needed to run ironic service in production
- test - needed to compile Python dependencies and for other tox targets,
  profile of this name is used by various gate jobs
- devstack - needed to setup VMs posing as HW nodes by the devstack
  plugin
- imagebuild - only needed to build various types of deploy ramdisk

Note that DevStack install procedure currently does not use 'bindep',
and still relies on files in `devstack/files/...` folder.

[0] http://docs.openstack.org/infra/bindep/index.html

(cherry picked from commit b2fcf480)

2. Update bindep.txt for doc builds

https://governance.openstack.org/tc/reference/project-testing-interface.html#documentation
specifies that doc dependencies need the doc tag in bindep, use it.

Leave test environment in for transition period until we use the new
scripts.

(cherry picked from commit 6a48eba4)

Change-Id: I42239e0ec960507b7345866d768b3ee06738799b

Following recent changes to global requirements [1], we need
to update sphinx requirements locally.

[1] If558f184c959e4b63b56dec3ca1571d1034cfe5c

(cherry picked from commit 9ed91013)

Change-Id: Ie806a891da6b28fe52f4b72bcd2e3609d4e95db8

Cirros can't mount a ephemeral partition if its enabled.

Change-Id: I7b56fcd0be1cd48a6b72a0c156d37b3f279a8da9
Story: #2004604
Task: #28499
(cherry picked from commit 71f0256b)

When installing devstack, found there is
a missing node id in devstack/lib/ironic
when root devices are less than 4Gb.

Change-Id: Ie101d1e312bd7820294fad8b317fc62faa8769e1
Story: #1730526
Task: #9773
(cherry picked from commit 4b6fe666)

Pyghmi was moved from openstack namespace to the x
namespace. As such the devstack plugin needs to match
up otherwise devstack fails in CI to clone as cloning
is blocked if not in the projects list... Which it is,
as x/pyghmi.

Change-Id: I87bdf65dd2dac189642d0af6def2d0ed155f3c00
(cherry picked from commit bb8101ca)

This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.

This fix is inconsistent with the stable branch policy [1]. It requires
python-dracclient version 1.5.0 and greater; however,
driver-requirements.txt specifies version 1.3.0 and greater can be used
on this branch.

A new fix which does not require a version bump is being developed.

This reverts commit bef4af3c.

[1] https://docs.openstack.org/project-team-guide/stable-branches.html

Change-Id: I63e248f804e4dfd9cf2e7c7cebc694ab633e3b40
Story: 2004340
Task: 27927

This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: Ia85c510b576045b87af25d0d3122d02b36bec5af

If list_cpus in dracclient, it returns below values -

[CPU(id='CPU.Socket.1', cores=14, ht__enabled=True),
CPU(id='CPU.Socket.2', cores=14, ht_enabled=True)]

Each CPU socket has cores with hyperthreading enabled, so if we
calculate the cpu per socket, we will get actual cpu count per socket,
So objective of this patch to fetch actual CPU's per socket and sum it

Story: #2004155
Change-Id: Ia86b0462a3e73af7a2ce0a6439286f7071f74caa
(cherry picked from commit 3e1fe5f7)

Baremetal nodes fail to pxe boot during OpenStack deployment.
Reason of this failure is because we are not checking the
actual pxe device interface and updating the port pxe_enabled field.

This patch make sure that everytime when port is created, it checks the
the BIOS and UEFI boot modes of node and fetch the pxe device interfaces
values matching to nic id and then update the port's pxe_enabled field.

Change-Id: I2890bf16110b713e269d4a4fe410f57273dc8e83
Story: 2004340
(cherry picked from commit 2b74d940)

We document that this can be disabled by setting the
instance_master_path config  to "<None>",
but don't actually support it in code. oslo.config doesn't actually
translate that value to the Python None as we expect.

Allow disabling the cache by setting the config to the empty string, as
in "instance_master_path=". This doesn't make sense as a directory to
use as a cache anyway, so it shouldn't break anyone.

Conflicts:
	ironic/drivers/modules/deploy_utils.py
Due to class InstanceImageCache being in iscsi_deploy.py instead of
deploy_utils.py.

Change-Id: I1bb62d55e3e18272fd5da355d63fd2c11a033acd
Story: 2004279
Task: 27829
(cherry picked from commit 0498e298)

The multinode job is failing again in stable branches pike
and queens. In queens the job is already non-voting, marking
it non-voting in pike as well until we are able to discover
the root cause and have a fix.
This will allow contributors to keep working on the pike branch
until we resolve the job issues.

Change-Id: I2f72b042996e3667315f5b79c132e1cb404b2639

When a timeout occurs when a node is in CLEANWAIT state, the conductor
puts it into the CLEANFAIL state. However, it tries to do that twice, and
our state machine doesn't support moving from a CLEANFAIL state to another
state via the 'fail' verb/event.

The code was changed so that it doesn't try to move it to CLEANFAIL twice,
and a check is put to prevent the node from being 'failed' frome a CLEANFAIL
state.

Change-Id: Ieeb77dd28a5d3053588c46fe2a700b5e6ceabbd7
Story: 2004299
Task: 27855
(cherry picked from commit 78ae60f1)

This template is only used on older branches and should not have been
imported, remove it again. It should stay in project-config.

Change-Id: Icdaa0f7102074be97e73d4361387084ec4bf5ae1
(cherry picked from commit 816ff1dd)

This is a mechanically generated patch to complete step 1 of moving
the zuul job settings out of project-config and into each project
repository.

Because there will be a separate patch on each branch, the branch
specifiers for branch-specific jobs have been removed.

Because this patch is generated by a script, there may be some
cosmetic changes to the layout of the YAML file(s) as the contents are
normalized.

See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html

Change-Id: Idc4c6057b7403889c1461123a328868458bf3b24
Story: #2002586
Task: #24302

This fixes an issue that caused the integrated Dell Remote Access
Controller (iDRAC) management hardware interface implementation,
'idrac', to fail to boot nodes in Unified Extensible Firmware Interface
(UEFI) boot mode. That interface is supported by the 'idrac' hardware
type. The following errors appear in the ironic conductor log:

ERROR ironic.drivers.modules.agent_base_vendor   File
"/opt/stack/new/ironic/ironic/drivers/modules/drac/management.py", line
106, in set_boot_device
ERROR ironic.drivers.modules.agent_base_vendor     in
drac_boot_devices[PERSISTENT_BOOT_MODE]
ERROR ironic.drivers.modules.agent_base_vendor KeyError: 'IPL'
ERROR ironic.drivers.modules.agent_base_vendor
ERROR ironic.conductor.utils [None
req-80b5a61c-4ae1-41ea-875f-5a722142be01 None None] Error rebooting node
cddff129-7353-4de1-b436-810b1ae0dd92 after deploy. KeyError: 'IPL'
DEBUG ironic.common.states [None
req-80b5a61c-4ae1-41ea-875f-5a722142be01 None None] Exiting old state
'deploying' in response to event 'fail' {{(pid=23850) on_exit
/opt/stack/new/ironic/ironic/common/states.py:263}}
DEBUG ironic.common.states [None
req-80b5a61c-4ae1-41ea-875f-5a722142be01 None None] Entering new state
'deploy failed' in response to event 'fail' {{(pid=23850) on_enter
/opt/stack/new/ironic/ironic/common/states.py:269}}

The issue is resolved for Dell EMC PowerEdge 13th and 14th generation
servers. It is not resolved for PowerEdge 12th generation and earlier
servers.

Prior to this change, the set_boot_device() function in
ironic.drivers.modules.drac.management could successfully set a boot
device that would persist during future boots only when the server's
boot mode was configured to be BIOS. The root cause was that function
understood only a single persistent boot mode, BIOS, and it was hard
coded.

PERSISTENT_BOOT_MODE = 'IPL'

Initial Program Load (IPL) is a synonym for legacy BIOS boot mode.

The iDRAC does not report nor make visible the boot source lists for
persistent boot modes that are not configured as the server's current
boot mode. Only the list associated with the configured boot mode is
reported and can be modified. So, when the boot mode is configured to be
UEFI, the iDRAC permits only its boot source list to be modified.
However, set_boot_device() had been capable of only modifying the boot
source list for BIOS (IPL), which was not reported. Its attempt to
access the wrong, unreported boot source list failed.

This change fixes the issue by determining the configured persistent
boot mode at run-time and modifying its boot source list. The hard
coding of a persistent boot mode has been eliminated. Now, nodes managed
by the iDRAC hardware type successfully boot in either persistent boot
mode, BIOS or UEFI. More information about using the iDRAC for boot
management is available in the "Dell EMC BIOS and Boot Management
Profile", Version 4.0.0, section 7.2 Boot Management, pp. 44-47 [1].

The way the iDRAC hardware type deploys the operating system, by copying
it to a disk, posed an additional challenge to this fix. UEFI discovers
new boot sources during the next boot. How could the server be
configured to directly boot into the newly deployed operating system
when its boot source is unknown? It was overcome by using new iDRAC BIOS
attributes named 'SetBootOrderFqddNN' [2]. Those make it possible to
specify the boot source for the next boot without requiring it to be
present in the current boot's boot source list. The server is configured
to directly boot from the disk on which the operating system was just
deployed. An additional reboot is not required. Note that those new
attributes are presently available on only 13th and 14th generation
servers. Also, this approach is only possible for configuring a
persistent boot source.

[1] http://en.community.dell.com/techcenter/extras/m/white_papers/20444495
[2] "Configuring server boot options on 14th generation Dell EMC
PowerEdge servers", section 3.5.2 SetBootOrderFqddNN, pp. 13-14
(http://en.community.dell.com/techcenter/extras/m/white_papers/20487489)

Conflicts:
	ironic/tests/unit/drivers/modules/drac/test_management.py

Story: 1656841
Task: 9711
Change-Id: Idad1db1807420eab62c44318afafe10ea8c52ee5
(cherry picked from commit eb14f575)

Fixes an issue seen during node tear down where a port being deleted by
the Bare Metal service could be deleted by the Compute service, leading
to an unhandled error from the Networking service. This change simply
ignores the PortNotFoundClient exception raises, as is done when
attempting to unbind a port that does not exist. See [1] for details.

[1] https://storyboard.openstack.org/#!/story/2002637>

Change-Id: Icd2aaa1276e75f08f99553bbc9a873102d896fb9
Story: 2002637
Task: 22285
(cherry picked from commit bfed31bb)

Before this patch, when unprovisioning a node, the console was left
running. This allowed a user to view the console even after the instance
was gone. Stop the console during unprovisioning to block this.

ConductorManager._set_console_mode will now bubble up any exceptions
raised, so that we can explode as needed during unprovisioning. This
does not have any side effects, as elsewhere it is run in it's own
thread and execution was complete after handling the exception.

Also change a few mock.ANY in the relevant unit tests to the actual task
object, as cleanup.

Conflicts:
	ironic/tests/unit/conductor/test_manager.py

Change-Id: Iec128444d692e0b0fbc1737eb21b0e6f69b7ec1e
Partial-Bug: #1769817
Story: #2002000
Task: #19634
(cherry picked from commit 7a8b26db)

When configdrive is too large, a node object cannot be saved to DB. If
it happens, the node cannot moved to DEPLOYFAIL because saving the node
is prevented again by the large configdrive in the object. In this
case, the node gets stuck in DEPLOYING, which doesn't allow any state
transition.

This patch removes the configdrive from a node object when storing the
configdrive fails. This also catches ConfigInvalid exception, which is
mentioned in the docsting, and any unexpected exception from
_store_configdrive() to avoid getting a node stuck in DEPLOYING.

(cherry picked from commit 927c487a)
Change-Id: I83cf3e02622fc3ed8f5b5389f533e374c1b985f3
Closes-Bug: 1745630

Rather than pinning pycodestyle, ignore the E402 and W503 error. Fix
issue with E731 (Do not assign a lambda expression, use a def).

W503 is something we will likely never enable as it is a personal
style decision and can change depending on the code. There is no one
right answer. Interestingly there is also a W504 which is the opposite
check.

E402 is one we should most likely fix. But it can be done in a follow
patch or patches.

List of various error codes:
https://pycodestyle.readthedocs.io/en/latest/intro.html#error-codes

Change-Id: Ie2c31a8a3c75beeef22453cab5878cf0094bdf3f
(cherry picked from commit ca91d4d8)

2.4.0 broke our pep8 tests pretty hard:
http://logs.openstack.org/20/531320/14/check/openstack-tox-pep8/c9d9d10/job-output.txt.gz#_2018-04-10_15_43_40_014738

Pin pycodestyle until we have time to fix all of those.

Note that this is in the global-requirements blacklist to allow
projects to manage those versions on their own:
https://github.com/openstack/requirements/blob/b94a4e8c8fadbdc4c5cac9c4da6755a3c06a7824/blacklist.txt#L13

Conflicts:
    test-requirements.txt

Change-Id: I6b0d458c702684779eb21e7022dd0f3193ac72d2
Story: 2001819
(cherry picked from commit fc15be6e)

We don't specify any regex for this job, so it runs some tests.
They only include test_network_basic_ops, which does not check that
multitenancy is in effect. We have a test for it in ironic-tempest-plugin,
but it does not run anywhere. There is a neutron test for the same,
but it's not a smoke test and it has been skipped for quite some time.

This change switches this job to the run the following tests:
* all ironic tests (that are not skipped for any reason)
* multinode tests from Nova

Unfortunately, this patch uncovers that we don't run any tests for
network attach/detach in practice. The devstack plugin has to be updated
first to support more than one port per node. A TODO is left in the playbook.

Change-Id: I46d94c29f858977604c0a3430257eca2bee977e3
(cherry picked from commit 5f03daf2)

Swiftclient uses public endpoint by default. Ironic uses the base URL
from Swift connection to build TempURLs for generated images.
Some drivers (e.g. iLO) use those TempURLs to mount images as vmedia.
With public URLs it will fail if the BMC doesn't have access to the
public network.

This change introduces an option to explicitly set the endpoint URL
used for Swift.

This is a stable-only change as the problem is fixed by refactoring
changes in later releases. This is the only version where Ironic uses
public Swift endpoints and there is no option to override this.

Change-Id: I639a421fa06fff7ab07b8eab557531b8f36c5ed9
Closes-Bug: #1755164
Related-Bug: #1699547

There is difference between dhcp option names in different backends.
This patch changes options name to code according to [0].

[0] https://www.iana.org/assignments/bootp-dhcp-parameters/bootp-dhcp-parameters.xhtml

Closes-Bug: 1717236

This is an updated version of c377f5cb
which problems with PXE and dnsmasq due to buggy dnsmasq code which uses
siaddr field to specify tftp server. They are addressed now by always
sending server-ip-address to make sure that dnsmasq works.

More information about siaddr and option 150,66 can be found in
informational RFC https://tools.ietf.org/html/rfc5859

Change-Id: I55487d867979bf6bb4cf228fcf6408beae955d2b
(cherry picked from commit 228a2a78)

Infiniband ports do not require the local_link_connection field to be
populated as the network topology is discoverable by the Infiniband
Subnet Manager.

This change removes the requirement for local_link_connection for
Infiniband ports. Infiniband ports have a client-id in their extra
field.

Closes-Bug: #1753222
Change-Id: I2bfac4ccaf825bd9aa8ea0d2b447fcd7767acbc5
(cherry picked from commit dcebb77d)

When a conductor managing a node dies abruptly mid cleaing, the
node will get stuck in the CLEANING state.

This also moves _start_service() before creating CLEANING nodes
in tests. Finally, it adds autospec to a few places where the tests
fail in a mysterious way otherwise.

Change-Id: Ia7bce4dff57569707de4fcf3002eac241a5aa85b
Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com>
Partial-Bug: #1651092
(cherry picked from commit 2921fe68)

These additions will allow us to filter nodes
by node uuid and id. This filter API is used
in many places throughout the code base. It is
natural to expect that this API would allow us to
filter by node id and uuid in addtion to the other
supported parameters. This also fixes a 3 year old bug.

This change from lucasagomes has a bug:
https://review.openstack.org/#/c/197141/

In conductor/manager.py, he calls _fail_if_in_state()
and uses the node_id as a filter. However this filter
effectively does nothing because sqalchemy does not
know how to filter by node id on the backend. My changes
fix this problem and make the API more intuitive

Closes-Bug: #1749755

Change-Id: I4efc0d5cd5d5d6108a334f954e1718203b47da0a
(cherry picked from commit 366a44a1)

Zuul no longer requires the project-name for in-repo configuration.
Omitting it makes forking or renaming projects easier.

Change-Id: I0778500de4479d3654a998639a4e3b1e69d2d5ae