In Zed cycle, we have dropped the python 3.6/3.7[1] testing
and its support. Moving the py36 job to py3 based so that they run
on latest supported python 3. Updating the python classifier also to reflect the same.

[1] https://governance.openstack.org/tc/reference/runtimes/zed.html

Change-Id: Ida582daf7db6fb3cedb3c7e37fc512d094b37c6e

There is a syntax error in the exception logging when attaching a
volume fails in attachment state manager. This patch corrects it
and adds a test to guard against similar changes in future.

Closes-Bug: #1970698

Change-Id: I43c407046a49bb37631113e2ea65d05450f9365d

When we try to simultaneously attach same volume multiple times
(multiattach), there are multiple attachments created, suppose
attachA and attachB. If attachA marks the volume "reserved" then
attachB can't proceed until the volume is in "in-use" or "available"
state. We retry until we reach any of these states for which we use
the retrying library.

The retrying library defaults to 1 second retry[1] (5 times) which causes
the original failure as the volume takes time to transition between
"reserved" -> "in-use" state. This patch corrects it by adding an
exponential retry time and max exponential retry i.e.

1: 2 ** 1 = 2 seconds
2: 2 ** 2 = 4 seconds
3: 2 ** 3 = 8 seconds
4: 2 ** 4 = 16 seconds (but max wait time is 10 seconds)
5: 2 ** 5 = 32 seconds (but max wait time is 10 seconds)

[1] https://github.com/rholder/retrying/blob/master/retrying.py#L84

Closes-Bug: #1969373

Change-Id: I0094b044085d7f92b07ea86236de3b6efd7d67ea

This patch adds test coverage of code paths like extend volume,
exception blocks, when image_size is zero, volume delete call etc

Change-Id: I59b1579dc9877668b82d4195431c14cc41cfe892

This patch adds coverage for the various exceptions handled and
raised in get, get_size and delete methods.
It also corrects the behavior of _test_cinder_get_size method where
client.volumes returned dictionary instead of a MagicMock and it
worked due to the existing method "get" in dictionary which has
same name as "get" method in cinderclient. The dictionary object
is replaced with the appropriate MagicMock object in this patch.

Change-Id: If63a6c810b5aab992e54857bc81f5052c2c593c4

Add coverage for helper methods like ``get_hash_str``,
``_get_mount_path`` and ``_get_host_ip``.

Change-Id: Idb232f2d7b0f74e9c65e42955c767fb7bb6ae004

This patch adds coverage for some parts in get_cinderclient
method like ``cinder_endpoint_template`` option and keystone
exception case when catalog info for cinder could not be found.
Also it adds tests for complete coverage of _check_context
method.

Change-Id: I18b1e5e8fd818824a2dda2ad14d9456190fe9ff4

The cinder volume create call was moved to the cinder_utils
module to keep the driver code cleaner with commit[1].
A decorator was also added to handle NotFound exception which
made the existing try/except block redundant.
Also the current try/except block has misleading messages:

1) "Invalid volume type configured"
2) "Failed to create image-volume due to invalid
   `cinder_volume_type` configured."

A volume create could fail because of many reasons and "invalid
volume type" is just one of them. These messages will confuse
the operators even if the configured volume type is correct.
This patch removes them since the NotFound exception is already
handled at[2] and the messages are misleading.

[1] 1178f113
[2] https://github.com/openstack/glance_store/blob/c0c4969a4a6ce77c61f84fcb37b3efdac71d6418/glance_store/common/cinder_utils.py#L37

Change-Id: I39c9caec85f574fc717e60f11e5681860a20ed30

This patch adds code coverage (UTs) for methods in StoreLocation class:
1) process_specs
2) get_uri
3) parse_uri

It also adds a missing method (_set_url_prefix) coverage in
Multistore cinder tests (test_multistore_cinder).

Change-Id: I8ced5af11669fb131c665ce53be57143f2c7b518

Add test coverage for two config options used to create a keystone
session object for cinderclient namely ``cinder_api_insecure`` and
``cinder_ca_certificates_file``.

Change-Id: Idb6db9f87ef7290441dfe60ca2eb9a59c9ee7bd3

six has been used to maintain compatibility between python2 and
python3 code. Since current openstack development branches doesn't
support python2, we don't need the six library and can use python3
code directly.
This patch removes six usage from cinder store tests.

Change-Id: I4deb193b44394ae9f99f57e31fb81fe04394e247

This patch aims at a refactoring effort that would remove
duplicate tests (current and future) by moving them into a
common base class which is called via both single and multi store
test modules with their specific configurations.

This has a lot of benefits:

1) Removes duplicate code
2) Makes addition of new tests easier and cleaner
3) Ensuring a new method/code path added is tested in both
   single and multi store configurations
4) Fixing issues detected while refactoring methods
   (Eg: tests for add method in test_multistore_cinder were not
    passing the hashing_algo parameter which is currently handled
    by the backward compat code (back_compat_add decorator) but
    those tests will break when we remove backward compatibility)

Change-Id: I12569af5623f1cd7803c00a6c3b9eb211f15b6fd

This patch aims at a refactoring effort that would remove
duplicate tests (current and future) by moving them into a
common base class which is called via both single and multi store
test modules with their specific configurations.

This has a lot of benefits:

1) Removes duplicate code
2) Makes addition of new tests easier and cleaner
3) Ensuring a new method/code path added is tested in both
   single and multi store configurations
4) Fixing issues detected while refactoring methods
   (Eg: fake_chown accepted an optional backend parameter which
    does not match the signature of original method temporary_chown)

Change-Id: Iaacb3e117cac2a661750bdb21bd0a7496078ea26

FakeObject is used to set dynamic attributes to a fake class which
works in most cases but fails when an attribute is accessed which
is not defined/passed explicitly.
This causes failure of tests whenever a new attribute is accessed
and it's hard to maintain the set of attributes for every object
used.
MagicMock provides the same functionality with additional features
and handles the case effectively when an attribute is accessed
which is not explicitly defined in the tests.
This change will help safeguard against cases seen earlier in[1].
This patch replaces FakeObject with MagicMock in test_multistore_cinder.py

[1] https://review.opendev.org/c/openstack/glance/+/805974

Change-Id: I193bb03d174e6c010132702dedef1dea05f40e41

FakeObject is used to set dynamic attributes to a fake class which
works in most cases but fails when an attribute is accessed which
is not defined/passed explicitly.
This causes failure of tests whenever a new attribute is accessed
and it's hard to maintain the set of attributes for every object
used.
MagicMock provides the same functionality with additional features
and handles the case effectively when an attribute is accessed
which is not explicitly defined in the tests.
This change will help safeguard against cases seen earlier in[1].
This patch replaces FakeObject with MagicMock in test_cinder_store.py

[1] https://review.opendev.org/c/openstack/glance/+/805974

Change-Id: Ifd0905b139cc90c2bd7444bc9e7638f3af6879a7

Add file to the reno documentation build to show release notes for
stable/yoga.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/yoga.

Sem-Ver: feature
Change-Id: I99772281621ebce5a57ddd37cee879d7cc387c47

When we have an image with size > 1 GB, we follow the following steps
to accomodate the image:
1) Detach the volume
2) extend the volume
3) Attach the volume
4) Open the volume device as a file and resume writing the image

Sometimes due to several reasons (mostly network related), the
size of the device file could mismatch with the actual volume size
(or the backend LUN size). This can happen if the extend was performed
(i.e. the control path) but it takes the time to reflect that into
the mapped device (i.e. the data path). This mismatch can cause the
issue "IOError: [Errno 28] No space left on device".
To avoid this scenario, we check if the device size is less than the
volume size, we wait for the extended LUN to show up in mapped device
and then continue the image writing operation.

Closes-Bug: #1959913

Change-Id: I206580f6be615ebc5e15b546b9c23728d4116a5d

Cinder's attachment_complete API is called to mark the volume from
"attaching" state to "in-use" state stating that the volume is
ready to use.
In the current code, attachment_complete is called before the
brick connection is complete. This should be done after the
connection to brick is successful and we have a volume path
to write the image into. This patch corrects the behavior.

The wrong sequence of execution won't have any issues functionally
since we open the volume after all steps are completed but flow
wise it is incorrect.
This change won't have any end user impact.

Change-Id: Ia9652a4ff6d7efbabb58511f0ce93a87b3a4dfa8

While fetching connector information the cinder driver of
glance-store is passing 'hostname' as IP address whereas it
actually needs IP address.

This path fetches the IPV4 or IPV6 address from the available
'hostname' and passes it to os-brick.

This worked for other cinder backends which don't rely specifically
on IP address for initializing connection like fibre channel, nfs etc
and even for some iscsi backends since, in some environments,
the hostname is same as the ip address. This is not the case always
and we should pass the correct IPv4/IPv6 address of the host to
avoid this issue.

Closes-Bug: #1955668

Change-Id: Ic79815972e654a8bfe2775f57c68cfa0bf115e2f

Recently cinder has utilized Ceph clone v2 support for its
RBD backend, since then if you attempt to delete an image from
glance that has a dependent volume, all future uses of that
image will fail in error state. Despite the fact that image
itself is still inside of Ceph/Glance. This issue is reproducible
if you are using ceph client version greater than 'luminous'

To resolve this issue glance RBD driver now checks whether snapshot
of original image has any external references before deleting/removing
it's snapshot and returns 409 Conflict response if it has any.

NOTE: To check this dependency glance osd needs 'read' access to
cinder and nova side RBD pool.

Closes-Bug: #1954883
Depends-on: https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/819476
Change-Id: If30b7bd7acac148b6f89ce46abbe128c678c90e7

Yoga testing runtime[1] has been updated to add py39
testing as voting. Unit tests update are handled by the
job template change in openstack-zuul-job

- https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/820286

this commit updates the classifier in setup.cfg file.

[1] https://governance.openstack.org/tc/reference/runtimes/yoga.html

Change-Id: I2241143bb4936d7258f156b8e9d760607ed61d0e

For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I8c8283efd0e7bb030b886e4a4e27a29d56e71288

Correct debug log when initializing _AttachmentStateManager.

Change-Id: Ifa909a67c76b897f2ab4b1ee5926046bc7045d3d

Both Python 3.x and Keystone v3 are hardcoded settings in devstack for
quite some time, drop the old variables from the job definitions.

Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I06ae79267159f3769dfcba860115e2c45ba2a8c6

Change-Id: I07b14385290cbaba570115a85233d35a88b4d586
Closes-Bug: #1946789

Change-Id: I7432abc646acf3dce7a6b5965dab83d3732f09d2

This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for yoga.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I7d6e6666e56c1a20cad3153cb84a4290f6ca3692