When we have an image with size > 1 GB, we follow the following steps
to accomodate the image:
1) Detach the volume
2) extend the volume
3) Attach the volume
4) Open the volume device as a file and resume writing the image

Sometimes due to several reasons (mostly network related), the
size of the device file could mismatch with the actual volume size
(or the backend LUN size). This can happen if the extend was performed
(i.e. the control path) but it takes the time to reflect that into
the mapped device (i.e. the data path). This mismatch can cause the
issue "IOError: [Errno 28] No space left on device".
To avoid this scenario, we check if the device size is less than the
volume size, we wait for the extended LUN to show up in mapped device
and then continue the image writing operation.

Closes-Bug: #1959913

Change-Id: I206580f6be615ebc5e15b546b9c23728d4116a5d

Cinder's attachment_complete API is called to mark the volume from
"attaching" state to "in-use" state stating that the volume is
ready to use.
In the current code, attachment_complete is called before the
brick connection is complete. This should be done after the
connection to brick is successful and we have a volume path
to write the image into. This patch corrects the behavior.

The wrong sequence of execution won't have any issues functionally
since we open the volume after all steps are completed but flow
wise it is incorrect.
This change won't have any end user impact.

Change-Id: Ia9652a4ff6d7efbabb58511f0ce93a87b3a4dfa8

While fetching connector information the cinder driver of
glance-store is passing 'hostname' as IP address whereas it
actually needs IP address.

This path fetches the IPV4 or IPV6 address from the available
'hostname' and passes it to os-brick.

This worked for other cinder backends which don't rely specifically
on IP address for initializing connection like fibre channel, nfs etc
and even for some iscsi backends since, in some environments,
the hostname is same as the ip address. This is not the case always
and we should pass the correct IPv4/IPv6 address of the host to
avoid this issue.

Closes-Bug: #1955668

Change-Id: Ic79815972e654a8bfe2775f57c68cfa0bf115e2f

Recently cinder has utilized Ceph clone v2 support for its
RBD backend, since then if you attempt to delete an image from
glance that has a dependent volume, all future uses of that
image will fail in error state. Despite the fact that image
itself is still inside of Ceph/Glance. This issue is reproducible
if you are using ceph client version greater than 'luminous'

To resolve this issue glance RBD driver now checks whether snapshot
of original image has any external references before deleting/removing
it's snapshot and returns 409 Conflict response if it has any.

NOTE: To check this dependency glance osd needs 'read' access to
cinder and nova side RBD pool.

Closes-Bug: #1954883
Depends-on: https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/819476
Change-Id: If30b7bd7acac148b6f89ce46abbe128c678c90e7

Yoga testing runtime[1] has been updated to add py39
testing as voting. Unit tests update are handled by the
job template change in openstack-zuul-job

- https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/820286

this commit updates the classifier in setup.cfg file.

[1] https://governance.openstack.org/tc/reference/runtimes/yoga.html

Change-Id: I2241143bb4936d7258f156b8e9d760607ed61d0e

For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I8c8283efd0e7bb030b886e4a4e27a29d56e71288

Correct debug log when initializing _AttachmentStateManager.

Change-Id: Ifa909a67c76b897f2ab4b1ee5926046bc7045d3d

Both Python 3.x and Keystone v3 are hardcoded settings in devstack for
quite some time, drop the old variables from the job definitions.

Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I06ae79267159f3769dfcba860115e2c45ba2a8c6

Change-Id: I07b14385290cbaba570115a85233d35a88b4d586
Closes-Bug: #1946789

Change-Id: I7432abc646acf3dce7a6b5965dab83d3732f09d2

This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for yoga.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I7d6e6666e56c1a20cad3153cb84a4290f6ca3692

Add file to the reno documentation build to show release notes for
stable/xena.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/xena.

Sem-Ver: feature
Change-Id: I9bda41a2d9ab44b97a1095600629ae2c048aac04

Change-Id: I595207658e812556d726121c1fe4400668c819ec

Swift driver reraises the client exception causing glance-api
to return 500. Lets raise g_s StoreFull correctly instead.

Change-Id: I5be151afe242a75142e74d488c4787a35929e189
Closes-bug: #1926404

We implemented cinder's new attachment API support with patch[1].
It was needed to add multiattach volume handling added with this
patch.
There is no special configuration change or user interference needed.
If a volume is of a multiattach type, then it will be handled as
a multiattach volume.

[1] https://review.opendev.org/c/openstack/glance_store/+/782200

Implements: blueprint attachment-api-and-multiattach-support
Closes-Bug: #1904546

Change-Id: Iffb825492a20fd877476acad05f817b399072f01

Change-Id: I1de76a70e07bbf2bd70900b4d93104a1b0fda74b

There's an issue when cinder nfs is configured as glance backend and
the image is > 1GB i.e. extend operation is performed on volume.
Currently glance writes the qcow2 header in the raw volume and
cinder nfs driver used to work on format autodetection which is
changed to manual detection here[1]
This fixes the extend volume problem for raw volumes, however, if
cinder nfs is configured to create qcow2 volumes then we will run into
the same problem.
This patch blocks creating images when the nfs volume is qcow2.

Refactoring work:
1) attachment_delete call is removed in nfs related code as it was
already done in the finally block
2) handle_exceptions decorator is removed from volume delete method
as it raises BackendException and incase when volume is not found
(possibly deleted manually), the image deletion fails so we don't
want to raise any error in this case

[1] https://review.opendev.org/c/openstack/cinder/+/761152

Closes-Bug: #1901138
Change-Id: I8ce6f36f1cb4b0ed6bcc5f3869fab3bb64fe3390

... because v2 API was deprecated during Pike and was already removed
during Xena[1].

[1] e05b261af7dcd24096b229860df65dff1d385910

Change-Id: I0e6d72ce82ce89aaaa8c56862bb8266d0351cd86

Cinder introduced new attachment API flow in microversion 3.27
(also attachment_complete added in mv 3.44 and support for passing
mode added in mv 3.54) which provides a clean interface to
interact with cinder for attachments and is also required for
multiattach volume support (Related future work).
Nova uses it since a long time and is proven to be stable,
this patch implements the same for glance.
The create volume and delete volume calls are also moved to
cinder_utils file to use the generic exception handler
and keep similar code together for consistency.

Partially Implements: blueprint attachment-api-and-multiattach-support

Change-Id: I2758ed1d5b8e0981faa3eff6f83e1ce5975a01d2

When processing the image data from a client, the previous and small
chunk size had a significant overhead. Let's be aligned with other chunk
size related parameters such as DEFAULT_LARGE_OBJECT_MIN_CHUNK_SIZE
(=5MiB) and DEFAULT_LARGE_OBJECT_CHUNK_SIZE (=10MiB).

The performance difference should be tangible especially with a
singlepart upload. And the upload time can be shortened for example as
follows:

100 MiB:   36.5s -> 4.0s
200 MiB: 2m16.8s -> 6.1s
300 MiB: 4m50.4s -> 9.1s

Closes-Bug: 1934849
Change-Id: Icecac80dd9e4e9f7ffa76bb7ca63d8d112036b70

Setuptools v54.1.0 introduces a warning that the use of dash-separated
options in the metadata section of 'setup.cfg' will not be supported in a future version [1].
Get ahead of the issue by replacing the dashes with underscores. Without
this, we see 'UserWarning' messages like the following on new enough
versions of setuptools:

  UserWarning: Usage of dash-separated 'description-file' will not be
  supported in future versions. Please use the underscore name
  'description_file' instead

[1] https://github.com/pypa/setuptools/commit/a2e9ae4cb

Change-Id: I75fe935a3b154c5eebf9d8b9550f21227e824867

add two new config options for cinder store

- cinder_store_user_domain_name
- cinder_store_project_domain_name

that allow to set the internal user and project to Keystone domains
other that the 'Default' one.

Closes-Bug: #1930299
Change-Id: I1d6c07b6c0e7e6a4da9adabaa026f024b64bb029

With changing the SOAP library backing oslo.vmware [1], the cookiejar of
a session must be accessed differently. Therefore, oslo.vmware
introduced a property on the client to abstract this change away. This
commit uses the new place to access the attribute.

[1]
https://specs.openstack.org/openstack/oslo-specs/specs/victoria/oslo-vmware-soap-library-switch.html

Change-Id: Iec6c4be5c61710f2a7a0056168d0c31ade987b1f

Currently we only pass the configured multipath option when getting
the connector properties and do not pass it when creating the
connector object which in turn does the multipathing in os-brick.
So we are only passing multipathing option to cinder for exporting &
mapping the volume to paths but not while creating the connector object
and hence doing multipathing partially.
This patch passes that option and fixes the stated issue.

Related-Bug: #1863983

Co-Authored-By: Gorka Eguileor <geguileo@redhat.com>

Change-Id: I82979c47ec9ec1dd6cc864d3827ab219177251f8

This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for xena.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: Ibd1cce5992992b474bf667cd18b14586c28d4e5f

Add file to the reno documentation build to show release notes for
stable/wallaby.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/wallaby.

Sem-Ver: feature
Change-Id: I642302a357abe16562a63cd233c1526211970279