oslo.concurrency 5.0.1 release

meta:version: 5.0.1
meta:diff-start: -
meta:series: independent
meta:release-type: release
meta:pypi: yes
meta:first: no
meta:release:Author: Hervé Beraud <hberaud@redhat.com>
meta:release:Commit: Hervé Beraud <hberaud@redhat.com>
meta:release:Change-Id: Iba860bc47cd40a4b61a8e77c6aaa361b9dff1299
meta:release:Code-Review+2: Elod Illes <elod.illes@est.tech>
meta:release:Code-Review+2: Thierry Carrez <thierry@openstack.org>
meta:release:Workflow+1: Thierry Carrez <thierry@openstack.org>

Change-Id: I82d120f2a04eebf40f309a5d301ad253744d9712

The fasteners lib in version 0.15.0 removed the
threading.current_thread workaround for eventlet[1] because eventlet
seemed to fixed the current_thread issues tracked in [2]. However the
fix for [2] only fixed half of the problem. The threading.current_thread
call works if it is called from thread created by eventlet.spawn.
However if the thread is created with eventlet.spawn_n then
threading.current_thread is still broken and returns the ID of the
python native thread.

The fasteners' ReaderWriterLock depends heavily on
threading.current_thread to decide which thread holds a lock and to
allow re-entry of that thread. This leads to the situation that
multiple threads created from spawn_n could take the same
ReaderWriterLock at the same time.

The fair internal lock in oslo.concurrency uses ReaderWriterLock and
as a result such lock is broken for threads created with spawn_n.

Note that this issue was raised with eventlet in [3] when the nova team
detected it via a direct usage of ReaderWriterLock in the nova test
code. As [3] did not lead to a solution in eventlet nova implemented a
nova local fix for the test code in [4].

However now we detected that oslo.concurrency is affected by this issue
as well.

This patch restores the workaround that was removed by [1].

Note that a fasteners issue [5] also opened to restore the
workaround[1].

[1] https://github.com/harlowja/fasteners/commit/467ed75ee1e9465ebff8b5edf452770befb93913
[2] https://github.com/eventlet/eventlet/issues/172
[3] https://github.com/eventlet/eventlet/issues/731
[4] https://review.opendev.org/c/openstack/nova/+/813114
[5] https://github.com/harlowja/fasteners/issues/96

Closes-Bug: #1988311
Change-Id: Ia873bcc6b07121c9bd0b94c593567d537b4c1112

The fasteners lib in version 0.15.0 removed the
threading.current_thread workaround for eventlet[1] because eventlet
seemed to fixed the current_thread issues tracked in [2]. However the
fix for [2] only fixed half of the problem. The threading.current_thread
call works if it is called from thread created by eventlet.spawn.
However if the thread is created with eventlet.spawn_n then
threading.current_thread is still broken and returns the ID of the
python native thread.

The fasteners' ReaderWriterLock depends heavily on
threading.current_thread to decide which thread holds a lock and to
allow re-entry of that thread. This leads to the situation that
multiple threads created from spawn_n could take the same
ReaderWriterLock at the same time.

The fair internal lock in oslo.concurrency uses ReaderWriterLock and
as a result such lock is broken for threads created with spawn_n.

Note that this issue was raised with eventlet in [3] when the nova team
detected it via a direct usage of ReaderWriterLock in the nova test
code. As [3] did not lead to a solution in eventlet nova implemented a
nova local fix for the test code in [4].

However now we detected that oslo.concurrency is affected by this issue
as well.

This patch adds tests to show the scope of the problem.

Note that the coverage tox target is changed to explicitly enable native
threading otherwise it runs eventlet specific tests in a native
environment.

Also note that [5] was opened to reintroduce the workaround[1] in fasteners.

[1] https://github.com/harlowja/fasteners/commit/467ed75ee1e9465ebff8b5edf452770befb93913
[2] https://github.com/eventlet/eventlet/issues/172
[3] https://github.com/eventlet/eventlet/issues/731
[4] https://review.opendev.org/c/openstack/nova/+/813114
[5] https://github.com/harlowja/fasteners/issues/96

Related-Bug: #1988311
Change-Id: Ibc193c855b49b95b46ebd2aac82ea89e33f885f0

oslo.concurrency 5.0.0 release

meta:version: 5.0.0
meta:diff-start: -
meta:series: independent
meta:release-type: release
meta:pypi: yes
meta:first: no
meta:release:Author: Hervé Beraud <hberaud@redhat.com>
meta:release:Commit: Hervé Beraud <hberaud@redhat.com>
meta:release:Change-Id: Iff79935d876bf7996419c13adb39a2d7c3afe5fd
meta:release:Code-Review+2: Elod Illes <elod.illes@est.tech>
meta:release:Code-Review+1: Daniel Bengtsson <dbengt@redhat.com>
meta:release:Code-Review+2: Thierry Carrez <thierry@openstack.org>
meta:release:Workflow+1: Thierry Carrez <thierry@openstack.org>

We currently have no log entry when we start trying to acquire a lock.

In general this is ok, but there are cases where it can be problematic,
for example if we have a deadlock situation or if a lock takes a very
long time to be acquired.

In those scenarios looking at the logs we would see the operation
proceed normally and suddenly go completely silent without knowing that
it's waiting for a lock to be freed.

This patch adds a debug log message right before trying to acquire the
lock so we can detect those situations.

Change-Id: I1354dfb98b0927ae167802ecc4ab1d34f6b4d720

For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: Ic6a8cb3efdddaa809b1381a7d9b96e50b7e9da9f

Change-Id: Icd55bbb5111c12f150b8792c36e201d114ab14dc

In Zed cycle testing runtime, we are targetting to drop the
python 3.6/3.7 support, project started adding python 3.8 as minimum,

example nova:
- https://github.com/openstack/nova/blob/56b5aed08c6a3ed81b78dc216f0165ebfe3c3350/setup.cfg#L13

Change-Id: I1f947ad541b7417fd43ac10feef591fa939f19da

Change-Id: I56651c1b83baaf443551686d7e8545eba2a0c7a1

When lock_file is not present on the [oslo_concurrency] section of the
config file, we are getting a misleading error saying that it is not
preset on the DEFAULT group. This commit fixes it by providing the group
arg to the RequiredOptError constructor.

Closes-Bug: #1966320

Signed-off-by: Thiago Brito <thiago.brito@windriver.com>
Change-Id: Idee6987739917c7eae4faee15a66085a9ef7f6d2

As part of the migration of this project to the independent release
model, we failed to notice that the job template was still tied to a
specific release. We've now introduced a new unversioned job template,
'openstack-python3-jobs' [1], which was can and should use. Do this.

[1] https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/833286/

Depends-On: https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/833286/


Change-Id: Ifc12f060d9d78dc04b6addbaa441667b00e5bd69
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

This patch updates the project template to use the Yoga project template
to ensure unit testing is in place for all of the tested runtimes
for yoga. For more please refer to [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I47f85706b2b5ad8ac12ac67e97b5bb7179572615

processutils.execute() is documented to take a python_exec kwarg and
default to using sys.executable if that argument isn't set.  It is
convenient (and more intuitive) for this behavior to also occur if
the argument is present with value None.

Closes-bug: #1962603
Related-bug: #1962581
Change-Id: I2e1f187feaf4bf9fbfaf04fce78efa0cba49fc07

oslo.concurrency 4.5.0 release

meta:version: 4.5.0
meta:diff-start: -
meta:series: independent
meta:release-type: release
meta:pypi: yes
meta:first: no
meta:release:Author: Hervé Beraud <hberaud@redhat.com>
meta:release:Commit: Hervé Beraud <hberaud@redhat.com>
meta:release:Change-Id: I32efe9f03829a1f07b4fdd997f36528749090f83
meta:release:Code-Review+2: Elod Illes <elod.illes@est.tech>
meta:release:Code-Review+2: Thierry Carrez <thierry@openstack.org>
meta:release:Workflow+1: Thierry Carrez <thierry@openstack.org>

Yoga testing runtime[1] has been updated to add py39
testing as voting. Unit tests update are handled by the
job template change in openstack-zuul-job

- https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/820286

this commit updates the classifier in setup.cfg file.

[1] https://governance.openstack.org/tc/reference/runtimes/yoga.html

Change-Id: I27c8214b7e56066c1989788a17f49264c86cba6c

This adds support for a non-blocking behavior of the lock : the context-based
lock now raises an AcquireLockFailedException if the lock can't be acquired on
the first try.

At a higher layer, we just caught this exception inside the 'synchronized'
decorator to prevent calling the wrapped function. In which case, we then
trace this acquisition failure.

For now at least, disabling blocking is not supported when using fair locks
because the ReaderWriterLock.write_lock() provided by the fasteners module
doesn't implements this behavior.

Change-Id: I409da79007c9ba4fb8585da881e3d56998b0b98b

Setuptools v54.1.0 introduces a warning that the use of dash-separated
options in 'setup.cfg' will not be supported in a future version [1].
Get ahead of the issue by replacing the dashes with underscores. Without
this, we see 'UserWarning' messages like the following on new enough
versions of setuptools:

  UserWarning: Usage of dash-separated 'description-file' will not be
  supported in future versions. Please use the underscore name
  'description_file' instead

[1] https://github.com/pypa/setuptools/commit/a2e9ae4cb

Change-Id: I4f58234e6f2d0b0b8548d1cf53c266c562b411eb

We support Python 3.6 as a minimum now, making these checks no-ops.

Change-Id: I1ff344d345b56246e6a86f7761ef900e42daefa2

The goal here is to avoid conflicts between flake8 and hacking version each
2 days.

Inspired from nova's approach[1].

The flake8 version to install will be determined by hacking and
requirements[2] will stay aligned instead of relying on different versions.

[1] https://opendev.org/openstack/nova/src/branch/master/.pre-commit-config.yaml#L26-L35
[2] https://opendev.org/openstack/hacking/src/branch/master/requirements.txt#L1

Change-Id: I2486771d784b84eeccdc3a71ab53ce707d2c8f73

oslo.concurrency 4.4.0 release

meta:version: 4.4.0
meta:diff-start: -
meta:series: wallaby
meta:release-type: release
meta:pypi: yes
meta:first: yes
meta:release:Author: Hervé Beraud <hberaud@redhat.com>
meta:release:Commit: Hervé Beraud <hberaud@redhat.com>
meta:release:Change-Id: I3062abf38d2581a66c363cc7a6292dc0bc16b4a2
meta:release:Code-Review+2: Thierry Carrez <thierry@openstack.org>
meta:release:Code-Review+2: Sean McGinnis <sean.mcginnis@gmail.com>
meta:release:Workflow+1: Sean McGinnis <sean.mcginnis@gmail.com>

These were missed in change: Ic086f5e9a7a8b9c9050ef27993f339d52ad1e236

Change-Id: Ifbfb33e18c95a3bf01dd522835d1246989e78242

oslo.concurrency 4.3.1 release

meta:version: 4.3.1
meta:diff-start: -
meta:series: victoria
meta:release-type: release
meta:pypi: yes
meta:first: no
meta:release:Author: Daniel Bengtsson <dbengt@redhat.com>
meta:release:Commit: Daniel Bengtsson <dbengt@redhat.com>
meta:release:Change-Id: I3d5c504ec9402be846c12aae3832d1aa580292cf
meta:release:Code-Review+2: Hervé Beraud <hberaud@redhat.com>
meta:release:Code-Review+2: Sean McGinnis <sean.mcginnis@gmail.com>
meta:release:Workflow+1: Sean McGinnis <sean.mcginnis@gmail.com>

We facing errors related to the new pip resolver, this
topic was discussed on the ML and QA team proposed to
to test lower-constraints [1].

I propose to drop this test because the complexity and recurring pain needed
to maintain that now exceeds the benefits provided by this mechanismes.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019390.html

Change-Id: Ic086f5e9a7a8b9c9050ef27993f339d52ad1e236

UPPER_CONSTRAINTS_FILE is old name and deprecated
This allows to use upper-constraints file as more
readable way instead of UPPER_CONSTRAINTS_FILE=<lower-constraints file>.

[1] https://review.opendev.org/#/c/722814/
[2] https://zuul-ci.org/docs/zuul-jobs/python-roles.html#rolevar-tox.tox_constraints_file

Change-Id: Ib833cd6818e7071cf3e85f77966e9c02a4026efd