Change-Id: I82d120f2a04eebf40f309a5d301ad253744d9712

The fasteners lib in version 0.15.0 removed the
threading.current_thread workaround for eventlet[1] because eventlet
seemed to fixed the current_thread issues tracked in [2]. However the
fix for [2] only fixed half of the problem. The threading.current_thread
call works if it is called from thread created by eventlet.spawn.
However if the thread is created with eventlet.spawn_n then
threading.current_thread is still broken and returns the ID of the
python native thread.

The fasteners' ReaderWriterLock depends heavily on
threading.current_thread to decide which thread holds a lock and to
allow re-entry of that thread. This leads to the situation that
multiple threads created from spawn_n could take the same
ReaderWriterLock at the same time.

The fair internal lock in oslo.concurrency uses ReaderWriterLock and
as a result such lock is broken for threads created with spawn_n.

Note that this issue was raised with eventlet in [3] when the nova team
detected it via a direct usage of ReaderWriterLock in the nova test
code. As [3] did not lead to a solution in eventlet nova implemented a
nova local fix for the test code in [4].

However now we detected that oslo.concurrency is affected by this issue
as well.

This patch restores the workaround that was removed by [1].

Note that a fasteners issue [5] also opened to restore the
workaround[1].

[1] https://github.com/harlowja/fasteners/commit/467ed75ee1e9465ebff8b5edf452770befb93913
[2] https://github.com/eventlet/eventlet/issues/172
[3] https://github.com/eventlet/eventlet/issues/731
[4] https://review.opendev.org/c/openstack/nova/+/813114
[5] https://github.com/harlowja/fasteners/issues/96

Closes-Bug: #1988311
Change-Id: Ia873bcc6b07121c9bd0b94c593567d537b4c1112

The fasteners lib in version 0.15.0 removed the
threading.current_thread workaround for eventlet[1] because eventlet
seemed to fixed the current_thread issues tracked in [2]. However the
fix for [2] only fixed half of the problem. The threading.current_thread
call works if it is called from thread created by eventlet.spawn.
However if the thread is created with eventlet.spawn_n then
threading.current_thread is still broken and returns the ID of the
python native thread.

The fasteners' ReaderWriterLock depends heavily on
threading.current_thread to decide which thread holds a lock and to
allow re-entry of that thread. This leads to the situation that
multiple threads created from spawn_n could take the same
ReaderWriterLock at the same time.

The fair internal lock in oslo.concurrency uses ReaderWriterLock and
as a result such lock is broken for threads created with spawn_n.

Note that this issue was raised with eventlet in [3] when the nova team
detected it via a direct usage of ReaderWriterLock in the nova test
code. As [3] did not lead to a solution in eventlet nova implemented a
nova local fix for the test code in [4].

However now we detected that oslo.concurrency is affected by this issue
as well.

This patch adds tests to show the scope of the problem.

Note that the coverage tox target is changed to explicitly enable native
threading otherwise it runs eventlet specific tests in a native
environment.

Also note that [5] was opened to reintroduce the workaround[1] in fasteners.

[1] https://github.com/harlowja/fasteners/commit/467ed75ee1e9465ebff8b5edf452770befb93913
[2] https://github.com/eventlet/eventlet/issues/172
[3] https://github.com/eventlet/eventlet/issues/731
[4] https://review.opendev.org/c/openstack/nova/+/813114
[5] https://github.com/harlowja/fasteners/issues/96

Related-Bug: #1988311
Change-Id: Ibc193c855b49b95b46ebd2aac82ea89e33f885f0

We currently have no log entry when we start trying to acquire a lock.

In general this is ok, but there are cases where it can be problematic,
for example if we have a deadlock situation or if a lock takes a very
long time to be acquired.

In those scenarios looking at the logs we would see the operation
proceed normally and suddenly go completely silent without knowing that
it's waiting for a lock to be freed.

This patch adds a debug log message right before trying to acquire the
lock so we can detect those situations.

Change-Id: I1354dfb98b0927ae167802ecc4ab1d34f6b4d720

For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: Ic6a8cb3efdddaa809b1381a7d9b96e50b7e9da9f

Change-Id: Icd55bbb5111c12f150b8792c36e201d114ab14dc

In Zed cycle testing runtime, we are targetting to drop the
python 3.6/3.7 support, project started adding python 3.8 as minimum,

example nova:
- https://github.com/openstack/nova/blob/56b5aed08c6a3ed81b78dc216f0165ebfe3c3350/setup.cfg#L13

Change-Id: I1f947ad541b7417fd43ac10feef591fa939f19da

Change-Id: I56651c1b83baaf443551686d7e8545eba2a0c7a1

When lock_file is not present on the [oslo_concurrency] section of the
config file, we are getting a misleading error saying that it is not
preset on the DEFAULT group. This commit fixes it by providing the group
arg to the RequiredOptError constructor.

Closes-Bug: #1966320

Signed-off-by: Thiago Brito <thiago.brito@windriver.com>
Change-Id: Idee6987739917c7eae4faee15a66085a9ef7f6d2

As part of the migration of this project to the independent release
model, we failed to notice that the job template was still tied to a
specific release. We've now introduced a new unversioned job template,
'openstack-python3-jobs' [1], which was can and should use. Do this.

[1] https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/833286/

Depends-On: https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/833286/


Change-Id: Ifc12f060d9d78dc04b6addbaa441667b00e5bd69
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

This patch updates the project template to use the Yoga project template
to ensure unit testing is in place for all of the tested runtimes
for yoga. For more please refer to [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I47f85706b2b5ad8ac12ac67e97b5bb7179572615

processutils.execute() is documented to take a python_exec kwarg and
default to using sys.executable if that argument isn't set.  It is
convenient (and more intuitive) for this behavior to also occur if
the argument is present with value None.

Closes-bug: #1962603
Related-bug: #1962581
Change-Id: I2e1f187feaf4bf9fbfaf04fce78efa0cba49fc07

Yoga testing runtime[1] has been updated to add py39
testing as voting. Unit tests update are handled by the
job template change in openstack-zuul-job

- https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/820286

this commit updates the classifier in setup.cfg file.

[1] https://governance.openstack.org/tc/reference/runtimes/yoga.html

Change-Id: I27c8214b7e56066c1989788a17f49264c86cba6c

This adds support for a non-blocking behavior of the lock : the context-based
lock now raises an AcquireLockFailedException if the lock can't be acquired on
the first try.

At a higher layer, we just caught this exception inside the 'synchronized'
decorator to prevent calling the wrapped function. In which case, we then
trace this acquisition failure.

For now at least, disabling blocking is not supported when using fair locks
because the ReaderWriterLock.write_lock() provided by the fasteners module
doesn't implements this behavior.

Change-Id: I409da79007c9ba4fb8585da881e3d56998b0b98b

Setuptools v54.1.0 introduces a warning that the use of dash-separated
options in 'setup.cfg' will not be supported in a future version [1].
Get ahead of the issue by replacing the dashes with underscores. Without
this, we see 'UserWarning' messages like the following on new enough
versions of setuptools:

  UserWarning: Usage of dash-separated 'description-file' will not be
  supported in future versions. Please use the underscore name
  'description_file' instead

[1] https://github.com/pypa/setuptools/commit/a2e9ae4cb

Change-Id: I4f58234e6f2d0b0b8548d1cf53c266c562b411eb

We support Python 3.6 as a minimum now, making these checks no-ops.

Change-Id: I1ff344d345b56246e6a86f7761ef900e42daefa2

The goal here is to avoid conflicts between flake8 and hacking version each
2 days.

Inspired from nova's approach[1].

The flake8 version to install will be determined by hacking and
requirements[2] will stay aligned instead of relying on different versions.

[1] https://opendev.org/openstack/nova/src/branch/master/.pre-commit-config.yaml#L26-L35
[2] https://opendev.org/openstack/hacking/src/branch/master/requirements.txt#L1

Change-Id: I2486771d784b84eeccdc3a71ab53ce707d2c8f73

These were missed in change: Ic086f5e9a7a8b9c9050ef27993f339d52ad1e236

Change-Id: Ifbfb33e18c95a3bf01dd522835d1246989e78242

We facing errors related to the new pip resolver, this
topic was discussed on the ML and QA team proposed to
to test lower-constraints [1].

I propose to drop this test because the complexity and recurring pain needed
to maintain that now exceeds the benefits provided by this mechanismes.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019390.html

Change-Id: Ic086f5e9a7a8b9c9050ef27993f339d52ad1e236

UPPER_CONSTRAINTS_FILE is old name and deprecated
This allows to use upper-constraints file as more
readable way instead of UPPER_CONSTRAINTS_FILE=<lower-constraints file>.

[1] https://review.opendev.org/#/c/722814/
[2] https://zuul-ci.org/docs/zuul-jobs/python-roles.html#rolevar-tox.tox_constraints_file

Change-Id: Ib833cd6818e7071cf3e85f77966e9c02a4026efd

Moving on py3 as the default runtime for tox to avoid to update this at
each new cycle.

Wallaby support officially the following runtimes [1]:
- Python 3.6
- Python 3.8

During Victoria Python 3.7 was used as the default runtime [2] however this
version isn't longer officially supported.

[1] https://governance.openstack.org/tc/reference/runtimes/wallaby.html#python-runtimes-for-wallaby
[2] https://governance.openstack.org/tc/reference/runtimes/victoria.html#python-runtimes-for-victoria

Change-Id: Id19b6e38400f7b644faeabcac6d8037603cac459

Monkey patch the original current_thread to use the up-to-date _active
global variable. This solution is based on that documented at:
https://github.com/eventlet/eventlet/issues/592

Change-Id: I0a2c1e0d8a8cad99d68100d25e88e0d3a2eb8f5c
Related-Bug: #1863021

This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for wallaby.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I6989299c0a913bc1110fc6d186975734065f98ac

Add file to the reno documentation build to show release notes for
stable/victoria.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/victoria.

Change-Id: Iabda8e0b114a46fb765249b55dbabf06429b9b06
Sem-Ver: feature

Introduced changes:
- pre-commit config and rules
- Add pre-commit to pep8 gate, Flake8 is covered in the pre-commit hooks.
- Applying fixes for pre-commit compliance in all code.

Also commit hash will be used instead of version tags in pre-commit to
prevend arbitrary code from running in developer's machines.

pre-commit will be used to:
- trailing whitespace;
- Replaces or checks mixed line ending (mixed-line-ending);
- Forbid files which have a UTF-8 byte-order marker (check-byte-order-marker);
- Checks that non-binary executables have a proper
  shebang (check-executables-have-shebangs);
- Check for files that contain merge conflict strings (check-merge-conflict);
- Check for debugger imports and py37+ breakpoint()
  calls in python source (debug-statements);
- Attempts to load all yaml files to verify syntax (check-yaml);
- Run flake8 checks (flake8) (local)

For further details about tests please refer to:
https://github.com/pre-commit/pre-commit-hooks



Change-Id: I6ae02d4e52fd812360414e7b8c1c9c4f17230812
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>

This patch bumps bandit allowed version to >=1.6.0,<1.7.0 in order to
avoid the errors detailed here https://github.com/PyCQA/bandit/pull/393



Change-Id: I358b62b45096cfbcafe0e665d92e6b05fb36b153
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>

For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I6ab357a5dca1e8f6e1a8d2b3febe79e961228fcc

This is a standard Python feature since 3.3.

Change-Id: Ib13af5aab0ebbae532f1e46309ad6023ca94d6b9

This updates lower constraints to versions that will work with py38 so
that when we move to running on focal nodes, which has py38 as its
default py3 runtime, the lower-constraints job will continue to pass.

It also cleans out some secondary requirements that are no longer needed
due to our direct dependencies being updated.

Linters are removed that are kept in the global requirements blacklist
as those are not version tracked and are not relevant for our
lower-constraints unit test runs.

Change-Id: I60af0a8c5241190e5c889ff8f9aba3a4b3acc04c
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>

New theme of docs respects pygments_style.

more info: http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html

Change-Id: I8847feba7669feed9edf9d7f7ba48af26760c298