oslo.limit 1.5.0 release

meta:version: 1.5.0
meta:diff-start: -
meta:series: yoga
meta:release-type: release
meta:pypi: yes
meta:first: yes
meta:release:Author: melanie witt <melwittt@gmail.com>
meta:release:Commit: melanie witt <melwittt@gmail.com>
meta:release:Change-Id: Ib13eb3644c35cbf1d011ccc54264f73f896cf768
meta:release:Code-Review+2: Hervé Beraud <herveberaud.pro@gmail.com>
meta:release:Code-Review+2: Elod Illes <elod.illes@est.tech>
meta:release:Workflow+1: Elod Illes <elod.illes@est.tech>

We currently have a public Enforcer interface for getting limits and
calculating usage but it is not yet possible to retrieve only limits
without calculating usage.

While working on unified limits support in nova, we realized we need a
way to get limits only. In nova there are legacy APIs for showing quota
limits and initially we will provide compat by proxying to keystone to
get the limits.

This adds public interfaces for getting limits to Enforcer.

Related to blueprint unified-limits-nova

Change-Id: I22234e0bb6b3a1cecb29a6b99a3afcd02ffdbf5f

This allows a caller to pass None for the project_id if it only wants
it to check the registered limit for a given resource. This is useful
for non-project-scoped resourced where we just want to make sure some
global limit hasn't been exceeded. This would also be relevant for
resources that are created by system-scoped users, such as host
aggregates.

Change-Id: I5fea0143b6a96b5f79bc273961e3e284a260e25e

The calculate_usage interface was added recently to allow consumers
to probe limits and usage without requiring the enforce behavior
workflow. If a limit was passed to it that was not registered in
keystone, get_project_limits() would raise a ProjectOverLimit
exception itself to abort the process immediately, providing the
"unregistered means zero" behavior. This works fine for the enforce
workflow, but not the calculate one.

This changes get_project_limits() to just return a zero limit for
a missing one, which will be considered by the enforce workflow in
the same way, keeping the existing behavior. It will merely be
reported by the calculate workflow, which is the desired change.

Change-Id: Iaab1f0d5eb0da9a667267537d86f6c70bc8db51d

This adds caching of resource limits for an Enforcer in order to
improve performance when repeated limit checks are needed. The cache
lasts the lifetime of an Enforcer and is enabled by default. It can be
disabled by passing cache=False when instantiating an Enforcer.

One usage pattern for a caching Enforcer would be to create an Enforcer
per service request so that the caching lives only as long as the
request.

Change-Id: I8e43dceec76aecd2b2ae23a137e56519efe29777

Currently the oslo.config.opts entry_points provided by oslo.limit
doesn't include options for auth plugins, thus the parameters to
define credentials like username, password and etc are not picked up by
oslo-config-generator.

This adds the options for auth plugin options to the entry point so
that the auth parameters are included by the generated config files
and users can easily find the parameters to define the required user
credential.

Note that keystoneauth provides several plugins but this change covers
only password plugins, assuming the password authentication is most
popularly used.

Change-Id: Ib440f58b589076677be9e90dd960cd4459e63746

This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for yoga.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: Ic2bf80ec8f183476facb9710be968050641532e0

Add file to the reno documentation build to show release notes for
stable/xena.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/xena.

Sem-Ver: feature
Change-Id: If079ffb950122121a9cd9254cce28bf62106ed57

Setuptools v54.1.0 introduces a warning that the use of
dash-separated options in 'setup.cfg' will not be supported
in a future version [1].
Get ahead of the issue by replacing the dashes with underscores.
Without this, we see 'UserWarning' messages
like the following on new enough
versions of setuptools:

UserWarning: Usage of dash-separated 'description-file' will not be
supported in future versions. Please use the underscore name
'description_file' instead
[1]https://github.com/pypa/setuptools/commit/a2e9ae4cb

Change-Id: I2ea9570dcf018c484054d8078f6e5a767b23dd82

The patch bumps min version of tox to 3.18.0 in order to
replace tox's whitelist_externals by allowlist_externals option:
https://github.com/tox-dev/tox/blob/master/docs/changelog.rst#v3180-2020-07-23

Change-Id: I1d65fb9d8d302ff1e3f33e92b668b2241741e7e0

This adds a fixture that can be used by consuming projects to
simulate a set of limits in keystone, without requiring actual
keystone. Currently, consumers have to mock oslo.limit internals (at
least) in order to do testing.

Change-Id: If72050e90ca8b03e26d128c7bbcef6bbea92b501

In multiple situations, it is necessary to be able to probe the limits
set for a project without actually enforcing. Examples:

1. Exposing a usage API where we want to not only report the current
   usage, but the limit as well. Otherwise clients have to do their
   own calls to keystone and correlation to get a single integer
   limit value, which we should be able to expose for them.
2. When checking quota as part of a long-running process of consuming
   an unbounded data stream, we need to be able to determine how much
   quota remains so that we can stop the transfer if we exceed the
   limit. Without this, we have to periodically call to keystone
   during the transfer, which is expensive and could fail.

This patch adds a calculate_usage() method to the Enforcer which
calculates the usage using the enforcement model and returns a
mapping of resource names to namedtuples that contain limit and usage
information.

Change-Id: Ic0632cc5ec52aefb85a04f879651963bfa54dcbe

[1] https://github.com/pre-commit/pre-commit-hooks/commit/9136088a246768144165fcc3ecc3d31bb686920a

Change-Id: I45f42b47cb3ba45416dc7e4325b4efd635df4a92

This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for xena.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I8a12f0dcdec43f3a5738858bafa210fdb6f1bac1

Add file to the reno documentation build to show release notes for
stable/wallaby.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/wallaby.

Sem-Ver: feature
Change-Id: I0ee8832bb010b0929ba0e7a0dd00fa9a399004e7

This patch is the merge of 2 current fixes that need to be solved in the
same time. Indeed each issue lock our gates independently.

Dropping lower constraints testing

We facing errors related to the new pip resolver, this
topic was discussed on the ML and QA team proposed to
to test lower-constraints [1].

I propose to drop this test because the complexity and recurring pain needed
to maintain that now exceeds the benefits provided by this mechanismes.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019390.html

Move flake8 as a pre-commit local target.

The goal here is to avoid conflicts between flake8 and hacking version each
2 days.

Inspired from nova's approach[1].

The flake8 version to install will be determined by hacking and
requirements[2] will stay aligned instead of relying on different versions.

[1] https://opendev.org/openstack/nova/src/branch/master/.pre-commit-config.yaml#L26-L35
[2] https://opendev.org/openstack/hacking/src/branch/master/requirements.txt#L1



Co-authored-by: Daniel Bengtsson <dbengt@redhat.com>
Change-Id: I04a845fd98f0d21f7a21dae9f184117263390dd1

UPPER_CONSTRAINTS_FILE is old name and deprecated
This allows to use upper-constraints file as more
readable way instead of UPPER_CONSTRAINTS_FILE=<lower-constraints file>.

[1] https://review.opendev.org/#/c/722814/
[2] https://zuul-ci.org/docs/zuul-jobs/python-roles.html#rolevar-tox.tox_constraints_file

Change-Id: I1a006cf4368cc18c34dd6f236ecfe966e6b09388

Change-Id: I09d641c27372bba638aa8be207d27d80c8ec01ab

Change-Id: I86ffd71d9c5cdda410d654fe9b0bb46ad12eeb05

Moving on py3 as the default runtime for tox to avoid to update this at
each new cycle.

Wallaby support officially the following runtimes [1]:
- Python 3.6
- Python 3.8

During Victoria Python 3.7 was used as the default runtime [2] however this
version isn't longer officially supported.

[1] https://governance.openstack.org/tc/reference/runtimes/wallaby.html#python-runtimes-for-wallaby
[2] https://governance.openstack.org/tc/reference/runtimes/victoria.html#python-runtimes-for-victoria

Change-Id: Idb0f7ff31a8fa788c940d70eddd1e7218691a31d

flake8 new release 3.8.0 added new checks and gate pep8
job start failing. hacking 3.0.1 fix the pinning of flake8 to
avoid bringing in a new version with new checks.

Though it is fixed in latest hacking but 2.0 and 3.0 has cap for
flake8 as <4.0.0 which mean flake8 new version 3.9.0 can also
break the pep8 job if new check are added.

To avoid similar gate break in future, we need to bump the hacking min
version.

- http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014828.html

Change-Id: I70ca30dd0a5bf0f1aa44f627d92038c826b67174

The docs requirements migrated to doc/requirements.txt
we need not install things from requirements.txt.

Change-Id: Ia65109bbb683489ba9a880d3a41baca5ef560d77

This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for wallaby.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I202c5e4bc869e3ac4f03acfca13d59f5e8e02c76

Add file to the reno documentation build to show release notes for
stable/victoria.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/victoria.

Change-Id: I73e3c0ae23f57c57cb3d55157df3cd8024914100
Sem-Ver: feature

Introduced changes:
- pre-commit config and rules.
- Add pre-commit to pep8 gate, Flake8 is covered in the pre-commit hooks.
- Applying fixes for pre-commit compliance in all code.

Also commit hash will be used instead of version tags in pre-commit to
prevend arbitrary code from running in developer's machines.

pre-commit will be used to:
- trailing whitespace;
- Replaces or checks mixed line ending (mixed-line-ending);
- Forbid files which have a UTF-8 byte-order marker
  (check-byte-order-marker);
- Checks that non-binary executables have a proper
  shebang (check-executables-have-shebangs);
- Check for files that contain merge conflict strings
  (check-merge-conflict);
- Check for debugger imports and py37+ breakpoint()
  calls in python source (debug-statements);
- Attempts to load all yaml files to verify syntax (check-yaml);
- Run flake8 checks (flake8) (local)

For further details about tests please refer to:
https://github.com/pre-commit/pre-commit-hooks



Change-Id: Ic00da1340d695c7a109f41a09929b654baf995a7
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>

This patch bumps bandit allowed version to >=1.6.0,<1.7.0 in order to
avoid the errors detailed here https://github.com/PyCQA/bandit/pull/393



Change-Id: I568123c93c32cb224628c4a8371a9a0e7c19cf67
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>