1. 15 Jul, 2019 1 commit
  2. 13 Jul, 2019 2 commits
  3. 12 Jul, 2019 3 commits
  4. 11 Jul, 2019 7 commits
    • Zuul's avatar
      f03b0386
    • Tim Burke's avatar
      sharding: Cache shard ranges for object writes · a1af3811
      Tim Burke authored
      Previously, we issued a GET to the root container for every object PUT,
      POST, and DELETE. This puts load on the container server, potentially
      leading to timeouts, error limiting, and erroneous 404s (!).
      
      Now, cache the complete set of 'updating' shards, and find the shard for
      this particular update in the proxy. Add a new config option,
      recheck_updating_shard_ranges, to control the cache time; it defaults to
      one hour. Set to 0 to fall back to previous behavior.
      
      Note that we should be able to tolerate stale shard data just fine; we
      already have to worry about async pendings that got written down with
      one shard but may not get processed until that shard has itself sharded
      or shrunk into another shard.
      
      Also note that memcache has a default value limit of 1MiB, which may be
      exceeded if a container has thousands of shards. In that case, set()
      will act like a delete(), causing increased memcache churn but otherwise
      preserving existing behavior. In the future, we may want to add support
      for gzipping the cached shard ranges as they should compress well.
      
      Change-Id: Ic7a732146ea19a47669114ad5dbee0bacbe66919
      Closes-Bug: 1781291
      a1af3811
    • zengjia's avatar
      Update auth_url in install docs · 0ae1ad63
      zengjia authored
      Beginning with the Queens release, the keystone install guide
      recommends running all interfaces on the same port.This patch
      updates the swift install guide to reflect that change
      
      Change-Id: Id00cfd2c921da352abdbbbb6668b921f3cb31a1a
      Closes-bug: #1754104
      0ae1ad63
    • Tim Burke's avatar
      reconciler: Enqueue right work for shard containers · f4bb1bea
      Tim Burke authored
      This fixes newly-enqueued work going forward, but doesn't offer anything
      to try to parse existing bad work items or even to kick shards so they
      reset their reconciler high-water marks.
      
      Change-Id: I79d20209cea70a6447c4e94941e5e854889cbec5
      Closes-Bug: 1836082
      f4bb1bea
    • Zuul's avatar
      e62f07d9
    • Zuul's avatar
      7e95e3d1
    • Zuul's avatar
      Merge "py3: add swift-dsvm-functional-py3 job" · 9367bff8
      Zuul authored
      9367bff8
  5. 10 Jul, 2019 8 commits
  6. 08 Jul, 2019 1 commit
  7. 04 Jul, 2019 1 commit
    • Tim Burke's avatar
      container-replicator: Add a timeout for get_shard_ranges · 4c4bd778
      Tim Burke authored
      Previously this had no timeout, which meant that the replicator might
      hang and fail to make progress indefinitely while trying to receive
      shard ranges.
      
      While we're at it, only call get_shard_ranges when the remote indicates
      that it has shard ranges for us to sync -- this reduces the number of
      requests necessary to bring unsharded replicas in sync.
      
      Change-Id: I32f51f42d76db38271442a261600089404a00f91
      Closes-Bug: #1835260
      4c4bd778
  8. 03 Jul, 2019 3 commits
    • Tim Burke's avatar
      s3token: fix conf option name · 345f577f
      Tim Burke authored
      Related-Change: Ica740c28b47aa3f3b38dbfed4a7f5662ec46c2c4
      Change-Id: I71f411a2e99fa8259b86f11ed29d1b816ff469cb
      345f577f
    • Zuul's avatar
    • Tim Burke's avatar
      py3: Be able to read and write non-ASCII headers · 76fde892
      Tim Burke authored
      Apparently Python's stdlib got more picky about what a header should
      look like. As a result, if an account, container, or object had a
      non-ASCII metadata name (values were fine), the proxy-server wouldn't
      parse all of the headers. See https://bugs.python.org/issue37093 for
      more information.
      
      This presented several problems:
      - Since the non-ASCII header aborts parsing, we may lose important
        HTTP-level information like Content-Length or Transfer-Encoding.
      - Since the offending header wouldn't get parsed, the client wouldn't
        even know what the problem was.
      - Even if the client knew what the bad header was, it would have no way
        to clear it, as the server uses the same logic to parse incoming
        requests.
      
      So, hack in our own header parsing if we detect that parsing was
      aborted. Note that we also have to mangle bufferedhttp's putheader so we
      can get non-ASCII headers to the backend servers.
      
      Now, we can run the test_unicode_metadata tests in
      test/functional/test_account.py and test/functional/test_container.py
      under py2 against services running under py3.
      
      Change-Id: I0f03c211f35a9a49e047a5718a9907b515ca88d7
      76fde892
  9. 02 Jul, 2019 2 commits
  10. 01 Jul, 2019 5 commits
  11. 29 Jun, 2019 1 commit
  12. 27 Jun, 2019 2 commits
    • Tim Burke's avatar
      s3api: Put more information in the str() for InternalError · 88cec2c3
      Tim Burke authored
      There are few things more frustrating when digging through a log than
      finding a traceback that ends with
      
          raise InternalError('unexpected status code %d' % status)
      
      while having no indication of what the status code was.
      
      Change-Id: I08e9f7c0643ae0469bd1937a30c1c4477b67c31f
      88cec2c3
    • Yuxin Wang's avatar
      Use get_container_info to check existence before container PUT · e02b9b79
      Yuxin Wang authored
      PUT request on an existing container will trigger an update on
      container db. When disks where container db landed are under
      heavy loads, update on the container db may fail due to LockTimout.
      
      Hence, we first check existence, if it's not there, we PUT.
      
      Change-Id: Ic61153948e35f1c09b05bfc97dfac3fb487b0898
      Closes-Bug: 1780204
      e02b9b79
  13. 25 Jun, 2019 2 commits
  14. 24 Jun, 2019 1 commit
  15. 22 Jun, 2019 1 commit