Train Series Release Notes

2.23.3

Bug Fixes

• Sharding improvements:

  • Prevent object updates from auto-creating shard containers. This ensures more consistent listings for sharded containers during rebalances.

  • Deleted shard containers are no longer considered root containers. This prevents unnecessary sharding audit failures and allows the deleted shard database to actually be unlinked.

  • The sharder daemon has been enhanced to better support the shrinking of shards that are no longer required. Shard containers will now discover from their root container if they should be shrinking. They will also discover the shards into which they should shrink, which may include the root container itself.

  • Improved performance of sharded container listings when performing prefix listings.

  • Improved container-sharder stat reporting to reduce load on root container databases.

  • The container sharder can now handle containers with special characters in their names.

  • swift-container-info now summarizes shard range information. Pass -v/--verbose if you want to see all of them.

  • Don’t inject shard ranges when user quits.

  • Various other minor bug fixes and improvements.

• Python 3 bug fixes:

  • Fixed a potential server error when uploading data via a tempurl.

  • Fixed a potential server error when getting symlink details.

• Added the ability to connect to Memcached over TLS. See the tls_* options in etc/memcache.conf-sample

2.23.2

Upgrade Notes

• If your cluster has encryption enabled and is still running Swift under Python 2, we recommend upgrading Swift before transitioning to Python 3. Otherwise, new writes to objects with non-ASCII characters in their paths may result in corrupted downloads when read from a proxy-server still running old swift on Python 2. See bug 1888037 for more information.

• The above bug was caused by a difference in string types that resulted in ambiguity when decrypting. To prevent the ambiguity for new data, set meta_version_to_write = 3 in your keymaster configuration after upgrading all proxy servers.

  If upgrading from Swift 2.20.0 or Swift 2.19.1 or earlier, set meta_version_to_write = 1 in your keymaster configuration prior to upgrading.

  See the provided keymaster.conf-sample for more information about this setting.

Bug Fixes

• Python 3 bug fixes:

  • Fixed an error when reading encrypted data that was written while running Python 2 for a path that includes non-ASCII characters.

  • Fixed an issue when reading or writing objects with a content-type like message/*. Previously, Swift would fail to respond.

  • Object expiration respects the expiring_objects_container_divisor config option.

  • fallocate_reserve may be specified as a percentage in more places.

  • The formpost middleware now works with Unicode file names.

  • Certain S3 API headers are now lower case as they would be coming from AWS.

• Improved how containers reclaim deleted rows to reduce locking and object update throughput.

• Fix a proxy-server error when retrieving erasure coded data when there are durable fragments but not enough to reconstruct.

• Fixed 500 from cname_lookup middleware. Previously, if the looked-up domain was used by domain_remap to update the request path, the server would respond Internal Error.

• The bulk extract middleware once again allows clients to specify metadata (including expiration timestamps) for all objects in the archive.

• Errors encountered while validating static symlink targets no longer cause BadResponseLength errors in the proxy-server.

• Fixed some SignatureDoesNotMatch errors when using the AWS .NET SDK.

• Various other minor bug fixes and improvements.

2.23.1

Bug Fixes

• On Python 3, the KMS keymaster now works with secrets stored in Barbican with a text/plain payload-content-type.

• Several utility scripts now work better on Python 3:

  • swift-account-audit

  • swift-dispersion-populate

  • swift-drive-recon

  • swift-recon

2.23.0

New Features

• Python 3.6 and 3.7 are now fully supported. If you’ve been testing Swift on Python 3, upgrade at your earliest convenience.

• Added “static symlinks”, which perform some validation as they follow redirects and include more information about their target in container listings. For more information, see the symlink middleware section of the documentation.

• Multi-character strings may now be used as delimiters in account and container listings.

Upgrade Notes

• Dependency update: eventlet must be at least 0.25.0. This also dragged forward minimum-supported versions of dnspython (1.15.0), greenlet (0.3.2), and six (1.10.0).

Bug Fixes

• Python 3 fixes:

  • Removed a request-smuggling vector when running a mixed py2/py3 cluster.

  • Allow fallocate_reserve to be specified as a percentage.

  • Fixed listings for sharded containers.

  • Fixed non-ASCII account metadata handling.

  • Fixed rsync output parsing.

  • Fixed some title-casing of headers.

  If you’ve been testing Swift on Python 3, upgrade at your earliest convenience.

• Sharding improvements

  • Container metadata related to sharding are now removed when no longer needed.

  • Empty container databases (such as might be created on handoffs) now shard much more quickly.

• The proxy-server now ignores 404 responses from handoffs that have no data when deciding on the correct response for object requests, similar to what it already does for account and container requests.

• Static Large Object sizes in listings for versioned containers are now more accurate.

• When refetching Static Large Object manifests, non-manifest responses are now handled better.

• S3 API now translates 503 Service Unavailable responses to a more S3-like response instead of raising an error.

• Improved proxy-to-backend requests to be more RFC-compliant.

• Various other minor bug fixes and improvements.

2.22.0

New Features

• Experimental support for Python 3.6 and 3.7 is now available. Note that this requires eventlet>=0.25.0. All unit tests pass, and running functional tests under Python 2 will pass against services running under Python 3. Expect full support in the next minor release.

• Log formats are now more configurable and include support for anonymization. See the log_msg_template option in proxy-server.conf and the Swift documentation for more information.

• Added an operator tool, swift-container-deleter, to asynchronously delete some or all objects in a container using the object expirers.

• Swift-all-in-one Docker images are now built and published to https://hub.docker.com/r/openstackswift/saio. These are intended for use as development targets, but will hopefully be useful as a starting point for other work involving containerizing Swift.

Upgrade Notes

• The object-expirer may now be configured in object-server.conf. This is in anticipation of a future change to allow the object-expirer to be deployed on all nodes that run the object-server.

• Dependency updates: we’ve increased our minimum supported version of cryptography to 2.0.2 and netifaces to 0.8. This is largely due to the difficulty of continuing to test with the old versions.

  If running Swift under Python 3, eventlet must be at least 0.25.0.

Bug Fixes

• Correctness improvements

  • The proxy-server now ignores 404 responses from handoffs without databases when deciding on the correct response for account and container requests.

  • Object writes to a container whose existence cannot be verified now 503 instead of 404.

• Sharding improvements

  • The container-replicator now only attempts to fetch shard ranges if the remote indicates that it has shard ranges. Further, it does so with a timeout to prevent the process from hanging in certain cases.

  • The proxy-server now caches ‘updating’ shards, improving write performance for sharded containers. A new config option, recheck_updating_shard_ranges, controls the cache time; set it to 0 to disable caching.

  • The container-replicator now correctly enqueues container-reconciler work for sharded containers.

• S3 API improvements

  • Unsigned payloads work with v4 signatures once more.

  • Multipart upload parts may now be copied from other multipart uploads.

  • CompleteMultipartUpload requests with a Content-MD5 now work.

  • Content-Type can now be updated when copying an object.

  • Fixed v1 listings that end with a non-ASCII object name.

• Background corruption-detection improvements

  • Detect and remove invalid entries from hashes.pkl

  • When object path is not a directory, just quarantine it, rather than the whole suffix.

• Various other minor bug fixes and improvements.