2023.2 Series (9.5.0 - 9.7.x) Release Notes¶

9.7.2¶

Deployers implementing their own HardwareManagers must to audit their code for unsafe uses of qemu-img and related methods.

Ironic-Python-Agent now checks any supplied image format value against the detected format of the image file and will prevent deployments should the values mismatch.

Images previously misconfigured as raw despite being in another format, in some non-default configurations, may have been mistakenly converted if needed. Ironic-Python-Agent will no longer perform conversion in any case for images with metadata indicating in raw format.

Ironic-Python-Agent always inspects any non-raw user image content for safety before running any qemu-based utilities on the image. This is utilized to identify the format of the image and to verify the overall safety of the image. Any images with unknown or unsafe feature uses are explicitly rejected. This can be disabled in both IPA and Ironic by setting [conductor]disable_deep_image_inspection to True for the Ironic deployment. Image inspection is the primary mitigation for CVE-2024-44082 being tracked in bug 2071740. Operators may desire to set [conductor]conductor_always_validates_images on Ironic conductors to mitigate the issue before they have upgraded their Ironic-Python-Agent.

Ironic-Python-Agent now explicitly enforces a list of permitted image types for deployment, defaulting to “raw” and “qcow2”. Other image types may work, but are not explicitly supported and must be enabled. This can be modified by setting [conductor]permitted_image_formats for all Ironic services.

Fixes an issue where configuration drive volumes which are mounted by the operating system could remain mounted and cause a lock to be held, which may conflict with actions such as rebuild. The agent now always makes sure the folder used by Glean and Cloud-init is not mounted.

Fixes multiple issues in the handling of images as it related to execution of the qemu-img utility. When using this utility to convert an unsafe image, a malicious user can extract information from a node while Ironic-Python-Agent is deploying or converting an image. Ironic-Python-Agent now inspects all non-raw images for safety, and never runs qemu-based utilities on raw images. This fix is tracked as CVE-2024-44082 and bug 2071740.

Images with metadata indicating a “raw” disk format may have been transparently converted from another format. Now, these images will have their exact contents imaged to disk without modification.

Fixes bug 2066308, an issue where Ironic Python Agent would call evaluate_hardware_support multiple times on hardware manager plugins. Scanning for hardware and disks is time consuming, and caused timeouts on badly-performing nodes.

Fixes a failure case where downloads would not be retried when the checksum fails verification. the agent now includes the checksum activity as part of the file download operation, and will automatically retry downloads when the checksum fails in accordance with the existing download retry logic. This is largely in response to what appears to be intermittent transport failures at lower levels which we cannot otherwise detect.

Fixes missing Content-Type header when sending inspection data back to ironic-inspector or ironic. While ironic-inspector tolerates the missing header, it may cause issues with the new inspection implementation.

The default timeout value for the agent to lookup itself in an Ironic deployment has been extended to 600 seconds from 300 seconds. This is to provide better stability for Ironic deployments under heavy load which may be unable to service new requests. This is particularly true when the backing database is SQLite for Ironic due to the limited write concurrency of the database.

Fixes referencing to raid_device variable before assignment, is replaced by blk variable.

Inspection is now retried on HTTP 409 (conflict), which can be returned by the new implementation in Ironic.

The error handling of the multipathd service startup/discovery process. IPA handles both scenario when the multipathd service is already started and the scenario when the service has not been started and in the second scenario IPA will try to start the service. IPA is not pre checking whether multipathd is running already or not, it will start the multipathd service even if it is already running and expects 0 error code . It has been noticed that with certain combinations of Linux distros and multipathd versions the error code is not 0 when IPA tries to start multipathd in case an instance of multipathd is already running. When the expected return code is not 0 an exception will be thrown and that will cause the multipath device discovery to terminate prematurely and if the selected root device is a multipath device then IPA won’t be able to provision. This fix discards the exception that is caused by the non 0 error code returned by the multipathd startup process. In case there is a genuine issue with the multipath service, that would be caught when the actual multipath device listing command is executed (multipath -ll).

Fixes an issue with rebuilding instances on Software RAID with RAIDed ESP partitions.

Adds a new service extension which facilitates command handling for Ironic to retrieve a list of service steps.

Adds a new base method to base HardwareManager, get_service_steps which works the same as get_clean_steps and get_deploy_steps. These methods can be extended by hardware managers to permit them to signal what steps are permitted.

Extends reasonable deploy/clean steps to also be service steps which are embedded in the Ironic agent. For example, CPU, Network, and Memory burnin steps are available as service steps, but not the disk burnin step as that would likely result in the existing disk contents being damaged.

Fixes a failure case where a deployed instance may be unable to access the configuration drive post-deployment. This can occur when block devices only support 4KB IO interactions. When 4KB block IO sizes are in use, the ISO9660 filesystem driver in Linux cannot be used as it is modeled around a 2KB block. We now attempt to verify, and rebuild the configuration drive on a FAT filesystem when we cannot mount the supplied configuration drive. Operators can force the agent to write configuration drives using the FAT filesystem using the [DEFAULT]config_drive_rebuild option.

Fixes, or at least lessens the case where a running Ironic agent can stack up numerous lookup requests against an Ironic deployment when a node is locked. In particular, this is beause the lookup also drives generation of the agent token, which requires the conductor to allocate a worker, and generate the token, and return the result to the API client. Ironic’s retry logic will now wait up to 60 seconds, and if an HTTP Conflict (409) message is received, the agent will automatically pause lookup operations for thirty seconds as opposed continue to attempt lookups which could create more work for the Ironic deployment needlessly.