Install Guide¶
Install from PyPI (you may want to use virtualenv to isolate your environment):
pip install ironic-inspector
Also there is a DevStack plugin for ironic-inspector - see How To Contribute for the current status.
Finally, some distributions (e.g. Fedora) provide ironic-inspector packaged, some of them - under its old name ironic-discoverd.
There are several projects you can use to set up ironic-inspector in production. puppet-ironic provides Puppet manifests, while bifrost provides an Ansible-based standalone installer. Refer to Configuration if you plan on installing ironic-inspector manually.
Note
Please beware of possible DNS issues when installing ironic-inspector on Ubuntu.
Version Support Matrix¶
ironic-inspector currently requires the Bare Metal API version
1.11
to be provided by ironic. This version is available starting
with the Liberty release of ironic.
Here is a mapping between the ironic versions and the supported ironic-inspector versions. The Standalone column shows which ironic-inspector versions can be used in standalone mode with each ironic version. The Inspection Interface column shows which ironic-inspector versions can be used with the inspection interface in each version of ironic.
Ironic Version |
Standalone |
Inspection Interface |
---|---|---|
Juno |
1.0 |
N/A |
Kilo |
1.0 - 2.2 |
1.0 - 1.1 |
Liberty |
1.1 - 2.2.7 |
2.0 - 2.2.7 |
Mitaka |
2.3 - 3.X |
2.3 - 3.X |
Newton |
3.3 - 4.X |
3.3 - 4.X |
Ocata+ |
5.0 - 5.X |
5.0 - 5.X |
Note
3.X
means there are no specific plans to deprecate support for this
ironic version. This does not imply that it will be supported forever.
Sample Configuration Files¶
To generate a sample configuration file, run the following command from the top level of the code tree:
tox -egenconfig
For a pre-generated sample configuration file, see Ironic Inspector Configuration Options.
To generate a sample policy file, run the following command from the top level of the code tree:
tox -egenpolicy
For a pre-generated sample configuration file, see Ironic Inspector Policy.
Installation options¶
Starting with Train release, ironic-inspector can run in a non-standalone mode, which means ironic-inspector API and ironic-inspector conductor are separated services, they can be installed on the same host or different hosts.
Following are some considerations when you run ironic-inspector in non-standalone mode:
Additional packages may be required depending on the tooz backend used in the installation. For example,
etcd3gw
is required if the backend driver is configured to useetcd3+http://
,pymemcache
is required to usememcached://
. Some distributions may provide packages likepython3-etcd3gw
orpython3-memcache
. Supported drivers are listed at Tooz drivers.For ironic-inspector running in non-standalone mode, PXE configuration is only required on the node where ironic-inspector conductor service is deployed.
Switch to a database backend other than sqlite.
Configuration¶
Copy the sample configuration files to some permanent place
(e.g. /etc/ironic-inspector/inspector.conf
).
Fill in these minimum configuration values:
The
standalone
in theDEFAULT
section - This determines whether ironic-inspector services are intended to be deployed separately.The
keystone_authtoken
section - credentials to use when checking user authentication.The
ironic
section - credentials to use when accessing ironic API.connection
in thedatabase
section - SQLAlchemy connection string for the database. By default ironic-inspector uses sqlite as the database backend, if you are running ironic-inspector in a non-standalone mode, please change to other database backends.dnsmasq_interface
in theiptables
section - interface on whichdnsmasq
(or another DHCP service) listens for PXE boot requests (defaults tobr-ctlplane
which is a sane default for tripleo-based installations but is unlikely to work for other cases).if you wish to use the
dnsmasq
PXE/DHCP filter driver rather than the defaultiptables
driver, see the dnsmasq PXE filter description.store_data
in theprocessing
section defines where introspection data is stored and takes one of three values:none
introspection data is not stored (the default)
database
introspection data is stored in the database (recommended for standalone deployments)
swift
introspection data is stored in the Object Store service (recommended for full openstack deployments)
Note
It is possible to create third party storage backends using the
ironic_inspector.introspection_data.store
entry point.
See comments inside the sample configuration for other possible configuration options.
Note
Configuration file contains a password and thus should be owned by root
and should have access rights like 0600
.
Here is an example inspector.conf (adapted from a gate run):
[DEFAULT]
debug = false
rootwrap_config = /etc/ironic-inspector/rootwrap.conf
[database]
connection = mysql+pymysql://root:<PASSWORD>@127.0.0.1/ironic_inspector?charset=utf8
[pxe_filter]
driver=iptables
[iptables]
dnsmasq_interface = br-ctlplane
[ironic]
os_region = RegionOne
project_name = service
password = <PASSWORD>
username = ironic-inspector
auth_url = http://127.0.0.1/identity
auth_type = password
[keystone_authtoken]
www_authenticate_uri = http://127.0.0.1/identity
project_name = service
password = <PASSWORD>
username = ironic-inspector
auth_url = http://127.0.0.1/identity_v2_admin
auth_type = password
[processing]
ramdisk_logs_dir = /var/log/ironic-inspector/ramdisk
store_data = swift
[swift]
os_region = RegionOne
project_name = service
password = <PASSWORD>
username = ironic-inspector
auth_url = http://127.0.0.1/identity
auth_type = password
Note
Set debug = true
if you want to see complete logs.
ironic-inspector requires root rights for managing iptables
. It
gets them by running ironic-inspector-rootwrap
utility with sudo
.
To allow it, copy file rootwrap.conf
and directory rootwrap.d
to the
configuration directory (e.g. /etc/ironic-inspector/
) and create file
/etc/sudoers.d/ironic-inspector-rootwrap
with the following content:
Defaults:stack !requiretty
stack ALL=(root) NOPASSWD: /usr/bin/ironic-inspector-rootwrap /etc/ironic-inspector/rootwrap.conf *
Danger
Be very careful about typos in /etc/sudoers.d/ironic-inspector-rootwrap
as any typo will break sudo for ALL users on the system. Especially,
make sure there is a new line at the end of this file.
Note
rootwrap.conf
and all files in rootwrap.d
must be writeable
only by root.
Note
If you store rootwrap.d
in a different location, make sure to update
the filters_path option in rootwrap.conf
to reflect the change.
If your rootwrap.conf
is in a different location, then you need
to update the rootwrap_config option in ironic-inspector.conf
to point to that location.
Replace stack
with whatever user you’ll be using to run
ironic-inspector.
Configuring IPA¶
ironic-python-agent is a ramdisk developed for ironic and support for ironic-inspector was added during the Liberty cycle. This is the default ramdisk starting with the Mitaka release.
Note
You need at least 2 GiB of RAM on the machines to use IPA built with diskimage-builder and at least 384 MiB to use the TinyIPA.
To build an ironic-python-agent ramdisk, use ironic-python-agent-builder. Alternatively, you can download a prebuild image.
For local testing and CI purposes you can use a TinyIPA image.
Configuring PXE¶
For the PXE boot environment, you’ll need:
TFTP server running and accessible (see below for using dnsmasq). Ensure
pxelinux.0
is present in the TFTP root.Copy
ironic-python-agent.kernel
andironic-python-agent.initramfs
to the TFTP root as well.Next, setup
$TFTPROOT/pxelinux.cfg/default
as follows:default introspect label introspect kernel ironic-python-agent.kernel append initrd=ironic-python-agent.initramfs ipa-inspection-callback-url=http://{IP}:5050/v1/continue systemd.journald.forward_to_console=yes ipappend 3
Replace
{IP}
with IP of the machine (do not use loopback interface, it will be accessed by ramdisk on a booting machine).Note
While
systemd.journald.forward_to_console=yes
is not actually required, it will substantially simplify debugging if something goes wrong. You can also enable IPA debug logging by appendingipa-debug=1
.IPA is pluggable: you can insert introspection plugins called collectors into it. For example, to enable a very handy
logs
collector (sending ramdisk logs to ironic-inspector), modify theappend
line in$TFTPROOT/pxelinux.cfg/default
:append initrd=ironic-python-agent.initramfs ipa-inspection-callback-url=http://{IP}:5050/v1/continue ipa-inspection-collectors=default,logs systemd.journald.forward_to_console=yes
Note
You probably want to always keep the
default
collector, as it provides the basic information required for introspection.You need PXE boot server (e.g. dnsmasq) running on the same machine as ironic-inspector. Don’t do any firewall configuration: ironic-inspector will handle it for you. In ironic-inspector configuration file set
dnsmasq_interface
to the interface your PXE boot server listens on. Here is an example dnsmasq.conf:port=0 interface={INTERFACE} bind-interfaces dhcp-range={DHCP IP RANGE, e.g. 192.168.0.50,192.168.0.150} enable-tftp tftp-root={TFTP ROOT, e.g. /tftpboot} dhcp-boot=pxelinux.0 dhcp-sequential-ip
Note
dhcp-sequential-ip
is used because otherwise a lot of nodes booting simultaneously cause conflicts - the same IP address is suggested to several nodes.
Configuring iPXE¶
iPXE allows better scaling as it primarily uses the HTTP protocol instead of slow and unreliable TFTP. You still need a TFTP server as a fallback for nodes not supporting iPXE. To use iPXE, you’ll need:
TFTP server running and accessible (see above for using dnsmasq). Ensure
undionly.kpxe
is present in the TFTP root. If any of your nodes boot with UEFI, you’ll also needipxe.efi
there.You also need an HTTP server capable of serving static files. Copy
ironic-python-agent.kernel
andironic-python-agent.initramfs
there.Create a file called
inspector.ipxe
in the HTTP root (you can name and place it differently, just don’t forget to adjust the dnsmasq.conf example below):#!ipxe :retry_dhcp dhcp || goto retry_dhcp :retry_boot imgfree kernel --timeout 30000 http://{IP}:8088/ironic-python-agent.kernel ipa-inspection-callback-url=http://{IP}>:5050/v1/continue systemd.journald.forward_to_console=yes BOOTIF=${mac} initrd=agent.ramdisk || goto retry_boot initrd --timeout 30000 http://{IP}:8088/ironic-python-agent.ramdisk || goto retry_boot boot
Note
Older versions of the iPXE ROM tend to misbehave on unreliable network connection, thus we use the timeout option with retries.
Just like with PXE, you can customize the list of collectors by appending the
ipa-inspector-collectors
kernel option. For example:ipa-inspection-collectors=default,logs,extra_hardware
Just as with PXE, you’ll need a PXE boot server. The configuration, however, will be different. Here is an example dnsmasq.conf:
port=0 interface={INTERFACE} bind-interfaces dhcp-range={DHCP IP RANGE, e.g. 192.168.0.50,192.168.0.150} enable-tftp tftp-root={TFTP ROOT, e.g. /tftpboot} dhcp-sequential-ip dhcp-match=ipxe,175 dhcp-match=set:efi,option:client-arch,7 dhcp-match=set:efi,option:client-arch,9 dhcp-match=set:efi,option:client-arch,11 # dhcpv6.option: Client System Architecture Type (61) dhcp-match=set:efi6,option6:61,0007 dhcp-match=set:efi6,option6:61,0009 dhcp-match=set:efi6,option6:61,0011 dhcp-userclass=set:ipxe6,iPXE # Client is already running iPXE; move to next stage of chainloading dhcp-boot=tag:ipxe,http://{IP}:8088/inspector.ipxe # Client is PXE booting over EFI without iPXE ROM, # send EFI version of iPXE chainloader dhcp-boot=tag:efi,tag:!ipxe,ipxe.efi dhcp-option=tag:efi6,tag:!ipxe6,option6:bootfile-url,tftp://{IP}/ipxe.efi # Client is running PXE over BIOS; send BIOS version of iPXE chainloader dhcp-boot=undionly.kpxe,localhost.localdomain,{IP}
First, we configure the same common parameters as with PXE. Then we define
ipxe
andefi
tags for IPv4 andipxe6
andefi6
for IPv6. Nodes already supporting iPXE are ordered to download and executeinspector.ipxe
. Nodes without iPXE booted with UEFI will getipxe.efi
firmware to execute, while the remaining will getundionly.kpxe
.
Managing the ironic-inspector Database¶
ironic-inspector provides a command line client for managing its database. This client can be used for upgrading, and downgrading the database using alembic migrations.
If this is your first time running ironic-inspector to migrate the database, simply run:
ironic-inspector-dbsync --config-file /etc/ironic-inspector/inspector.conf upgrade
If you have previously run a version of ironic-inspector earlier than
2.2.0, the safest thing is to delete the existing SQLite database and run
upgrade
as shown above. However, if you want to save the existing
database, to ensure your database will work with the migrations, you’ll need to
run an extra step before upgrading the database. You only need to do this the
first time running version 2.2.0 or later.
If you are upgrading from ironic-inspector version 2.1.0 or lower:
ironic-inspector-dbsync --config-file /etc/ironic-inspector/inspector.conf stamp --revision 578f84f38d
ironic-inspector-dbsync --config-file /etc/ironic-inspector/inspector.conf upgrade
If you are upgrading from a git master install of the ironic-inspector after rules were introduced:
ironic-inspector-dbsync --config-file /etc/ironic-inspector/inspector.conf stamp --revision d588418040d
ironic-inspector-dbsync --config-file /etc/ironic-inspector/inspector.conf upgrade
Other available commands can be discovered by running:
ironic-inspector-dbsync --help
Running¶
Running in standalone mode¶
Execute:
ironic-inspector --config-file /etc/ironic-inspector/inspector.conf
Running in non-standalone mode¶
API service can be started in development mode with:
ironic-inspector-api-wsgi -p 5050 -- --config-file /etc/ironic-inspector/inspector.conf
For production, the ironic-inspector API service should be hosted under a web service. Below is a sample configuration for Apache with module mod_wsgi:
Listen 5050
<VirtualHost *:5050>
WSGIDaemonProcess ironic-inspector user=stack group=stack threads=10 display-name=%{GROUP}
WSGIScriptAlias / /usr/local/bin/ironic-inspector-api-wsgi
SetEnv APACHE_RUN_USER stack
SetEnv APACHE_RUN_GROUP stack
WSGIProcessGroup ironic-inspector
ErrorLog /var/log/apache2/ironic_inspector_error.log
LogLevel info
CustomLog /var/log/apache2/ironic_inspector_access.log combined
<Directory /opt/stack/ironic-inspector/ironic_inspector/cmd>
WSGIProcessGroup ironic-inspector
WSGIApplicationGroup %{GLOBAL}
AllowOverride All
Require all granted
</Directory>
</VirtualHost>
You can refer to ironic installation document for more guides.
ironic-inspector conductor can be started with:
ironic-inspector-conductor --config-file /etc/ironic-inspector/inspector.conf