Adding Database Migrations

The migrations in sahara/db/migration/alembic_migrations/versions contain the changes needed to migrate between Sahara database revisions. A migration occurs by executing a script that details the changes needed to upgrade or downgrade the database. The migration scripts are ordered so that multiple scripts can run sequentially. The scripts are executed by Sahara’s migration wrapper which uses the Alembic library to manage the migration. Sahara supports migration from Icehouse or later.

Any code modifications that change the structure of the database require a migration script so that previously existing databases will continue to function when the new code is released. This page gives a brief overview of how to add the migration.

Generate a New Migration Script

New migration scripts can be generated using the sahara-db-manage command.

To generate a migration stub to be filled in by the developer:

$ sahara-db-manage --config-file /path/to/sahara.conf revision -m "description of revision"

To autogenerate a migration script that reflects the current structure of the database:

$ sahara-db-manage --config-file /path/to/sahara.conf revision -m "description of revision" --autogenerate

Each of these commands will create a file of the form revision_description where revision is a string generated by Alembic and description is based on the text passed with the -m option.

Follow the Sahara Naming Convention

By convention Sahara uses 3-digit revision numbers, and this scheme differs from the strings generated by Alembic. Consequently, it’s necessary to rename the generated script and modify the revision identifiers in the script.

Open the new script and look for the variable down_revision. The value should be a 3-digit numeric string, and it identifies the current revision number of the database. Set the revision value to the down_revision value + 1. For example, the lines:

# revision identifiers, used by Alembic.
revision = '507eb70202af'
down_revision = '006'

will become:

# revision identifiers, used by Alembic.
revision = '007'
down_revision = '006'

Modify any comments in the file to match the changes and rename the file to match the new revision number:

$ mv 507eb70202af_my_new_revision.py 007_my_new_revision.py

Add Alembic Operations to the Script

The migration script contains method upgrade(). Sahara has not supported downgrades since the Kilo release. Fill in this method with the appropriate Alembic operations to perform upgrades. In the above example, an upgrade will move from revision ‘006’ to revision ‘007’.

Command Summary for sahara-db-manage

You can upgrade to the latest database version via:

$ sahara-db-manage --config-file /path/to/sahara.conf upgrade head

To check the current database version:

$ sahara-db-manage --config-file /path/to/sahara.conf current

To create a script to run the migration offline:

$ sahara-db-manage --config-file /path/to/sahara.conf upgrade head --sql

To run the offline migration between specific migration versions:

$ sahara-db-manage --config-file /path/to/sahara.conf upgrade <start version>:<end version> --sql

To upgrade the database incrementally:

$ sahara-db-manage --config-file /path/to/sahara.conf upgrade --delta <# of revs>

To create a new revision:

$ sahara-db-manage --config-file /path/to/sahara.conf revision -m "description of revision" --autogenerate

To create a blank file:

$ sahara-db-manage --config-file /path/to/sahara.conf revision -m "description of revision"

This command does not perform any migrations, it only sets the revision. Revision may be any existing revision. Use this command carefully:

$ sahara-db-manage --config-file /path/to/sahara.conf stamp <revision>

To verify that the timeline does branch, you can run this command:

$ sahara-db-manage --config-file /path/to/sahara.conf check_migration

If the migration path does branch, you can find the branch point via:

$ sahara-db-manage --config-file /path/to/sahara.conf history