Rally v0.1.0¶

Information¶

Commits	355
Bug fixes	90
Dev cycle	132 days
Release date	25/September/2015

Details¶

This release contains new features, new 42 plugins, 90 bug fixes, various code and API improvements.

New Features & API changes¶

Improved installation script
- Add parameters:
  - --develop parameter to install rally in editable (develop) mode
  - --no-color to switch off output colorizing useful for automated output parsing and terminals that don't support colors.
- Puts rally.conf under virtualenv etc/rally/ so you can have several rally installations in virtualenv
- Many fixes related to access of different file, like: rally.conf, rally db file in case of sqlite
- Update pip before Rally installation
- Fix reinstallation
Separated Rally plugins & framework

Now plugins are here: https://github.com/openstack/rally/tree/master/rally/plugins

Plugins are as well separated common/* for common plugins that can be use no matter what is tested and OpenStack related plugins

New Rally Task framework

All plugins has the same Plugin base: rally.common.plugin.pluing.Plugin They have the same mechanisms for: discovering, providing information based on docstrings, and in future they will use the same deprecation/rename mechanism.
Some of files are moved:
- rally/benchmark -> rally/task
  
  This was done to unify naming of rally task command and actually code that implements it.
- rally/benchmark/sla/base.py -> rally/task/sla.py
- rally/benchmark/context/base.py -> rally/task/context.py
- rally/benchmark/scenarios/base.py -> rally/task/scenario.py
- rally/benchmark/runners/base.py -> rally/task/runner.py
- rally/benchmark/scenarios/utils.py -> rally/task/utils.py
This was done to:
- avoid doing rally.benchmark.scenarios import base as scenario_base
- remove one level of nesting
- simplify framework structure
Some of classes and methods were renamed
- Plugin configuration:
  - context.context() -> context.configure()
  - scenario.scenario() -> scenario.configure()
  - Introduced runner.configure()
  - Introduced sla.configure()
  This resolves 3 problems:
  - Unifies configuration of different types of plugins
  - Simplifies plugin interface
  - Looks nice with new modules path:
    >>> from rally.task import scenario >>> @scenario.configure()
Atomic Actions were changed:
- New rally.task.atomic module
  
  This allow us in future to reuse atomic actions in Context plugins
- Renames:
  
  rally.benchmark.scenarios.base.AtomicAction -> rally.task.atomic.ActionTimer
  
  rally.benchmark.scenarios.base.atomic_action() -> rally.task.atomic.action_timer()
Context plugins decide how to map their data for scenario

Now Context.map_for_scenario method can be override to decide how to pass context object to each iteration of scenario.

Samples of NEW vs OLD context, sla, scenario and runner plugins:

Context

# Old
from rally.benchmark.context import base

@base.context(name="users", order=100)
class YourContext(base.Context):

    def setup(self):
        # ...

    def cleanup(self):
        # ...

# New
from rally.task import context

@context.configure(name="users", order=100)
class YourContext(context.Context):

    def setup(self):
        # ...

    def cleanup(self):
        # ...

    def map_for_scenario(self):
        # Maps context object to the scenario context object
        # like context["users"] -> context["user"] and so on.

Scenario

# Old Scenario

from rally.benchmark.scenarios import base
from rally.benchmark import validation

class ScenarioPlugin(base.Scenario):

    @base.scenario()
    def some(self):
        self._do_some_action()


    @base.atomic_action_timer("some_timer")
    def _do_some_action(self):
        # ...

# New Scenario

from rally.task import atomic
from rally.task import scenario
from rally.task import validation

# OpenStack scenario has different base now:
# rally.plugins.openstack.scenario.OpenStackScenario
class ScenarioPlugin(scenario.Scenario):

    @scenario.configure()
    def some(self):
        self._do_some_action()

    @atomic.action_timer("some_action")
    def _do_some_action(self):
        # ...

Runner

## Old

from rally.benchmark.runners import base

class SomeRunner(base.ScenarioRunner):

     __execution_type__ = "some_runner"

    def _run_scenario(self, cls, method_name, context, args)
      # Load generation

    def abort(self):
      # Method that aborts load generation

## New

from rally.task import runner

@runner.configure(name="some_runner")
class SomeRunner(runner.ScenarioRunner):

    def _run_scenario(self, cls, method_name, context, args)
        # Load generation

    def abort(self):
        # Method that aborts load generation

SLA

# Old

from rally.benchmark import sla

class FailureRate(sla.SLA):
    # ...

# New

from rally.task import sla

@sla.configure(name="failure_rate")
class FailureRate(sla.SLA):
    # ...

Rally Task aborted command

Finally you can gracefully shutdown running task by calling:
```
rally task abort <task_uuid>
```
Rally CLI changes
- [add] rally --plugin-paths specify the list of directories with plugins
- [add] rally task report --junit - generate a JUnit report This allows users to feed reports to tools such as Jenkins.
- [add] rally task abort - aborts running Rally task when run with the --soft key, the rally task abort command is waiting until the currently running subtask is finished, otherwise the command interrupts subtask immediately after current scenario iterations are finished.
- [add] rally plugin show prints detailed information about plugin
- [add] rally plugin list prints table with rally plugin names and titles
- [add] rally verify genconfig generates tempest.conf without running it.
- [add] rally verify install install tempest for specified deployment
- [add] rally verify reinstall removes tempest for specified deployment
- [add] rally verify uninstall uninstall tempest of specified deployment
- [fix] rally verify start --no-use --no-use was always turned on
- [remove] rally use now each command has subcommand use
- [remove] rally info
- [remove] rally-manage tempest now it is covered by rally verify
New Rally task reports
- New code is based on OOP style which is base step to make pluggable Reports
- Reports are now generated for only one iteration over the resulting data which resolves scalability issues when we are working with large amount of iterations.
- New Load profiler plot that shows amount of iterations that are working in parallel
- Failed iterations are shown as a red areas on stacked are graphic.

Non backward compatible changes¶

[remove] rally use cli command
[remove] rally info cli command
[remove] --uuid parameter from rally deployment <any>
[remove --deploy-id parameter from: rally task <any>, rally verify <any>, rally show <any>

Specs & Feature requests¶

[feature request] Explicitly specify existing users for scenarios

[feature request] Improve install script and add --uninstall and --version

[feature request] Allows specific repos & packages in install-rally.sh

[feature request] Add ability to capture logs from tested services

[feature request] Check RPC queue perfdata

[spec] Refactoring Rally cleanup

[spec] Consistent resource names

Plugins¶

Scenarios:

[new] CinderVolumes.create_volume_backup

[new] CinderVolumes.create_and_restore_volume_backup

[new] KeystoneBasic.add_and_remove_user_role

[new] KeystoneBasic.create_and_delete_role

[new] KeystoneBasic.create_add_and_list_user_roles

[new] FuelEnvironments.list_environments

[new] CinderVolumes.modify_volume_metadata

[new] NovaServers.boot_and_delete_multiple_servers

[new] NeutronLoadbalancerV1.create_and_list_pool

[new] ManilaShares.list_shares

[new] CeilometerEvents.create_user_and_get_event

[new] CeilometerEvents.create_user_and_list_event_types

[new] CeilometerEvents.create_user_and_list_events

[new] CeilometerTraits.create_user_and_list_trait_descriptions

[new] CeilometerTraits.create_user_and_list_traits

[new] NeutronLoadbalancerV1.create_and_delete_pools

[new] NeutronLoadbalancerV1.create_and_update_pools

[new] ManilaShares.create_and_delete_share

[new] ManilaShares.create_share_network_and_delete

[new] ManilaShares.create_share_network_and_list

[new] HeatStacks.create_and_delete_stack

[new] ManilaShares.list_share_servers

[new] HeatStacks.create_snapshot_restore_delete_stack

[new] KeystoneBasic.create_and_delete_ec2credential

[new] KeystoneBasic.create_and_list_ec2credentials

[new] HeatStacks.create_stack_and_scale

[new] ManilaShares.create_security_service_and_delete

[new] KeystoneBasic.create_user_set_enabled_and_delete

[new] ManilaShares.attach_security_service_to_share_network

[new] IronicNodes.create_and_delete_node

[new] IronicNodes.create_and_list_node

[new] CinderVolumes.create_and_list_volume_backups

[new] NovaNetworks.create_and_list_networks

[new] NovaNetworks.create_and_delete_network

[new] EC2Servers.list_servers

[new] VMTasks.boot_runcommand_delete_custom_imagea

[new] CinderVolumes.create_and_update_volume
Contexts:

[new] ManilaQuotas

Add context for setting up Manila quotas: shares, gigabytes, snapshots, snapshot_gigabytes, share_networks

[new] ManilaShareNetworks

Context for share networks that will be used in case of usage deployment with existing users. Provided share networks via context option "share_networks" will be balanced between all share creations of scenarios.

[new] Lbaas

Context to create LBaaS-v1 resources

[new] ImageCommandCustomizerContext

Allows image customization using side effects of a command execution. E.g. one can install an application to the image and use these image for 'boot_runcommand_delete' scenario afterwards.

[new] EC2ServerGenerator

Context that creates servers using EC2 api

[new] ExistingNetwork

This context lets you use existing networks that have already been created instead of creating new networks with Rally. This is useful when, for instance, you are using Neutron with a dumb router that is not capable of creating new networks on the fly.
SLA:

[remove] max_failure_rate - use failure_rate instead

Bug fixes¶

90 bugs were fixed, the most critical are:

Many fixes related that fixes access of rally.conf and DB files
Incorrect apt-get "-yes" parameter in install_rally.sh script
Rally bash completion doesn't exist in a virtualenv
Rally show networks CLI command worked only with nova networks
RPS runner was not properly generating load
Check is dhcp_agent_scheduler support or not in network cleanup
NetworkContext doesn't work with Nova V2.1
Rally task input file was not able to use jinja2 include directive
Rally in docker image was not able to
Rally docker image didn't contain samples
Do not update the average duration when iteration failed

Documentation¶

Add plugin reference page

Rally Plugins Reference page page contains a full list with
Add maintainers section on project info page

Rally Maintainers section contains information about core contributors of OpenStack Rally their responsibilities and contacts. This will help us to make our community more transparent and open for newbies.
Added who is using section in docs
Many small fixes

OpenStack

Rally v0.1.0¶

Information¶

Details¶

New Features & API changes¶

Non backward compatible changes¶

Specs & Feature requests¶

Plugins¶

Bug fixes¶

Documentation¶

What is Rally?

Contents

Contacts

Useful links

OpenStack

Rally v0.1.0¶

Information¶

Details¶

New Features & API changes¶

Non backward compatible changes¶

Specs & Feature requests¶

Plugins¶

Bug fixes¶

Documentation¶

What is Rally?

Contents

Contacts

Useful links

Quick search