Setting up a development environment

This section describes how to setup a working Python development environment that you can use in developing Zaqar on Ubuntu or Fedora. These instructions assume that you are familiar with Git. Refer to GettingTheCode for additional information.

Virtual environments

Use virtualenv to track and manage Python dependencies for developing and testing Zaqar. Using virtualenv enables you to install Python dependencies in an isolated virtual environment, instead of installing the packages at the system level.

Note

Virtualenv is useful for development purposes, but is not typically used for full integration testing or production usage. If you want to learn about production best practices, check out the OpenStack Operations Guide.

Install GNU/Linux system dependencies

Note

This section is tested for Zaqar on Ubuntu 14.04 (Trusty) and Fedora-based (RHEL 6.1) distributions. Feel free to add notes and change according to your experiences or operating system. Learn more about contributing to Zaqar documentation in the Welcome new contributors manual.

Install the prerequisite packages.

On Ubuntu:

$ sudo apt-get install gcc python-pip libxml2-dev libxslt1-dev python-dev zlib1g-dev

On Fedora-based distributions (e.g., Fedora/RHEL/CentOS):

$ sudo yum install gcc python-pip libxml2-devel libxslt-devel python-devel

Install MongoDB

You also need to have MongoDB installed and running.

On Ubuntu, follow the instructions in the MongoDB on Ubuntu Installation Guide.

On Fedora-based distributions, follow the instructions in the MongoDB on Red Hat Enterprise, CentOS, Fedora, or Amazon Linux Installation Guide.

Note

If you are Contributor and plan to run Unit tests on Zaqar, you may want to add this line to mongodb configuration file (etc/mongod.conf or etc/mongodb.conf depending on distribution):

smallfiles = true

Many Zaqar’s Unit tests do not clean up their testing databases after executing. And database files consume much disk space even if they do not contain any records. This behavior will be fixed soon.

Getting the code

Get the code from git.openstack.org to create a local repository with Zaqar:

$ git clone https://git.openstack.org/openstack/zaqar.git

Configuration

  1. From your home folder create the ~/.zaqar folder. This directory holds the configuration files for Zaqar:

    $ mkdir ~/.zaqar

  2. Generate the sample configuration file zaqar/etc/zaqar.conf.sample:

    $ pip install tox
$ cd zaqar
$ tox -e genconfig

  3. Copy the Zaqar configuration samples to the directory ~/.zaqar/:

    $ cp etc/zaqar.conf.sample ~/.zaqar/zaqar.conf
$ cp etc/logging.conf.sample ~/.zaqar/logging.conf

  4. Find the [drivers] section in ~/.zaqar/zaqar.conf and specify mongodb as the message store:

    message_store = mongodb
management_store = mongodb

  5. Then find [drivers:message_store:mongodb] and [drivers:management_store:mongodb] sections and specify the URI to point to your local mongodb instance by adding this line to both the sections:

    uri = mongodb://$MONGODB_HOST:$MONGODB_PORT

    By default you will have:

    uri = mongodb://127.0.0.1:27017

    This URI points to single mongodb node which of course is not reliable, so you need to set in the [default] section of configuration file:

    unreliable = True

    For your reference, you can omit this parameter or set it to False only if the provided URI to your mongodb is actually the URI to mongodb Replica Set or Mongos. Also it must have “Write concern” parameter set to majority or to a number more than 1.

    For example, URI to reliable mongodb can look like this:

    uri = mongodb://mydb0,mydb1,mydb2:27017/?replicaSet=foo&w=2

    Where mydb0, mydb1, mydb2 are addresses of the configured mongodb Replica Set nodes, replicaSet (Replica Set name) parameter is set to foo, w (Write concern) parameter is set to 2.

  6. For logging, find the [handler_file] section in ~/.zaqar/logging.conf and modify as desired:

    args=('zaqar.log', 'w')

Installing and using virtualenv

  1. Install virtualenv by running:

    $ pip install virtualenv

  2. Create and activate a virtual environment:

    $ virtualenv zaqarenv
$ source zaqarenv/bin/activate

  3. Install Zaqar:

    $ pip install -e .

  4. Install the required Python binding for MongoDB:

    $ pip install pymongo

  5. Start Zaqar server in info logging mode:

    $ zaqar-server -v

    Or you can start Zaqar server in debug logging mode:

    $ zaqar-server -d

  6. Verify Zaqar is running by creating a queue via curl. In a separate terminal run:

    $ curl -i -X PUT http://localhost:8888/v2/queues/samplequeue -H "Content-type: application/json" -H 'Client-ID: 123e4567-e89b-12d3-a456-426655440000' -H 'X-PROJECT-ID: 12345'

    Note

    Client-ID expects a valid UUID.

    X-PROJECT-ID expects a user-defined project identifier.

  7. Get ready to code!

Note

You can run the Zaqar server in the background by passing the --daemon flag:

$ zaqar-server -v --daemon

But with this method you will not get immediate visual feedback and it will be harder to kill and restart the process.

Troubleshooting

No handlers found for zaqar.client (…)

This happens because the current user cannot create the log file (for the default configuration in /var/log/zaqar/server.log). To solve it, create the folder:

$ sudo mkdir /var/log/zaqar

Create the file:

$ sudo touch /var/log/zaqar/server.log

And try running the server again.

DevStack

If you want to use Zaqar in an integrated OpenStack developing environment, you can add it to your DevStack deployment.

To do this, you first need to add the following setting to your local.conf:

enable_plugin zaqar https://git.openstack.org/openstack/zaqar

Then run the stack.sh script as usual.

Running tests

See Running tests for details.

Running the benchmarking tool

See Running benchmark for details.

Contributing your work

See Welcome new contributors and Your first patch for details.