In our first post on database testing, we saw how to write and run tests in our sample web project. It's now time to see how to integrate all this within Jenkins!

Create a new job

First of all, create a New Job, enter its name (here djangotutorial) and select Build multi-configuration project before validating.

Basic setup

As usual, setup:

• The description.
• The source repository, here https://github.com/shiningpanda/djangotutorial-database.git.
• The build trigger policy: checking for modifications every five minutes in this example.

Axis

Axis provide parameters to the build. At least two axis are required for this project:

• The SQL one to specify the database server to use,
• The Python one to specify the Python interpreter running the tests.

SQL axis

To create an SQL axis, click on Add axis in Configuration Matrix section and select SQL.

Then select all the desired database servers.

Python axis

Click once more on Add axis and select Python to be able to select the required Python interpreters.

Feel free to select more than one interpreter to test your project on multiple versions of Python.

Builder

Create database

Let's go back to the source to have a look at the settings:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': 'djangotutorial',
    }
}

By default the name of the test database is computed by prefixing the NAME by test_, so in our example the test database will be called test_djangotutorial.

The test runner creates and deletes the test database on the fly, so an administrator account is required, but we will get back to that later.

Even though the test database is the one actually used by the tests, Django also tries to establish a connection with the nominal database, the one specified by NAME (djangotutorial in this example). If this nominal database is missing, Django will be unable to run the tests, hence the need to create it.

The best way is to use the Create database builder.

Step in the Build section, click on Add build step and select Create database. Then fill the fields as follows:

• Backend: the database server to target is the one provided by the SQL axis, so select SQL axis,
• Database name: the name of the nominal database, here djangotutorial,
• Username: whatever,
• Password: whatever.

Virtualenv Builder

To be able to install all the required packages, a Virtualenv Builder is recommended.

Click once more on Add build step and select Virtualenv Builder.

In the Command field enter all the required steps:

• Install dependencies with pip, we're using the DBA_SQL_PACKAGE environment variable to get the name of the Python package used to connect to the database,
• Warning: while testing against PostgreSQL, pip install $DBA_SQL_PACKAGE will install the latest version of the psycopg2 package. However Django does not support it yet (see Ticket #16250), so we need to override it with an older version of the package. This should be fixed in an upcoming Django release.
• Run the tests with:

python djangotutorial/manage.py jenkins

Here is the full Command content:

pip install django $DBA_SQL_PACKAGE django-jenkins coverage pylint
pip install psycopg2==2.4.1
python djangotutorial/manage.py jenkins 

As you may have noticed, the SQL axis exports a lot of environment variables, such as DBA_SQL_PACKAGE.

Let's detail the ones used in the project settings:

DATABASES['default'].update(dict(
    ENGINE=os.getenv('DBA_SQL_DJANGO_ENGINE'),
    USER=os.getenv('DBA_SQL_ADMIN'),
    PASSWORD=os.getenv('DBA_SQL_ADMIN_PASSWORD'),
    HOST=os.getenv('DBA_SQL_HOST'),
    PORT=os.getenv('DBA_SQL_PORT'),
))

• DBA_SQL_DJANGO_ENGINE is the Django engine for the current database server: django.db.backends.sqlite3, django.db.backends.mysql...
• DBA_SQL_ADMIN is the name of the database administrator,
• DBA_SQL_ADMIN_PASSWORD is the password of the database administrator,
• DBA_SQL_HOST is the host of the database server,
• DBA_SQL_PORT is the port of the database server.

More variables are available, have a look at the documentation for more details.

Post-build actions

Add all the desired post-build actions.

Here Jenkins is asked to parse the XML test, coverage and PyLint reports and to send emails on failure.

Results

Finally start a new build by clicking on Build Now: execution results by database and Python interpreter are directly available on the main page of the project.

Tips

Testing on multiple databases can be time and resource consuming.

To ensure that a revision is worthwhile to test on all environments, you can execute a touchstone build first:

• If this build is successful, builds on other versions will be triggered.
• If this build fails, other versions will not be tested.

To enable this option, look for a Execute a touchstone build first checkbox in the Configuration Matrix and check it.

The syntax for filter is SQL=="<backend>", where <backend> is the name of the database server you want to test on first.

Here the touchstone build is executed on SQLite (SQL=="sqlite") but it could also have been MySQL (SQL=="mysql") or PostgreSQL (SQL=="postgresql").

If you want to extend this touchstone build to the Python interpreter, you can also type SQL=="mysql" && PYTHON=="CPython-2.7".

What about NoSQL databases?

Use a NoSQL axis instead of the SQL one.

What about testing on a single database?

Have a look on the Start a database server wrapper, like the SQL axis it starts a database server on the fly.