Corriendo tests con instancia de PostgreSql en ram

Otra manera de optimizar la ejecución de tests de Django, pero SIN modificar la instalación de PostgreSql del sistema operativo es crear una nueva instancia, corriendo totalmente en ram.

Para crear una nueva instancia usamos initdb:

$ /usr/lib/postgresql/9.1/bin/initdb \
    --pgdata=/dev/shm/pgtesting \
    -U horacio

Y para lanzar la instancia en segundo plano:


$ /usr/lib/postgresql/9.1/bin/postgres \
    -F -i -p 5444 -S $((1024*64)) \
    -D /dev/shm/pgtesting \
    --unix_socket_directory=/dev/shm/pgtesting \
    -r /dev/shm/pgtesting/postgresql.log &


Luego creamos el usuario y BD para la aplicación Django:

psql -h 127.0.0.1 \
    -c "create user vindler with password 'x' superuser" postgres
psql -h 127.0.0.1 \
    -c "create database vindler owner vindler" postgres


Mejora en tiempos de ejecución de test: 28%

La ejecución de los tests con PostgreSql en disco, con las optimizaciones de mi artículo anterior tardan alrededor de 7 segundos:


Ran 18 tests in 7.071s
5.44user 0.58system 0:10.40elapsed 57%CPU (0avgtext+0avgdata 122880maxresident)k
0inputs+1184outputs (0major+16964minor)pagefaults 0swaps


Ran 18 tests in 6.843s
5.28user 0.65system 0:10.27elapsed 57%CPU (0avgtext+0avgdata 122976maxresident)k
0inputs+1176outputs (0major+16956minor)pagefaults 0swaps


Ran 18 tests in 7.032s
5.42user 0.64system 0:10.42elapsed 58%CPU (0avgtext+0avgdata 124304maxresident)k
0inputs+1168outputs (0major+17122minor)pagefaults 0swaps

Los mismos tests ejecutandose en la nueva instancia de PostgreSql totalmente en ram, tardan alrededor de 5 segundos:


Ran 18 tests in 5.220s
4.41user 0.65system 0:08.38elapsed 60%CPU (0avgtext+0avgdata 117904maxresident)k
8inputs+1176outputs (0major+16524minor)pagefaults 0swaps


Ran 18 tests in 5.047s
4.32user 0.53system 0:08.04elapsed 60%CPU (0avgtext+0avgdata 117888maxresident)k
0inputs+1168outputs (0major+16504minor)pagefaults 0swaps


Ran 18 tests in 4.665s
3.95user 0.64system 0:07.79elapsed 58%CPU (0avgtext+0avgdata 117856maxresident)k
0inputs+1176outputs (0major+16502minor)pagefaults 0swaps

Al ejecutar los tests de stress: 33%

Con PostgreSql en disco:


Ran 1 test in 24.601s
15.21user 2.72system 0:27.98elapsed 64%CPU (0avgtext+0avgdata 130944maxresident)k
0inputs+9616outputs (0major+18298minor)pagefaults 0swaps


Ran 1 test in 24.458s
15.16user 2.51system 0:27.57elapsed 64%CPU (0avgtext+0avgdata 131488maxresident)k
472inputs+9608outputs (0major+18457minor)pagefaults 0swaps


Ran 1 test in 24.552s
15.23user 2.63system 0:27.95elapsed 63%CPU (0avgtext+0avgdata 131088maxresident)k
232inputs+9616outputs (0major+18310minor)pagefaults 0swaps

Con PostgreSql en ram:

Ran 1 test in 16.082s

11.30user 2.46system 0:19.22elapsed 71%CPU (0avgtext+0avgdata 126080maxresident)k

0inputs+9616outputs (0major+17675minor)pagefaults 0swaps

Ran 1 test in 15.736s

11.25user 2.20system 0:18.82elapsed 71%CPU (0avgtext+0avgdata 126192maxresident)k

0inputs+9608outputs (0major+17684minor)pagefaults 0swaps

Ran 1 test in 15.408s

10.78user 2.27system 0:18.44elapsed 70%CPU (0avgtext+0avgdata 126384maxresident)k

0inputs+9608outputs (0major+17693minor)pagefaults 0swaps

Acelerando tests de Django con PostgreSql

La ejecucion inicial de los tests toma aproximadamente 15 segundos:

4.54user 0.64system 0:15.25elapsed 34%CPU (0avgtext+0avgdata 123600maxresident)k
4.70user 0.54system 0:14.70elapsed 35%CPU (0avgtext+0avgdata 122992maxresident)k
4.46user 0.65system 0:14.85elapsed 34%CPU (0avgtext+0avgdata 122912maxresident)k

Luego de ajustar los siguientes parametros:
- fsync = off
- synchronous_commit = off
- wal_sync_method = fsync

la ejecución toma aproximadamente 8.5 segundos:

4.22user 0.56system 0:08.62elapsed 55%CPU (0avgtext+0avgdata 123024maxresident)k
4.32user 0.58system 0:08.97elapsed 54%CPU (0avgtext+0avgdata 123008maxresident)k
4.29user 0.50system 0:08.74elapsed 54%CPU (0avgtext+0avgdata 123040maxresident)k

El tiempo se redujo casi a la mitad! Aunque esta configuración no es para nada recomendable para un equipo de producción, creo que vale la pena para ejecutar tests.

Introduction to Python/Django testing: Doctests

Nota: copié todo el artículo porque el link al post original no me funcionó... (este es el link original)

Eric Holscher: Introduction to Python/Django testing: Doctests

This is the first in a series of blog posts and screencasts that will walk you through how to test your Django application. These posts will focus more on how to get things done in Django, but note that a lot of the content is applicable to pure python as well. A lot of best practices are codified into Django's testing framework, so that we don't have to worry about them! I will try to point them out as we are using them through, because they are good things to know.

The currently planned structure for this series is below. Please comment if there is something that you think is missing, or something that I shouldn't do. This is subject to change, a lot, as well, so your feedback will help direct it. Also note that most or all of this content is available in the Django and Python documentation, and I will try and point there and not re-invent the wheel. I hope that these posts will take a more practical look, and try to point out some pit falls and other things that are useful.

Outline

• Basic Doc tests
• Basic Unit tests
• Comparison of Unit tests vs. Doc test
• Fixtures
• Using Mock objects
• Third party testing tools
• Writing your own test runner
• Getting code coverage for your tests

Where to start

I'm assuming that you already have a project that you're working on that you would like to test. There are two different ways of putting tests inside of your django project. You can add a tests.py file and put your tests inside of there. You can also define a tests/ directory and put your tests in files inside of that. For these tutorials it is assumed that the second is the way things are done. It makes it a lot easier when you can break your tests out into logical files.

Doctests

These can go in two places inside your django project. You can put them in your models.py file, in the Docstring for your modules. This is a good way to show usage of your models, and to provide basic testing. The official docs have some great examples of this.

The other place your Doctests can go is inside your tests directory. A doctests file is usually pretty simple. A doctest is just a large string, so there isn't much else to put besides a string. Usually you want to use the triple quote, multi-line string delimiter to define them. That way your " and 's inside of your doctests don't break anything.

"""
This is my worthless test.
>>> print "wee"
wee
>>> print False
False
"""

You can go ahead and put that in a file in your tests/ directory, I named it doctst. My application that I'm writing tests for is mine, because it's the code for my website. Make sure that directory has an __init__.py as well, to signify that it is a python module.

Now here is the tricky part; go ahead and try and run your test suite. In your project directory run ./manage.py test APPNAME. It will show you that you have passed 0 tests. 0 tests? We just defined one.

You need to go into your init.py file and put some stuff in there.

import doctst
__test__ = {
   'Doctest': doctst
   }

You are importing the doc test module and then adding it to the __test__ dictionary. You have to do this because of the way that python handles looking for doc tests. It looks for a __test__ dictionary inside of your module, and if that exists it looks through it, executing all docstrings as doctests. For more information look at the Python docs.

Now you should be able to go ahead and run the tests and see the magical Ran 1 test in 0.003s OK that all testers live for. This is little bit of overhead really threw me off when I was trying to break my tests.py out into the tests/ directory.

So now we have a test suite that is worthless, but you know how to use doc tests. If you didn't notice, the doctest format is simply the output of your default python shell, so when you are testing your code on the command line and it works, you can simple copy and paste it into your tests. This makes writing doc tests almost trivial. Note however, that they are somewhat fragile, and shouldn't be used for everything. In the next segment, we will talk about unit tests. Then we will compare the two and see when you should use each.

Herramientas de testing para Python

Encontré un link muy bueno con una listado bastante extenso de las herramientas existentes para realizar el testing de programas desarrollados con Python.

http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy