py.test - change in testing platform

Python's unittest module is decent for some quick testing. It's a bit cumbersome to use though. Especially when compared to something extremely simple yet powerful like py.test.

After dabbling around with py.test a while I came to realize that's the way I want to write tests. Let's take a look at small excerpt from parser_test.py and see how it works:

...

minimal_structure = '''
VerticalContainer:
'''

def test_StructureParser_parse_minimal():
    structure_parser = StructureParser(minimal_structure)
   
    assert isinstance(structure_parser, StructureParser)
    root = structure_parser.parse()
    assert isinstance(root, VerticalContainer)

...

The point of this test is to guarantee that first of all we can create StructureParser. Next we parse, get the root node of the parsing as result and check that the type of class gotten is right.

The main things here are the way the file was named (parser_test.py), the way the function was named (test_StructureParser_parse_minimal) and the way assert statements are used. If I want to execute the test(s), all I need to do is to type "py.test" in the terminal.

Based on where I executed py.test, it finds the tests and executes them. This is done recursively beginning from the directory in which it was executed. If the name of the file is in form test_foobar.py or foobar_test.py (we stick to this convention in Cassopi), it checks if the file contains any tests (test_StructureParser_parse_minimal for instance!) and executes them.

Here is another more comprehensive test found directly after the test presented above from the same file:

...

structure_with_padding_in_all_directions = '''
VerticalContainer:
    padding:
        top: 5
        left: 10
        right: 15
        bottom: 20
'''

def test_StructureParse_parse_vertical_container_with_padding():
    structure_parser = StructureParser(structure_with_padding_in_all_directions)
   
    root = structure_parser.parse()
    assert isinstance(root, VerticalContainer)
    assert root.padding.top == 5
    assert root.padding.left == 10
    assert root.padding.right == 15
    assert.root.padding.bottom == 20

...

This case is pretty much the same as the previous one except that we have added more data. Note that we still assert that we have the right instance type. The reason I did this because each test should be independent (see Kent Beck's rules of testing). In other words no test should depend on other.

I structure my tests so that each covers different aspect. I start by focusing on basic aspects and then iteratively move on to more focused concerns. As you can infer based on given examples, the concepts we are dealing with are fairly high level. We hide the crude implementation details and test parts having different abstraction level separately.

The main point about testing is that they help us to define the way the code should behave. The underlying implementation just implements these set of rules. This is one of the strong points of test driven approach. It makes refactoring code really easy and I claim that you end up with more simple and powerful design in the end.

One of the reasons for this is that when you want to add new functionality to the system, you write a test first. First you write piece of code that makes the test pass. After this you can see how it fits the design. If the fit is not that good, you can easily refactor the design. That's why the tests were written for.

When you are implementing a piece of functionality this way, you may see the design evolve radically. It is probably quite clunky at first but at the end up you might have something actually quite elegant and perhaps even reusable.

So far my usage of py.test has been fairly basic but it has proved already to be nice and fast to use. There are still a couple of things I need to figure out how to handle with py.test. First of all it would be nice to know if there was some easy way to check test coverage. Secondly it might be interesting to look into using mocks as this allows nice separation of modules. In other words it allows to test modules without having them to affect each other.

In case you want to find more Python testing tools, http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy is well worth checking out. Due to amount of available tools it may actually be quite a bit of challenge to find the right tools for the job. :)