Python Testing (Unittests)

I've never had an aversion to the newest and best ideas out there when it comes to making my coding productive and fun. One idea I've played with for the last few years is Agile development. I've never belonged to an official agile project but I have followed it over the years in blog posts by others and Scott Ambler's musings in Dr Dobbs magazine.

All I can really offer here is what I've done over the past with the python code that I've been developing for the last years and the tools and design patterns I use to make this work for me.So without further ado lets look at some of the tools out there that you will need in order to get this to work for you.

UnitTest & Coverage

A really good tutorial on unit testing that I would have liked to written can be found here and the official Python documentation of pyUnit is definitely a necessary read for additional insights.


These tools are going to form the backbone of your entire developmental system. Test Driven development requires that we create tests first before we even write a single line of production code. This requirement is easily met if we follow the agile development adage that we design only what we need at the present time to fulfill our requirements.

I'm going to stress this even more by offering a trivial example to walk through:

Design a class that has the same functionality as the pushd/popd unix calls

I like to have a unit test per file which makes running the unit test easier from the command line, you can easily comment out sections of your tests and just run the items that are failing without the distraction of everything else blowing up around you as a result of a low-level change you made. To this end I always have the following at the end of all my unittests

if __name__ == '__main__':
unittest.main()

Just run the unittest from the command line

python <nameOfTest>.py

I'm going to pretend that we wrote the following example iteratively by following these requirements or features

1. class to be instantiated with a directory entry, directory chosen is now the current directory


2. error to be thrown if directory entry is invalid or non-existent


3. once class is destroyed, the previous directory it was pointing to is 'pop'ed back

for every element in feature list:
-Write the test,
-Hack the code until it runs the test and passes

Here's the UnitTest:

########################################################################
class testChDir (unittest.TestCase):

#----------------------------------------------------------------------
def setUp(self):
    """Basic setup"""
    self.prevDir = os.getcwd()
#----------------------------------------------------------------------
def testChDirGood(self):
    ''' make that we can cd to a directory and pop it '''
    targDir = tempfile.mkdtemp()
    targObj = chDir(targDir)
    self.assertEquals(os.getcwd(), targDir)
    targObj = None
    self.assertEquals(os.getcwd(), oldDir)
    shutil.rmtree(targDir)
#----------------------------------------------------------------------
def testChDirBad(self):
    ''' change directory to something that doesn't exist'''
    oldDir = os.getcwd()
    targDir = '/none'
    self.assertRaises(OSError, chDir, targDir)
#----------------------------------------------------------------------
def tearDown(self):
    """typical tear down code"""
    os.chdir(os.oldDir)
   

Here's the code:

########################################################################
class chDir:
#----------------------------------------------------------------------
def __init__(self,dir):
    """Push to our directory entry"""
    #get the current directory and save it
    try:
        self.olddir = os.getcwd()
        os.chdir(dir)
    except (OSError, IOError), msg
        raise msg
#----------------------------------------------------------------------
def __del__(self):
    """pop our directory entry"""
    try:
        os.chdir(self.olddir)
    except (OSError, IOError), msg
        raise msg

After this iterative exercise, we're left with a unittest that should be exercising all the capabilities of the class in question.

That was a lot of work for a teensy-weensy class that all it does it push and pop directories but the result of all this work will come in handy if I every have to refactor the code. If I need to add capabilities to this code, all I have to do is change the code and run my unittests to determine that I haven't broken anything. Now I'm totally assured that once I run my unit test and it passes, all the code that uses this tested class will behave properly. Thats certainly a load off my mind when it comes to maintaining changes to code; because it allows me to have the confidence to refactor my code and know that it won't create nasty bugs in my application.

How many times have I encountered push back from developers when it comes to changing an already existent feature in an application. They cite concerns about the mountain of work for them to make sure that it doesn't create bugs and regression testing that needs to be done by the QA department for it to get approved. With the existance of unittests these arguments are moot. The regression tests are unittests and they can be run at anytime to gauge the health of your application's codebase. In fact having unittests will allow the developer the freedom of being able to refactor his code with the least amount of deleterious side effects.

Unittests also bring into sharp focus who is the most capable of testing the code one writes; the developer or the QA engineer. If you hear a traditional programmer explain it; QA engineers are not paid the big bucks coders get so it is beneath them to debug their code. All too often we have the phenomenon of throwing code over to the QA department, to have them validate it but is it really QA's job? Who really knows what a particular function call is doing? Who knows the possible problems that can arise from code that is being written but the programmer? A QA engineer will cull the high level bugs but we can't seriously expect a black box tester to somehow have white-box understanding, this seems to be a fallacy that management falls into to appease the programming prima donnas who are just plain lazy, irresponsible and may I daresay stupid and short sighted.

So Unittests are the answer right? Not exactly... we can have unittests up the yin-yang but if they don't test everything about our code, they're not telling us everything we need and worse yet, they're giving us a false sense of security. The answer to this quandary is our next topic:

Coverage!