Monday, April 8, 2013

Unit testing with nose Framework

         Unit testing is a software development process in which the smallest testable parts of an application, called units, are individually and independently tested for proper operation. Unit testing of software applications is done during the development (coding) of an application. Proper unit testing done during the development stage saves both time and money in the end. Unit tests find problems early in the development cycle. Some programming languages directly support unit testing, like Python, java etc. There are a couple of ways to integrate unit tests into your development style. These include Test Driven Development, where unit tests are written prior to the functionality they're testing; during refactoring, where existing code -- sometimes code without any automated tests to start with -- is retrofitted with unit tests as part of the refactoring process; bug fix testing, where bugs are first pinpointed by a targetted test and then fixed; and straight test enhanced development, where tests are written organically as the code evolves. In the end, I think it matters more that you're writing unit tests than it does exactly how you write them.For me, the most important part of having unit tests is that they can be run quickly, easily, and without any thought by developers.

nose

There are many unit test frameworks in Python, and more arise every day. I personally use nose, and it fits my needs fairly well. nose is simpler Unit Testing Framework for python. nose extends unit test to make testing easier. nose comes with a number of builtin plugins to help you with output capture, error introspection, code coverage, doctests, and more. It also comes with plugin hooks for loading, running, watching and reporting on tests and test runs.

Installation

Please refer the nose 1.3.0 documentation for installation of nose on Unix like systems.

simple examples:

Now let's start with a few examples. Here's the simplest nose test you can write:


def test_a():
      assert 'a' == 'a'

Put this in a file called 'test_me.py', and then run nosetest. You will see this output:
.
----------------------------------------------------------------------
Ran 1 test in 0.002s

OK

If you want to see exactly what test was run, you can use nosetest -v

test_me.test_a ... ok

----------------------------------------------------------------------
Ran 1 test in 0.002s

OK

A fairly common pattern for unit tests is something like this:


def test():
   setup_test()
   try:
      do_test()
      make_test_assertions()
   finally:
      cleanup_after_test()
Here, setup_test is a function that creates necessary objects, opens database connections, finds files, etc.Then do_test and make_test_assertions acually run the test code and check to see that the test completed successfully.Finally, the preconditions are cleaned up.


test discovery and execution

nose is a unit test discovery and execution package. Before it can execute any tests, it needs to discover them. nose has a set of rules for discovering tests, and then a fixed protocol for running them. While both can be modified by plugins, for the moment let's consider only the default rules.nose only looks for tests under the working directory, unless you specify one with the -w command line option.nose can only execute tests that it finds. If you're creating a new test suite, it's relatively easy to make sure that nose finds all your tests.


nose command line

Apart from the plugins, there are only a few options that I use regularly.

-w: Specifying the working directory
nose only looks for tests in one place. The -w flag lets you specify that location; e.g. nosetests -w my_nose_test/.  will run only those tests in the directory "/my_nose_test/"
As of the latest development version you can specify multiple working directories on the command line:
nosetests -w nose_test/ -w sample/

-s: Not capturing stdout
By default, nose captures all output and only presents stdout from tests that fail. By specifying '-s', you can turn this behavior off.

-v: Info and debugging output
nose is intentionally pretty terse. If you want to see what tests are being run, use '-v'.

-p: plugins
Output list of available plugins and exit. Combine with higher verbosity for greater detail

Tools for testing

           nose.tools provides a few convenience functions to make writing tests easier. You don’t have to use them; nothing in the rest of nose depends on any of these methods.

nose.tools.ok_(expr, msg=None)
Shorthand for assert. Saves 3 whole characters!

nose.tools.eq_(a, b, msg=None)
Shorthand for ‘assert a == b, “%r != %r” % (a, b)

nose.tools.make_decorator(func)
Wraps a test decorator so as to properly replicate metadata of the decorated function, including nose’s additional stuff (namely, setup and teardown).

nose.tools.raises(*exceptions)
Test must raise one of expected exceptions to pass.

nose.tools.set_trace()
Call pdb.set_trace in the calling frame, first restoring sys.stdout to the real output stream. Note that sys.stdout is NOT reset to whatever it was before the call once pdb is done!

nose.tools.timed(limit)
Test must finish within specified time limit to pass.

nose.tools.with_setup(setup=None, teardown=None)
Decorator to add setup and/or teardown methods to a test function:

nose.tools.istest(func)
Decorator to mark a function or method as a test

nose.tools.nottest(func)
Decorator to mark a function or method as not a test

Running nose programmatically

nose has a friendly top-level API which makes it accessible to Python programs. You can run nose inside your own code by doing this:


import nose

### configure paths, etc here

nose.run()

### do other stuff here


By default nose will pick up on sys.argv; if you want to pass in your own arguments, use nose.run(argv=args). You can also override the default test collector, test runner, test loader, and environment settings at this level. This makes it convenient to add in certain types of new behavior

For more details please refer the official site.