In PHPDBUnit - Testing DB interaction with PHPUnit - Digital Sandwich, Mike Lively announces a port of Java’s DBUnit to PHP. I gave a short(ish) reply on the blog but after posting it I realized I wanted to expand on the topic a bit .

Let me start by quoting my reply on his site. It’s Sunday morning and I am lazy -

Might be nice for acceptance testing, but I don’t see why you would want to talk to a real db during unit testing. It’s just too slow, and if you need anything resembling a biggish dataset for a *unit* test, your unit tests aren’t fine-grained enough.

Mocking the database is really where you want to go. Redirect the connection logic to give a mock connection object back that basically does string compares to pick which of a handful of precooked result sets (in-memory arrays) to return. It’ll cost you an hour or so to rig up some baseclasses, and keeping everything in memory (and not having to truncate tables) makes sure that your unit tests keep flying. Also, having your datasets in the code will force you to think about your unit tests once they get large - usually that’s a refactoring smell

I used database-based unit testing once, when I just started with XP 10 years ago, and I’ll never go back there.

If you check the original post, the example he gives would be exceedlingly simple to write a unit test for that doesn’t use a database. Roughly it goes like:

    function testDelete() {
        $pdo = new MockPDO(/* callback = */ $this);
        $this->seenDelete = false;
        $foo = new Foo($pdo);
        $foo->deleteByCol1('value1');
        $this->assertTrue($this->seenDelete);
    }
    function pdoCallback($query) {
        if ($query == "DELETE FROM foo WHERE col1 = 'value1'") {
            $this->seenDelete = true;
        }
        return null; // no resultset needed
    }

Look, Ma - no Database! This is a Unit Test. It tests a Unit of code. Not a whole stack of it.

Think of using a real live database as a very strong refactoring smell because you are having a lot of test coverage duplication - every test you do using DBUnit activates the same bits of code - like MySQL drivers, etcetera - that hopefully should be tested elsewhere. I assume that when I feed the above query to MySQL, it will do what I told it. When writing the test, I might even copy and paste it to the MySQL command line to make sure that the query I am comparing against works, and in real life you probably want a “sloppy compare” that ignores whitespace and character case.

By the way - this code has one callback per test class. This was the simplest thing when we started using it at work, but what also might work is a callback per test, especially using the quite cool mock class generation facilities of SimpleTest (which is what we use at the moment). However, we don’t tend to do a lot of database testing anyway: most CRUD code is generated, so we have a single test set to test the generator and exercise what the generated code is spitting out; live classes just get the additional (hand-coded) methods tested.

If you wonder why I am so adamant against using something like DBUnit in a unit test set (that is, unless you are actually developing the database code - then it will be very useful indeed), apart from the somewhat academic argument about test coverage, a very big issue is performance. Unit tests become more valuable the faster they run, and ideally you want them so fast that every time you press “save”, they run and you don’t even notice the lag (I have seen that demoed once and it was quite cool). As I said, it is sunday morning and I am lazy so I am going to guess that setting up the database will take you something like 250ms. A hundred database tests later just the overhead of setting up the database is 25 seconds. In that 25 seconds, I could have run an additional thousand or so “fast” unit tests. If your test suite gets slow enough that you get up for a coffee after pushing the “run all tests” button, you will not run them often enough - it becomes a ritual to execute before, say, committing your code instead of a friend that helps you guide you along every step of that big refactoring you were tasked with.

So, unit testing must be fast (err… well, relatively fast. Nothing is absolutely fast in PHP, alas). The best way to ensure this is to identify just what code you want to test, and then sandwich it: on the top, there’s the unit test, on the bottom, there is probably mock code, and in between just the code that you scrutinize. Keep the sandwich model in mind and you’ll find that you will write cleaner code (because all your code becomes “pluggable” in a sense, in order to accommodate it being able to run in the test sandwich as well as in the real environment), and your unit tests stay focused and fast. Focused unit tests also means that if you change one thing, only one unit test breaks - clearly that’s a win too.

This entry was posted on Sunday, June 24th, 2007 at 10:46 am and is filed under Software Development, Agile/Lean. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

Stumble it!  Post to del.icio.us