Comments on: Testing DB interaction http://www.cdegroot.com/blog/2007/06/24/testing-db-interaction/ Everything and the kitchen sink Mon, 12 May 2008 12:32:00 +0000 http://wordpress.org/?v=2.0.3 by: Mike Lively http://www.cdegroot.com/blog/2007/06/24/testing-db-interaction/#comment-8039 Wed, 04 Jul 2007 18:50:59 +0000 http://www.cdegroot.com/blog/2007/06/24/testing-db-interaction/#comment-8039 I am not sure if you checked my reply to your comment on my blog or not. But I thought I would re-iterate my point here. There are several reasons why I think it is beneficial to use a live database for testing. 1. It allows you to safely refactor your queries. While this is not a problem for smallish projects, when you are dealing with a large amount of data in a performance oriented database design (not necessarily fully normalized) it is very common to alter your queries to ensure that you are not breaking your queries. 2. It allows you to test the output of your data driven units. Testing the queries you send to the database is all fine and good, but that makes the very big assumption that the queries you are writing are returning exactly what you expect. I will be the first to admit that the query in my example was extremely simple and your way of testing that particular query works just fine. However we both know that queries get much more complicated than my example and when they begin getting complicated it helps being able to test the output of them. 3. It provides for real regression testing when you find bugs in queries. If you find a bug in one of your queries, write a test that exposes the bug, hopefully squish that bug for good. If you just use query comparison you will be forced to change your test everytime you alter your query. This means that the next change you make you open the door for all of your old bugs to sneak back in. 4. Provides a very simple means to test your code against different database platforms. If you want to keep your code working on several different database platforms then using something like DBUnit makes that very easy to test. If you want to see where your code breaks in Postgres you simply need to change the database your tests use and you'll have a very good idea of what works and what doesn't. I can understand the desire to keep tests slim, but you always have the option of segregating database test cases into their own suites that can be ran only after the affected code has been changed and once a night otherwise. There is usually a way to work around slow tests. Personally I would rather have slow tests that test all of the critical parts of my application than quick tests that just do the bare minimum. It basically boils down to testing what is important to your application. Your application may be more concerned with complicated business logic that is driven soley by php. The project I am working on is a financial application that has extremely intense data interaction that has to be 100% correct all the time. So it is very important to me to ensure that all of our queries are correct :). I am not sure if you checked my reply to your comment on my blog or not. But I thought I would re-iterate my point here.

There are several reasons why I think it is beneficial to use a live database for testing.

1. It allows you to safely refactor your queries. While this is not a problem for smallish projects, when you are dealing with a large amount of data in a performance oriented database design (not necessarily fully normalized) it is very common to alter your queries to ensure that you are not breaking your queries.

2. It allows you to test the output of your data driven units. Testing the queries you send to the database is all fine and good, but that makes the very big assumption that the queries you are writing are returning exactly what you expect. I will be the first to admit that the query in my example was extremely simple and your way of testing that particular query works just fine. However we both know that queries get much more complicated than my example and when they begin getting complicated it helps being able to test the output of them.

3. It provides for real regression testing when you find bugs in queries. If you find a bug in one of your queries, write a test that exposes the bug, hopefully squish that bug for good. If you just use query comparison you will be forced to change your test everytime you alter your query. This means that the next change you make you open the door for all of your old bugs to sneak back in.

4. Provides a very simple means to test your code against different database platforms. If you want to keep your code working on several different database platforms then using something like DBUnit makes that very easy to test. If you want to see where your code breaks in Postgres you simply need to change the database your tests use and you’ll have a very good idea of what works and what doesn’t.

I can understand the desire to keep tests slim, but you always have the option of segregating database test cases into their own suites that can be ran only after the affected code has been changed and once a night otherwise. There is usually a way to work around slow tests. Personally I would rather have slow tests that test all of the critical parts of my application than quick tests that just do the bare minimum. It basically boils down to testing what is important to your application. Your application may be more concerned with complicated business logic that is driven soley by php. The project I am working on is a financial application that has extremely intense data interaction that has to be 100% correct all the time. So it is very important to me to ensure that all of our queries are correct :) .

]]>