What's your unit testing philosophy?

A member of the Austin Software Mentorship group asked this question on the mailing list.  The kinds of questions being asked, is remarkable to me, and from college students! Just wow.  I had recently been a part of a great discussion at Lean Software Austin, and had been thinking about this stuff a lot.  So I took some time to reply, and helped me to capture my thoughts too...

When you are writing tests, one concern is to know if the code you wrote or are writing works or not.  But by automating the tests as a permanent addition to the code base, you have another set of responsibilities.

 

What's going to happen when one of your tests break and someone else on your team (or someone who will be on your team later or even you in a year or 2) has to understand what your tests were intending, whether that concern is still valid or needs to be modified or moved elsewhere?  If someone were to modify your code, what might they be likely to misunderstand and make a mistake? What conditions are not obvious from the design that might trip someone up? Can you change the code/design to make it obvious and clear? If you are delaying or unable to make the design more easy to understand, can you document the non-obviousness in a test that will fail in a way that will inform the developer of the mistake they made? 

 

More often than not, incomprehensible automated tests will end up getting dragged along on a project for years.  When they break and are either not understood or misunderstood, the bars just get turned green by flipping asserts.  The original intent and knowledge that went with the tests just get lost, and the tests just eat up time.  Even if the intent is clear, tests that break frequently when the app isn't broken, especially from non-functional changes are really annoying.  How can you write tests that will generally break when the code is actually broken, and not just because the code changed? How can you change your design so the code design so that your tests will be less brittle?  How can you simplify your design so that the things you are trying to test can no longer break?

 

For example, if you end up with a massive amount of test setup to control all of the dependencies for your test, something is wrong with your design.  It should trigger your spidey senses.  How can you restructure your design so that you don't have to be concerned about so many things at the same time?  Let your tests help you improve your design.

 

If all you care about is "seeing if it works now", there's no point in checking the tests into source control.  Just delete them. Once you check them in, your tests begin a whole new life, and if you aren't aware of the implications of that, you can create quite an expensive long-term burden.