Testing your tests. Making sure you're safe.

Tuomas Karkkainen and mhjort

Developer Community Track
Scheduled Time: 
Monday 19 November 2007, 01:30 to 03:00
Room: 
Southwark Cathedral, The John Trevor Williams Room
Session type: 
interactive presentation
Intended audience and experience level: 

Developers, intermediate to advanced, experienced with or desiring to learn about assessing quality and coverage of automated tests.

Prerequisites: 

Laptop optional. Exercise done in pairs. No development environment needed, a Live CD with all necessary software will be distributed.

Although developer testing has reached the mainstream, when looked at closer, we often find that these tests are in fact not helpful. Low quality tests prevent refactoring, create a false sense of security, and confuse the developer. This interactive presentation addresses the assessment of developer test quality.

We begin by introducing what we mean by quality of tests and the methodologies we use to assess it. Participants will then work through a series of exercises helping them evaluate the quality of tests. The exercises demonstrate common test problems found in real world applications.

During the session attendees will consider the following:

  • Do the tests cover enough of the behavior of the production class? Can the behavior be changed to do something unintended without getting a red bar.
  • Can the production code be refactored so that the tests accurately reflect the change? i.e. tests break if the behavior changes, but pass if only the design changes.
  • Is the granularity of the tests correct? Are the tests focused on the behavior of one production class, and its interaction with other objects, or does it depend on the implementation of its collaborators?
  • Are the tests written in a way to communicate the intent of the production code so they eliminate the need for javadoc or other documentation?
  • What to do when the version control for a project has tests that do not pass. Are the tests broken or is the production code broken? How to tell the difference?
  • Does it look like the tests were written earnestly or were they only written to satisfy an outside observer?

The session will end with a playful competition similar to the other exercises.

Tuomas Karkkainen

Tuomas Kärkkäinen is a programmer at Reaktor Innovations. He has over two years experience in agile methods. He is currently writing the Java bytecode mutation testing tool Test Police.

mhjort

Scrum Master and Agile Developer at Reaktor Innovations, Markus has been in the industry for over seven years with experience from various technologies including Java, Microsoft, and Symbian platforms. Certified Scrum Master with extensive experience on agile methods and a long-time active participant in process improvement wherever he’s worked at, Markus kick-started the local Coding Dojo events in Finland back in 2005.