How to report what you do

One of the most important parts of testing is the ability to report what have been done. I think this is one reason for the popularity of scripted test cases and formal tools, a means for management to get some report of the testing. What I have found is that it is very easy to make false assumptions and come to the wrong conclusions using this method, one common mistake is that test cases is not a documentation of what have been done, but a documentation of what is intended to do. The different is subtle but important.

Lets go back to the entire purpose of testing, one of the main reasons is to gain knowledge. How does the product perform, does the product fulfill requirements, does the product behave as expected and so on. The receiver of this knowledge is not the test organization but the product owner. The test organization of course also have an interest in this knowledge, but that is secondary to the product owner. It is also secondary to the needs of the developer organization for which the feedback from the product is invaluable, both in order to improve the quality by correcting errors, but also to help improve the development in order to increase the chance of improved quality in the future as well. This is why I like to think of the test organization as a service to the product owner and to the developers. A service to provide feedback.

With this in mind how do we report what is done in testing? The simple answer is to provide the information needed for the product owner to know if the product is good enough and for the developers to realize what to do in order to improve quality. This is of course different for different organization.

Currently we are looking into different methods to report what has been done without having test cases to report. More on that soon to come.