When To Get Serious About Testing

Some years back, I taught a course on software configuration management.  In that course, we discussed the benefits and drawbacks to integrating change control within the software development process considering that the software is being actively developed or matured during that process.  A concept that I presented in the course was to apply only the level of control that was appropriate for the phase or maturity the project/software had reached.

This same concept is applicable when deciding how to establish the most useful balance between providing early feedback and performing “serious” testing.

Planning, Preparing, and Executing

When describing the activities to be undertaken during a testing effort (whether Agile, Iterative or Waterfall), I tend to categorize those activities by the following:

• • Planning: we are investigating and designing approach options to find what will best fit the needs of a particular release
  • Preparation: we are busy with all the tasks that will let us best execute testing according to the agreed plan
  • Execution: we are running tests, recording the results, reporting the issues, and interpreting the data and trends to allow for informed decision-making

Adapted from “Test Management: Leading Your Team To Success (Course Extract)”
http://thinktesting.wordpress.com/2011/12/12/

Preparation activities can start before planning is complete.  Likewise, execution can start before preparation is complete.  In fact, it really should – to minimize rework while reducing risk.  But, how can we do this practically?

A Creative Process

Consider: if you are creating something, do you wait until it is perfect before the unveiling to anyone else in the world?  Is that amazing thing “done” the first time you show it to someone?  Perhaps in cases where you yourself hold the entirety of the vision and your satisfaction comes from accomplishing the creation itself.  However, if you want someone else to like it…to use it…?

Feedback is an essential component to obtaining verification that an idea or an expression of a concept is perceived to be useful, valid, or valuable.  When is this feedback the most useful?  When it is not too late to make a change…early.

So then for software, when is it the right time to:

• • Start looking at a feature?
  • Start logging bugs?
  • Start measuring test coverage (across functionality, platforms)?
  • Start counting the bugs and test results towards trends, metrics?

But It’s Not Ready For Testing Yet

“Serious” testing – the kind that comes with or needs artifacts such as test cases/scenarios, checklists, charters, test data, reports, bug logging, etc – is not directly oriented at providing feedback.  Such testing is providing an appraisal – an appraisal or evaluation of how a system behaves versus the description of how that system should or is expected to behave. (Ref: “Quality Cost Analysis: Benefits and Risks”, Cem Kaner, 1996)

For that appraisal or evaluation to be valid and comparable, we want to have certain things understood and complete – the requirements, the functionality, the user interface, etc.  We would also like to have certain conditions – time, stability.  But, we can’t expect this early in the project, can we?

So how can we get involved early and best provide maximum value (ie: appraisal + timely feedback) to the project without applying a net drag on the project?  And how can we best leverage that involvement to compress the effort needed to complete our test activities by minimizing rework and staying off the critical path?

Providing Early Feedback & Performing “Serious” Testing

The following illustrates, at a high-level, one approach on how to structure giving valuable early feedback while gaining the useful insights/inputs needed for pre-execution activities.

Conclusion

When is it time to get serious about testing?  Delay completing the planning and preparation activities as long as responsibly possible.  Delay starting the heavier aspects of formalized execution to the point(s) that fit best with the needs of the rest of the project team.  Bring forward the ability to get involved early and provide timely and useful feedback.

Undertaking an approach such as that described above will help you be efficient and effective at the same time, and it will help you stay off the critical path while you are uncovering important issues early in the project lifecycle.