20 approaches to Smart Test Design

T Ashok @ash_thiru on Twitter

Summary

This article outlines twenty approaches to smart test design based on seven views of user, logic/analysis, construction, test, experience, operational and evolution.

---

Let us talk a bit of test design now. We focus a lot of execution, and therefore the ability to cover more. The focus has veered to how frequently we are able to execute the tests and therefore on automation. Let’s step back and ask to what the objective was. Well it was to primarily deliver clean code. So we need to have a deeper sensitivity to the quality of tests. This is where design becomes important.

So what is a smart way to design to come with good scenarios, ideally few that can uncover issues that matter most.

Smart Test Design is about looking at the system from multiple views to:

I want | expect | would-like behaviours to satisfy needs that are implemented well and comprehensively covered to help me do well on my environments with no side effects.

Then decide what you want to prevent, statically detect or test, be it via human or a machine. Focus on intent and then the activity.  

Let me list down TWENTY approaches to smart test design looking at the system from SEVEN views.

1. User view based

1.1 Use the requirement specification to design.
1.2 See actual users of how they work and then use this to design.
1.3 With users doing experience sessions with system and use this information to design.

2. Analytical view based

2.1 Use software test techniques (black or white) on spec/structure to design.
2.2 Construct behavior models and use this to design. 
2.3 If the system needs to comply with a standard, use the standards information to design. 

3. Construction view based

3.1 Use the code properties like lines/conditions/path to design.
3.2 Exploit your deep understand of technology to understand potential mechanisms/flaws and use this design.
3.2 Understand how the system has been architected, composed & integrated to design. 

4. Test view based

4.1 Hypothesise potential faults probable given your understand of usage, structure, architecture, environment, conditions to design.
4.2 Use potential error return codes, exceptions, deliberate bad inputs, violations of system states to design.
4.3 Identify potential end failures and failure modes and use this to design.
4.4 Explore the system to understand its behaviour in various contexts and use this to design .
4.5 Probe the system with a series of questions (say what-if) and use this to design.

5. Experience-view

5.1 Use the past history of issues encountered with various customers and design.
5.2 Apply the learning of various situations or deeper knowledge to come u with fault patterns and use this to design. 

6. Operational-view

6.1 Use the understanding of actual business flows, usage profiles of features on various environments to design.
6.2 Identify various deployment configurations and use this information to design.

7. Evolution-based

7.1 Use the information of what code has been changed, to understand how these may propagate and may cause issues to design.
7.2 Use the information of changes in the environment to understand their impact on the system and design 

Smart Design is about:

I want | expect | would-like behaviours to satisfy needs that are implemented well and comprehensively covered to help me do well on my environments with no side effects.

---

Signup to receive SmartQA digest that has something interesting weekly
to becoming smarter in QA and delivering great products.