Three aids for Smart Regression Testing

In the current world of rapid development, software is constantly updated with new features, incremental additions and bug fixes. While features (new & incremental) are the focus for revenue generation and market expansion, bug fixes are necessary to ensure that customers stay.

“As product grows, so does regression,
increasing costs & threatening to slow down releases”

So what can be done to solve this challenge? Automate, right? But can we automate all the scenarios right from the early dev tests all the way to end-to-end flow tests? A significant of the latter is pretty difficult, right? And these do consume time and effort to build. Not that all the earlier tests will be completely automated either. Also automation is a catch up game with the software updated continually. 

So what else can we do smartly ? Well in addition to whatever we can do via automation to relegate the effort to a machine we can look at two ways to lessen regression (1) Do less (2) Don’t do !


What would be a good way to analyse the impact of change? i.e What other entities may be affected and what aspects of the entities may be impacted? The technique “Fault Propagation Analysis” of HBT (Hypothesis Based Testing) can be very handy. 

Fault Propagation Analysis
The first aid uses this technique and is illustrated in this picture:

Aid #1

Given that one of the most preferred approach to efficient regression is automation, how do you ensure the automated scenarios can be kept in sync with the constant updates in the software? Well the logical answer is short and less complex scripts.

So how can we achieve this? By ‘level-ising’ scenarios that implies scenarios are decomposed into multiple scenarios each focused on uncovering a specific type of issue of a specific entity. What we want to do is prevent  ‘spaghetti’  scenarios that attempts to uncover multiple types of defects over various entity.

For example a scenario that attempts to validate inputs, validate the UI aspects of an input in terms of say colour or format, validate  logical behavior correctness, and compatibility on different devices is a sure short recipe for disaster. Automating this scenario would result in a complex script making this fragile and potentially maintenance heavy. 

The second aid titled “Automation fitness analyser” illustrates how to analyse the fitness of scenario(s) for automation so that we can re-test efficiently. Remember this is about ‘Do-less’.

Aid #2

Moving on to the third part of tackling change smartly, the objective is “what scenarios can we avoid executing” especially those that are not yet automated.

This is illustrated in the third aidYield Analyser” illustrated below. The gist is to analyse test cases that have continually passed over the last few cycles and use this information to judge what parts of the system may have ‘hardened’ and therefore not execute these scenarios.

Aid #3
In summary it is about “How can we do less to do more”?  

Do less regression or Don’t regress and Do less automation maintenance is what “Smart Regression Testing ” is all about. The three aids outlined enable you to smartly accomplish this.

All three aids

If all regression test scenarios have been automated, then it is real smart use of technology!

Signup to receive SmartQA digest that has something interesting weekly to becoming smarter in QA and delivering great products.

The Buddha Story: Good understanding

by T Ashok

Good understanding is not about knowing everything in detail, it is about just knowing what is required and learning to discard what is not necessary or deferring to a later stage.

Let me tell you a beautiful Buddha story that illustrates this.
Once upon a time there lived a wise and learned man with mastery over various languages, philosophies and religions. He was extremely proud of his knowledge, but eager to learn more. He came to know that a person named Buddha lived far away and was a very learned man. Keen to further his knowledge, he went to Buddha, introduced himself as a man well versed with various philosophies, expressed his wish to enhance his knowledge and requested to teach him.

Buddha said he would be happy to do and requested him to kindly sit in the corner of the room. A few hours passed and the learned man was getting fidgety while Buddha continued to do his work. As the sun went down, Buddha suggested that they resume the next day. The learned gentleman after a good night’s rest came back promptly. Buddha again requested him sit down in the corner. The morning turned into afternoon and the dusk set in. The man was throughly upset that Buddha was ignoring him. He went up to him and said “Sir, I am very upset that you are ignoring me, I hope you understand that I am also a man of knowledge”.

Buddha calmly said “Come my dear friend, sit down let us have a cup of tea”. He took the tea pot and poured the tea onto the cup while watching the man. The cup filled to the brim and Buddha continued pouring, spilling the tea on the floor. The man watching this shouted “The cup is full, please stop pouring!”.

Buddha fully aware of what he was doing said “My friend, your mind is like this, filled with knowledge, with no space to acquire additional knowledge. Empty it and you will be ready to learn.” And only then did the learned man realise his folly.

As a test practitioner, understanding a complex system is always always interesting. The big challenge that I face is when passionate folks explain the system sparing no detail. That is when I say “STOP, I do not need to know this, will certainly ask when I need it”.Good understanding requires that the tea cup be only partially full.

Defer what is not needed now to later. Don’t attempt to understand everything at one go.

Smart QA is about having great clarity, a result of good understanding. This occurs when you are actively questioning with an agile mind, not passively absorbing and getting weighed down. Lessen your burden and your mind is like a nimble goat climbing the mountain of complexity. At the top, view the system in totality clearly. And the issues pop out.

HBT (Hypothesis Based Testing) is based on this principle of deferment and active questioning to understand complex systems rapidly and test smartly.

About SmartQA The theme of SmartQA is to explore various dimensions of smartness to leapfrog into the new age of software development, to accomplish more with less by exploiting our intellect along with technology.  Towards this, we will strive to showcase interesting thoughts, expert industry views through high quality content as articles, posters, videos, surveys outlined as a SmartQA Digest weekly emailer. SmartBites is soundbites from smart people”. Ideas, thoughts and views to inspire you to think differently.

Signup to receive SmartQA digest that has something interesting weekly to becoming smarter in QA and delivering great products.