Tooling and automation are great, but Smart QA is brilliant

In today’s world of extreme agility, intelligent systems, SAAS, automated test and deployments, what is the role of humans in QA? How has the way we test software changed?

We live in the age of speed and smarts. Will machines take over testing? Will testing be subsumed into dev? With systems becoming very intelligent, it is only natural that testing become far more smarter. The theme of Smart QA explores the various dimensions of smartness and how we need to reinvent what we do.

To meet today’s demand for speed of delivery, automation seems to be the way forward. With a wide range of testing tools spanning across the entire lifecycle covering all design, execution, management, data creation, structural assessment spanning all types of tests from early stage development test to system test supporting a plethora of technologies, automation is deeply entrenched in how we test software. Is testing therefore about creating automated suites and setting up a pipeline to execute these periodically and automatically?

Yes, automated suites are the order of the day as we move into a continuous delivery mode. The continual enhancement of software demands a mechanism that enables a frequent system health check ensuring nothing is broken as we continually enhance the software. This is where tools and automation come in very handy. 

Let us step back as to what we are attempting to accomplish when we test. Well, we are looking for potential issues that would upset the end users experience and risk the business outcomes. So would the automated suites that run periodically uncover potential newer issues as they seep in? Hmmm, they cannot as they are programmed to execute the same paths/flows with the primary focus of ensuring good health. So what do we do? Add/update the automated suites continually and also test humanly. Note newer scenarios are created and some converted into suites, however not all can be figured out and automated as creativity cannot be boxed. It is necessary to be smarter to outwit the elusive bugs. As software is continually modified becoming richer, automated suites need to be synched with them too. A well-formed test strategy and a well-architected test assets are necessary to exploit the power of automation continually. 

So where smartness figure in this? Is it not just plain tooling tech to come up with scripts/suites? Let us take a deeper look. What we are doing really is converting our scenarios into scripts and also using appropriate tools & frameworks to enable static assessment and incorporate this into the tool chain. So never forget that the test scenarios to  are really the key to effective testing. To be able to this well, we need to be  able to understand the system context well, decompose the system into various sized entities to test so was to come up a great strategy, to create well-organised test assets(scenarios/cases/data) covering all the different-sized entities and ensuring these ‘tightly’ cover the software ensuring high coverage spanning all types of tests right from early dev stage to whole system stage. All this up front work demands brilliant smartness, before we exploit tech and tools to make the execution efficient. And this is why I think “Tooling and automation are great, but Smart QA is brilliant”.

Smart QA is about doing less, doing quick and achieving far more. In every facet of the lifecycle be it from the earliest stage of understanding, to exploring the context, charting out the scenarios, scripting suites, intelligent regression, doing static assessment to continuous course correction, it is necessary to do the least, do the fastest and get great outcomes.  Let us take a look at the “smarts” in the various parts of the lifecycle in detail.

In today’s world, information as to what is expected is not spelt in great detail. It is terse and simplistic, that is at best clarified by ‘acceptance criteria’. It is no more elaborately articulated as was done in the past. It is expected that the tester explore the system far more carefully and creatively to discern what may have been intended, what would be most meaningful for end users and then ask questions to clarify, to understand, or come up with scenarios to evaluate. Smart exploration is what is needed to rapidly understand the context, and how visual thinking can aid is discussed in “Exploit visual thinking – Smart exploration of software”.

The UI of today’s software is very rich and contains very many features packaged densely on a small screen. I have noticed that how one decomposes a rich UI to into various entities to test, decides how well and how quick one can evaluate. Smart modelling is key to this, this is dealt with in detail in the article “How to test rich UI – Smart Modelling”.

Software is continuously modified necessitating constant retesting/regression. Automated testing is very handy but not everything can be automated. It may not be really necessary to test all every time as systems harden with time. How can we do regress less is where Smart Regression comes in and this is examined in detail in “Three Aids for Smart Regression Testing.

Bugs seep in right from earliest stage of code development/modification, which we believe should be uncovered by automated unit tests. This does seem right, but what a minute – Can we become sensitive to issues that may seep at the early dev stage and prevent these, or detect this as cheaply as possible using ‘smart checklists’ as is done in other mature industries rather than resort to unit test?  What a smart checklist is and how this can help is the crux of the article “Smart checklists make you think”.

Smart QA is really about applying intellect to deliver ‘clean software’ by doing the least amount of work and exploiting tools to do the grunt work or be the enablers. 

“Tooling and automation are great, but Smart QA is brilliant”

Signup to receive SmartQA digest that has something interesting weekly to becoming smarter in QA and delivering great products.