eT Archives - Page 3 of 3

50+ Software Testing Terms that are most often abused!

Summary
Terms used in the software testing discipline are often confusing leading to them being abused. Most of the phrases seem to have appendage of ‘testing’ like a surname that seems to make it belong to a family. Some terms are test types while some are approaches, and some practices and so on. This article partitions 50+ software testing terms into cohesive groups to sharpen the clarity depicting it as simple mind map and then giving a clear definition to each one.

Jargons abound in software, only to be misused most often! And I think in our discipline of testing, they are abused even more! In the testing discipline, most of the terms seem to have an appendage of ‘testing’ making it far more confusing.

In this article I have attempted to put together a mind map
attempting to group terms cohesively based on a phrase
to minimise confusion and sharpen clarity.

50+ software testing terms grouped cohesively to enhance clarity of understanding

Here are quick explanation of these terms with details in the associated link after the terms.

Levels of testing
Unit, Integration, System, Acceptance refer to the levels of the testing, implying as when in the SDLC, the earliest being the Unit testing.

Types of Testing
Functional, Performance,Load, Security, Reliability, Stress, Volume and others are really specific types of tests designed to uncover specific types of issues, the functional being the behaviour while the others are about the various attributes.

Test Techniques
Black, box, White box are really test techniques that use external behavioural information to design test cases whilst white box uses internal structural information to design test cases. Pair-wise(All-pairs) Orthogonal array are specific techniques to combine of test inputs to generate optimal number of test cases.

All pairs testing
… is a combinatorial method of software testing that, for each pair of input parameters to a system (typically, a software algorithm), tests all possible discrete combinations of those parameters. Using carefully chosen test vectors, this can be done much faster than an exhaustive search of all combinations of all parameters, by “parallelizing” the tests of parameter pairs. (From Wikipedia )
Click here to read in detail.

Orthogonal array testing
… is a black box testing technique that is a systematic, statistical way of software testing. It is used when the number of inputs to the system is relatively small, but too large to allow for exhaustive testing of every possible input to the systems. It is particularly effective in finding errors associated with faulty logic within computer software systems.Orthogonal arrays can be applied in user interface testing, system testing, regression testing, configuration testing and performance testing. The permutations of factor levels comprising a single treatment are so chosen that their responses are uncorrelated and therefore each treatment gives a unique piece of information. The net effects of organizing the experiment in such treatments is that the same piece of information is gathered in the minimum number of experiments.

(From Wikipedia)

Click here to read in detail.

Execution Approach
The terms “Manual testing” and “Automated testing” really indicate the method of execution of test cases, the former implying using a human being whilst the latter is about using a machine to execute the test cases.

Black-box testing
… is a method of software testing that examines the functionality of an application without peering into its internal structures or workings. This method of test can be applied virtually to every level of software testing: unit, integration, system and acceptance. It is sometimes referred to as specification-based testing. (From Wikipedia)
Click here to read in detail.

White-box testing
… is a method of testing software that tests internal structures or workings of an application, as opposed to its functionality. White-box testing can be applied at the unit, integration and system levels of the software testing process. Although traditional testers tended to think of white-box testing as being done at the unit level, it is used for integration and system testing more frequently today. It can test paths within a unit, paths between units during integration, and between subsystems during a system–level test. Though this method of test design can uncover many errors or problems, it has the potential to miss unimplemented parts of the specification or missing requirements. (From Wikipedia)
Click here to read in detail

Model-based testing
… is an application of model-based design for designing and optionally also executing artifacts to perform software testing or system testing. Models can be used to represent the desired behavior of a system under test (SUT), or to represent testing strategies and a test environment. (From Wikipedia)
Click here to read in detail.

Adhoc testing
… an approach to testing a software in a unstructured manner with consciously applying any techniques to design test cases, to break the system using unconventional ways.
Click here to read in detail.

Exploratory testing
… is an approach to software testing that is concisely described as simultaneous learning, test design and test execution. It is defined as”a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of his/her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.” (From Wikipedia)
Click here to read in detail. Another great read on this is here.

Risk-based testing (RBT)
… is a testing approach which considers risks of the software product as the guiding factor to support decisions in all phases of the test process.
Click here to read the full article.

Shift left testing
While early testing has been highly recommended by software testing and QA experts, there has been a special emphasis on this agile approach of Shift Left testing that recommends reversing the testing approach and involving system/software testing earlier in the lifecycle. Practically, move the testing approach to the left end on the project timeline.
(From Cigniti blog, Click here to read the blog).

Continuous integration (CI)
… the practice of merging all developer working copies to a shared mainline several times a day. Extreme programming (XP) adopted the concept of CI and did advocate integrating more than once per day. (From Wikipedia)
Click here to read in detail.

Continuous delivery (CD)
... a software engineering approach in which teams produce software in short cycles, ensuring that the software can be reliably released at any time and, when releasing the software, doing so manually. It aims at building, testing, and releasing software with greater speed and frequency. The approach helps reduce the cost, time, and risk of delivering changes by allowing for more incremental updates to applications in production. A straightforward and repeatable deployment process is important for continuous delivery. CD contrasts with continuous deployment, a similar approach in which software is also produced in short cycles but through automated deployments rather than manual ones. (From Wikipedia)
Click here to read in detail.

Test-driven development (TDD)
… is a software development process that relies on the repetition of a very short development cycle: requirements are turned into very specific test cases, then the software is improved to pass the new tests, only. This is opposed to software development that allows software to be added that is not proven to meet requirements. (From Wikipedia)
Click here to read in detail.

Acceptance test–driven development (ATDD)
… is a development methodology based on communication between the business customers, the developers, and the testers. ATDD encompasses many of the same practices as specification by example (SBE), behavior-driven development (BDD),example-driven development (EDD), and support-driven development also called story test–driven development (SDD). All these processes aid developers and testers in understanding the customer’s needs prior to implementation and allow customers to be able to converse in their own domain language.

ATDD is closely related to test-driven development (TDD). It differs by the emphasis on developer-tester-business customer collaboration. ATDD encompasses acceptance testing, but highlights writing acceptance tests before developers begin coding. (From Wikipedia)
Click here to read in detail.

Behavior-driven development (BDD)
… is a software development process that emerged from test-driven development (TDD).Behavior-driven development combines the general techniques and principles of TDD with ideas from domain-driven design and object-oriented analysis and design to provide software development and management teams with shared tools and a shared process to collaborate on software development. (From Wikipedia)
Click here to read in detail.

Health checks
Sanity testing is primarily a basic evaluation for the system to ascertain if it basically good enough to go ahead so that we may proceed to do thorough testing. Regression testing on the other hand is primarily about if checking prior health of the system has not not compromised due to changes done now. It simply means that was was available and working in the system continues to do so.

About SmartQA The theme of SmartQA is to explore various dimensions of smartness to leapfrog into the new age of software development, to accomplish more with less by exploiting our intellect along with technology. Towards this, we will strive to showcase interesting thoughts, expert industry views through high quality content as articles, posters, videos, surveys outlined as a SmartQA Digest weekly emailer. SmartBites is soundbites from smart people”. Ideas, thoughts and views to inspire you to think differently.

Signup to receive SmartQA digest that has something interesting weekly to becoming smarter in QA and delivering great products.

7 Thinking Tools to Test Rapidly

by T Ashok

Summary
The act of testing is a scientific exploration of a system done in three phases – RECONNAISSANCE to understand and plan, SEARCH to look for issues, REST&RECOVER to analyse and course correct. To enable the various activities in each phase to be done quickly and effectively, is where the SEVEN Thinking Tools outlined in this article help. How to apply these tools in a session-based approach is also briefly outlined.

When I hear people talking about testing as Manual or Automated with the latter being the need of the hour, I am flabbergasted. All the word ‘manual’ conjures in my brain is that of me doing a menial job of painful scrubbing!

Definition of manual from Cambridge Dictionary

It is time we used Intellectual & Tool-supported. “Think well. Exploit tools to do.”Enough of rant!

In current times, speed is everything, right? What can we do to test quickly ? Use tools. Automate. Right? Wait a minute – This is about execution, right? What about prior activities?

To answer, let us ask the basic question what is testing after all? Testing is exploration. Let me correct it. Testing is scientific exploration. And exploration is a human activity that is aided by tools & technology. How can we do scientific exploration rapidly? By using tools that help us think better and do faster.

Let us say you want to explore the nearby mountain range by foot. Would you just pick up your backpack and go on? I bet not, unless it is a really short trip. Otherwise I think you will study the geography/terrain, read others’ experiences, do a reconnaissance, create various maps of terrain, of pit stops, of food joints etc before you chalk out the full route. Once the route is setup, you will pack your bags and go. As you explore, you will discover that “the map is not the terrain” and be taunted, surprised, challenged and you will learn, adjust, improvise, revise the maps, routes as needed. Tired, you will rest, analyse, replan & recover to continue on your journey. This is not ad-hoc nor driven by sheer bravado. This requires logical thinking(scientific), planning and ability to observe, adjust continuously and also some bravado and good luck!

This is what we can apply too in testing our software/systems. This article distils this and provides you with SEVEN THINKING TOOLS to enable you do these easily and scientifically.

Applying the above analogy, we look act of testing as being done in THREE phases: RECONNAISANCE, EXPLORATION, REST&RECOVER.

RECONNAISANCE : Do survey and create maps
Survey : Get to understand the system under test by reading documents, playing with the software/system, discussing with people, to clearly understand who the end users are, what the entities (e.g. features, requirements..) to test are, what the various attributes the end users expect are, and the environment in which it will be deployed. In a nutshell we want to know the Who, What-to-test, What-to-test-for and Where. This is done by the Tool #1 “Landscaper”.

Create Maps: Now that you know the key information, connect them to four useful maps: Persona map, Scope map, Interaction map and Environment map.

(Tool #2) Persona map : A list that clearly connects the “Who” to What”. This helps us understand who uses what and therefore helps us prioritise testing and certainly enables us to get user centric view to validation.

(Tool #3 )Scope map: A list that connects the ‘What’ to ‘What-for’. This helps us to understand what the expectations of the various entities are i.e. That for even feature F1, we have an expectation of performance. What does this help us do? This helps us identify the various types of tests to be done.

(Tool #4) Interaction map: No entity is an island i.e. each entity may affect one or more entities. i.e a feature F1 may affect another feature F2 and therefore a modification of F1 may require retesting of F2. How does his map help us? Well, this helps plan our regression strategy intelligently.

(Tool #5) Environment map : This lists out the various environments on which the final system may be run so that the functionality and attributes may be evaluated on various deployment environments.

Now that we have done done the reconnaissance, we should have good idea of the system under test and therefore be ready to explore.

SEARCH : Now that we have the maps, the next step would be chalk out the routes and then we are ready to commence our search for issues. This is done using the “Scenario creator” tool. Once this is done we commence our search for issues. When doing this we will encounter things we don’t know, things that we did not anticipate, issues and therefore will need to revise and course correct revise the landscape, maps and routes. This is accomplished via the Dashboard tool in the Rest&Recover phase.

(Tool #6) Scenario creator: This tool helps to design the various test scenarios that would serve as the starting point. Note that these will be continuously revised as we explore and gain a deeper understand of the system and its context and usage. What is important would be segregate the scenarios into Levels so that the test scenarios are focused and clear in their objective. Robust Test Design approach of HBT helps you design scenarios that may be done using a mix of formal techniques, past experience, domain knowledge, context but clearly segregated into various HBT Quality Levels.

REST& RECOVER: In this phase, the objective is to analyse the exploration phase results and improve what we can do, track progress of doing and judge of the quality of the system-under-test. This is done by the tool ‘Dashboard’

(Tool #7) Dashboard: This tool helps you to do there things : (a) judge adequacy by look at the map and route information and improve the same (b) track progress of work done by checking the what has been done vs.planned as far as routes are concerned (c) judge quality by looking at the execution outputs of the scenarios level wise.

So how do we apply this tools?
We saw that these 7 tools could be used in the THREE phases of RECONNAISSANCE, SEARCH, REST&RECOVER by a session based approach “Immersive Session Testing”. Each session is suggested to be short and focused say 60-90 minutes with a session objective to one or a mix of the phases.

Note that a session could be an exclusive RECONNAISSANCE or SEARCH or REST&RECOVER on a combination of these. Why is the session time suggested to be 60-90 minutes? Well this is to ensure razor sharp focus on each on the activity done. Also a short focussed sessions allow one to get into a state of flow enabling higher productivity and enjoy the activity!

Click here to play the webinar video.

Click here to see the slides in SlideShare

Signup to receive SmartQA digest that has something interesting weekly to becoming smarter in QA and delivering great products.

Automation SmartQA testing

The Buddha Story: Good understanding

by T Ashok

Good understanding is not about knowing everything in detail, it is about just knowing what is required and learning to discard what is not necessary or deferring to a later stage.

Let me tell you a beautiful Buddha story that illustrates this.
Once upon a time there lived a wise and learned man with mastery over various languages, philosophies and religions. He was extremely proud of his knowledge, but eager to learn more. He came to know that a person named Buddha lived far away and was a very learned man. Keen to further his knowledge, he went to Buddha, introduced himself as a man well versed with various philosophies, expressed his wish to enhance his knowledge and requested to teach him.

Buddha said he would be happy to do and requested him to kindly sit in the corner of the room. A few hours passed and the learned man was getting fidgety while Buddha continued to do his work. As the sun went down, Buddha suggested that they resume the next day. The learned gentleman after a good night’s rest came back promptly. Buddha again requested him sit down in the corner. The morning turned into afternoon and the dusk set in. The man was throughly upset that Buddha was ignoring him. He went up to him and said “Sir, I am very upset that you are ignoring me, I hope you understand that I am also a man of knowledge”.

Buddha calmly said “Come my dear friend, sit down let us have a cup of tea”. He took the tea pot and poured the tea onto the cup while watching the man. The cup filled to the brim and Buddha continued pouring, spilling the tea on the floor. The man watching this shouted “The cup is full, please stop pouring!”.

Buddha fully aware of what he was doing said “My friend, your mind is like this, filled with knowledge, with no space to acquire additional knowledge. Empty it and you will be ready to learn.” And only then did the learned man realise his folly.

As a test practitioner, understanding a complex system is always always interesting. The big challenge that I face is when passionate folks explain the system sparing no detail. That is when I say “STOP, I do not need to know this, will certainly ask when I need it”.Good understanding requires that the tea cup be only partially full.

Defer what is not needed now to later. Don’t attempt to understand everything at one go.

Smart QA is about having great clarity, a result of good understanding. This occurs when you are actively questioning with an agile mind, not passively absorbing and getting weighed down. Lessen your burden and your mind is like a nimble goat climbing the mountain of complexity. At the top, view the system in totality clearly. And the issues pop out.

HBT (Hypothesis Based Testing) is based on this principle of deferment and active questioning to understand complex systems rapidly and test smartly.

Signup to receive SmartQA digest that has something interesting weekly to becoming smarter in QA and delivering great products.

HBT Hypothesis Based Testing Software Testing