Mutation Software Testing

Back to: Software Testing Tutorials

Mutation Software Testing

In this article, I am going to discuss Mutation Software Testing. Please read our previous article, where we discussed Globalization Software Testing. At the end of this article, you will understand the following essential pointers related to Mutation Software Testing.

1. What is Mutation Software Testing?
2. What is Mutation?
3. What are the Objectives of Mutation Testing?
4. How to do a Mutation Test?
5. Types of Mutation Software Testing
6. Mention tools used for Mutation Testing
7. What are the Advantages of Mutation Testing?
8. What are the Disadvantages of Mutation Testing?

What is Mutation Software Testing?

Mutation testing is a white-box approach to software testing where mistakes are purposefully introduced into the program that is being tested to see if the test cases already in place can catch the problems or not. In this testing, the original program is modified to create the mutant version of the program. Mutation testing’s primary goal is to determine whether each mutant produced an output that differs from the output of the original program. We will only make minor adjustments to the mutant program because significant changes would impact the overall strategy.

The fact that we found so many mistakes suggests that either the program is sound or the test case is ineffective at finding the problem. The purpose of mutation testing is to assess the strength of the test cases that should be able to fail the mutant code; consequently, this method is also referred to as fault-based testing because it is used to cause a bug in the program. For this reason, we can say that mutation testing is carried out to evaluate the effectiveness of the test cases.

What is Mutation?

The term “mutation” refers to a modest adjustment made to a program to address common low-level faults that occur throughout the coding process. In most cases, we plan the mutation operators in the form of rules that match the data and create some effective conditions for the mutant to be produced.

What are the Objectives of Mutation Testing?

Testing for mutations has the following goal:

• To locate faulty code that has not been thoroughly tested.
• To find hidden flaws that other testing techniques are unable to see.
• To identify novel faults or problems.
• The mutation score must be calculated.
• To investigate state infection and error propagation within the program.
• To evaluate the test cases’ level of quality.

How to do a Mutation Test?

We shall carry out mutation testing as described below:

• We will create various known mutant versions to add the flaws to the program’s source code. Every mutant in this situation has one defect, which causes the mutant types to fail and confirms the effectiveness of the test instances.
• The actual application will then be used to detect the flaws in the code with the assistance of the test cases in the mutant program.
• We will compare the output of the actual code with the mutant code after we have located the errors.
• The test cases then run the mutant if the output comparison between the natural and mutant programs yields inconsistent results. As a result, the test case must be adequate for detecting differences between the mutant and original programs.
• Additionally, the mutant is saved if the results of the original program and the modified programs’ results are identical. And because they enable us to carry out all the mutants, those cases serve as more active test cases.

Explain Types of Mutation Software Testing

There are three primary forms of mutation testing:

Value Mutations:

In this testing method, values are modified to find program faults. Essentially, a significant value is changed to a smaller value, or a small value is changed to a larger one. Constants are effectively altered in this testing.

For example:
Initial Code:
int mod = 1000000007;
int x = 12345678;
int y = 98765432;
int z = (x + y) % mod;

Changed Code:
int mod = 1007;
int x = 12345678;
int y = 98765432;
int z = (x + y) % mod;

Decision Mutations:

To find program faults, decision mutations modify logical or arithmetic operators.

For example:
Initial Code:
if(x < y)
   z = 10;
else
   z = 20;

Changed Code:
if(x > y)
   z = 10;
else
   z = 20;

Statement Mutations:

Statement mutations include the deletion or replacement of a statement with another statement.

For example:
Initial Code:
if(x < y)
   z = 10;
else
   z = 20;

Changed Code:
if(x < y)
   d = 10;
else
   d = 20;

Mention tools used for Mutation Testing

• Judy
• Jester
• Jumble
• PIT
• MuClipse

What are the Advantages of Mutation Testing?

The benefits of Mutation Testing are:

• It improves the program’s error detection capabilities.
• The source code’s ambiguities are found.
• It locates and fixes problems with program loopholes.
• It aids testers in creating or automating more effective test scenarios.
• It offers programming source code that is more effective.

What are the Disadvantages of Mutation Testing?

The drawbacks of Mutation Testing are:

• It requires a significant investment of both finances and time.
• Black Box Testing is not possible with it.
• Some mutations are complex, making developing or applying them to various test situations challenging.
• Here, the team members doing the tests should be knowledgeable in programming.
• It is crucial to choose the right automation tool while testing applications.

In the next article, I am going to discuss Security Software Testing. Here, in this article, I try to explain Mutation Software Testing. I hope you enjoy this Mutation Software Testing article.