Standard Disclaimer: This discussion is about high reliability software for situations where software failures can have significant impacts. The ideas discussed are not necessarily appropriate or cost effective for most software products.

When refactoring high reliability software it is often important not to introduce any observable behavioural changes – no bugs. But how do you go about it?

One answer is to have very high test coverage. Ensuring a high line and branch coverage is not enough to ensure that your refactoring is safe. The reason for this is that even 100% branch coverage might only test a very small subset of the possible executions of a program. A simple example:

int myFunction(int x)
{
   int result = 0;
   if(x % 2 == 0)
   {
       // A (even)
       result++;
   }
   else
   {
       // B (odd)
       result--;
   }
 
   if(x % 3 == 0)
   {
       // C (divisible by 3)
       result++;
   }
   else
   {
       // D (indivisible by 3)
       result--;
   }
 
   return result;
}

So, we can test all the branches with the tests values 2,3,7.

• 2 – branches A,D
• 3 – branches B,C
• 7 – branches B,D

So, even with this simple code where there is 100% branch and statement coverage, it does not cover the path “A,C”. An additional test case is required. For example the value (x == 6) would cover “A,C”.

In practice, there can be a very large number of paths through a program, so exhaustively testing them can be either very expensive or completely impractical.