A random number generator can be used to generate test cases when writing a regression test. This technical article describes the methods and details on using a random number generator for test case generation.
A random number generator is a function that returns a random looking number when called. Each time the random number function is called, a different random looking number is returned.
Typically a random number generator is implemented using a pseudo random number algorithm. A pseudo random number generator's output is entirely depended on its seed value. Given a seed, its list of generated numbers is deterministic.
There are also hardware based true random number generators that will generate a non-deterministic sequence. A typical application is running a lottory.
For generating test cases, we want pseudo random numbers. We need to have two runs of the test program to be repeatable. Because a pseudo random number is deterministic, the generated test cases are naturally repeatable.
Just as an illustration, you can build a very simple random generator by doing integer multiplication. This generator is not very statistically random; I will show some better ones later on.
public class rand1
{
public int seed = 1;
public int nextInt()
{
seed = seed * 69069;
return seed;
}
}
This is a pseudo random number generator due to George Marsaglia. This generator has good statistical random property, so it is suitable for test case generation.
static int s1 = 0x52f7d319;
static int s2 = 0x6e28014a;
public static int nextInt()
{
// MWC generator, period length 1014595583
final int v1 = s1;
final int v2 = s2;
return ((s1 = 36969 * (v1 & 0xffff) + (v1 >>> 16)) << 16) ^
(s2 = 30963 * (v2 & 0xffff) + (v2 >>> 16));
}
public static long nextLong()
{
return (((long) nextInt()) << 32) + nextInt();
}
public static float nextFloat() /* from 0.0 to 0.99999999 */
{
return 5.9604645e-8f * (0x00ffffff & nextInt());
}
public static double nextDouble() /* from 0.0 to 0.99999999 */
{
return 2.22044604925031e-16 * (0x000fffffffffffffL & nextLong());
}
Java has a built-in pseudo random generator located in java.util.Random. It has the usual nextInt(), nextFloat(), and nextDouble() methods.
Random gen = new Random(17L); System.out.println(gen.nextInt()); System.out.println(gen.nextFloat()); System.out.println(gen.nextDouble());
The implementation of java.util.Random should not change in principle in different JDK versions. However, if you want iron-clad guarantee of no change, use your own random number generator from what I have listed above.
For example, we want to test Math.min(int a, int b) to see if it is working correctly or not. We generate this test case by writing:
public void mytestcase(Random gen)
{
int a = gen.nextInt();
int b = gen.nextInt();
int c = Math.min(a,b);
System.out.println("Math.min(" + a + ',' + b + ") = " + c);
}
The beauty of this system is that a new test case is generated each time mytestcase() is called. Without writing thousands of hand-written test cases, you can generate all the test cases you want by simply calling mytestcase() in a loop.
There are three ways to verify the output correctness of the test cases. They are 'direct-verification', 'cross-checking', and 'output-recording'.
If the function to be tested is very simple, we can directly verify its output.
c = Math.min(a,b);
if (((c != a) && (c != b)) || (c > a) || (c > b))
{
System.out.println("FAILED");
}
Cross-checking is a way to check for correctness by comparing the method under test with a related function. If we are testing Math.min(), then Math.max() can be used for cross-checking.
if (Math.min(a,b) > Math.max(a,b))
{
System.out.println("FAILED");
}
Output recording is an approach where we record the testcase output under a good JDK, then we record the testcase output of the JDK under measurement. If the two outputs are identical, then the test case passes.
Testing with all legal input is straight forward. However, if we decide to test for cases with illegal inputs, then we will need to deal with possibility of triggering Java exceptions.
Usually we can get a list of possible exceptions from the comment section of the method under test.
Integer.java:
/**
* Parses the string argument as a signed decimal integer. The
* characters in the string must all be decimal digits, except that
* the first character may be an ASCII minus sign '-'
* ('\u002d') to indicate a negative value. The resulting
* integer value is returned, exactly as if the argument and the radix
* 10 were given as arguments to the
* {@link #parseInt(java.lang.String, int)} method.
*
* @param s a string.
* @return the integer represented by the argument in decimal.
* @exception NumberFormatException if the string does not contain a
* parsable integer.
*/
public static int parseInt(String s) throws NumberFormatException {
return parseInt(s,10);
}
From looking at the comments, we know if we generate an illegal input to Integer.parseInt(), we will get a NumberFormatException.
For some other cases, an illegal input may generate an unspecified RuntimeException. For these cases, we have to catch the RuntimeException, and verify the input is illegal.
Although the random number generator is helping you generating test cases, you still have to be aware of what exactly is being tested. Think what values you are not testing by design. Perhaps you are limiting a random string's length to 200 characters, then that means you are not testing cases with more than 200 characters.
Boundary testing is lacking with random number based tests, because unless you manually program the boundary values in the test cases, a random roll may not hit on the boundary values during the test run. Some hand written boundary value tests can supplement random number based testing.
In this article, we discussed the practice of using random numbers to generate test cases. This can be a valuable tool in writing a regression package.