Vladimir Prus


vladimirprus.com

Tuesday, October 04, 2005

Black or white?

What's better: black box testing, or white box testing? In general, neither, but sometimes peeking at internals can be a great help.

One of the project at work now is writing a compiler for simplified C (basically, C without structures). And one recent addition was declaring several variables in one declaration. The existing test was:

int32 v, v2, v3;

int32 main()
{
    v3 = 15;
    v2 = v = 10;
    printf("Result = %d\n", v + v2 + v3);
    return 0;
}
and it worked immediately. But there's was a bug, and just as experiment, I've asked the author of the test to find the bug. His attempts were:
  1. Using duplicate name of variable: int32 v, v2, v3, v;. That produced an error, as expected.
  2. Defining extra local variable "v2". Still no problems.
  3. Initializing some of the variables at definition point, not inside the function.
  4. Trying different type of variables.
  5. Moving the code from main to another function.

At this point, he gave up. The real problem occured at this example:

int32 xi = 100, i = xi + 5;

int32 main()
{
    printf("Result = %d\n", i);
}
and was caused by creating variables in the reverse order. The language grammar is written in the way that makes traversal in the reverse order more natural. And so variable i was initialized before variable xi. Honestly, I can't blaim the tester. It's a kind of bug you can think about only if you know that nonterminals can be left-recursive and right-recursive. Or see the code.

No comments: