Friday, August 12, 2011

Overflow without warning

Testing might also be kind of learning programming. E.g. there's this programming resp. transformation language for structured (text) data, MetaMorphosis.
As I'm trying to figure out how that scripting language works I found myself with little documentation about datatypes. As a mathematician with interest in prime number testing I'm always interested in big integers as in java or C#. So I wrote the following hoping the loop NOT to end with an overflow:

*count := 1, *lastcount:=*count,
!while *count >= *lastcount !do
   *lastcount := *count,
   *count := 10*(*count)

The loop really didn't end by itself throwing any error. But what I saw on my screen kind of surprised me. When I paused it I found that 
1946039808 <= -1348927488  

seemed to be true. So I asked in pace mode (option -p):

> i *result := (-1348927488) - 1946039808;
> i *result;
> 1000000000

So supposing a < b will be true if and only if 0 < b - a we see that really 1946039808 <= -1348927488.
What have I learned? Well, first you cannot rely on any programming language to tell you about its limitations. Second: overflows do not only influence our results when we calculate with numbers like subtracting or adding, but go also further as seen in the comparison of two of them.

Finding only two a, b such that a-b > max(numberdomain) we cannot be sure a < b will give the expected result.

No comments:

Post a Comment