Roman Leshchinskiy rl at cse.unsw.edu.au
Wed Feb 13 20:28:22 EST 2008

```Richard A. O'Keefe wrote:
>
> On 12 Feb 2008, at 5:14 pm, jerzy.karczmarczuk at info.unicaen.fr wrote:
>> Would you say that *no* typical floating-point software is reliable?
>
> With lots of hedging and clutching of protective amulets around the
> word "reliable", of course not.  What I *am* saying is that
> (a) it's exceptionally HARD to make reliable because although the
> operations
>     are well defined and arguably reasonable they do NOT obey the laws that
>     school and university mathematics teach us to expect them to obey

Ints do not obey those laws, either. It is not exceptionally hard to
write reliable software using ints. You just have to check for
exceptional conditions. That's also the case for floating point.

That said, I suspect that 90% of programs that use float and double
would be much better off using something else. The only reason to use
floating point is performance.

> This is leaving aside all sorts of machine strangeness, like the student
> whose neural net program started running hundreds of times slower than
> he expected.  I advised him to replace
>     s = 0;
>     for (i = 0; i < n; i++) s += x[i]*x[i];
> by
>     s = 0;
>     for (i = 0; i < n; i++)
>         if (fabs(x[i]) > 1e-19)
>         s += x[i]*x[i];
> and the problem went away.  Dear reader: do you know why I expected this
> problem, what it was, and why this is NOT a general solution?

I guess it trapped on creating denormals. But again, presumably the
reason the student used doubles here was because he wanted his program
to be fast. Had he read just a little bit about floating point, he would
have known that it is *not* fast under certain conditions. As it were,
he seems to have applied what he though was an optimisation (using
floating point) without knowing anything about it. A professional
programmer would get (almost) no sympathy in such a situation.

Roman
```