Floating-point round-off errors - MS Excel example
I was looking for an Excel spreadsheet the other day that showed how easy it is to come unstuck doing floating-point arithmetic, due to rounding errors. I couldn’t find one easily, so I ended up writing my own. Basically, it shows the cumulative round-off error when iteratively calculating:
s(i+1) = [s(i)-0.9]/0.1
It also shows what happens for 0.99, 0.999 etc. You get the idea. s(1) is 1, obviously. There are plenty of other ways of showing this problem.
Here’s also a nice comic strip on rounding errors.
Comments
Leave a Reply