Learn math with flash, flex or javascript!

On December 20, 2007, Flash / Flex - 1 Comment

Flex / Flash has a neat feature: doing math for you.

So when you want to know what 0.1 + 0.2 is, you don’t need your calculator, just use flex. The result: 0.30000000000000004.

WTF is that about?

I’m currently trying to find out a fix for this. ATM the only workaround is doing a

?View Code ACTIONSCRIPT
var x:Number = 0.1 + 0.2
Math.round(x*10) / 10

How hard is it to add to two numbers and give the correct result?

or of course

?View Code ACTIONSCRIPT
var x:Number = (0.1 + 0.2).toFixed(1)

Works for actionscript as well as JavaScript

By the way, it’s not only 0.3 which is causing problems.

Doing a simple for loop reveals the size of this bug:

?View Code ACTIONSCRIPT
for(var i = 0; i < 20; i++) {
    trace (i * .3);
}

gives

0
0.3
0.6
0.8999999999999999
1.2
1.5
1.7999999999999998
2.1
2.4
2.6999999999999997
3
3.3
3.5999999999999996
3.9
4.2
4.5
4.8
5.1
5.3999999999999995
5

Hmm… studying it further reveals that it’s not only an actionscript problem. Also Javascript gives these strange results. That tells me that it’s an ECMAScript problem, sinds both languages are based on ECMAScript.

I’ve heard of strange rounding / precision errors, but never knew they were that simple to produce. I really hope that someone at adobe addresses this problem and fixes it for the next release of flash.

It seems to have something to do with the number 3 or .3



What others have to say:

1

Here is a quote I got my post on this subject:

“It is caused by the way floating points are stored:

http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems

Leave a Reply

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Submit comment