floating point arithmetic

[jbergeron]

Greetings,

I'm a bit new to javascript, so excuse my ignorance here.

var n1 = -0.049999237060546875;
var n2 = 0.05000019073486328;

var p = n1 + n2; 

The result of this operation is something along the lines of: "9.53674316406E-07". What does that number mean exactly, and why does it occur?

This result also occurs in the PHP language. I came across this situation when logging my trades to a database. I thought it may be due to the length of the number, but it seems to occur with the values in my database which are rounded to 2 places! Which doesn't seem to make much sense...because I can't reproduce it.

Any thoughts?

Cheers,
Joshua C. Bergeron

[Doji3333]

As I understand it...

This number means the following...

Take the first portion of the returned value...

9.53674316406

Now, look at the second portion...

E-07

"E-07" means..

NEGATIVE ("10" to the power of 7) = or...

-100,000,000

or...

9.53674316406 * -100,000,000 =

0.00000000953674316406.

This is a way for the computer/calculator to show REALLY large or small numbers...

Hope this helps...

Brad

[Doji3333]

Ooops...

Screwed that one up...

See what happens when you take three phone calls while trying to write an answer....

Here is the real solution...

----------------------------------------------
Take the first portion of the returned value...

9.53674316406

Now, look at the second portion...

E-07

"E-07" means..

1/10 = .1
.1/10 = .01
.01/10 = .001
... seven times.....

( which can be simplified to 1/10000000) - notice there are "7" ZEROs in the "divide by" value - ie: "E-07".
results in
0.0000001

Now multiply the numbers...

9.53674316406 * 0.0000001 =

0.000000953674316406.

if the "E" number were positive, you would use the previous posts calculations (for really large numbers).

Hope this helps...

Brad

[jbergeron]

Thanks, that does help. I should have known that

*smacks head*

Cheers,
Josh