#209406 - 2014-08-11 11:02 PM
Math Question
|
ShaneEP
MM club member
Registered: 2002-11-29
Posts: 2125
Loc: Tulsa, OK
|
Am I missing something stupid here? Can someone explain to me why the following code results in an output of 1.4210854715202E-014 ?
$num = 729.95-629.96-99.99
? $num
get $
I would expect it to be simple 0.00.
|
Top
|
|
|
|
#209410 - 2014-08-12 05:08 PM
Re: Math Question
[Re: Glenn Barnas]
|
ShaneEP
MM club member
Registered: 2002-11-29
Posts: 2125
Loc: Tulsa, OK
|
And unfortunately the FormatNumber() in kix doesn't even fix it.
results in a long string of 0's, and not just 0.00 like it should. Having to use the Round() function to get around it for now.
|
Top
|
|
|
|
#209411 - 2014-08-12 05:35 PM
Re: Math Question
[Re: ShaneEP]
|
BradV
Seasoned Scripter
Registered: 2006-08-16
Posts: 686
Loc: Maryland, USA
|
Yes, I was going to post earlier, but can't from work. It has to do with how floating point numbers are stored in memory. It is usually just an approximation. You have to keep track of your significant digits. In your case, you probably want to multiply by 1000, convert to an integer, add 0.00 plus your variable, divide by 1000 to get a more accurate answer.$num=....
$num2=$num*1000
$num3=int($num2)
$num4=0.00+$num3
$num5=$num4/1000 Of course, you can condense that. Just want to illustrate that this will give you 0.0. You can change your "-99.99" to "-99.97" and your answer will come out 0.02.
|
Top
|
|
|
|
#209412 - 2014-08-12 06:03 PM
Re: Math Question
[Re: BradV]
|
Henriques
Fresh Scripter
Registered: 2007-09-13
Posts: 43
|
$num = 729.95 - (629.96 + 99.99) ? $num get $
Will give you the right answer.
|
Top
|
|
|
|
#209436 - 2014-08-20 01:33 AM
Re: Math Question
[Re: Lonkero]
|
NTDOC
Administrator
Registered: 2000-07-28
Posts: 11623
Loc: CA
|
Well if this is a math question....
|
Top
|
|
|
|
Moderator: Jochen, Allen, Radimus, Glenn Barnas, ShaneEP, Ruud van Velsen, Arend_, Mart
|
2 registered
(morganw, mole)
and 414 anonymous users online.
|
|
|