[futurebasic] Re: [FB] FB^3 math accuracy

Message: < previous - next > : Reply : Subscribe : Cleanse
Home   : September 1999 : Group Archive : Group : All Groups

From: Chris Stasny <staz@...>
Date: Wed, 22 Sep 1999 11:44:51 -0500
>The BCD math in FB II could induce math errors in simple addition and
>subtraction calculations. For example, my program works extensively with
>dollar and cent calculations, but sometimes simple calculations would
>incorrectly produce fractional cent results. Thus, I have to always use
>extra steps to truncate or round results to eliminate incorrect
>fractional cents.
>On page 29 of the FB^3 conversion document it cautions that instead of
>using IF a#=10000.01 one should use IF a#>10000.01 to test for matches.
>Unfortunately, I need the ability to have exact matches to the penny
>(like IF a#=10000.01). Does this statement in the document imply that
>FB^3 will have problems with this?
>Will FB^3 still suffer the same BCD math errors I experienced with FB
>Doug Stemen

I think you have this backwards. BCD math was accurate up to 2048 digits
(according to prefs settings). SANE and PPC math are base on fractional
approximations of numbers. BCD is and alwasy has been exact. SANE & PPC
never will be.


-STAZ  ~)~

800.348.2623 Orders  http://www.stazsoftware.com
228.255.7086 FAX     mailto:staz@...