[futurebasic] Re: [FB] FB^3 math accuracy

Message: < previous - next > : Reply : Subscribe : Cleanse
Home   : September 1999 : Group Archive : Group : All Groups

From: "Michael Evans" <mikonic@...>
Date: Wed, 22 Sep 1999 07:54:16 -0400
> The BCD math in FB II could induce math errors in simple addition and
> subtraction calculations. For example, my program works extensively with
> dollar and cent calculations, but sometimes simple calculations would
> incorrectly produce fractional cent results. Thus, I have to always use
> extra steps to truncate or round results to eliminate incorrect
> fractional cents.
>
> On page 29 of the FB^3 conversion document it cautions that instead of
> using IF a#=10000.01 one should use IF a#>10000.01 to test for matches.
>
> Unfortunately, I need the ability to have exact matches to the penny
> (like IF a#=10000.01). Does this statement in the document imply that
> FB^3 will have problems with this?
>
> Will FB^3 still suffer the same BCD math errors I experienced with FB
> II?
>
> Thanks!
>
> Doug Stemen
>
>
> --
> To unsubscribe, send ANY message to <futurebasic-unsubscribe@...>
>

I was under the impression that, when working with currencies, the best way
to preserve accuracy was to use integer math working with the smallest unit
of the currency and then converting to the largest unit when displaying the
value:

Thus:

myCents& = 300536                        'good?
myDollars$ = "$" + str$(300536/100) 'needs rounding?

rather than:

myDollars# = 3005.36                     'bad?
myDollars$ = "$" + str$(myDollars#)

I suppose if you need accuracy to 1/100 of a cent you could

myCents& = 30053600 'good
myDollars$ = "$" + str$(30053600/10000) 'needs rounding?

Cheers,


----------------------------------------------------------
Michael Evans
Manager of Software Development * Photo Systems, Inc.
3301 Wood Valley Road, NW * Atlanta, GA, 30327-1515
Voice: (404) 846-9386
Fax: (404) 240-0878 * Cell: (404) 229-3930
E-mail: evans@... * michael_evans@...
----------------------------------------------------------