[futurebasic] Re: [FB] FB^3 math accuracy

Message: < previous - next > : Reply : Subscribe : Cleanse
Home   : September 1999 : Group Archive : Group : All Groups

From: Mel Patrick <mel@...>
Date: Wed, 22 Sep 1999 12:03:07 -0700
> > The BCD math in FB II could induce math errors in simple addition and
>> subtraction calculations. For example, my program works extensively with
>> dollar and cent calculations, but sometimes simple calculations would
>> incorrectly produce fractional cent results. Thus, I have to always use
>> extra steps to truncate or round results to eliminate incorrect
> > fractional cents.
>
>I was under the impression that, when working with currencies, the best way
>to preserve accuracy was to use integer math working with the smallest unit
>of the currency and then converting to the largest unit when displaying the
>value:

The ABS (American Banking System) used to use something akin to long 
ints for everything to do with the money stuff because of the 
accuracy. Any accounting software I ever worked on also used integers 
but for display, they shoved in the decimal so the human side of the 
equation wouldn't be confused.

For example $109.33, is one hundred nine dollars and 33 cents. But 
the .33 in computerese is 33/100 (a fraction that is infinite). So if 
you simply stored the number as 10933 and put in the decimal for 
display purposes only, there'd be a lot more accuracy and less hassle.

Just my .02 cents worth.

Mel Patrick
mel@...