FiveTech Software tech support forums

Sometimes if I compare two decimal point values also if they seem to be the same they have a slightly difference.

I do like this:

if INT(value1 * 1000) - INT(value* 1000) <> 0
endif

But I believe there is a build in function. Does someone know one?
Thanks in advance
Otto

Hello Otto,

I think You have to use the ROUND()-Function for exact results.

Some samples :

SET DECIMALS TO 2
SET FIXED ON

// Function ROUND( nValue, nDec ) => rounded
// nDec = how many Dec.
// Negative = round to 10, 100 ......

ROUND( 10.4, 0 ) => Result : 10.00
ROUND( 10.5, 0 ) => Result : 11.00
ROUND( 10.51, 0 ) => Result : 11.00
ROUND( 10.499999999999, 2 ) => Result 10.50

To round before dec, the argument < nDec > is negative

ROUND(101.99, -1) => Result : 100.00
ROUND(109.99, -1) => Result : 110.00
ROUND(101.99, -2 => Result : 100.00

Regards
Uwe

I compare this way for approximate equality:

Code: Select all  Expand view

#define TOLERANCE 0.0001   // your value


if ABS( n1 - n2 ) < TOLERANCE
   // approximately equal
else
  // not even approx equal
endif



The best way to compare two decimal values:

if ROUND(value1,2) = ROUND(value,2)
endif

Regards,
Birol Betoncu

I also have problems with INT()!
If I call INT(nParam/aArray[1]) and both params are variables with the same value as decimal-number the result should be 1. But in some cases this brings as result 0! I increase the value from the first param with 0.000001 and all is OK.
Is the number-format from the variables different?? INT-function buggy?

Hello Günther,

there is nothing wrong with INT().

The function INT() just cuts the decimals ( the values are not rounded ).

var1 := 1.56
var2 := 1.63
var3 := var1 + var2 = 3.19

var4 := INT(var1) + INT(var2) = 2

After the calculation of var1 + var2 => INT(var3) = 3 is OK

If You don't want to use Decimals, You have to use the function ROUND() for exact values.
INT(), I use only for Number-display.
Sample :
I want to show a browser with running numbers inside a listbox.
I start with < nNumber := 1 > than nNumber++ and so on.
The var nNumber is shown in the listbox => 1.00, 2.00 ........
in relation to SET DECIMALS. That is the reason I use INT(nNumber) for display.

Regards
Uwe

I confirm what Uwe said.

EMG

This wrong result of 0 of the following code are only, if one of the values are from an array.
In my actual situation both values in a loop became 0.01.
The result from 0.01/0.01 = 1. And INT(1) should be also 1! But it is 0!

Code: Select all  Expand view

INT(nParam/aArray[1])

Please post a reduced and self-contained sample of the problem.

EMG

I had the same problem in the past and use STRZERO(nVar,10,2) .
I compare strzero of var1 with strzero of var2 .

Hi all

Isn't this essentially just a reflection of the fact that (x)Harbour numeric variables are handled as C double precision floats which means binary representations of floating point numbers and thus the results of (particularly division) operations may not be able to be represented exactly. You can see the problem is equivalent to us using base 10 decimals: Take 10.0000 / 3 and we would get 0.3333. Multiply that by 3 and we would get 0.9999. In other words, seen like this ( 10.000 / 3 ) * 3 = 0.9999 which isn't (exactly) equal to 10.0000. And INT( 0.9999 ) = 0.

Some variance of the tolerance idea should solve the problem. Evaluating the difference between two values as being less than a given tolerance is at least theoretically more robust (and more configurable) than rounding to a given number of decimal places, although the latter method is probably acceptable for most monetary calculations (dollars and cents) where the answer has to come out eventually to two decimal places. That's because depending upon the rounding function either 5.4999999 or 5.50000001 will round differently from 5.5000000 ! Also, although I haven't tested it, I would be pretty sure that testing approximate equality by comparing rounded values would require substantially more CPU clocks.

betoncu wrote:The best way to compare two decimal values:

if ROUND(value1,2) = ROUND(value,2)
endif

Regards,
Birol Betoncu

This is not the correct logic. Please consider the following numbers.


a := 1.01
b := 1.01499999999999
c := 1.015

Difference between a and b is almost 0.005, where as difference between b and c is near zero.
But the above logic shows a is approx equal to b and b is not even approx equal to c, because a,b and c rouned are 1.01, 1.01 and 1.02.

So, ROUND( a, x ) - ROUND( b, x ) <> 0 also is not the correct logic.

Comparing difference of the two numbers with a tolerance value is the right logic, which can be written in many ways.

Even ROUND( a - b, nDecAccuracy ) == 0 also is another way of writing.

This is the code i use :

Code: Select all  Expand view

x = n1 - n2
return ( x > -epsilon .and. x < epsilon )  // epsilon is my constant value


Yes, you are right.
ROUND( a - b, nDecAccuracy ) == 0 is the best.

Birol:)

Oops, discard my previous message.