Decimal hell.

I'm very frustrated cos what I thought would a simple task turned out to be hellish.

I'm trying to make a simple function which finds percentages. I am doing this by having the user input a number on a box and the other number on another box.. EG: What percentage is 50 of 100? The result should be a simple 50, but it returns 0. I already tried changing 100 to 100.0 and it just doesn't work, some numbers just won't come out and others will give erratic results. This is my formula:

First Box / Second Box * 100.0

As far as I know I am doing nothing wrong. Thank you for yer help.

Try:
First box / (second Box * 100.0)

This will make the second number into a float before the calculation.

Now, it probably does this:

Integer / Integer = Integer
Integer * Float = Float

...which as you see, converts the result into a float AFTER the first calculation.

Quote:

---

Originally Posted by Popcorn

Try:
First box / (second Box * 100.0)

This will make the second number into a float before the calculation.

Now, it probably does this:

Integer / Integer = Integer
Integer * Float = Float

...which as you see, converts the result into a float AFTER the first calculation.

---

Tried and it partially fixes the problem. Now it does show decimals, but let's say we use the easy what % 50 is of 100, it will only show 0.005 as if it was dividing by 100 instead of multiplying.

:crazy:


EDIT: Seems after doing that I had to multiply it by 10000 for it to work... I did not know MMF worked like this? Why is that?

EDIT 2: Darn! Sometimes it gives wrong percentages now... I did randomly try to do what percentage 60 was of 2.99999 and it said 3000 when when I do it on a number of other ways it says 2000.006666688889. I'm kind of really confused about this...

The correct formula is: (First box * 100.0) / second Box. Always do multiplies first if you can, it's more accurate.

Also popcorn had the multiply grouped under the divide, so it effectively became a divide by 100...

1. You should do : (First box * 100.0) / second Box

EDIT: ah, Dynasoft was faster :)

2. It's due to the way computers store floating point values. Already discussed here.

Haha, it was early in the morning :)
Sorry for the confusion!

Thanks for all the help guys, really. By doing that it works just fine, but still gives some erratic values. Like that 60 of 2.99999
it returns 3000% which I know is very incorrect given the fact that, any other method returns 2000.006666688889%. Why is this? Any way to correct this?

It definitely sounds like something else is wrong - maths just isn't this... erratic! I would check any other events that could possibly be affecting any of those counters or values...

You'd get 60 of 2.99999 = 3000% if you were rounding down the 2.99999 to 2. You're not using "value of editbox" are you? Use "Val()" on the "text of editbox" instead.

Yes the expression of the edit box to get the nummeric value only returns integers. As Dynasoft sais you have to use Val() on the text of the edit object.