CODEWITHSUNDEEP
excelexcel-formulagreatest-common-divisor
BoltBait
BoltBait

Reputation: 11489

Excel GCD function giving strange results

I'm working on a formula to reduce fractions in Excel. The numerator and denominator are in separate cells. The numerator is in cell I38 and has a value of 1015. The denominator is in cell I40 and has a value of 100000.

I'm trying to use the GCD function to determine when I can reduce the fraction.

However, the GCD function is returning crazy results: enter image description here

Am I misunderstanding the GCD function and how it works or is there a bug in Excel?

MS Excel 2016 on Windows 10, if that helps.

EDIT: When cell I38 has a number in it, the formula works fine. When it has a formula in it, the wrong answer is returned. Here is a screenshot of the formula: enter image description here Here is the formula:

=G38*10^IF(C39=INT(C39),0,LEN(MID(C39-INT(C39),FIND(".",C39,1),LEN(C39)-FIND(".",C39,1))))

Upvotes: 0

Views: 312

Answers (1)

BoltBait
BoltBait

Reputation: 11489

The problem was I forgot to cast the result of that formula to INT (even though the result of the formula was always going to be an INT, Excel didn't see it that way). Once I added that, everything worked fine:

=INT(G38*10^IF(C39=INT(C39),0,LEN(MID(C39-INT(C39),FIND(".",C39,1),LEN(C39)-FIND(".",C39,1)))))

Thanks goes out to user @JvdV for helping me track that down in the comments.

Upvotes: 1

Related Questions