Division Returning 0
Hello everyone,
For some reason this line of code will result in 0:
I have already confirmed that the values going in are correct.
Even this code will result in 0 (where the values are what the original variables could possibly be set as):
After debugging (the original code), I found this:
Everything seems to be going right until IDIV is called. That call sets EAX to 0. Is that where the problem lies? I'm sure there is a simple solution. I am using Microsoft's Visual C++ 2008 Express.
For some reason this line of code will result in 0:
float scrollheight = (drawheight/textheight)*drawheight;I have already confirmed that the values going in are correct.
Even this code will result in 0 (where the values are what the original variables could possibly be set as):
float scrollheight = (124/192)*124;After debugging (the original code), I found this:
|
|
Everything seems to be going right until IDIV is called. That call sets EAX to 0. Is that where the problem lies? I'm sure there is a simple solution. I am using Microsoft's Visual C++ 2008 Express.
Integer division results in integer quotients.
124/192 = 0 in integer arithmetic.
Cast either drawheight or textheight to a float to fix.
124/192 = 0 in integer arithmetic.
Cast either drawheight or textheight to a float to fix.
Thank you!
I figured the solution would be simple. I thought it had something to do with integers, but I thought declaring scrollheight as a float would be enough to stop rounding.
I figured the solution would be simple. I thought it had something to do with integers, but I thought declaring scrollheight as a float would be enough to stop rounding.
Topic archived. No new replies allowed.