floating division issue

[thisisfun]

Hey yall, thanks for reading.

I got what is hopefully a very simple question for everyone. I just returned to c++ from a journey to nodejs and C#

I have a simple app that I am trying to upgrade to lower the amount of floating points extending.

I am coming across an issue with c++ and Visual Studio 2015 that I do not remember in the past. I.E. float testNum = 10.0F / 100.0F; -> I would expect the answer to be 0.1 but instead I am getting 0.1000000001. I have the same issue with doubles as well.

Is there a simple work around for this issue as the round off error at the end is causing me some issues.

Thanks again everyone for your time.

[IMG]http://i66.tinypic.com/50mi5x.png[/IMG]

[Cubbi]

The type float and double are binary. Just like how you can't represent 1/3 in a finite-sized decimal, you can't write 1/10 in finite-sized binary.

For floats, the value of 1/10 lies between 0x1.99999ap-4 (binary) aka 0.100000001490116119384765625 (decimal) and 0x1.999998p-4 (binary) aka 0.0999999940395355224609375 (decimal).

It is closer to the first one, so when you write "float f = 0.1f;", you're actually writing "float f = 0.100000001490116119384765625f;"

For doubles, 0.1 lies between 0x1.999999999999ap-4 (binary) aka 0.1000000000000000055511151231257827021181583404541015625 (decimal) and 0x1.9999999999999p-4 (binary) aka
0.09999999999999999167332731531132594682276248931884765625 (decimal).

Again, it is closer to the first one, so when you write "double d = 0.1;", you're actually writing "double d = 0.1000000000000000055511151231257827021181583404541015625 ;".

You deal with this the same way you deal with representing 1/3 in decimal.

[thisisfun]

that makes since thanks for the great explanation Cubbi I really appreciate it.

Building on top of my original question, this is the true issue that I am trying to work around.

in this case, I have an float of 0.1234567

i multiplied converted to int to drop off the last three percision points -> 1234 (Don't care about rounding)

now i need to put the number back to a float to be stored as a JSON number with only 4 precision points -> what I want is 0.1234

using division yields in the result you mentioned most of the time. Is there a way to shift the decimal and casting back to a float (or double) without gaining the extra precision points back.

the reason for this is to cut down on the packet size being sent via TCP and retaining a number format instead of using a string.

Thanks again Cubbi and everyone else for your time.

[Cubbi]

There are a couple dozen C++ JSON libraries,, perhaps you can find one that allows setting precision of floats? (or write up and submit a patch to the one you're using?). For example, this library recognizes the issue but says they won't do it: https://github.com/nlohmann/json/issues/345

Formally (as that issue mentions), the appropriate type would be a decimal-base floating-point type. There are detailed standard specifications and libraries and even compiler extensions that implement those, but they aren't in popular use outside some finance applications.. I would look for/patch a json library to enforce precision on float fields.

[thisisfun]

thanks again Cubbi for the info and advice. I honestly thought this was a simple issue on my side but looks like a feature/limitation. I will follow your lead as suggested and find or generate a patch for precision.

Thanks again for everything!!!