Math problem

Development Programming & Scripting
1

Hi,
I ran into a strange problem with calculations:

Why does this GEngine->AddOnScreenDebugMessage(-1, 5.f, FColor::Cyan, FString::SanitizeFloat(1 - (0.1f * 3))); not equal to this GEngine->AddOnScreenDebugMessage(-1, 5.f, FColor::Cyan, FString::SanitizeFloat(1 - ((1 / 10) * 3)));

From what I can calculate they should both yield the same answer. However the former give 0.7 as an answer while the latter give 1.

Can someone explain why is this happening?

2

1 - integer
10 - integer

You use operator “/” with two int params: [int] / [int]. It return integer. So 1/10 = 0.1f → cast to int → 0

1 - 0 = 1. Your answer.

[int] / [int] = [int]

[float] / [int] = [float]

[int] / [double] = [double]

3

So if I were to specify them as floats then there won’t be any problem?

4

Yep…

  • 1 - ((1 / 10) * 3) = 1
  • 1 - ((1.f / 10.f) * 3) = 0.7f
5

To attempt to clarify, integers are not very friendly with decimal values or anything related to fractions. It drops anything on the right side of the decimal.