Skip to content

Add/Subtract never round to tens or hundreds: 1.2E3 + 34 = 1234 instead of 1.2E3 #107

Description

@matt-edmondson

What's wrong

CountDecimalDigits (SignificantNumber/SignificantNumber.cs ~line 152) clamps a positive exponent to 0:

private static int CountDecimalDigits(PreciseNumber value) =>
    value.Exponent > 0 ? 0 : int.Abs(value.Exponent);

Add and Subtract round the result to the lower of the two operands' decimal places. When an operand's least-significant digit is left of the decimal point (tens, hundreds, …), its place is reported as "units". The result is then never rounded to the coarser place.

Reproduction

(invariant culture)

Expression Actual Expected (addition rule: round to the coarsest least-significant place)
Parse("1.2E3") + Parse("34") 1234 1.2E3
Parse("1.2E3") - Parse("34") 1166 1.2E3
Parse("1200") + Parse("34") 1234 1.2E3

The library contradicts itself. It stores 1200 as 12e2 with SignificantDigits == 2, and Multiply honors that: 1200 * 1.234 gives 1.5E3. Addition, however, treats the same value as exact to the units.

Existing Add/Subtract tests only combine operands with the same positive exponent (e.g. 12300 + 45600). There the clamped value happens not to matter, so this case is untested.

Why it matters

The type exists to propagate significant figures correctly. Adding a measurement known to the hundreds and a small correction reports false precision: 1234 claims four significant figures from a two-figure input.

Suggested fix / acceptance criteria

  • Track the least-significant place as a signed value (-value.Exponent, negative for tens/hundreds), still ignoring operands with unlimited precision (-1, 0, 1).
  • Round the sum or difference to that place. Rounding to a negative number of decimal places depends on PreciseNumber.Round handling negative digit counts. PreciseNumber just merged a fix for that (Round to negative decimal places correctly for integers PreciseNumber#108, "round-negative-digits"), so this may need a PreciseNumber bump. The alternative is to round manually: scale the significand by 10^k, round, and rebuild with CreateFromComponents.
  • Add tests for 1.2E3 + 34 == 1.2E3, 1.2E3 - 34 == 1.2E3, and a case that must still keep units, such as 1234 + 5.6 == 1240.

Confirmed by running against a scratch build of main (e1ebb5d) with ktsu.PreciseNumber 2.6.2 on .NET 10.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't workingreadyFully specified; implement as written

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions