The code for determining the default denominator stumbles on the following requirement: https://github.com/38/d4-format/blob/master/d4tools/src/create/main.rs#L244
The threshold of 1e-10 is too strict when dealing with high coverage (e.g. when dealing with MT coverage in WGS).
For example, a coverage value of "8284.95" ends up with a denominator of 10000 (instead of 100).
Furthermore, really large values of the denominator can also happen (example: "33262.09"), which breaks the coverage calculations completely (where value * denominator > i32::MAX).
See https://play.rust-lang.org/?version=stable&mode=debug&edition=2015&gist=b69a0f4bfaffddead14fb68c088157c3
A simple fix is to loosen the criteria of 1e-10 to a lower value. Another fix is to ensure that value * denominator < i32::MAX.
(Working on a PR)
The code for determining the default denominator stumbles on the following requirement: https://github.com/38/d4-format/blob/master/d4tools/src/create/main.rs#L244
The threshold of 1e-10 is too strict when dealing with high coverage (e.g. when dealing with MT coverage in WGS).
For example, a coverage value of "8284.95" ends up with a denominator of 10000 (instead of 100).
Furthermore, really large values of the denominator can also happen (example: "33262.09"), which breaks the coverage calculations completely (where value * denominator > i32::MAX).
See https://play.rust-lang.org/?version=stable&mode=debug&edition=2015&gist=b69a0f4bfaffddead14fb68c088157c3
A simple fix is to loosen the criteria of 1e-10 to a lower value. Another fix is to ensure that value * denominator < i32::MAX.
(Working on a PR)