Migrate from f32 and f64 to rust_decimal::Decimal

[pan93412]

Abstract

f32 and f64 has some precision issues¹, which may be critical for finance calculation². You can find it with this interactive converter.

crypto-crawler currently converts all the decimal float numbers (String) to binary float numbers (f32/f64), and stores values in Float/f32/f64. It makes the numbers inaccurate and may introduce issues when using this in production (for example, the wrong determination and time point to buy and sell³.)

Solution

To prevent the potential issue, I propose to use rust_decimal::Decimal instead of f32 or f64. It stores float numbers in decimal and thus prevents infinite precision due to the binary representation.

I've tried this library in my project (though has not been in production yet)⁴, and I can help rewrite this to Decimal.

TBDs (To Be Discussed)

• It introduces breaking changes. A safer non-breaking way is to write a feature flag and manually enable Decimal; however, it complexes the codebase.
• Error handling when we failed to convert a number to Decimal or convert a Decimal to a number. Note that Decimal::to_f64(self) returns a Option<f64>, and the Option may be None if f64 is not enough to represent this Decimal.

Footnotes

1. (Chinese) https://medium.com/starbugs/see-why-floating-point-error-can-not-be-avoided-from-ieee-754-809720b32175 ↩

2. (Chinese, ⚠️ Content Farm) https://www.twblogs.net/a/5eec07e211be511c38938805 ↩

3. (Chinese) https://zh.wikipedia.org/zh-tw/捨入誤差#举例 ↩

4. wmjtyd/libstock (1) wmjtyd/libstock (2) ↩

[soulmachine]

In practice I found that f32 is enough, do you really need that high precision?

[pan93412]

In practice I found that f32 is enough, do you really need that high precision?

I think it is still better to prevent such a precision loss even though we don't need such a high precision at this moment, as the retrieved data is a decimal float number instead of a binary one.

[soulmachine]

The crypto-crawler library doesn't do any parsing at all, while the crypto-msg-parser does. The crypto-crawler library outputs original messages without information loss.

In which case do you need high precision? I don't see additional value of Decimal.