Peter Bindels 0a0089fc11 LibC strtod: Reduce incremental error to nearly nothing ...

Instead of scaling by 1/10th N times, scale 10^N and then divide by
that. Avoid doing this beyond double-infinity. This decreases the
progressive error for numbers outside of integer range immensely. Not
a full 100% fix; there is still a single ULP difference detected by a
Javascript test

2021-07-18 12:45:10 +01:00

31 KiB

Raw Blame History

View Raw