Shannon Booth f589bedb0d LibJS: Improve JS::modulo precision for large floating values ...

JS::modulo was yielding a result of '0' for the input:
```
modulo(1., 18446744073709551616.)
```

Instead of the expected '1'.

As far as I can tell the reason for this is that the repeated calls to
fmod is losing precision in the calculation, leading to the wrong
result. Fix this by only calling fmod once, and preserving the negative
value behaviour by an 'if' check.

Without this, the LibWeb text test:
`/Streams/ReadableByteStream-enqueue-respond.html`

Would hang forever after using this function in the IDL conversion of a
u64 in ConvertToInt.

This should also be more efficient :^)

2024-01-02 10:01:26 +01:00

14 KiB

Raw Blame History

View Raw