Skip to content
This repository was archived by the owner on May 18, 2024. It is now read-only.
This repository was archived by the owner on May 18, 2024. It is now read-only.

Round-to-even bug in modp_dtoa2 #52

@jeroen

Description

@jeroen

We found that modp_dtoa2 has odd cases where it does not properly round-to-even. For example x.665 should be rounded down to x.66 but sometimes it gets rounded to x.67

modp_dtoa2(9.8665, digits = 3)
# 9.867
modp_dtoa2(1.8665, digits = 3)
# 1.866

Or

modp_dtoa2(111.665, digits = 2)
# 111.67
modp_dtoa2(222.665, digits = 2)
# 222.66 
modp_dtoa2(333.665, digits = 2)
# 333.67
modp_dtoa2(444.665, digits = 2)
# 444.67

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions