Comments (4)
Problem is only with "9.8".
9.7.abs <= BigDecimal('9.7')
=> true
from bigdecimal.
It's likely related to the precision of converting floats to BigDecimal, see below:
# It's not only with `#abs`, also without it
[40] pry(main)> 9.8 <= BigDecimal('9.8')
=> false
# when we calculate the delta we see a small, but non-zero result
[41] pry(main)> 9.8.to_d - BigDecimal('9.8')
=> 0.1e-14
# if we use the `Float::DIG` constant it's OK
[42] pry(main)> 9.8.to_d(15) - BigDecimal('9.8')
=> 0.0
# if we use the `Float::DIG+1` constant it's no OK (the new default) after https://github.com/ruby/bigdecimal/commit/aa536cd4b585ff6045cf91f396bc3eef993b91f2
[43] pry(main)> 9.8.to_d(16) - BigDecimal('9.8')
=> 0.1e-14
# because of how the representation looks like
[44] pry(main)> 9.8.to_d(16)
=> 0.9800000000000001e1
seems to be related to aa536cd
from bigdecimal.
It's fixed in the newest version of the gem:
[8] pry(main)> 9.8.abs <= BigDecimal('9.8')
=> true
[9] pry(main)> BigDecimal::VERSION
=> "3.1.2"
from bigdecimal.
@mrzasa - Thank you for your response and investigation.
from bigdecimal.
Related Issues (20)
- BigDecimal#round raises FloatDomainError regardless of exception mode HOT 2
- Inconsistent max precision affecting dump&load result HOT 4
- Inconsistency between Bigdecimal 3.0.x and 3.1.x HOT 4
- BigDecimal#n_significant_digits and #precision do not return the correct value HOT 1
- I would expect BigDecimal#divmod to return type [Int, BigDecimal] (like Integer#divmod, Float#divmod)
- mode(BigDecimal::EXCEPTION_ALL,true) did not throw an exception when bigdecimal#exponent overflowed HOT 1
- Integrate JRuby implementation into the gem
- Loss of precision on very large numbers in `to_d` HOT 1
- Incorrect calculation result after upgrading from BigDecimal 3.1.1 to 3.1.2 HOT 1
- Difference in behavior with and without BigDecimal#dev 'digit' argument
- nil.to_d inconsistent (bigdecimal/util)
- Installation Error of BigDecimal 3.1.5 in JRuby HOT 3
- ruby 3.0.6 compilation error on MacOS Sonoma 14 related to maybe_unused HOT 1
- Ruby 3.4 and BigDecimal. HOT 10
- Memory leak with BigDecimal::EXCEPTION_OVERFLOW HOT 2
- BigDecimal#whole? HOT 3
- Use RUBY_TYPED_EMBEDDABLE in BigDecimal
- Please publish 3.1.8-java gem HOT 2
- bigdecimal install error HOT 26
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bigdecimal.