Inconsistent max precision affecting dump&load result about bigdecimal HOT 4 CLOSED

@atitan Thank you for your report.

We've observed that BigDecimal treats number and string differently on max precision allocation.

The numbers BigDecimal#precs returns are internal detail so you shouldn't use them. This method has already been deprecated. You can see the following deprecation warning with -w option.

$ ruby -w -rbigdecimal -e "BigDecimal(1).precs"
-e:1: warning: BigDecimal#precs is deprecated and will be removed in the future; use BigDecimal#precision instead.

As this warning said, BigDecimal provides BigDecimal#precision method as of the version 3.0.0. Use this method instead of BigDecimal#precs.

Especially when taking serialized result for comparison to check for duplicates or data changes.

>> Marshal.load(Marshal.dump(BigDecimal(123))) == BigDecimal(123)
=> true
>> Marshal.load(Marshal.dump(BigDecimal(123456))) == BigDecimal(123456)
=> true
>> Marshal.load(Marshal.dump(BigDecimal(123456789))) == BigDecimal(123456789)
=> true
>> Marshal.load(Marshal.dump(BigDecimal(123456789012))) == BigDecimal(123456789012)
=> true

What is the problem?

from bigdecimal.

I think you don't understand the problem.

This is an internal issue of bigdecimal, so I use precs to describe the issue.

The following code modified from yours for better understanding.

irb(main):001:0> require 'bigdecimal'
=> true
irb(main):002:0> RUBY_VERSION
=> "3.1.2"
irb(main):003:0> Marshal.dump(Marshal.load(Marshal.dump(BigDecimal(123)))) == Marshal.dump(BigDecimal(123))
=> false
irb(main):004:0> Marshal.dump(Marshal.load(Marshal.dump(BigDecimal(123456)))) == Marshal.dump(BigDecimal(12345
6))
=> false
irb(main):005:0> Marshal.dump(Marshal.load(Marshal.dump(BigDecimal(123456789)))) == Marshal.dump(BigDecimal(12
3456789))
=> false
irb(main):006:0> Marshal.dump(Marshal.load(Marshal.dump(BigDecimal(123456789012)))) == Marshal.dump(BigDecimal
(123456789012))
=> true

I have a real world usage that stores bigdecimal object into database using ActiveRecord's serialize method. Sometimes I got column changed error when I tried to lock the record, but I've changed nothing. After digging into ActiveRecord, I found that it dumps object into string every time trying to compare serialized column change.

And the inconsistent string dump result from bigdecimal is the source of error.

from bigdecimal.

@atitan

This is an internal issue of bigdecimal, so I use precs to describe the issue.

No, it's not an issue. BigDecimal does not guarantee that a number has always the same internal state. The numbers that equal each other can have different internal states when they are created by the different ways. The reason why we don't guarantee the same internal state for equal numbers is to simplify the internal implementation and to avoid needless scanning of digits.

I have a real world usage that stores bigdecimal object into database using ActiveRecord's serialize method.

If you state the internal state of BigDecimal should always be the same between two equal numbers, I cannot agree with you. But, if you state that the result of Marshal.dump of two equal BigDecimal numbers should be the same, I think it can be valuable to consider. If Marshal.dump guarantees the same dump string for two equal objects, BigDecimal must follow the specification.

Note that the internal state of BigDecimal number and the result of Marshal.dump is not directly linked, in theory. The current behavior is not the specification, but just due to the implementation.

from bigdecimal.

@atitan
By the way, the format of Marshal.dump may change among the different Ruby versions. This is the specification of Marshal module (see the document). The format of BigDecimal#_dump may also change in the same way.
Moreover, BigDecimal#_dump and BigDecimal#_load may be replaced with BigDecimal#marshal_load and BigDecimal#marshal_dump some day in the future because these methods are expected to be called only from Marshal. If you use these methods, please do not expect any specifications for public use.
Therefore, I strongly recommend avoiding storing the result of Marshal.dump and BigDecimal#_dump in any kind of storage except for temporary ones.

from bigdecimal.