Support for Decimal128 type in Ming about ming HOT 3 CLOSED

As far as MappedClass is involved, safe_bson is not used at all.
safe_bson is only involved when using the low level Document layer outside of the ODM.
So for TurboGears applications themselves it should be fairly safe to remove that cast from Decimal to float as it's never executed.

For custom applications that used the Document layer, the best practice is still to declare the schema of the fields, as there was no Decimal schema there should be no past application that was saving Decimal data relying on the implicit cast to float.

Only way I can think of to rely on that implicit cast is by using Document with a field that is S.Anything.
At that point you would be bypassing the schema and thus you would be able to set a decimal.Decimal but as you are on Document you would to go through _safe_bson and thus you would end up with a float.

I feel we have too many validation steps and different behaviours here.
Object.make_safe is probably not used in 90% of the cases, in the remaining 9% it won't do a thing because it's used with a schema already enforcing type and for the remaining 1% of cases it's more valuable to simplify the code base than to handle the corner case of a corner case.

I'd just get rid of Object.make_safe and _safe_bson at all. We would just gain a simpler codebase and far faster insertion of documents because it doesn't have to traverse them every time.

If you plan to add Decimal support, we would also require to add it to the schema btw, as it's currently missing.

from ming.

I've been trying to add support for Decimal128, but I'm stuck trying to figure out if there is a way to differentiate the validation performed before the documents are flushed to the database, and right after they're retrieved from it. My issue is that I would like to automatically convert bson.decimal128.Decimal128 objects to python's decimal.Decimal on get, and the opposite on set. Can you think of any simple way to achieve this without adding complexity to the current code?
Thanks!

from ming.

I'm not fond of reverse implicit conversion.
While converting from decimal.Decimal to bson.decimal128.Decimal128 makes sense and can easily be achieved by declaring a schema.Decimal128 ( See how schema.ObjectId works for an example: https://github.com/TurboGears/Ming/blob/master/ming/schema.py#L696-L709 ) the reverse is not so obvious and poses some major problems:

• When the data is coming from the db, was it originally provided as Decimal or Decimal128? If the user provided a Decimal128 explicitly on insertion, it shouldn't get back a Decimal.
• When querying what I'm supposed to provide? {'$eq': Decimal} or {'$eq': Decimal128}? Are we supposed to parse every query and convert Decimal->Decimal128? We don't for str->ObjectId, if you are querying for _id you are expected to provide a bson.ObjectId

There is a whole bunch of inconsistencies and unexpected behaviours that open up if we try to hide the fact that underlying mongo will always only store and serve Decimal128.

I believe we should only have a schema.Decimal128 that makes clear we are only going to store Decimal128, but that for convenience automatically converts Decimal->Decimal128 when a Decimal is provided. That makes clear that in the object a Decimal128 is what you are storing but allows you to store data that comes from Python functions.

from ming.