The intersection logic of Compare has problems. about datacompy HOT 3 CLOSED

I think about it twice, this is even harder than I thought.

The big challenge is the floating columns. For example 0.99 and 1 can be considered same with certain atol and rtol. So the question is should we even compute the intersections? It's so expensive to make it right.

from datacompy.

@goodwanghan This might actually be covered in the docs: https://capitalone.github.io/datacompy/pandas_usage.html#duplicate-rows

This is sort of a known limitation and if you have a lot of dups you would really need to join on more fields to get more accurate results. I think the example in the docs sort of outlines what you have above and is the expected behaviour.

from datacompy.

Closing this. @goodwanghan feel free to reopen but I think this is known and documented.

from datacompy.