It seems `SparkCompare` object has no attribute 'sample_mismatch` ? about datacompy HOT 2 OPEN

Hey @pangjac first off thank you for supporting the package!

sample_mismatch doesn't exist for the SparkCompare class in that version of datacompy. We have a branch which is waiting review where we are shifting to pandas on pyspark if you are ok using that instead. v0.8.4 is fairly old so I'd highly recommend bumping up if you are able to. That old version of SparkCompare doesn't inherit from the base class as it was built aside from it. It has been something which has been bugging me hence the new branch waiting review and deprecating the old Spark class.

If you look at the new implementation (which aligns better to the pandas, polars, and fugue logic) we will have that function natively for Spark.

Alternatively I wonder if the internal dataframe: _all_rows_mismatched would give you what you need. you can filter on the column you are interested in since its just a Spark DF.

from datacompy.

@pangjac Just wanted to follow up and see if this was solved for you? Thanks!

from datacompy.