Pandas 1.3.0 (but not 1.3.1) support on purpose? about reciprocalspaceship HOT 7 CLOSED

This is @JBGreisman 's department, but I recall that Pandas has been mucking around with custom dtypes lately and breaking everything. I would not be surprised if the very specific versioning requirement was just self defense.

from reciprocalspaceship.

This was mostly done in self defense as @kmdalton suggested. There are two places that rs has historically run into issues during pandas updates:

The first is with attributes for our subclassed DataFrame (rs.DataSet). In the olde days of this project, there were a few pandas operations that would lead to loss of the subclass , and therefore loss of cell and spacegroup attributes. That has largely stabilized in the last couple of years and hasn't caused trouble lately.

The second place is with our ExtensionDtypes that are used to implement all the MTZ datatypes. Pandas adjusts that API a bit more regularly, and also sometimes changes behavior in small ways during patch releases. I had set a maximal version to protect against short-term issues, but I think that was the wrong approach to this problem.

I think it makes the most sense to have a GitHub Action that tests the build on a set schedule to detect possible issues with any latest pandas version. This sort of "detect early" strategy is a bit more flexible, and should allow us to keep up to date with pandas in a more seamless manner.

from reciprocalspaceship.

To do list:

• Extend supported version to latest pandas (1.3.2)
• Add renovate to repo to automatically test against dependency updates

from reciprocalspaceship.

We use renovate to track and test against dependency updates, eg. DiamondLightSource/python-workflows#65

from reciprocalspaceship.

awesome -- good to hear. I had been looking into that as a possible solution for this

from reciprocalspaceship.

I tested renovate in my personal fork and it seems accomplish this task exactly as I hoped. I tested it by reverting the pandas version to "pandas >= 1.2.0, <= 1.3.0", and renovate filed a PR to change the line to read "pandas >= 1.2.0, <= 1.3.2".

from reciprocalspaceship.

I'm closing this issue because renovate has now been added to the repo, and appears to be working as intended

from reciprocalspaceship.