Giter VIP home page Giter VIP logo

Comments (5)

cosmicBboy avatar cosmicBboy commented on May 31, 2024

@amosaikman can you provide a minimally reproducible example? the snippet you provided is a good start, but it needs to be copy-pastable to reproduce.

from pandera.

amosaikman avatar amosaikman commented on May 31, 2024

@cosmicBboy Is this better?

df2 = pd.DataFrame(
    {
        "sample_id": ["a", "b", "c", "d", "e", "f", "g", "h", "i", "j"],
        "Cu_ppm": [10, 20, -1, 40, 50, 60, 70, 80, 90, 100],
        "Cs_ppb": [100, 200, 300, 400, 500, 600, 700, 800, 900, 1000],
        "spam": [100, 200, 300, 400, 500, 600, 700, 800, 900, 1000],
    }
)


schema7 = pa.DataFrameSchema(
    {
        r"^[A-Z][a-z]*_ppm$": pa.Column(
            float,
            nullable=True,
            coerce=True,
            checks=pa.Check.in_range(0, 1e6),
            required=False,
            regex=True,
        ),
    },
    index=pa.Index(str, nullable=False, unique=True, coerce=True),
    checks=valid_column_labels,
    strict='filter',
)


schema7.validate(df2)

For me that produces a DataFrame with a default integer index, one column labelled Cu_ppm containing the supplied data coerced to float. I expected it to produce a SchemaError because -1 is outside the in_range bounds.

from pandera.

cosmicBboy avatar cosmicBboy commented on May 31, 2024

what checks are contained in valid_column_labels?

from pandera.

amosaikman avatar amosaikman commented on May 31, 2024

Sorry, missed that.

valid_column_labels = [
    pa.Check(lambda df: df.columns.is_unique, element_wise=False, error="Columns must be unique"),
    pa.Check(
        lambda df: df.columns.str.split("_").str[0].isin(valid_prefixes).all(),
        element_wise=False,
        error="Element symbols must be valid",
    ),
]

valid_prefixes in this dummy context is just a long list of element symbols e.g. ['H', 'He', …]

from pandera.

cosmicBboy avatar cosmicBboy commented on May 31, 2024

#1248 should address this issue

from pandera.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.