Checks <input type="c

Rolling_mean with weights gives wrong result when min_periods is not None about polars HOT 1 OPEN

jackaixin commented on August 22, 2024

Rolling_mean with weights gives wrong result when min_periods is not None

from polars.

Comments (1)

Julian-J-S commented on August 22, 2024

oh yeah, this is tricky 🤔. The "problem" is only at the start of the window

another easy example:

pl.DataFrame({"x": [1, 2, 3, 4]}).with_columns(
    without_weight=pl.col("x").rolling_mean(
        window_size=2,
        min_periods=0,
    ),
    with_weight_50_50=pl.col("x").rolling_mean(
        window_size=2,
        min_periods=0,
        weights=[0.5, 0.5],
    ),
    with_weight_20_80=pl.col("x").rolling_mean(
        window_size=2,
        min_periods=0,
        weights=[0.2, 0.8],
    ),
)

# shape: (4, 4)
# ┌─────┬────────────────┬───────────────────┬───────────────────┐
# │ x   ┆ without_weight ┆ with_weight_50_50 ┆ with_weight_20_80 │
# │ --- ┆ ---            ┆ ---               ┆ ---               │
# │ i64 ┆ f64            ┆ f64               ┆ f64               │
# ╞═════╪════════════════╪═══════════════════╪═══════════════════╡
# │ 1   ┆ 1.0            ┆ 0.5               ┆ 0.2               │ 🤔  window [1]
# │ 2   ┆ 1.5            ┆ 1.5               ┆ 1.8               │ ✅  window [1, 2]
# │ 3   ┆ 2.5            ┆ 2.5               ┆ 2.8               │ ✅ 
# │ 4   ┆ 3.5            ┆ 3.5               ┆ 3.8               │ ✅ 
# └─────┴────────────────┴───────────────────┴───────────────────┘

First window is only [1]

without weights: knows that this window has only length of 1 and divides by 1 ✅
with weights: multiplies by weight without considering missing weights (should always add to 100%?!) 🤔

Not sure if this is a bug or intended behaviour and requires more documentation.
But one can surely argue that the 50/50 weighted mean of 1 and nothing else is still 1 and not 0.5

from polars.

Recommend Projects