Skip to content

Use column-valued parameters

Give each row its own distribution, either from columns you already have or from parameters you compute in the same query.

Pass columns as parameters

Any parameter accepts a scalar, a column name (str), a pl.Expr, or a pl.Series, and you can mix them in one constructor:

import polars as pl
import polars_stats as ps

ps.Normal(mu=pl.col("mu"), sigma=1.0)  # column mean, scalar scale
ps.Normal(mu=0.0, sigma=1.0)  # all scalar
ps.Uniform(min="min", max="max")  # column names as strings
ps.Binomial(n=pl.col("size"), p=pl.col("probas"))  # all expressions

The instance then evaluates one distribution per row:

import polars as pl
import polars_stats as ps

df = pl.DataFrame({"mu": [0.0, 10.0], "sigma": [1.0, 2.0], "x": [0.5, 11.2]})

print(df.with_columns(p=ps.Normal(mu="mu", sigma="sigma").pdf("x")))

shape: (2, 4)
┌──────┬───────┬──────┬──────────┐
│ mu   ┆ sigma ┆ x    ┆ p        │
│ ---  ┆ ---   ┆ ---  ┆ ---      │
│ f64  ┆ f64   ┆ f64  ┆ f64      │
╞══════╪═══════╪══════╪══════════╡
│ 0.0  ┆ 1.0   ┆ 0.5  ┆ 0.352065 │
│ 10.0 ┆ 2.0   ┆ 11.2 ┆ 0.166612 │
└──────┴───────┴──────┴──────────┘

Method arguments follow the same rule: pdf("x") reads column x, exactly like pdf(pl.col("x")). A bare string is always a column reference, never a literal.

A constant parameter is faster as a plain number

ps.Normal(mu=0.0, sigma=1.0) and ps.Normal(mu=pl.lit(0.0), sigma=pl.lit(1.0)) return the same rows, but only the first takes the constant-parameter fast path, where the parameters ride as plugin kwargs and build one distribution per call. An Expr parameter gives that up and rebuilds it per row.

Floats are strict

A float parameter rejects an int: ps.Normal(mu=0, sigma=1) raises TypeError. Write 0.0 and 1.0. Count parameters are the mirror image (Binomial(n=10, p=0.3) wants an int for n). The full table is in Reference / Parameters and contracts.

Keep parameters in a lookup table

When parameters belong to an entity rather than a row, join them in and pass the joined columns:

readings = pl.DataFrame(
    {
        "sensor": ["a", "a", "b", "b"],
        "reading": [9.8, 10.4, 100.0, 135.0],
    }
)
baselines = pl.DataFrame({"sensor": ["a", "b"], "mu": [10.0, 100.0], "sigma": [0.5, 2.0]})

print(
    readings.join(baselines, on="sensor", maintain_order="left").with_columns(
        upper_tail=ps.Normal(mu="mu", sigma="sigma").sf("reading")
    )
)

shape: (4, 5)
┌────────┬─────────┬───────┬───────┬────────────┐
│ sensor ┆ reading ┆ mu    ┆ sigma ┆ upper_tail │
│ ---    ┆ ---     ┆ ---   ┆ ---   ┆ ---        │
│ str    ┆ f64     ┆ f64   ┆ f64   ┆ f64        │
╞════════╪═════════╪═══════╪═══════╪════════════╡
│ a      ┆ 9.8     ┆ 10.0  ┆ 0.5   ┆ 0.655422   │
│ a      ┆ 10.4    ┆ 10.0  ┆ 0.5   ┆ 0.211855   │
│ b      ┆ 100.0   ┆ 100.0 ┆ 2.0   ┆ 0.5        │
│ b      ┆ 135.0   ┆ 100.0 ┆ 2.0   ┆ 7.1635e-69 │
└────────┴─────────┴───────┴───────┴────────────┘

Derive parameters from the data

There is no fit method. Estimate the parameters with ordinary Polars expressions, then feed those columns in. A per-group Normal fit is a mean and a standard deviation:

history = pl.DataFrame(
    {
        "sensor": ["a", "a", "a", "a", "b", "b", "b", "b"],
        "reading": [1.0, 1.2, 0.9, 1.1, 10.0, 10.5, 9.5, 10.2],
    }
)

print(
    history.with_columns(
        mu=pl.col("reading").mean().over("sensor"),
        sigma=pl.col("reading").std().over("sensor"),
    ).with_columns(upper_tail=ps.Normal(mu="mu", sigma="sigma").sf("reading"))
)

shape: (8, 5)
┌────────┬─────────┬───────┬──────────┬────────────┐
│ sensor ┆ reading ┆ mu    ┆ sigma    ┆ upper_tail │
│ ---    ┆ ---     ┆ ---   ┆ ---      ┆ ---        │
│ str    ┆ f64     ┆ f64   ┆ f64      ┆ f64        │
╞════════╪═════════╪═══════╪══════════╪════════════╡
│ a      ┆ 1.0     ┆ 1.05  ┆ 0.129099 ┆ 0.650732   │
│ a      ┆ 1.2     ┆ 1.05  ┆ 0.129099 ┆ 0.122639   │
│ a      ┆ 0.9     ┆ 1.05  ┆ 0.129099 ┆ 0.877361   │
│ a      ┆ 1.1     ┆ 1.05  ┆ 0.129099 ┆ 0.349268   │
│ b      ┆ 10.0    ┆ 10.05 ┆ 0.420317 ┆ 0.547346   │
│ b      ┆ 10.5    ┆ 10.05 ┆ 0.420317 ┆ 0.14217    │
│ b      ┆ 9.5     ┆ 10.05 ┆ 0.420317 ┆ 0.904654   │
│ b      ┆ 10.2    ┆ 10.05 ┆ 0.420317 ┆ 0.360593   │
└────────┴─────────┴───────┴──────────┴────────────┘

The same shape works for a rolling baseline (rolling_mean / rolling_std instead of mean / std) or for method-of-moments estimates of other distributions (Exponential(rate=1 / pl.col("reading").mean().over("sensor"))).

Estimated parameters can be invalid: a group with one row gives sigma = null, and that propagates to null rather than raising. See Handle nulls and errors.