feat: Support inline calculation transforms by joelostblom · Pull Request #3991 · vega/altair

joelostblom · 2026-04-05T09:33:04Z

Altair already supports inline aggregation transforms via magic string syntax such as 'mean(field_name)'. This PR adds support for inline calculation transforms by passing expressions to encoding fields. The corresponding calculation transform is automatically generated and a hash-based field name is used, since the idea is that these calculations are just used for a single field. This only works with the alt.expr module and not with js expression strings.

Expression syntax now supported on this branch:

encode(x=alt.expr.random())
encode(x=alt.datum.a + alt.datum.b)
encode(x=alt.X(alt.datum.a + 1).type("quantitative"))
Commit f437815 also added support for encode(alt.expr('random()') for consistency with how these are used interchangeably elsewhere, but I would be ok to drop this in favor of always using the expr submodule.

There are other options for the syntax, so let's discuss in #3992 before considering merging this PR.

Closes #3992 and #3602.

Examples

# "strip chart" with random offset values
import altair as alt
from altair.datasets import data

source = data.penguins.url

chart = (
  alt.Chart(source, height=alt.Step(30)).mark_point(size=20, filled=True).encode(
      x="Body Mass (g):Q",
      y=alt.Y("Species:N"),
      yOffset=alt.expr.random()
    ) 
)
print(chart.to_dict()["transform"])
chart

# arithmetic expression in two channels (dedup transform)
import altair as alt

expr = alt.datum.a + alt.datum.b
chart = (
    alt.Chart({"values": [{"a": 1, "b": 2}]})
    .mark_point()
    .encode(
        x=alt.X(expr).type("quantitative"),
        y=alt.Y(expr).type("quantitative"),
    )
)
spec = chart.to_dict()
print(spec["transform"])         # one calculate entry
print(spec["encoding"]["x"])
print(spec["encoding"]["y"])
chart

# explicit type overrides inferred type
import altair as alt

chart = (
    alt.Chart({"values": [{"a": 1, "b": 2}]})
    .mark_point()
    .encode(x=alt.X(alt.datum.a + alt.datum.b).type("nominal"))
)
print(chart.to_dict()["encoding"]["x"])
chart

Details

FieldChannelMixin.to_dict() detects expression-valued channel inputs and:
- emits a deterministic calculated field name (calc)
- records a corresponding {"calculate": , "as": } transform
- returns an encoding referencing that generated field
- uses explicit channel type when provided, otherwise attempts expression type inference
Auto-generated transforms are deduplicated by generated field name.
Top-level to_dict() collects queued auto-calc transforms from context and appends them to spec["transform"].
Added expression-focused tests for:
- expression shorthand support
- type inference behavior
- explicit type override behavior
- deduplication behavior
- uninferrable expression types (omits type)

altair/utils/core.py (_wrap_in_channel): - Removed try/except ImportError around from altair.vegalite.v6.api import Parameter — it's a same-package import that can never fail - Collapsed the two separate Expression and Parameter branches into clean sequential checks altair/vegalite/v6/schema/channels.py (FieldChannelMixin.to_dict): - Added import hashlib at module level (was a local import inside the method) - Added from altair.expr.core import Expression, GetItemExpression at module level - Removed from altair.utils.schemapi import Undefined as AltairUndefined alias — uses Undefined directly - Removed try/except ImportError around Parameter import (kept as function-local to avoid circular import, but without the try/except) - Removed dead elif isinstance(val, Expression) branch (unreachable — _param_to_expr already converts expressions) - Removed redundant else: shorthand_or_kwds = shorthand branch - Changed auto_calc_transforms from a list to a dict[str, dict] keyed by field name → deduplicates identical expressions - Added context.pop("parsed_shorthand", None) before early return → fixes a bug where the leftover parsed_shorthand in context was leaking into subsequent SchemaBase.to_dict() calls (manifested as a spurious field key appearing in serialized params) altair/vegalite/v6/api.py (TopLevelMixin.to_dict): - Restored pre_transform=False in the sub-call context (regression from previous session) - Pre-initializes auto_calc_transforms as a shared dict object on context before creating the dict(context, pre_transform=False) copy — this ensures children write into the same shared dict object that the top-level code reads back - Changed .extend(auto_calc_transforms) → .extend(auto_calc_transforms.values()) to match the new dict type

What I corrected/simplified: - In FieldChannelMixin.to_dict (altair/vegalite/v6/schema/channels.py:163), I moved expression/variable-param handling before parsed-shorthand context setup. - This removes the need for the previous context.pop("parsed_shorthand", None) workaround and avoids context leakage by design. - Behavior is unchanged for normal field-string paths, but this is cleaner and less fragile. What I added to tests: - Added a missing regression test for the bare channel case: - encode(x=param) (not just encode(x=alt.X(param, ...))) - File: tests/vegalite/v6/test_api.py:2259 - This ensures _wrap_in_channel path for variable params is covered.

Limit inline transform_calculate generation to expression-valued encodings and remove variable-parameter sugar from this branch. This keeps expression support (e.g. x=alt.expr.random()) while deferring bare-parameter behavior (e.g. x=param) to a dedicated follow-up branch, with tests updated accordingly.

joelostblom · 2026-04-05T09:54:45Z

                for sh in shorthand
            ]

+        shorthand_or_kwds = shorthand


We intentionally inspect raw channel kwargs for Expression values before shorthand parsing so expression encodings can bypass normal field-shorthand parsing.

joelostblom · 2026-04-05T09:55:23Z

+
+        if isinstance(shorthand_or_kwds, Expression):
+            vega_expr = repr(shorthand_or_kwds)
+            field_hash = hashlib.md5(vega_expr.encode()).hexdigest()[:8]


Calculated field names are deterministic (md5(repr(expr))[:8]) so repeated expressions across channels reuse the same transform.

joelostblom · 2026-04-05T09:55:52Z

+            vega_expr = repr(shorthand_or_kwds)
+            field_hash = hashlib.md5(vega_expr.encode()).hexdigest()[:8]
+            calc_field_name = f"_calc_{field_hash}"
+            transforms: dict[str, dict] = context.setdefault("auto_calc_transforms", {})


auto_calc_transforms is a dict (not list) to deduplicate transforms by generated field name.

joelostblom · 2026-04-05T09:56:07Z

+            )
+
+            result: dict[str, Any] = {"field": calc_field_name}
+            explicit_type = self._get("type")  # type: ignore[attr-defined]


If channel type is explicitly provided, it wins over inferred type; inference is best-effort fallback.

joelostblom · 2026-04-05T09:56:34Z

+            elif inferred := _infer_expr_type(shorthand_or_kwds):
+                result["type"] = inferred
+            return result
+


Expression path returns early above; normal shorthand parsing continues unchanged for non-expression encodings below.

joelostblom · 2026-04-05T09:56:55Z


        # remaining to_dict calls are not at top level
        context["top_level"] = False
+        # pre-initialize so children can write into it by reference,


Pre-initialize shared transform accumulator so child serialization can mutate the same object even when receiving dict(context, pre_transform=False).

joelostblom · 2026-04-05T09:57:30Z

        )

+        context.pop("auto_calc_transforms", None)
+        if auto_calc_transforms and is_top_level:


Only top-level charts inject accumulated auto-calc transforms into final spec

joelostblom · 2026-04-05T09:58:28Z

    return result


+def _infer_expr_type(expr: Any) -> str | None:  # noqa: C901


Type inference is intentionally heuristic and conservative; unknown/ambiguous expressions return None to avoid incorrect schema typing. It's also possible to override manually by using alt.X(expr).type(...)

joelostblom · 2026-04-05T09:59:15Z

    it = chain.from_iterable(info.all_names for info in channel_infos.values())
    all_ = sorted(chain(it, COMPAT_EXPORTS))
    imports = [
+        "import hashlib",


These imports must include Expression, hashlib, and _infer_expr_type because generated channels.py depends on them.

The autogenerated field names are not useful as axis titles, so instead of always having to change them, they are now set to empty/None. If vega-lite supports params for axis titles in the future we could maybe do something smarter

Avoid that it is picked up as a public API symbol, which causes docs to fail: - Expression was imported into altair.vegalite.v6.schema.channels at module scope. - API docs generation scans public names in that module and incorrectly added Expression to autosummary. - Sphinx then tried to import altair.Expression, which does not exist.

Trying to reference it from multiple places so that it is more discoverable

alt.datum(alt.expr...) should not be converted into transform as that breaks some old behavior like the identiy line example

mattijn · 2026-04-06T15:18:23Z

+        alt.Chart()
+        .mark_rule()
+        .encode(
+            x=alt.datum(alt.expr("domain('x')[0]"), type="quantitative"),


Just reading the diff here, will alt.expr.domain('x')[0] also work here?

Yup, I actually just fixed that in 2a87ca1 and will update this example to use the simpler syntax. I changed it so both 'domain' and alt.expr.domain will generate an ExprRef when wrapped in alt.datum() (rather than generated in Expression or being converted into calculate transforms)

joelostblom added 4 commits April 5, 2026 11:33

Support calculations inline to fields

5cfa39a

joelostblom force-pushed the feat/inline-calc branch from 7a9f25a to 9d4f57f Compare April 5, 2026 09:33

joelostblom marked this pull request as draft April 5, 2026 09:39

joelostblom added 3 commits April 5, 2026 11:41

Add additional tests

ca6d864

Include changes in templates

8f9d815

Remove redundant test file

968fb0c

joelostblom commented Apr 5, 2026

View reviewed changes

joelostblom marked this pull request as ready for review April 5, 2026 10:05

joelostblom mentioned this pull request Apr 5, 2026

Support calculations inline in encodings #3992

Open

joelostblom added 2 commits April 5, 2026 13:49

Add support for alt.expr('<js_expr_str>')

f437815

Set default title to None

fc95022

The autogenerated field names are not useful as axis titles, so instead of always having to change them, they are now set to empty/None. If vega-lite supports params for axis titles in the future we could maybe do something smarter

joelostblom mentioned this pull request Apr 5, 2026

feat: Support params to be passed inline to encodings #3993

Open

joelostblom added 5 commits April 5, 2026 15:54

Add docs for inline encoding expressions

9065530

Update docs

122782b

Add inline expression sections in docs

9d58afa

Trying to reference it from multiple places so that it is more discoverable

Avoid converting expr passed to datum into transform_calc

e2a33bd

alt.datum(alt.expr...) should not be converted into transform as that breaks some old behavior like the identiy line example

mattijn reviewed Apr 6, 2026

View reviewed changes

joelostblom added 3 commits April 6, 2026 17:23

Allow alt.expr... to be used as ExpRef when wrapped in alt.datum

2a87ca1

Prefer alt.expr where possible

33576c2

Correct test and remove redundancy

6495608

Navigation Menu

Uh oh!

Uh oh!

feat: Support inline calculation transforms - #3991

feat: Support inline calculation transforms#3991
joelostblom wants to merge 17 commits into
mainfrom
feat/inline-calc

joelostblom commented Apr 5, 2026 •

edited

Loading

Uh oh!

joelostblom Apr 5, 2026

Uh oh!

joelostblom Apr 5, 2026

Uh oh!

joelostblom Apr 5, 2026

Uh oh!

joelostblom Apr 5, 2026

Uh oh!

joelostblom Apr 5, 2026

Uh oh!

joelostblom Apr 5, 2026

Uh oh!

joelostblom Apr 5, 2026

Uh oh!

joelostblom Apr 5, 2026

Uh oh!

joelostblom Apr 5, 2026

Uh oh!

mattijn Apr 6, 2026

Uh oh!

joelostblom Apr 6, 2026

Uh oh!

mattijn Apr 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		return result


		def _infer_expr_type(expr: Any) -> str \| None: # noqa: C901

Uh oh!

Uh oh!

Conversation

joelostblom commented Apr 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Examples

Details

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

joelostblom commented Apr 5, 2026 •

edited

Loading