Skip to content

Fix just() conversion for container fields of dataclass types (#858)#875

Open
martinez-hub wants to merge 1 commit into
mit-ll-responsible-ai:mainfrom
martinez-hub:fix/858-just-container-dataclass-field
Open

Fix just() conversion for container fields of dataclass types (#858)#875
martinez-hub wants to merge 1 commit into
mit-ll-responsible-ai:mainfrom
martinez-hub:fix/858-just-container-dataclass-field

Conversation

@martinez-hub

Copy link
Copy Markdown

Summary

Fixes #858. just failed to convert a dataclass to a config when one of its fields is annotated with a container of a dataclass type (e.g. dict[str, Inner] or list[Inner]).

import hydra_zen as hz
from dataclasses import dataclass

@dataclass
class Inner:
    x: int = 0

@dataclass
class Outer:
    outer: dict[str, Inner]

hz.to_yaml(hz.just(Outer(outer={"a": Inner(x=1)})))
# omegaconf.errors.ValidationError: Invalid type assigned:
# Builds_Inner is not a subclass of Inner. value: <class 'types.Builds_Inner'>

Root cause

just recursively converts the dataclass values into Builds_Inner configs, but the field's annotation keeps the original element type (dict[str, Inner]). OmegaConf type-checks container elements before instantiation and rejects the Builds_Inner config since it isn't a subclass of Inner.

_retain_type_info already broadens a field's annotation to Any when its value is a structured config — but it only inspected the top-level value, not configs nested inside a container. A second gap: mutable container defaults are stored via default_factory, so the value reaching that check was MISSING.

Fix

  • _value_contains_builds(value) — detects a structured config nested (possibly deeply) inside a list/tuple/mapping.
  • _resolve_field_default(field) — resolves a field's default_factory so the real container value can be inspected.
  • _retain_type_info now broadens the annotation to Any when the value is a container holding a structured config.

Broadening to Any only loosens OmegaConf validation — it never tightens an annotation, so previously-working configs are unaffected.

Tests

Added to tests/test_dataclass_conversion.py:

  • test_just_on_container_field_of_typed_dataclassesdict[str, Inner], list[Inner], and nested dict[str, list[Inner]], each round-tripped through to_yamlinstantiate.
  • Unit tests for both new helpers.

Verification

  • Full test suite passes locally (test_launch/test_with_hydra_submitit/test_third_party excluded — they run in CI).
  • 100% coverage on the changed lines.
  • pyright basic: 0 errors on the changed source.
  • ruff format --check, ruff check, and autoflake all clean (ruff 0.14.1).
  • Changelog entry added under a new 0.16.1 section.

`just` failed to convert a dataclass to a config when one of its fields was
annotated with a container of a dataclass type (e.g. `dict[str, Inner]` or
`list[Inner]`). The field's values were converted to `Builds_Inner` configs
while the field's annotation retained the original element type, which
OmegaConf's type-checking rejected with a `ValidationError` during `to_yaml`.

`_retain_type_info` now broadens such a field's annotation to `Any` when the
value is a container holding a structured config. `default_factory` values are
resolved so mutable container defaults can be inspected.

Closes mit-ll-responsible-ai#858
@martinez-hub

Copy link
Copy Markdown
Author

The failing tests (3.9) and test-minimum-dependencies checks here are unrelated to this change. They're the pre-existing, order-dependent flaky failure tracked in #574MissingConfigException: In 'hydra/config': Could not find 'hydra/sweeper/basic' in tests/test_launch/test_validation.py — which #876 fixes at its root cause.

All other checks pass, including coverage, run-pyright, and tests on 3.10–3.14. Once #876 is merged, this branch can be rebased onto main and CI will go fully green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cannot convert to YAML with nested dataclass type-hints for dict values

1 participant