Declarative schema rules, and the search space as a declared type - #867
Merged
Conversation
A field constraint describes one value; some constraints are relations between several. The holdout proportions must sum to one, chunk_overlap must stay under chunk_size, random_state is meaningless when shuffling is off. None of those can be expressed by a per-field keyword, and writing them as a model_validator body leaves no trace in the JSON Schema, so the frontend re-implements them by hand and drifts: the sum-to-one rule currently exists four times at three different tolerances, and the exact-equality copy in the pipeline editor means a valid 60/30/10 split can never be saved. Rules become data instead. A schema declares `rules = [Check(...), Relevance(...)]` over a closed expression algebra, and BaseSchema turns that single declaration into two enforcers: an inherited model_validator(mode="after") on the server, and an additive `x-dashai-rules` root key that the browser replays with the same semantics, before submit and with no network round trip. Design points worth knowing: - No grammar, so there is no parser to keep in parity between the two runtimes. Operators where CPython and JavaScript disagree (//, %, string coercion, truthiness of non-booleans) are left out on purpose, and ruleConformance.fixture.json pins the agreement: 1241 cases generated by pytest and replayed by jest, currently at zero divergences. - Three verdicts, not two. A rule reading a field that is empty, or a context key that was not fetched, is pending rather than failed. Mid edit is the normal state of a form, and a rule that fires while someone is still typing is worse than no rule. - Emission is conditional, so a schema that declares no rules serves a byte-identical document. The 194 in-tree schema classes and the dashai-* packages on PyPI need no change, which the existing test_schemas.py and holdout tests confirm unedited. - Subclass rules merge with their parents' rather than replacing them. 25 in-tree schemas subclass another schema, and a silently discarded inherited rule is a dead constraint nobody notices. - The authoring surface is Python objects, never expression strings, so the wire format stays an implementation detail: if the algebra turns out too tight it can be swapped for CEL or JSONLogic without touching a single plugin. Migrates C1 and C2 on HoldoutSplitter as the first case. The "it is ignored when shuffling is disabled" sentence that random_state carried in five languages is now an enforced Relevance rule, so the renderer can disable the control instead of asking the user to read about it. Also stops BaseSchema's docstring from being served as a component description. ConfigObject.SCHEMA defaults to BaseSchema, and until now five registered components shipped a commented-out method body to the browser as their user-facing text. Not wired into the renderer yet: reading x-dashai-rules in utils/schema.js and honouring the disable effect in FormSchemaField is the next step.
The rule set a component declares now reaches the browser and drives the form, so a cross-field constraint is reported before submit instead of coming back as a 422 with a pydantic URL in it. The plumbing had one real obstacle. formattedModel flattens the served JSON Schema down to its properties, which is where anything at the root gets dropped, so the rules travel on a symbol key: every consumer walks a formatted schema with Object.keys or for...in to render one field per entry, and both ignore symbols, so the rules ride along without becoming a seventh field to draw. It stays enumerable so the consumers that spread the object keep them. From there the two halves of a rule go to the two places that can act on them: - Checks become part of the Yup schema, through withRules, so their messages arrive as ordinary formik errors under every field the relation names. The sum-to-one rule now reports under train, validation and test at once, with the offending total interpolated. - Relevance is render state, so FormSchemaRenderFields resolves it and either disables the control or leaves it out. `disable` also shows the rule's reason under the input, because a field that cannot be typed into and does not say why is worse than one that is merely wrong. The leaf inputs gained a `disabled` prop to make that possible. TextInput already spread its extra props through, the other four did not. Tests cover the seam end to end from the payload the backend actually serves (captured in holdoutSchema.fixture.js rather than hand-written, so it fails if the wire format drifts): the rules survive the flattening and a spread, a schema declaring none behaves exactly as before, 60/30/10 is accepted where an exact comparison rejects it, and the seed stops being required once shuffling is off. The render tests pin the property the rival server-echo design failed on: with shuffling off the seed is inert but still shows the 7 the user typed, and resolving relevance calls neither setFieldValue nor handleUpdateSchema. Read-only over form state is the whole reason that design was rejected, so it is worth a test rather than a comment. One scoped compromise: react-markdown is ESM only and CRA's jest does not transform node_modules, so importing any form component fails to parse before a single assertion runs. It is stubbed in the render test with a passthrough, which renders the plain text those assertions look at. Teaching jest to transform the unified/remark tree is a single config change and belongs in its own commit.
scikit-learn reads SimpleImputer's fill_value only when the strategy is
"constant". Measured on 1.7.2:
strategy="mean", fill_value=99 -> imputes 1.5, the 99 is discarded
strategy="median", fill_value=99 -> imputes 1.5
strategy="constant", fill_value=99 -> imputes 99
strategy="constant", fill_value=None -> imputes 0, without asking
So for three of the four strategies a value the user typed and dashAI
persisted is silently dropped, and the fourth invents a default. Neither
fact was written down anywhere: unlike the holdout seed, which at least
said "it is ignored when shuffling is disabled" in its description, this
dependency was not documented at all.
Both are now rules. The Check deliberately carries no strategy condition
of its own, because a check whose target a relevance rule has marked
irrelevant does not run, so it applies exactly when fill_value is
meaningful. That composition is what keeps a rule set from turning into a
pile of guard clauses, and it is asserted rather than assumed.
Leaving the ignored value in the payload is also deliberate. The natural
way to reach that state is to type a fill value under "constant" and then
change your mind about the strategy, so rejecting it would be hostile;
sklearn ignores it, and the form disables the control and says why.
This is the first relevance rule on a field the generic renderer draws
through FormSchemaFieldWithOptions rather than a plain input, which turned
up a gap: the disable effect reached the input but not the type-selector
chips, so a user could still switch Int/Float/String/Null and write a
value into a field that means nothing. SingleSelectChipGroup and
FormSchemaFieldWithOptions now take `disabled` too.
Optimizer fields and nested component fields still cannot be disabled. No
shipped schema asks for it, and passing the prop to a component that
ignores it would only make it look handled, so the loop says so in a
comment instead. `hide` and `omit` already work everywhere, because they
are decided before a branch is chosen.
One out-of-reach case is asserted rather than left implicit:
fill_value="x" against a numeric column raises inside sklearn at fit time,
in a Huey worker. Catching that needs the dataset's column dtypes, which
is the context channel and a later phase.
A cleared text box hands back "", and for a field like group_column an
empty string is not "no value", it is a column named "". That reached
scikit-learn, or got persisted and raised a KeyError in a Huey worker much
later. 77 fields accept string-or-null today, including every class_weight
and max_features in the sklearn wrappers.
The int half of this was fixed years ago in the leaf inputs, where
IntegerInput and NumberInput each coerce "" to null themselves, and it was
the visible half because "" fails int validation loudly. The string half
was never fixed, because it fails silently.
The historical attempt went the other way, translating "" to None during
validation, and there was no good place for it: from a bare string the
schema layer cannot tell whether the author meant "unset" or "the empty
string". What made it decidable is that the schema already says both
things, and nobody was reading them:
- the field does not admit null -> "" is a value, leave it
- it admits null and its placeholder is "" -> empty means "", which is the
14 negative_prompt fields across the diffusion models and exactly why a
global rule would have been wrong for them
- it admits null and its placeholder is anything else -> empty means unset
So emptyValueFor derives it per field with no new keyword, no authoring
burden and no backend change, and normalizeEmptyValue is applied at the one
point where a form reports a change, so every input type is covered and no
leaf component has to know about it. The pipelines dispatcher calls the same
shared helper rather than growing its own copy, since the two must not
disagree.
Also removes the display of the literal word "none" for a null value
(TextInput, a work-in-progress line from March 2024). It stayed harmless
only because the null branch renders that input disabled: with the input
enabled it would be a submittable string, and pydantic does not parse
"none" to None, it keeps it as the string 'none'. And passes the
`placeholder` prop that FormSchemaFieldWithOptions documents and reads but
neither of its two call sites ever provided, which is why leaving the Null
chip used to set the value to undefined.
Not changed: the backend still accepts "" for a nullable string. Tightening
that would reject params already persisted, so it is a separate decision
with a data-migration dimension rather than something to ride along here.
A dropdown that lists a value the wrapped library refuses is worse than a
missing feature: the option looks supported, the form validates it, the
parameters get persisted, and the run dies with an InvalidParameterError
inside a Huey worker minutes later.
Cross-checking every enum option of the 55 components that wrap a
scikit-learn estimator against that estimator's own
`_parameter_constraints` found five that reach sklearn invalid:
- GradientBoostingR.criterion offered "mse" and "mae", deprecated in
scikit-learn 1.0 and removed in 1.2. Replaced by "squared_error".
- RandomForestRegression.max_features offered "auto", deprecated in 1.1
and removed in 1.3 — and it was the first entry in the list, so it is
the one a user trying the dropdown reaches first.
- OrdinalEncoder.unknown_value offered "int" and "np.nan", which are type
names rather than values. cast_string_to_type turns "np.nan" into nan,
but "int" reached sklearn as the class `int`. sklearn wants an actual
number here, so the int branch is a real number field now.
- TruncatedSVD.power_iteration_normalizer offered "QR", and scikit-learn
1.7 accepts "OR" instead: its StrOptions are {"auto","OR","LU","none"}.
That is a typo upstream — the algorithm is QR decomposition — so the
schema keeps the correct name and the misspelling is confined to one
line in the converter.
Also moves `None` out of two str-typed enums (GradientBoostingR and
RandomForestRegression max_features). enum_field returns Annotated[str,
...], so a None member is advertised in the JSON Schema and then rejected
by the field's own validator: the schema was describing an option it could
not accept, which is why GradientBoostingR.max_features had an
unsubmittable default. Both now spell it none_type(union_type(...)), and
max_features has left the set of fields whose placeholder fails its own
schema.
The check is mechanical, so it is a test rather than a note: 158
parametrized cases, one per option per component. It constructs the
component instead of reading the schema, which is what keeps it from
flagging the translations that are correct — random_state="RandomState"
becoming a real numpy RandomState in six converters,
unknown_value="np.nan" becoming nan, MLPClassifier's hidden_layer_size
becoming the (n,) tuple sklearn expects. The TruncatedSVD quirk is pinned
explicitly, so if scikit-learn ever fixes the spelling the test tells us
to drop the translation.
The diffusion models' image sizes have to be multiples of 8, because the VAE downsamples by that factor. That was stated in the description of 28 properties, in five languages, and enforced nowhere: a width of 513 passed the form, got persisted, and went into the pipeline unrounded. Only one model rounds it down itself (sd15_depth_controlnet_model.py:225); the other fourteen forward it as typed. `multipleOf` is standard JSON Schema and pydantic already emits and enforces it, so this needed no new mechanism at all — just one parameter on int_field and the keyword read on the way out. The 28 emitted properties come from 12 declarations across 6 schema classes, since fourteen registered models share them by inheritance. Every one of the 28 defaults was already a multiple of 8, so nothing that used to be accepted becomes invalid except values the description already called invalid. On the frontend Yup has no built-in for it, so applyMultipleOf rides a test in getValidator, which covers both a plain field and the selected branch of a union. Worth stating why this is not a rule: it is a constraint on one value, not a relation between fields, so it belongs in the schema's own vocabulary rather than in the rule engine. Conflating the two is how a rule layer grows until nobody can predict what it does — the algebra deliberately has no modulo operator, and this is the case that would have tempted someone to add one. Also adds a test for the class rather than for these 28 fields: it reads descriptions the way a user would and asserts the schema agrees, for "multiple of N" and for "between X and Y". 37 cases pass, and the one skip is named with its reason — DescribeExplorer.percentiles describes a range for the integers parsed out of a comma-separated text box, which no keyword on a string field can express.
Plotly's histnorm offers "" as a real value meaning raw counts, and it is the field's default. Two independent defects made that option unusable. SelectInput translated a selection of "" into null, on the assumption that an empty string from a dropdown means "nothing selected". For a field that does not admit null the backend then rejected it, so once a user changed the normalization there was no way back to the default. The empty string is only emptiness when it is not itself one of the options, which the control can see, so it now checks. The same rule goes into emptyValueFor, so the two layers cannot disagree about the same field. The option also rendered as a blank row, because the label falls back to the raw value and `enumNames` — the key the renderer has always read — had no producer anywhere in the backend. Across 219 components it was emitted zero times, which means every dropdown in the product has been showing raw Python values in all five languages, and an option whose value is the empty string showed nothing at all. So the option nobody could see was also the default. enum_field now takes `labels`, a mapping from option to MultilingualString, emitted as enumNames aligned with enum because the renderer reads it by index. Options left out fall back to their own value, so a partial mapping is fine and only the options that need a name get one, and a label naming an option that does not exist raises rather than silently never showing. It rides the existing server-side localize() path, so a labelled dropdown is translated with no client-side machinery. histnorm is the first user; the rest of the enums keep behaving exactly as before, since a schema that passes no labels emits no key.
When the underlying library takes a range as one tuple — feature_range,
ngram_range, percentiles, a pair of Canny thresholds — the schema cannot
express a tuple, so the pair becomes two form fields. Splitting it drops the
one thing the tuple guaranteed: that the low end comes first.
Ten pairs did that and none of them checked. This is the case the rule layer
was built for, and it turns out the holdout proportions were never the
exceptional one: with these ten and the conditional check inside
ExponentialSmoothing's __init__, it is a family of twelve.
The failure modes, measured rather than assumed:
- MinMaxScaler, the vectorizers and the n-gram converters raise a ValueError
from inside scikit-learn. For a converter or a model that means inside a
Huey worker, long after the form said the configuration was fine.
- partial_dependence raises at explanation time.
- cv2.Canny raises nothing and returns the same edges either way, so an
inverted pair is silently wrong. That is the worse one: the user never
finds out that the configuration they chose did nothing.
The relation is not the same everywhere and each was checked against the
library rather than guessed. min_df/max_df and ngram_range accept equal
values (sklearn compares the resulting document counts, and (1,1) is a valid
unigram range); feature_range and percentiles require the low end to be
strictly smaller ("must be smaller than maximum", "percentiles[0] must be
strictly less than percentiles[1]"). All ten pairs are plain numeric fields,
so the comparison is unambiguous — none of them is the int-or-proportion
union that would have made ordering meaningless.
Each rule names both fields as targets, so the message appears under both
halves of the relation rather than under whichever one the user was not
looking at.
The test is written against the pairs rather than the rule ids, so renaming
a rule does not break it and adding a new split range without an ordering
rule does. It also asserts what each library actually does about equal
values, and that every shipped default satisfies its own rule, since a
default that broke it would block the form before the user typed anything.
The exploration wizard's explorer picker showed an error and no options at
all. Two defects in the same function, both from the same cause: the
contract for which dataset columns a component accepts was reimplemented
four times in the frontend, and one copy went stale.
- It read `restricted_dtypes`, a key renamed to `non_allowed_dtypes` and
explicitly popped by both metadata builders, with no fallback. Calling
`.some()` on undefined threw inside the `.then()`, so every render landed
in the catch that reports "error while fetching explorers".
- It tested `allowed_dtypes.includes("*")` to mean "no restriction". The
backend normalizes `["*"]` to `[]` before serving, so the test was always
false: an explorer with no dtype restriction filtered its columns against
an empty allow-list, ended up with none, and was disabled for failing its
own minimum of one column. Every unrestricted explorer, every dataset.
It also ignored `allowed_types` entirely, which the other three copies
apply.
So the contract now lives in one place, mirroring `validate_columns` on the
backend, and the four consumers read it: the exploration wizard, the column
dialog, and both pickers in the notebook sidebar. Two more silent gaps fell
out of consolidating them:
- The column dialog's "does not accept these data types" chips read the
retired key too, so they never rendered. The backend has been enforcing a
blacklist that nine components declare and the user was never shown.
- The notebook's converter picker never applied the blacklist at all, and
two of those nine are converters: SMOTE and SMOTEENN refuse string, bool
and unnamed dtypes, so both were offered for datasets whose columns the
backend would then refuse.
And `base_converter` now defaults `input_cardinality` to `{"min": 1}`, the
way `base_explorer` already did. 41 converters served it absent or None, the
picker read that as "no requirement", and no converter was ever disabled for
a dataset it cannot accept — the frontend code for it was there, it just
never had data to work with.
Two suites pin the contract from both ends rather than trusting a comment:
16 cases in columnEligibility.test.js for the browser's reading of it, and
533 in test_component_metadata_contract.py for what the payload contains and
what it must not contain, including a named list of the nine blacklists so
growing that set is a decision rather than an accident.
While in ConfigureExplorersStep: its validateOptions callback read
datasetColumns with an empty dependency list, so it kept whatever columns
were loaded on the first render.
…t cannot work
ExponentialSmoothing carried both halves of one dependency where the form
could not see either.
The half that moves: a season length means nothing without a seasonal
component. That was a sentence in the field's description, in five
languages, enforced nowhere. It is now a Relevance rule, so the control is
disabled and says why. The condition compares against the string "none"
because that is how statsmodels spells "no seasonality" — one of the three
spellings of absence in this codebase, and the reason the comparison cannot
just be a null check.
The half that does not move, and the reason it is worth a commit: a seasonal
component needs a season length of at least 2, since a season of 1 repeats
every observation. That check stays in __init__. It cannot become a Check
because season_length is an optimizer field, so its value is the {optimize,
fixed_value, lower_bound, upper_bound} envelope rather than a number, and the
algebra refuses a non-number by design. The rule would evaluate to pending
forever and never fire.
Pending is reported rather than passed, so nothing would have been validated
wrongly — but nobody reads a pending report, so a rule that can only ever be
pending is indistinguishable from a rule that works. validate_rules now
refuses one outright, with a message that says what the value model is, that
the rule would never fire, and that the constraint has to stay in a validator
until an optimizable hyperparameter is a declared type. So the comment
explaining this is enforced: write that Check and the class fails to import.
The guard distinguishes reading a field from targeting one, because
Relevance("season_length", when=...) evaluates its condition and never the
field's own value. Targeting an optimizer field stays allowed, which is what
makes the half above possible at all.
This is the first case where the sequencing stopped being a preference and
started being a blocker, which is worth having as a test rather than as a
paragraph in a plan: nine cases pin both halves, including that refusing the
rule did not lose the constraint.
Fifteen fields stated a relevance dependency in their description, in five languages, enforced nowhere. The useful question was not how many there are but which of them a rule can express today, so that got measured before any of it got written: - Five can, and now do. The seed-and-shuffle dependency in the five splitters that take both, and the covariance explorer's delta degrees of freedom. Their conditions read a plain boolean sibling, which is all the algebra needs. - Two cannot, because the condition is not in the schema at all. ColumnArithmetic and ColumnConcat say the constant is "only used (and required) when a single column is selected", and how many columns the user picked is session state. Those wait for the context channel, and a test now says so rather than leaving the omission looking like an oversight. - One could not because of the value model, which is the ExponentialSmoothing case from the previous commit. - Three were never dependencies. The sweep matched "at 0.0 the depth map has no effect" and "when keep_binary_mask is enabled, mask_i columns...", which describe what a value means and which columns get produced, not what the field depends on. Retracted rather than converted. The five splitters share one rule object instead of five copies of the same sentence. A rule is stateless, so one instance can be listed by several schemas, and each still has its field names checked against its own fields, so listing it somewhere without a shuffle field fails at import. Holdout's inline copy from the first commit is now that shared one too. The alternative was a shared base schema for five components, which is a structural change for one shared sentence. The covariance explorer is the interesting one: delta_degree_of_freedom is declared 30 lines above the numeric_only it depends on. A @field_validator there would read info.data, find its controller absent and quietly do nothing — the same silent no-op the RAG chunkers avoid only by luck of declaration order. A rule runs on the complete model, so the order cannot disable it, and there is a test asserting the order is in fact the bad one. 31 cases, including that the prose is gone from all ten descriptions in the five splitters, and that the two repeated fold splitters correctly declare no such rule: they have a seed but no shuffle flag, because sklearn always shuffles them.
`enumNames` is the key the renderer has always read to decide what to show
for an enum option, and nothing produced it: zero emissions across 219
components. So every dropdown in the product displayed the raw Python value
in all five languages. `friedman_mse`, `char_wb`, `balanced_subsample`, `C`
and `F` are the right values to send and the wrong words to show.
The shape of the problem made the fix small. 216 unlabelled fields used only
110 distinct option sets, and `["sqrt", "log2"]` means the same thing in
every component that offers it, so the name belongs to the option set rather
than to the field. enum_field consults a shared table when a schema declares
no labels of its own, which turns 126 call-site edits into one file and no
churn at all. An explicit `labels=` still wins.
66 of 217 enum fields now carry names, up from one.
Two families are deliberately left showing their raw value, and there is a
test for each. Model checkpoints, because `sentence-transformers/all-MiniLM-
L6-v2` is the identifier a user wants to see and renaming it loses
information. And device lists, 72 fields, because the value is already
readable and one of them is built at import time from the machine's own GPU
name.
Two option sets are deliberately absent from the table for a different
reason: `("auto",)` is offered by iterated_power, where it means "pick the
iteration count for me", and by sampling_strategy, where it means "resample
every class but the largest". One name cannot be right for both. Keying the
table on the option set is what makes that a decision rather than an
accident, and a test pins it.
Some of the names carry information the raw value hid. The distance metric
set offers cityblock, l1 and manhattan, which are three names for one
metric; labelling them says so. `C` and `F` become row-major and
column-major. `RandomState`, the sentinel standing in for an object a form
cannot express, becomes "a new random generator each run".
Nothing about the submitted value changes: MenuItem still carries the enum
member, the label is display only, so there is no migration dimension.
HPO worked for integers and floats only. That was not an oversight in the
optimizers so much as a consequence of how a search space was declared: the
only carrier of "this parameter can be optimized" was the shape of its
placeholder dict, and that shape has two keys, `lower_bound` and
`upper_bound`. It is a vocabulary that can only describe a scale, and `hinge`
is not between `squared_hinge` and anything else.
Of the 204 fields in the 31 models that already do numeric search, 109 are
searchable today, 38 are enums and 18 are booleans. Nothing in that set is
impossible to optimize; the enums and booleans had no way to say what they
wanted. A boolean is an enum with two options, so one mechanism covers 56.
`search_space` makes the space the field's type:
loss: search_space(
enum_field(["squared_hinge", "hinge"]),
fixed="squared_hinge",
description=...,
)
The four-key envelope already in the database is unchanged, and so is the
emitted property apart from one added key, so no migration and no rewrite of
the numeric forms. What changes is that the envelope is now checked: an
interval or a set of choices but never both, `lower_bound < upper_bound`
enforced where the value lives rather than in a yup test, and every bound and
option validated against the field's own constraints. A declaration that
contradicts itself fails at import instead of at some trial in the middle of
a study.
The dtype is declared as `x-dashai-search-dtype` rather than read off the
JSON Schema `type`, which is absent whenever a field admits null and cannot
say "categorical" at all.
Two bugs surfaced on the way and are fixed here:
* Hyperopt's space builder was an `if`/`elif` with no `else`, so a parameter
of any other kind was left out of the space and silently never optimized:
the study ran, reported trials, and moved nothing.
* Hyperopt reports the *index* of a chosen option in everything it hands back
after `fmin`, and the results were read through `float(raw_value)`. That
would have set `loss = 1.0` on the final model. All three read sites now go
through `space_eval`, which also means the trials table shows `5` rather
than `5.0` for an integer.
An unsupported dtype is now a TypeError. `study.optimize` catches ValueError
as an unfittable trial, so it used to surface as "every one of the N trials
failed, narrow the ranges and try again", which is the wrong advice for a
declaration naming a kind of space that does not exist.
Two form-layer fixes come with it. The renderer tested `anyOf` before
`isOptimizable`, so a field that admits null always went to the union picker,
which is a second reason the 31 fields declared `none_type(optimizer_*)`
could never be optimized. And the option list for a categorical control comes
from the declared search space rather than from `enum`: `enum` sits inside an
`anyOf` branch that does not mention null, so a user who unticked "None" on
`class_weight` could not tick it back.
48 fields migrated across 20 models. Null counts as an option where the field
admits one, which is what makes `class_weight` searchable at all: unweighted
against balanced is the comparison worth running, and read as a one-option
enum there is nothing to search. 13 option sets gained shared names, since
those values are now listed in the options-to-search control where a raw
`squared_epsilon_insensitive` is least readable.
Left alone, with reasons: `mlp_regression.device`, whose options are built
from the machine's own GPU at import; `exponential_smoothing.trend` and
`seasonal`, because a Relevance rule reads `seasonal` and a rule over an
envelope is always pending, which `validate_rules` refuses, so migrating them
would stop the class from importing; and `copy_X`, `oob_score`, `warm_start`
and `dual`, which are not hyperparameters or are constrained against other
fields in ways a blind search would violate.
Two things finished here: the audit of the thirty-one fields that claimed to
be optimizable and never could be, and the migration of the remaining
hand-written envelopes. With no declaration left using them,
`optimizer_int_field` and `optimizer_float_field` are deleted; they were
byte-identical to `int_field` and `float_field`, which is what made the
placeholder's shape the only carrier of the signal in the first place.
THE AUDIT
The thirty-one were `none_type(optimizer_int_field(...))`, which forces
`placeholder=None` and erases the optimize signal. The first reading was that
thirty-one searches had been lost. Read by name it is not that:
random_state 16 max_depth 7
n_jobs 2 max_leaf_nodes 4
n_iter_no_change 1
max_samples 1
A seed does not have a better value, it has a value that happens to score
better on this split. A thread count changes how long a fit takes and nothing
about the fit. Eighteen of the thirty-one should never have been declared
searchable, and the accident that erased the signal is the only reason the
product never offered it. So: thirteen revived with a declared range, eighteen
demoted to ordinary fields.
Three more of the same mistake were live rather than dead, and only turned up
because the migration made every declaration explicit:
`RandomForestClassifier.random_state` was searchable over `0..10`, and
`verbose` over `0..100` on two models. Demoted too — twenty-one in total.
`GradientBoostingR.min_samples_leaf` was declared over `(0, 0.5]`, the
fraction-of-sample form, while carrying counts and a `1..20` range: every
trial would have proposed a value the field forbids. The other three models
offering that parameter declare it `int_field(ge=1)`, and its own description
says "the minimum number of samples", so the type was the odd one out. This
is the contradiction `search_space` refuses at import, and it is why it
refuses at import.
THE MIGRATION
106 numeric declarations became `search_space`. The wire diff across all 219
components falls in four buckets and nothing else moved:
105 the dtype key added, placeholder byte-identical
13 revived: placeholder and dtype
3 demoted: placeholder only, envelope to scalar
1 min_samples_leaf retyped
Two form-layer gaps the revivals exposed, both about a field that admits null
keeping its information one level down inside `anyOf`:
* Yup read `minimum`/`maximum` off the property only, so the thirteen revived
nullable parameters had no bounds enforced in the form at all and a depth of
-3 reached the backend before anything objected.
* The categorical control took its option list from `enum`, which for
`class_weight` does not mention null, so a user who unticked "None" could
not tick it back. It now comes from the declared search space, and the
option names are matched by option rather than by index, since `enumNames`
aligns with `enum` and no longer with the options.
STILL PROSE, NOT CODE
Two declared ranges cannot produce a converged model, and both are on SVC.
`max_iter` is searched over `-1..10` where the other twelve models offering it
search 50..10000, and -1 means unlimited, so the range mixes "no limit" with
"ten iterations" as if they were points on one scale. `tol` is searched over
`1.0..10.0` where eight siblings search 1e-6..0.1, and
`LogisticRegression.tol` starts at 0.0. These pass every constraint, so no
mechanism will object; changing a search range changes results, so they are
reported rather than quietly fixed.
The plugin guide documented `int_field`, `float_field`, `enum_field` and
`schema_field` and never the two deleted factories, so a plugin following it
is unaffected. One importing them directly breaks at import.
cristian-tamblay
commented
Sep 7, 2026
Irozuku
approved these changes
Sep 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Schema fields could not depend on each other. A rule like "the partitions sum to at most 1" had no home, so it lived in a bespoke form, in prose nobody enforced, or nowhere. This branch gives that kind of statement a place to live: rules are declared as data next to the fields they constrain, published on the wire under
x-dashai-rules, and evaluated by two interpreters — one in Python, one in JS — kept honest by a generated conformance fixture both consume.The same idea then applies to the second thing a field could not declare: a hyperparameter's search space. That was carried by the shape of its placeholder dict, which is why HPO worked for ints and floats only, why 31 fields claimed to be optimizable and could not be, and why no component's schema accepted its own declared defaults.
Most of what this enables was already reachable and switched off.
enumNameshad been the key the renderer reads for dropdown labels since forever and nothing produced it, so every dropdown in the product showedfriedman_msein all five languages.multipleOfwas described in prose instead of declared. Options scikit-learn rejects were being offered. A field emptied by the user sent""where a number was wanted.Nothing here is a rewrite. Rule emission is additive, the four-key search envelope already in the database is unchanged, and no component or plugin is required to declare anything.
Type of Change
Check all that apply like this [x]:
Changes (by file)
The rule layer
DashAI/back/core/schema_fields/rules.py(new): the authoring API.CheckandRelevance, and a closed 16-node expression AST (F,Ctx,Sum,Approx,And,In,Len, …) with operator overloads. Being a closed set of nodes rather than a grammar is what removes any parser-parity risk between the two runtimes.validate_rulesrefuses a rule that cannot fire, so a mistake fails at import instead of silently doing nothing.DashAI/back/core/schema_fields/rules_eval.py(new): the Python interpreter. Verdicts are three-valued — aPENDINGsentinel whose__bool__raises, so a half-filled form cannot be read as a violation.DashAI/front/src/utils/ruleEngine.js(new): the mirror interpreter pluswithRules, which folds the rules into the Yup schema so violations arrive through the channel formik already reads.DashAI/back/core/schema_fields/base_schema.py: merges parent and own rules, validates them at class definition, publishes them underx-dashai-rules, and runs them as an inherited validator.tests/back/core/test_rules_conformance.py(new): generatesruleConformance.fixture.json(1,241 cases) from the Python side; the JS suite consumes the same file, so the two interpreters cannot drift apart quietly.The search space
DashAI/back/core/schema_fields/search_space.py(new):search_space(inner, fixed=, low=, high=, choices=)makes the space the field's type. The placeholder is derived, so it cannot disagree with the field it describes, andfixed/low/high/every choice are checked against the field's own constraints when the class is defined.DashAI/back/core/schema_fields/optimizer_{int,float}_field.py: deleted. They were byte-identical toint_fieldandfloat_field, which is exactly why the placeholder's shape ended up carrying the signal.DashAI/back/optimizers/optuna_optimizer.py:suggest_categoricalarm, so an enum or a boolean can be searched.DashAI/back/optimizers/hyperopt_optimizer.py: same, plus two bugs. Its space builder was anif/elifwith noelse, so a parameter of any other kind was left out of the space and silently never optimized. Andfminreports the index of a chosen option, which the code read throughfloat(raw_value)— that would have setloss = 1.0on the final model. All three read sites now go throughspace_eval.DashAI/back/models/model_factory.py: reads a declaredx-dashai-search-dtypeinstead of the JSON Schematype, which is absent whenever a field admits null and cannot say "categorical" at all.DashAI/front/src/components/configurableObject/Inputs/SelectInputOptimize.jsx(new): the third optimizer input, sibling to the two numeric ones. An interval makes no sense for something picked out of a set, so this offers the options instead.Forms that were available and switched off
DashAI/back/core/schema_fields/enum_labels.py(new): a hand-written table of 53 option-set vocabularies, 154 names in five languages, which now name 79 of the 217 enum fields in the tree.enumNameshad been the key the renderer reads and was emitted zero times across 219 components, so every dropdown showed the raw Python value in every language. The 217 fields use only 111 distinct option sets, so the name belongs to the option set rather than to the field, which is what makes this a table rather than 126 edits. Sets whose meaning is not the same everywhere stay out on purpose:("auto",)means "pick the iteration count for me" in one field and "resample every class but the largest" in another.DashAI/front/src/utils/schema.js:emptyValueForderives what an emptied field means from its own schema, which is the elegant home the"" → Nonepatch never had;applyMultipleOf; bounds now read from theanyOfbranch too.DashAI/front/src/components/shared/FormSchemaRenderFields.jsx: relevance effects, andisOptimizablenow tested beforeanyOf— the union picker used to win, which is a second reason a nullable field could never show its optimize toggle.DashAI/front/src/utils/columnEligibility.js(new): one implementation of the column metadata contract, replacing three that disagreed.SelectInput.jsx:disabledsupport, and""coerced to null only when""is not itself an option (Plotly'shistnormoffers it as a real value meaning raw counts, and coercing it made that option unselectable).Schemas migrated
167 search spaces across 31 components (69 integer, 50 number, 48 categorical); relevance rules in 5 splitters sharing one rule object,
SimpleImputer,ExponentialSmoothingand the covariance explorer;multipleOfin 6 diffusion schemas; option fixes in 4 sklearn models.Testing
Backend 2740 pass (1 skip, 1 xfail); frontend 208 pass (up from 187 — there were no form tests at all before, because
react-markdownis ESM-only and CRA's jest does not transformnode_modules, so importing any form component was a parse error).Two things worth a reviewer's eye, both about failing loudly:
withRulesinruleEngine.jsuses Yup's{is, then, otherwise}form deliberately. The bare-function builder is called asfn(...values, schema, options), so reading the schema off the end of the argument list picks upoptions, Yup throws "conditions must return a schema object", and formik turns that into no errors at all. There is a guard that fails loudly rather than silently disabling every validation.TypeError.study.optimizecatchesValueErroras an unfittable trial, so it used to surface as "every one of the N trials failed, narrow the ranges and try again" — the wrong advice for a declaration naming a kind of space that does not exist.Notes
The audit of the 31 went both ways. The fields declared
none_type(optimizer_int_field(...))could not be optimized however they were declared. Read by name, 16 arerandom_stateand 2 aren_jobs. A seed does not have a better value, it has a value that happens to score better on this split. So 13 were revived with a declared range and 18 demoted to ordinary fields — and 3 more of the same mistake turned up live (RandomForestClassifier.random_statewas searchable over0..10,verboseover0..100on two models).Two declared ranges cannot produce a converged model, and I did not change them. Both are on SVC:
max_iteris searched over-1..10where the other twelve models offering it search 50..10000, and-1means unlimited, so the range mixes "no limit" with "ten iterations" as if they were points on one scale.tolis searched over1.0..10.0where eight siblings search1e-6..0.1, andLogisticRegression.tolstarts at0.0. These pass every constraint, so no mechanism objects; changing a search range changes results, so they want a decision rather than a patch.Deliberately left as prose, with a test saying so. The two
constantconverters depend on how many columns the user selected, which is session state and not a schema field: that needs the context channel.ExponentialSmoothing.seasonalcannot be searched because aRelevancerule reads it, and a rule over a search envelope can only ever be pending —validate_rulesrefuses such a rule, so trying stops the class from importing. That guard working is why the omission is a decision and not an oversight.Known and unfixed, found while measuring:
TrainNode.jsx:476compares partition sums with exact equality, so 60/30/10 is unsaveable;validator.py:89hardcodes the holdout splitter names;utils/schema.jsgives ananyOffieldYup.mixed().nullable(), leaving 241 properties unvalidated;applyMinMaxmapsexclusiveMinimumto.min()at 12 sites;IntegerInput.jsx:24parses"1e3"as 1.Plugins: the plugin guide documented
int_field,float_field,enum_fieldandschema_field, never the two deleted factories, so a plugin following it is unaffected. One importing them directly breaks at import; a 6-line deprecated alias would give a grace period if that is wanted.Still open by design: the optimizer's own UI, and
run.parametersis not validated at read time, so the new runtime checks only bite where something callsmodel_validate.