Multi-fidelity surrogate models by jpenn2023 · Pull Request #721 · emdgroup/baybe

jpenn2023 · 2026-01-08T09:03:28Z

Adding multi-fidelity properties for the SearchSpace class and multi-fidelity Gaussian process classes with the required fidelity kernels.

AVHopp

Some comments on everything but the multi_fidelity.py file, as I guess that we will discuss this today in a bit more detail.

AdrianSosic · 2026-03-03T20:28:28Z

Hey @jpenn2023, I just wanted to have a look at your PR but then noticed that something went wrong with your rebase. I know you asked me about the weird diff shown on Github (and yes, that is generally still a problem), but it turns out that your diff is wrong for a different reason – namely because you somehow rebased your commits together with an entire bunch of other commits 😬 Can you quickly fix it and ping me once ready for review?

AdrianSosic · 2026-03-06T17:28:33Z

Hey @jpenn2023, have a look, this should now give you a clean picture of your PR content, right?

jpenn2023 · 2026-03-09T09:31:14Z

Hey @jpenn2023, have a look, this should now give you a clean picture of your PR content, right?

Hi @AdrianSosic. Yes, this view looks right to me.

Scienfitz

my biggest question is why the concepts of TL and MF are now deedply intertwined by using a single searchspace property for combining them isntead of two properties characterizing the searchspaces task / fidelity character

AVHopp

Some more thoughts from my end

AVHopp · 2026-03-27T12:07:28Z

@jpenn2023 it seems like the continuous search space example is currently failing which is why the doc building does not run (just FYI as reading the doc building error logs can be a bit annoying)

AVHopp

I've found several comments that were marked as resolved without them actually being addressed. Please ONLY resolve comments when the changes are actually visible and have been pushed. Also, it would be great if you could give context when resolving a comment. For example, I asked an explicit question at one point and the comment was just being resolved without providing an answer and without changing the corresponding code.

EDIT: Whoopsie, this was on me. Ignore :)

Scienfitz

just threadwork left

AdrianSosic

Hey @AVHopp , here some first input. I'll now look at the kernel constructions in particular, as well as the tests

AdrianSosic · 2026-08-13T06:59:00Z

        return len(task_param.values)

+    @property
+    def n_fidelities(self) -> int:


I know you're just following the pattern of the task parameter, but I think we need to make a decision now. Much of the task stuff grew over time, but here we're now deliberately adding new functionality without having really decided how we handle the "multiple task/fidelity" case. Half of the properties/methods are private because they explicitly say private since not yet decided how to handle multiple params but then the other half is public, effectively ignoring the problem. So one of two things will happen:

We decide that support for multi params is coming soon --> much deprecation work, i.e. all should be private

We decide that support is far down the road --> why making it private then

Let's discuss this point in a meeting.

Agreed to have everything related to the search space and its properties introduced here private.

Changed this in 3d4bd5d. Please comment if something else/additional should be private, resolve if happy.

the analogous task properties also need to be privatized, the code is otherwise in an inconsistent state

I think this can be done without deprecation as super unimportant and super unimpactful "breaking" change, but if you disagree then do it with a deprecation

AdrianSosic · 2026-08-13T07:00:32Z

        NotImplementedError: If more than one
            :class:`baybe.parameters.categorical.TaskParameter` is requested.
+        NotImplementedError: If more than one fidelity parameter is present.
+        NotImplementedError: If task and fidelity parameters are combined.


I think "combined" is not a good word because it sounds like you are producing one new parameter type from the two others. Same in error message

Changed in 840dfff, please resolve if this wording is better or provide an alternative.

AdrianSosic · 2026-08-13T07:24:30Z

+        if not any(
+            i not in (searchspace.task_idx, searchspace.fidelity_idx)
+            for i in range(len(searchspace.comp_rep_columns))
+        ):
+            raise IncompatibleSurrogateError(
+                f"'{self.__class__.__name__}' requires at least one "
+                f"non-task/non-fidelity parameter."
+            )


This is very cumbersome. Also, I'd appreciate if you could avoid access to low-level attributes like exp_rep and comp_rep since otherwise I'll have to recode this after the candidates PR 😬

In fact, no need to access such low level stuff. Simply check e.g. that there is at least one REGULAR parameter in the search space. And then also turn the message around, i.e. instead of listing non-regular types (which will break once the list get's longer), just mention that we need at least one regular parameter.

Check has been simplified in 383e86a. I also took another look at potential changes that touch comp_rep or expr_rep, and added two comments in places where we'll need to adjust after the merge of candidates: a492139 Please resolve if this is fine.

AdrianSosic · 2026-08-13T07:27:03Z

+        ):
+            raise IncompatibleSurrogateError(
+                f"'{self.__class__.__name__}' does not support custom components "
+                f"for numerical multi-fidelity search spaces, which are delegated "


I think the "which" makes no sense. What should it refer to? The costum components? No --> they are not compatible. The search space? No --> botorch doesn't use our search spaces.

Changed the formulation in d28b7e5. Please resolve if you are satisfied with the text now.

AdrianSosic

I'll review the kernel tests once we've aligned on the decorator stuff

AdrianSosic · 2026-08-13T08:39:41Z

+    searchspace = SearchSpace.from_product(parameters)
+    measurements = create_fake_input(


why create these objrects inside the test if we have them already defined outside (keep test logic minimal)

Can you elaborate on this? The corresponding search space objects which do not use any REGULAR parameters are not yet defined outside of the function, but the parameters are (with the excepction of the task parameter). Does it really make sense to have those search spaces and measurements all defined outside of the test on the module level or am I misunderstanding you?

Co-authored-by: AdrianSosic <adrian.sosic@merckgroup.com>

The decorator now represents modeling mechanisms (transfer learning, multi-fidelity) rather than the ICM kernel artifact it builds, and each mechanism can be enabled independently.

Scienfitz · 2026-08-18T08:29:53Z


+### Deprecations
+- The `task_kernel_or_factory` argument of `ICMKernelFactory` has been renamed to
+  `index_kernel_or_factory` since it now also carries fidelity indices. The old name is


The last sentence The old... is obsolete as this is always implied when something is listed under Deprecations

Scienfitz · 2026-08-18T08:30:59Z



-def _enable_transfer_learning(
+def _enable_index_kernel(


I'm fine with this as it closes my original question, leaving it for you two to close this thread

Scienfitz · 2026-08-18T08:33:40Z

    """The factory for the base kernel operating on numerical input features."""

-    task_kernel_factory: KernelFactoryProtocol = field(
+    task_kernel_factory: KernelFactoryProtocol | None = field(


I'm not a huge fan of the new name as it expresses the architecture and not what is its used for (which is a more reasonable naming approach if there are choices involved)

eg we could have kernels intended for task/fidelity that are not index. but give that this is currently not the case and I have no better name suggestion feel free to resolve

Scienfitz · 2026-08-18T08:36:26Z

        return len(task_param.values)

+    @property
+    def n_fidelities(self) -> int:


the analogous task properties also need to be privatized, the code is otherwise in an inconsistent state

I think this can be done without deprecation as super unimportant and super unimpactful "breaking" change, but if you disagree then do it with a deprecation

Scienfitz · 2026-08-18T08:39:22Z


        return result

+    def _validate_fit_context(


personally I thought this name made sense

validate_X_context = gets everyhtign as input that X sees (or X itself as input)

Or is Adrian suggesting to make this method use _ModelContext (and rename the method accordingly)? Could also be an option if its 100% equivalent

jpenn2023 requested review from AVHopp, AdrianSosic and Scienfitz as code owners January 8, 2026 09:03

jpenn2023 force-pushed the dev-mfbo-main-surrogate-models branch from d062293 to 8ed0f2a Compare January 8, 2026 09:43

Scienfitz assigned jpenn2023 Jan 14, 2026

jpenn2023 commented Feb 3, 2026

View reviewed changes

Comment thread baybe/surrogates/gaussian_process/core.py Outdated

jpenn2023 commented Feb 3, 2026

View reviewed changes

Comment thread baybe/surrogates/gaussian_process/core.py Outdated

jpenn2023 commented Feb 3, 2026

View reviewed changes

Comment thread baybe/surrogates/gaussian_process/core.py Outdated

jpenn2023 commented Feb 3, 2026

View reviewed changes

Comment thread baybe/surrogates/gaussian_process/multi_fidelity.py Outdated

AdrianSosic force-pushed the dev-mfbo-main-surrogate-models branch from ccac1f8 to 178772a Compare February 4, 2026 16:58

jpenn2023 commented Feb 5, 2026

View reviewed changes

Comment thread baybe/surrogates/gaussian_process/multi_fidelity.py Outdated

AVHopp reviewed Feb 6, 2026

View reviewed changes

jpenn2023 force-pushed the dev-mfbo-main-surrogate-models branch from 2e64082 to 8e8547b Compare February 27, 2026 07:29

jpenn2023 force-pushed the dev-mfbo-main-surrogate-models branch from d96cf75 to 2c513b0 Compare March 6, 2026 12:25

AdrianSosic force-pushed the dev-mfbo-main-surrogate-models branch from 2c513b0 to 754d5da Compare March 6, 2026 13:48

AdrianSosic changed the base branch from dev/mfbo to main March 6, 2026 13:54

AdrianSosic changed the base branch from main to dev/mfbo March 6, 2026 13:55

AdrianSosic force-pushed the dev-mfbo-main-surrogate-models branch from 754d5da to 563b86a Compare March 6, 2026 17:21

AdrianSosic changed the base branch from dev/mfbo to main March 6, 2026 17:22

AdrianSosic changed the base branch from main to dev/mfbo March 6, 2026 17:22

AdrianSosic force-pushed the dev-mfbo-main-surrogate-models branch from 563b86a to 88db97f Compare March 6, 2026 17:26

AdrianSosic force-pushed the dev/mfbo branch from 5b1aa42 to ae31906 Compare March 6, 2026 17:27

Scienfitz requested changes Mar 11, 2026

View reviewed changes

AVHopp reviewed Mar 13, 2026

View reviewed changes

Scienfitz mentioned this pull request Mar 16, 2026

Multi fidelity acquisition functions and recommender logic #756

Draft

AVHopp requested changes Mar 27, 2026

View reviewed changes

Addd explicit _supported_parameter_kinds to index kernel factory

85e3a65

AVHopp marked this pull request as draft August 6, 2026 08:36

Scienfitz modified the milestones: 0.16.0, 0.17.0 Aug 6, 2026

AVHopp marked this pull request as ready for review August 6, 2026 14:47

Scienfitz approved these changes Aug 6, 2026

View reviewed changes

AdrianSosic requested changes Aug 13, 2026

View reviewed changes

AVHopp and others added 5 commits August 14, 2026 09:41

Update docstring of supports_multi_fidelity

f379623

Co-authored-by: AdrianSosic <adrian.sosic@merckgroup.com>

Remove unnecessary comment in test

69d132b

Co-authored-by: AdrianSosic <adrian.sosic@merckgroup.com>

Shorten error message

8f1974c

Co-authored-by: AdrianSosic <adrian.sosic@merckgroup.com>

Move comment into docstring

dad7203

Co-authored-by: AdrianSosic <adrian.sosic@merckgroup.com>

Rename Enums to SCREAMING_SNAKE_CASE

567a113

AVHopp force-pushed the dev-mfbo-main-surrogate-models branch from 65e2110 to 567a113 Compare August 14, 2026 08:10

AVHopp added 15 commits August 14, 2026 10:13

Use assert_never instead of raising an error

2bc03ef

Improve wording on using task and fidelity parameters

840dfff

Use optional_v instead of plain optional

50ef873

Remove mention of caching in docstring

02bb051

Improve docstrings of validate_fit_context

23337a9

Improve error message within _validate_fit_context

d28b7e5

Update CHANGELOG

df8574f

Dedoplicate code

47c2097

Simplify cumbersome check

383e86a

Add comments to signify rework after candidates merge

a492139

Make fidelity-related properties of search space private

3d4bd5d

Relocate tests

33bcbe8

Simplify tests by using constants instead or parametrization

bf931f1

Expand rejection test for custom components

8821e0c

Rework _enable_index_kernel into concept-level _enable_mechanism

fc8756f

The decorator now represents modeling mechanisms (transfer learning, multi-fidelity) rather than the ICM kernel artifact it builds, and each mechanism can be enabled independently.

Scienfitz reviewed Aug 18, 2026

View reviewed changes

		searchspace = SearchSpace.from_product(parameters)
		measurements = create_fake_input(

Uh oh!

Conversation

jpenn2023 commented Jan 8, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AVHopp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AdrianSosic commented Mar 3, 2026

Uh oh!

AdrianSosic commented Mar 6, 2026

Uh oh!

jpenn2023 commented Mar 9, 2026

Uh oh!

Scienfitz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AVHopp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AVHopp commented Mar 27, 2026

Uh oh!

AVHopp left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Scienfitz left a comment

Choose a reason for hiding this comment

Uh oh!

AdrianSosic left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AVHopp Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

AVHopp left a comment •

edited

Loading

AVHopp Aug 14, 2026 •

edited

Loading