Skip to content

Add support for calling input plane functions #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from

Conversation

thecodingwizard
Copy link
Contributor

This PR adds libmodal support for calling input plane functions. This requires a few changes:

  1. Support multiple ModalClientStubs pointing to different server URLs.
  2. Propagate x-modal-auth-token from FunctionGet to the input plane RPCs.
  3. Implement the new set of RPCs for the input plane.

We will need to add support for retrying internal failures next.

Note that this PR cannot be merged until we enable input plane functions in production; otherwise, the tests will fail.

@@ -13,6 +13,8 @@ import {
import { ClientType, ModalClientDefinition } from "../proto/modal_proto/api";
import { profile, Profile } from "./config";

let modalAuthToken: string | undefined;
Copy link
Contributor Author

@thecodingwizard thecodingwizard May 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that the way we propagate auth tokens is a little different in Go vs. Typescript.

In Typescript (and Python), we propagate auth tokens via auth middleware / a grpc interceptor in client.ts.

In Go, we propagate auth tokens via context in function.go.

I'm open to changing Go's implementation to also propagate auth tokens via grpc interceptors in client.go for consistency if we think that's better, but it might be a bit trickier since Go is multithreaded? Regardless, we can always change our minds later.

@thecodingwizard thecodingwizard force-pushed the nathan/inputplane branch 3 times, most recently from 52b14c5 to bb7b679 Compare May 19, 2025 16:00
@thecodingwizard thecodingwizard requested a review from ekzhang May 19, 2025 16:02
@thecodingwizard thecodingwizard marked this pull request as ready for review May 19, 2025 16:02
@rculbertson
Copy link
Contributor

rculbertson commented Jun 9, 2025

@ekzhang I rebased this PR on main, and updated it to be compatible with the changes made since it was first opened. So far I've made the changes only to the typescript part. If those changes seem good, I'll go ahead and take the same approach in the Go code.

In the code in main, #execFunctionCall returns a function call id to .remote. That then calls pollFunctionOutput with the function call id, which calls FunctionGetOutputs. For the input plane, we don't have a function call id - we have an attempt token. We need .remote to call AttemptAwait and pass it the attempt token.

To handle these two cases, I created a class called FunctionOutputPoller. This replaces the pollFunctionOutput function. Within #execFunctionCall, we create an instance which is configured to call either the input plane or the control plane. In .remote we call FunctionOutputPoller.poll() which will call the right one.

Hope that makes sense! Happy to chat IRL if that's easier!

@rculbertson rculbertson force-pushed the nathan/inputplane branch 2 times, most recently from fcc1f50 to 00654ec Compare June 9, 2025 22:07
@rculbertson
Copy link
Contributor

After starting to implement client retries, I realized some more refactoring was needed. I opened this PR #30, which is based on top of this one. In that one, I removed the FunctionOutputPoller class and replaced it with the InputStrategy interface (and the classes which implement it). See that PR's description for more info.

@rculbertson
Copy link
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants