Skip to content

How can I deploy bentoDeployment with both remote runner and embedded runner? #507

@KimSoungRyoul

Description

@KimSoungRyoul

As far as I know, this feature is not currently supported, can you please consider this for yatai 2.0?


a_runner = bentoml.xxx.to_runner(embedded=True)
b_runner = bentoml.xxx.to_runner()

svc = bentoml.Service()

@svc.api(...)
async predict(...): 
    a_result = await a_runner.async_run(...) # embedded runner
    b_result = await b_runner.async_run(...) # remote runner ( other pods )

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions