Create Playground for Deployments

[Cahllagerfeld]

[znegrin]

@Cahllagerfeld I guess for this first version, this will be closer to this design, showing the schema with the different types and the fields based on the code (I used JSON for the example):

I am not sure if, apart from the schema, we need to show endpoints, which then will make the design slightly different.

I used a JSON example from a model I found during my research, but I don't know if we have an example of how this will look in ZenML, more specifically. Maybe @stefannica has one, so we can prepare a more accurate design for the playground MVP.

[stefannica]

This is an example from the weather agent we've been using in our demos (extracted from the snapshot's pipeline_spec field). The important sections are input_schema and output_schema, which are both using the JSON schema format (similar to the stack component flavor configuration used to render the "create/update stack component" screens):

    "input_schema": {
        "additionalProperties": false,
        "properties": {
            "city": {
                "default": "London",
                "title": "City",
                "type": "string"
            }
        },
        "title": "PipelineInput",
        "type": "object"
    },
    "output_schema": {
        "properties": {
            "weather_analysis": {
                "title": "Weather Analysis",
                "type": "string"
            }
        },
        "required": [
            "weather_analysis"
        ],
        "title": "PipelineOutput",
        "type": "object"
    },
    "outputs": [
        {
            "output_name": "weather_analysis",
            "step_name": "analyze_weather_with_llm"
        }
    ],
    "parameters": {
        "city": "London"
    },
    "source": {
        "attribute": "weather_agent",
        "module": "pipelines.weather_agent",
        "type": "user"
    },
    "steps": [
        {
            "inputs": {},
            "invocation_id": "get_weather",
            "source": {
                "attribute": "get_weather",
                "module": "steps.weather_agent",
                "type": "user"
            },
            "upstream_steps": []
        },
        {
            "inputs": {
                "weather_data": {
                    "output_name": "weather_data",
                    "step_name": "get_weather"
                }
            },
            "invocation_id": "analyze_weather_with_llm",
            "source": {
                "attribute": "analyze_weather_with_llm",
                "module": "steps.weather_agent",
                "type": "user"
            },
            "upstream_steps": [
                "get_weather"
            ]
        }
    ],
    "version": "0.5"
}

This information, plus the outputs field are also important when representing the runs created from invoking the deployment. The pipeline outputs are in fact step outputs with the same name.

[znegrin]

@stefannica @Cahllagerfeld based on the code example, it would look similar to this:

So I guess pretty similar to what we already had in this version, just a small change to have a similar toggle to switch between the form and the code, like the one I designed for the snapshots.

I also added a "Re-run" button that will trigger the run again with the same parameters, and a "Delete" button for the output, with the idea of using it to remove this run in case the output is not useful. If you think it doesn't make sense to have it here, I can remove it.

Am I missing anything here? There are details on the JSON that I'm not sure if I should show there, as they are not part of the input/output.

[Cahllagerfeld]

Okay Im having multiple questions @znegrin , maybe we can even have a quick call, also including @stefannica:

• I think in the json-view of this particular thing, there is only the item called "city", as this is the payload that needs to be sent do the endpoint
• In the result, I understand that Im getting back the Weather-Analysis from the endpoint. Do I also get some reference to the produced run-entity that was created after I invoked the endpoint?
• I’m assuming the parameters could, in theory, be infinitely complex. In this example, it’s just a simple string like "city", but technically it could be anything. The same goes for the output — right now it’s the straightforward “happy path,” where the output is just a string that can be rendered somehow, but I assume it could also become much more complex.
• How does an error state look like, if there is no response or so?
• What happens if the deployment is not running, and someone tries to use the playground?
• How does the view look like, if no run was triggered yet?

[znegrin]

• I think in the json-view of this particular thing, there is only the item called "city", as this is the payload that needs to be sent do the endpoint
• In the result, I understand that Im getting back the Weather-Analysis from the endpoint. Do I also get some reference to the produced run-entity that was created after I invoked the endpoint?

My idea is to show the run card as you see in the design.

• I’m assuming the parameters could, in theory, be infinitely complex. In this example, it’s just a simple string like "city", but technically it could be anything. The same goes for the output — right now it’s the straightforward “happy path,” where the output is just a string that can be rendered somehow, but I assume it could also become much more complex.

Maybe we can get a more complex example to see how it could be. I have no idea how long and complex these inputs can be. I checked a lot of different model inputs in apps like Replicate or Hugging Face, and these are usually not that complex, but I don't know how it applies here. That's why I asked for an example. Maybe @stefannica has an example that has a lot more of inputs/outputs.

• How does an error state look like, if there is no response or so?

As this is a run, I guess it will show the run card with the error state, but I can design something additional.

• What happens if the deployment is not running, and someone tries to use the playground?

I guess if the deployment is not running, the playground shouldn't be accessible.

• How does the view look like, if no run was triggered yet?

I need to design this one. I wanted to know first if this approach made sense before adding all the different cases.

[Cahllagerfeld]

Key Notes:

• The Parameters from the schema an json inputs are embedded in the request that is send to the /invoke endpoint of the deployment
• The response contains the embeded output, as well as metadata, like runId, runname etc...
• There should be a copiable version of the outputs, but also a cleaner "preview" viz
• It will always return 200, but with success as a flag
• Each Output is technically an artifact version

[znegrin]

@Cahllagerfeld @stefannica After our discussion, I've been trying to show a more complex output dict including a card linking to the artifact version. I added an example of a very long string at the beginning. Apart from identifying when a string is very long to show it with the key on top and the value taking the full width, I thought maybe we can limit the height of the text to something around 120px with a text button to expand/collapse the text.

I also tried to make everything more compact by reducing the paddings between elements, making the collapsible headers also more compact, and removing additional boxes inside the dict. For the dropdowns, I think we can show all expanded by default and have this button to collapse all, so users can focus on the relevant stuff and expand it on request