First Asserts tool #105
First Asserts tool #105
Conversation
|
Not sure how to do the cloud test against |
tools/asserts.go
Outdated
| StartTime int64 `json:"startTime" jsonschema:"description=The start time of the assertion status in unix milliseconds format"` | ||
| EndTime int64 `json:"endTime" jsonschema:"description=The end time of the assertion status in unix milliseconds format"` |
There was a problem hiding this comment.
FWIW elsewhere in the server we use time.Time and mention that the timestamp should be in RFC3339 in the description, which tends to work quite well (it's easier for a model to get an RFC3339 timestamp correct than a unix timestamp, I guess)
There was a problem hiding this comment.
Oh. If that's the case, I can revert back to use RFC3339 and do the *1000 when calling Asserts API.
|
There was a problem hiding this comment.
Could you un-export all of the structs (other than GetAssertionsParams and GetAssertions) so we keep the tools package clean when imported? 🙏
tools/asserts.go
Outdated
|
|
||
| type GetAssertionsParams struct { | ||
| StartTime int64 `json:"startTime" jsonschema:"description=The start time of the assertion status in RFC3339 format"` | ||
| EndTime int64 `json:"endTime" jsonschema:"description=The end time of the assertion status in RFC3339 format"` |
There was a problem hiding this comment.
Hmm I don't think the jsonschema description matches the type any more; should the type be time.Time instead so the model can provide an RFC3339 string? (you can convert it to millis using t.UnixMilli)
There was a problem hiding this comment.
Eventually the API expects Millis. Which way do you think will work better? Ask the LLM to come up with Millis or use RFC3399 format and convert to Millis in our code?
tools/asserts.go
Outdated
| EntityType string `json:"entityType" jsonschema:"description=The type of the entity to list"` | ||
| EntityName string `json:"entityName" jsonschema:"description=The name of the entity to list"` | ||
| Env string `json:"env" jsonschema:"description=The env of the entity to list"` | ||
| Site string `json:"site" jsonschema:"description=The site of the entity to list"` | ||
| Namespace string `json:"namespace" jsonschema:"description=The namespace of the entity to list"` |
There was a problem hiding this comment.
for each of these string types, is the model expected to know the possible options in advance? If it's a closed set (e.g. entity type) we should provide a list of options in the description really; for the others, perhaps we should tell the LLM how to obtain valid values either in the field description or tool description.
There was a problem hiding this comment.
We can supply a list of know types. This will help LLM.
Adding the first Asserts tool for an API call. It's not quite important what this tool does. It will probably go through more iterations in the future and we will add more tools. This is just a simple PR as the first step.
Tested with Claude Desktop: