-
Notifications
You must be signed in to change notification settings - Fork 3.9k
Add Metabase AI Tools #17673
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Add Metabase AI Tools #17673
Conversation
Thank you for the update! 🎉 Due to our current reduced availability, the initial review may take up to 10-15 business days |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
This PR adds AI tools to the Metabase extension, enabling natural language querying of databases and saved questions through Raycast's AI interface.
- The
tools
section inpackage.json
needsevals
in theai
section - please add more test cases following the evals documentation - API calls in
api.ts
should handle failed responses with proper error handling before JSON parsing launchCommand
inrun-query.ts
andrun-question.ts
should be wrapped in try/catch blocks- Consider using
showFailureToast
from@raycast/utils
instead ofshowToast
for error handling in the API calls - Since there are multiple commands, consider adding "Metabase" as subtitle to each command for better context
💡 (1/5) You can manually trigger the bot by mentioning @greptileai in a comment!
8 file(s) reviewed, 11 comment(s)
Edit PR Review Bot Settings | Greptile
export async function runQuery(input: { query: string; databaseId: number }) { | ||
const endpoint = new URL(`/api/dataset`, preferences.instanceUrl); | ||
|
||
return fetch(endpoint.toString(), { | ||
method: "POST", | ||
headers: { | ||
accept: "application/json", | ||
"content-type": "application/json", | ||
"x-api-key": preferences.apiToken, | ||
}, | ||
body: JSON.stringify({ | ||
database: input.databaseId, | ||
type: "native", | ||
native: { | ||
query: input.query, | ||
}, | ||
}), | ||
}) | ||
.then((res) => res.json() as Promise<{ data: Query }>) | ||
.then((res) => res.data); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logic: No error handling for failed requests. Should check res.ok before parsing JSON and handle network errors in the catch block. Consider using showFailureToast from @raycast/utils for error handling.
export async function runQuery(input: { query: string; databaseId: number }) { | |
const endpoint = new URL(`/api/dataset`, preferences.instanceUrl); | |
return fetch(endpoint.toString(), { | |
method: "POST", | |
headers: { | |
accept: "application/json", | |
"content-type": "application/json", | |
"x-api-key": preferences.apiToken, | |
}, | |
body: JSON.stringify({ | |
database: input.databaseId, | |
type: "native", | |
native: { | |
query: input.query, | |
}, | |
}), | |
}) | |
.then((res) => res.json() as Promise<{ data: Query }>) | |
.then((res) => res.data); | |
} | |
export async function runQuery(input: { query: string; databaseId: number }) { | |
const endpoint = new URL(`/api/dataset`, preferences.instanceUrl); | |
try { | |
const response = await fetch(endpoint.toString(), { | |
method: "POST", | |
headers: { | |
accept: "application/json", | |
"content-type": "application/json", | |
"x-api-key": preferences.apiToken, | |
}, | |
body: JSON.stringify({ | |
database: input.databaseId, | |
type: "native", | |
native: { | |
query: input.query, | |
}, | |
}), | |
}); | |
if (!response.ok) { | |
throw new Error(`HTTP error! status: ${response.status}`); | |
} | |
const json = await response.json() as { data: Query }; | |
return json.data; | |
} catch (error) { | |
showFailureToast(error instanceof Error ? error.message : "Failed to run query"); | |
throw error; | |
} | |
} |
accept: "application/json", | ||
contentType: "application/json", | ||
"x-api-key": preferences.apiToken, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
syntax: Header name should be 'content-type' (hyphenated) instead of 'contentType' to match other requests and HTTP standards
accept: "application/json", | |
contentType: "application/json", | |
"x-api-key": preferences.apiToken, | |
accept: "application/json", | |
"content-type": "application/json", | |
"x-api-key": preferences.apiToken, |
"evals": [ | ||
{ | ||
"input": "@metabase How many users do I have?", | ||
"expected": [] | ||
}, | ||
{ | ||
"input": "@metabase What databases do I have connected?", | ||
"expected": [] | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logic: Evals are empty arrays. Need at least one eval with expected output to validate AI behavior. See https://developers.raycast.com/ai/write-evals-for-your-ai-extension
export default async function () { | ||
const questions = await getQuestions(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logic: getQuestions() call should be wrapped in try/catch to handle API errors gracefully
export default async function () { | |
const questions = await getQuestions(); | |
export default async function () { | |
try { | |
const questions = await getQuestions(); |
@@ -0,0 +1,23 @@ | |||
import { getQuestions } from "../lib/api"; | |||
|
|||
export default async function () { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
style: Function lacks TypeScript return type annotation
try { | ||
const ddl = await runQuery({ | ||
query: ddlQueries[database.engine], | ||
databaseId: database.id, | ||
}); | ||
|
||
schema.tables = JSON.parse(ddl.rows[0][0]); | ||
if (ddl.rows[0][1]) { | ||
schema.enums = JSON.parse(ddl.rows[0][1]); | ||
} | ||
} catch { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
style: runQuery should be wrapped in try/catch with showFailureToast from @raycast/utils for better error handling
try { | |
const ddl = await runQuery({ | |
query: ddlQueries[database.engine], | |
databaseId: database.id, | |
}); | |
schema.tables = JSON.parse(ddl.rows[0][0]); | |
if (ddl.rows[0][1]) { | |
schema.enums = JSON.parse(ddl.rows[0][1]); | |
} | |
} catch { | |
try { | |
const ddl = await runQuery({ | |
query: ddlQueries[database.engine], | |
databaseId: database.id, | |
}); | |
schema.tables = JSON.parse(ddl.rows[0][0]); | |
if (ddl.rows[0][1]) { | |
schema.enums = JSON.parse(ddl.rows[0][1]); | |
} | |
} catch (error) { | |
showFailureToast("Failed to fetch database schema", error); |
export default async function (input: Input) { | ||
const result = await runQuery(input); | ||
|
||
return { result }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logic: runQuery should be wrapped in try/catch to handle potential API errors gracefully
export default async function (input: Input) { | |
const result = await runQuery(input); | |
return { result }; | |
export default async function (input: Input) { | |
try { | |
const result = await runQuery(input); | |
return { result }; | |
} catch (error) { | |
throw new Error(`Failed to run query: ${error.message}`); | |
} |
export const confirmation: Tool.Confirmation<Input> = async (input) => { | ||
const database = await getDatabase(input.databaseId); | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logic: getDatabase call should be wrapped in try/catch to handle API errors
export const confirmation: Tool.Confirmation<Input> = async (input) => { | |
const database = await getDatabase(input.databaseId); | |
export const confirmation: Tool.Confirmation<Input> = async (input) => { | |
try { | |
const database = await getDatabase(input.databaseId); |
export default async function (input: Input) { | ||
const result = await runQuestion(input); | ||
|
||
return { result }; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logic: runQuestion should be wrapped in try/catch to handle potential API errors gracefully
export default async function (input: Input) { | |
const result = await runQuestion(input); | |
return { result }; | |
} | |
export default async function (input: Input) { | |
try { | |
const result = await runQuestion(input); | |
return { result }; | |
} catch (error) { | |
throw new Error(`Failed to run question: ${error.message}`); | |
} | |
} |
value: question.name, | ||
}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logic: question.name could be undefined - needs null check like other fields
I'm not sure if I should do the AI changes pointed out @pernielsentikaer |
Description
As can be seen in the screencast, the first question I ask uses an existing query stored in Metabase. The second one is not stored, and the AI is able to figure out the PostgreSQL query.
I plan for future contributions to add more DDL queries besides PostgreSQL to improve accuracy.
Screencast
https://share.cleanshot.com/X7QXDL1Z
Checklist
npm run build
and tested this distribution build in Raycastassets
folder are used by the extension itselfREADME
are located outside the metadata folder if they were not generated with our metadata tool