Skip to content

[FDC] Generate schema during init dataconnect flow #8596

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 46 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
d2f9774
Generate schema in fdc init
fredzqm May 15, 2025
f8cdc0f
tested
fredzqm May 15, 2025
ae7e61e
Merge remote-tracking branch 'origin/master' into fz/gen-schema-init
fredzqm May 15, 2025
9239426
Merge branch 'master' into fz/gen-schema-init
fredzqm May 15, 2025
6208153
Merge branch 'master' into fz/gen-schema-init
fredzqm May 15, 2025
091706d
Merge branch 'master' into fz/gen-schema-init
fredzqm May 15, 2025
909ef53
Merge remote-tracking branch 'origin/master' into fz/gen-schema-init
fredzqm May 15, 2025
c3c08c9
tos
fredzqm May 15, 2025
e9eec2a
tweaks
fredzqm May 15, 2025
68ea00f
m
fredzqm May 15, 2025
ac584ce
m
fredzqm May 15, 2025
7290205
vaib code a test
fredzqm May 15, 2025
1f84dc3
CHANGELOG
fredzqm May 15, 2025
bc2647e
m
fredzqm May 15, 2025
1beed8d
Merge branch 'master' into fz/gen-schema-init
fredzqm May 15, 2025
74d8710
Merge branch 'master' into fz/gen-schema-init
fredzqm May 15, 2025
ae6868d
m
fredzqm May 15, 2025
a59900f
Merge remote-tracking branch 'origin/master' into fz/gen-schema-init
fredzqm May 15, 2025
adfa8b8
Update src/init/features/dataconnect/index.ts
fredzqm May 15, 2025
1923130
Update src/init/features/dataconnect/index.ts
fredzqm May 15, 2025
d41bd43
format
fredzqm May 15, 2025
220a79d
m
fredzqm May 15, 2025
9bead7f
clean
fredzqm May 15, 2025
ee4c8c0
Merge branch 'master' into fz/gen-schema-init
fredzqm May 15, 2025
345d54a
Update ensureApis.ts
fredzqm May 15, 2025
5a82f27
Merge branch 'master' into fz/gen-schema-init
fredzqm May 15, 2025
eefd4ce
Merge branch 'master' into fz/gen-schema-init
fredzqm May 16, 2025
8f452f5
Merge branch 'master' into fz/gen-schema-init
fredzqm May 16, 2025
46967ed
Show GiF ToS in firebase login
fredzqm May 22, 2025
d5e0621
Merge branch 'master' into fz/show-gif-tos
fredzqm May 22, 2025
2fa36c0
tweaks
fredzqm May 22, 2025
d1d1100
Merge remote-tracking branch 'origin/master' into fz/show-gif-tos
fredzqm May 22, 2025
b0f3bc5
Merge remote-tracking branch 'origin/master' into fz/show-gif-tos
fredzqm May 22, 2025
1ef575f
Merge branch 'master' into fz/show-gif-tos
fredzqm May 23, 2025
3db2767
Merge remote-tracking branch 'origin/master' into fz/show-gif-tos
fredzqm May 23, 2025
dd092dd
tweaks
fredzqm May 24, 2025
a1f20db
Merge remote-tracking branch 'origin/fz/show-gif-tos' into fz/show-gi…
fredzqm May 24, 2025
6b7a8da
Merge branch 'master' into fz/show-gif-tos
fredzqm May 27, 2025
b19986d
Merge remote-tracking branch 'origin/master' into fz/show-gif-tos
fredzqm May 29, 2025
129c382
Merge branch 'master' into fz/show-gif-tos
fredzqm May 29, 2025
f884056
Merge remote-tracking branch 'origin/fz/show-gif-tos' into fz/show-gi…
fredzqm May 29, 2025
55d87d0
Merge branch 'fz/show-gif-tos' into fz/gen-schema-init
fredzqm May 29, 2025
2e09e7e
merge
fredzqm Jun 6, 2025
cd14029
m
fredzqm Jun 6, 2025
04b1c42
merge
fredzqm Jun 6, 2025
2403de2
adjust question
fredzqm Jun 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
- Fixed multi tenancy support for SSO users in the auth emulator (#8544)
- Add SDK autoinit capabilities to App Hosting emulator. (#8582)
- Provision App Hosting compute service account during init flow. (#8580)
- `firebase init dataconnect` is now integrated with Gemini in Firebase API to generate Schema based on description. (#8596)
- Updated the Firebase Data Connect local toolkit to v2.6.1, which includes the following changes: (#8598)
- Fixed a bug where `@transaction` continued to execute after errors and produces more confusing errors.
- Fixed a bug where rolled-back fields in `@transaction` returned bogus data that had been invalidated by the rollback. They now correctly return null with a corresponding error "(rolled back)".
Expand Down
9 changes: 7 additions & 2 deletions src/dataconnect/ensureApis.ts
Original file line number Diff line number Diff line change
@@ -1,15 +1,20 @@
import * as api from "../api";
import { ensure } from "../ensureApiEnabled";

const prefix = "dataconnect";

export async function ensureApis(projectId: string): Promise<void> {

Check warning on line 6 in src/dataconnect/ensureApis.ts

View workflow job for this annotation

GitHub Actions / lint (20)

Missing JSDoc comment
const prefix = "dataconnect";
await ensure(projectId, api.dataconnectOrigin(), prefix);
await ensure(projectId, api.cloudSQLAdminOrigin(), prefix);
await ensure(projectId, api.computeOrigin(), prefix);
}

export async function ensureSparkApis(projectId: string): Promise<void> {

Check warning on line 12 in src/dataconnect/ensureApis.ts

View workflow job for this annotation

GitHub Actions / lint (20)

Missing JSDoc comment
const prefix = "dataconnect";
// These are the APIs that can be enabled without a billing account.
await ensure(projectId, api.cloudSQLAdminOrigin(), prefix);
await ensure(projectId, api.cloudAiCompanionOrigin(), prefix);
}

export async function ensureGIFApis(projectId: string): Promise<void> {

Check warning on line 18 in src/dataconnect/ensureApis.ts

View workflow job for this annotation

GitHub Actions / lint (20)

Missing JSDoc comment
await ensure(projectId, api.cloudAiCompanionOrigin(), prefix);
}
67 changes: 67 additions & 0 deletions src/gif/fdcExperience.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
import { expect } from "chai";
import { extractCodeBlock } from "./fdcExperience.js"; // Adjust the path to your file

describe("extractCodeBlock", () => {
it("should extract a basic GraphQL query block", () => {
const text =
'Here is a GraphQL query:\n```graphql\nquery GetUser { user(id: "1") { name email } }\n```\nThanks!';
const expected = 'query GetUser { user(id: "1") { name email } }';
expect(extractCodeBlock(text)).to.eq(expected);
});

it("should extract a multi-line GraphQL mutation block", () => {
const text = `
Some preamble.
\`\`\`graphql
mutation CreatePost($title: String!, $content: String!) {
createPost(title: $title, content: $content) {
id
title
}
}
\`\`\`
Followed by some description.
`;
const expected = `mutation CreatePost($title: String!, $content: String!) {
createPost(title: $title, content: $content) {
id
title
}
}`;
expect(extractCodeBlock(text)).to.eq(expected);
});

it("should extract a GraphQL fragment block", () => {
const text = "```graphql\nfragment UserFields on User { id name }\n```";
const expected = "fragment UserFields on User { id name }";
expect(extractCodeBlock(text)).to.eq(expected);
});

it("should extract an empty GraphQL code block", () => {
const text = "```graphql\n\n```";
const expected = "";
expect(extractCodeBlock(text)).to.eq(expected);
});

it("should extract a GraphQL schema definition block", () => {
const text = `
\`\`\`graphql
type Query {
hello: String
}

schema {
query: Query
}
\`\`\`
`;
const expected = `type Query {
hello: String
}

schema {
query: Query
}`;
expect(extractCodeBlock(text)).to.eq(expected);
});
});
15 changes: 15 additions & 0 deletions src/gif/fdcExperience.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { Client } from "../apiv2";
import { cloudCompanionOrigin } from "../api";
import { FirebaseError } from "../error";

const apiClient = new Client({ urlPrefix: cloudCompanionOrigin(), auth: true });
const schemaGeneratorExperience = "/appeco/firebase/fdc-schema-generator";
Expand Down Expand Up @@ -128,3 +129,17 @@
);
return res.body.output.messages[0].content;
}

/**
* extractCodeBlock extracts the code block from the generated response.
* @param text the generated response from the service.
* @return the code block from the generated response.
*/
export function extractCodeBlock(text: string): string {
const regex = /```(?:[a-z]+\n)?([\s\S]*?)```/m;
const match = text.match(regex);

Check warning on line 140 in src/gif/fdcExperience.ts

View workflow job for this annotation

GitHub Actions / lint (20)

Use the `RegExp#exec()` method instead
if (match && match[1]) {
return match[1].trim();
}
throw new FirebaseError(`No code block found in the generated response: ${text}`);
}
31 changes: 26 additions & 5 deletions src/init/features/dataconnect/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
import { provisionCloudSql } from "../../../dataconnect/provisionCloudSql";
import { checkFreeTrialInstanceUsed, upgradeInstructions } from "../../../dataconnect/freeTrial";
import * as cloudsql from "../../../gcp/cloudsql/cloudsqladmin";
import { ensureApis, ensureSparkApis } from "../../../dataconnect/ensureApis";
import { ensureApis, ensureGIFApis, ensureSparkApis } from "../../../dataconnect/ensureApis";
import {
listLocations,
listAllServices,
Expand All @@ -19,10 +19,11 @@
import { parseCloudSQLInstanceName, parseServiceName } from "../../../dataconnect/names";
import { logger } from "../../../logger";
import { readTemplateSync } from "../../../templates";
import { logBullet, envOverride } from "../../../utils";
import { logBullet, envOverride, promiseWithSpinner } from "../../../utils";
import { isBillingEnabled } from "../../../gcp/cloudbilling";
import * as sdk from "./sdk";
import { getPlatformFromFolder } from "../../../dataconnect/fileUtils";
import { extractCodeBlock, generateSchema } from "../../../gif/fdcExperience";

const DATACONNECT_YAML_TEMPLATE = readTemplateSync("init/dataconnect/dataconnect.yaml");
const CONNECTOR_YAML_TEMPLATE = readTemplateSync("init/dataconnect/connector.yaml");
Expand All @@ -33,7 +34,7 @@
// serviceEnvVar is used by Firebase Console to specify which service to import.
// It should be in the form <location>/<serviceId>
// It must be an existing service - if set to anything else, we'll ignore it.
const serviceEnvVar = () => envOverride("FDC_SERVICE", "");

Check warning on line 37 in src/init/features/dataconnect/index.ts

View workflow job for this annotation

GitHub Actions / lint (20)

Missing return type on function

export interface RequiredInfo {
serviceId: string;
Expand Down Expand Up @@ -76,7 +77,7 @@

// askQuestions prompts the user about the Data Connect service they want to init. Any prompting
// logic should live here, and _no_ actuation logic should live here.
export async function askQuestions(setup: Setup): Promise<void> {

Check warning on line 80 in src/init/features/dataconnect/index.ts

View workflow job for this annotation

GitHub Actions / lint (20)

Missing JSDoc comment
const hasBilling = await isBillingEnabled(setup);
if (setup.projectId) {
hasBilling ? await ensureApis(setup.projectId) : await ensureSparkApis(setup.projectId);
Expand Down Expand Up @@ -110,8 +111,7 @@
default: true,
}));
if (shouldConfigureBackend) {
// TODO: Prompt for app idea and use GiF backend to generate them.
info = await promptForService(info);
info = await promptForSchema(setup, info);
info = await promptForCloudSQL(setup, info);

info.shouldProvisionCSQL = !!(
Expand All @@ -130,10 +130,10 @@

// actuate writes product specific files and makes product specifc API calls.
// It does not handle writing firebase.json and .firebaserc
export async function actuate(setup: Setup, config: Config, options: any): Promise<void> {

Check warning on line 133 in src/init/features/dataconnect/index.ts

View workflow job for this annotation

GitHub Actions / lint (20)

Missing JSDoc comment

Check warning on line 133 in src/init/features/dataconnect/index.ts

View workflow job for this annotation

GitHub Actions / lint (20)

Unexpected any. Specify a different type
// Most users will want to persist data between emulator runs, so set this to a reasonable default.
const dir: string = config.get("dataconnect.source", "dataconnect");

Check warning on line 135 in src/init/features/dataconnect/index.ts

View workflow job for this annotation

GitHub Actions / lint (20)

Unsafe assignment of an `any` value
const dataDir = config.get("emulators.dataconnect.dataDir", `${dir}/.dataconnect/pgliteData`);

Check warning on line 136 in src/init/features/dataconnect/index.ts

View workflow job for this annotation

GitHub Actions / lint (20)

Unsafe assignment of an `any` value
config.set("emulators.dataconnect.dataDir", dataDir);

const info = setup.featureInfo?.dataconnect;
Expand Down Expand Up @@ -425,12 +425,33 @@
return info;
}

async function promptForService(info: RequiredInfo): Promise<RequiredInfo> {
async function promptForSchema(setup: Setup, info: RequiredInfo): Promise<RequiredInfo> {
if (info.serviceId === "") {
info.serviceId = await input({
message: "What ID would you like to use for this service?",
default: basename(process.cwd()),
});
logBullet(
`Check out the terms of service of Gemini in Firebase https://firebase.google.com/docs/gemini-in-firebase/set-up-gemini#required-permissions`,
);
if (
await confirm({
message: `Do you want Gemini to help generate the schema?`,
default: true,
})
) {
await ensureGIFApis(setup.projectId!);
const prompt = await input({
message: "Describe the app you are building:",
default: "movie rating app",
});
const schema = await promiseWithSpinner(
() => generateSchema(prompt, setup.projectId!),
"Generating the Data Connect Schema...",
);
info.schemaGql = [{ path: "schema.gql", content: extractCodeBlock(schema) }];
info.connectors = [emptyConnector];
}
}
return info;
}
Expand Down
Loading