Skip to content
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .changeset/soft-meals-listen.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
"@gradio/client": patch
"@self/app": patch
"@self/spa": patch
"gradio": patch
---

fix:Fix ZeroGPU handling for `gr.Server`
3 changes: 3 additions & 0 deletions client/js/src/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ import {
parse_and_set_cookies
} from "./helpers/init_helpers";
import { check_and_wake_space, check_space_status } from "./helpers/spaces";
import { initialize_zerogpu_handshake } from "./helpers/zerogpu";
import { open_stream, readable_stream, close_stream } from "./utils/stream";
import {
API_INFO_ERROR_MSG,
Expand All @@ -41,6 +42,8 @@ import {
} from "./constants";
declare const BROWSER_BUILD: boolean;

initialize_zerogpu_handshake();
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this happening here?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where else could it go? I put it module init because it needs to finish before the first user request. Otherwise that first request can race the iframe auth handshake and not include the forwarded headers.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking it'd go in submit.ts or the client init. But let me know if I'm misunderstanding!

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should put it in submit.ts because either we'd have to block until we have the headers or it might not get the headers quickly enough. We could do it in Client init I think (happy to move it there), but this is the closest to what we had previously (when we had it happen as part of the page initialization itself)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok let's keep as is then! But it's also currently being called in submit.ts. Is it intentional?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that seems unnecessary, will remove

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't see this sorry.

I'd put this in Client init, that way user has some control over when it gets called. As it is, this would fire as soon as the module is imported and means that the entire module has side-effects which can cause issues when bundling.


export class Client {
app_reference: string;
options: ClientOptions;
Expand Down
40 changes: 40 additions & 0 deletions client/js/src/helpers/zerogpu.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
const ZEROGPU_HEADERS_MESSAGE = "supports-zerogpu-headers";

let zerogpu_handshake_initialized = false;

function supports_browser_handshake(): boolean {
return (
typeof window !== "undefined" &&
typeof document !== "undefined" &&
typeof window.addEventListener === "function"
);
}

export function get_zerogpu_origin(hostname: string): string | null {
if (hostname.includes(".dev.")) {
return `https://moon-${hostname.split(".")[1]}.dev.spaces.huggingface.tech`;
}
if (hostname.endsWith(".hf.space")) {
return "https://huggingface.co";
}
return null;
}

export function initialize_zerogpu_handshake(): void {
if (!supports_browser_handshake() || zerogpu_handshake_initialized) {
return;
}

window.addEventListener("message", (event) => {
if (event.data === ZEROGPU_HEADERS_MESSAGE) {
window.supports_zerogpu_headers = true;
}
});

zerogpu_handshake_initialized = true;

const origin = get_zerogpu_origin(window.location.hostname);
if (origin && window.parent !== window) {
window.parent.postMessage(ZEROGPU_HEADERS_MESSAGE, origin);
}
}
7 changes: 3 additions & 4 deletions client/js/src/utils/submit.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import type {
} from "../types";

import { skip_queue, post_message, handle_payload } from "../helpers/data";
import { get_zerogpu_origin } from "../helpers/zerogpu";
import {
handle_message,
map_data_to_params,
Expand Down Expand Up @@ -412,15 +413,13 @@ export function submit(
hostname = window?.location?.hostname;
}

let hfhubdev = "dev.spaces.huggingface.tech";
const origin = hostname.includes(".dev.")
? `https://moon-${hostname.split(".")[1]}.${hfhubdev}`
: `https://huggingface.co`;
const origin = get_zerogpu_origin(hostname);

const is_zerogpu_iframe =
typeof window !== "undefined" &&
typeof document !== "undefined" &&
window.parent != window &&
!!origin &&
window.supports_zerogpu_headers;
const zerogpu_auth_promise = is_zerogpu_iframe
? post_message<Map<string, string>>("zerogpu-headers", origin)
Expand Down
2 changes: 2 additions & 0 deletions guides/09_gradio-clients-and-lite/08_server-mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,8 @@ python run.py

Then open `http://localhost:7860` in your browser. The custom HTML page uses the `@gradio/client` JavaScript library to call the Gradio API endpoints. Meanwhile, the same endpoints are available as MCP tools and through the REST API at `/gradio_api/call/add` and `/gradio_api/call/multiply`.

Note: if your `Server` app uses ZeroGPU, you _must_ call Gradio API endpoints through `@gradio/client` from the browser. The JavaScript client forwards the Hugging Face iframe auth headers needed for ZeroGPU quota handling.

## Concurrency and Streaming

`app.api()` supports all of the same concurrency and streaming options as `gr.api()`:
Expand Down
16 changes: 0 additions & 16 deletions js/app/src/routes/[...catchall]/+page.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -293,22 +293,6 @@

await add_custom_html_head(config.head);

const supports_zerogpu_headers = "supports-zerogpu-headers";
window.addEventListener("message", (event) => {
if (event.data === supports_zerogpu_headers) {
window.supports_zerogpu_headers = true;
}
});
const hostname = window.location.hostname;
const is_hf_host =
hostname.includes(".dev.") || hostname.endsWith(".hf.space");
if (is_hf_host) {
const origin = hostname.includes(".dev.")
? `https://moon-${hostname.split(".")[1]}.dev.spaces.huggingface.tech`
: `https://huggingface.co`;
window.parent.postMessage(supports_zerogpu_headers, origin);
}

if (config.js) {
try {
const script = document.createElement("script");
Expand Down
16 changes: 0 additions & 16 deletions js/spa/src/Index.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -356,22 +356,6 @@
css_ready = true;
window.__is_colab__ = config.is_colab;

const supports_zerogpu_headers = "supports-zerogpu-headers";
window.addEventListener("message", (event) => {
if (event.data === supports_zerogpu_headers) {
window.supports_zerogpu_headers = true;
}
});
const hostname = window.location.hostname;
const is_hf_host =
hostname.includes(".dev.") || hostname.endsWith(".hf.space");
if (is_hf_host) {
const origin = hostname.includes(".dev.")
? `https://moon-${hostname.split(".")[1]}.dev.spaces.huggingface.tech`
: `https://huggingface.co`;
window.parent.postMessage(supports_zerogpu_headers, origin);
}

dispatch("loaded");

pages = config.pages;
Expand Down
Loading