Still crashing using WebGPU with translation pipeline #1518
Description
System Info
@huggingface/transformers@4.0.0-next.1
Environment/Platform
- Website/web-app
- Browser extension
- Server-side (e.g., Node.js, Deno, Bun)
- Desktop app (e.g., Electron)
- Other (e.g., VSCode extension)
Description
WASM executor works, webgpu doesn't. Same as #1380 but has a different error message:
transformers.web.js:16471 dtype not specified for "encoder_model". Using the default dtype (fp32) for this device (webgpu).
transformers.web.js:16471 dtype not specified for "decoder_model_merged". Using the default dtype (fp32) for this device (webgpu).
ort.webgpu.bundle.min.mjs:10 Uncaught Error: Can't create a session. ERROR_CODE: 6, ERROR_MESSAGE: std::bad_alloc
at z (ort.webgpu.bundle.min.mjs:10:26538)
at er (ort.webgpu.bundle.min.mjs:11:23219)
at async yr.loadModel (ort.webgpu.bundle.min.mjs:11:36125)
at async br.createInferenceSessionHandler (ort.webgpu.bundle.min.mjs:11:37849)
at async a.create (ort.webgpu.bundle.min.mjs:6:19885)
Reproduction
import { pipeline } from '@huggingface/transformers';
async function translate(text: string) {
const translator = await pipeline(
'translation',
'Xenova/nllb-200-distilled-600M',
{
device: "webgpu",
},
);
return await translator(text, {
src_lang: 'heb_Hebr',
tgt_lang: 'eng_Latn',
});
}
console.log(await translate("בראשית ברא אלהים את השמים ואת הארץ׃"))