Unable to use @tensorflow/tfjs-tflite with a bundler

[Michael-F-Bryan]

System information

• OS Platform and Distribution: Arch Linux
• TensorFlow.js installed from (npm or script link): @tensorflow/tfjs-tflite version 0.0.1-alpha.7

Describe the problem

The @tensorflow/tfjs-tflite package can't be imported by a project that uses a bundler unless the end user manually copies the dist/ folder out of node_modules/ and serves its contents themselves.

This causes usability issues where developers will get non-obvious errors (404 response, "Unexpected token <", etc.) when trying to do inference, and can be particularly difficult to troubleshoot when @tensorflow/tfjs-tflite is a transitive dependency and the end user may not know anything about tflite.

Provide the exact sequence of commands / steps that you executed before running into the problem

First, create a new project and add @tensorflow/tfjs-tflite as a dependency. For convenience, I used create-react-app because it comes with webpack already configured.

$ cd /tmp
$ yarn create react-app repro
$ cd $_
$ yarn add @tensorflow/tfjs-tflite @tensorflow/tfjs-core

Next, update a file to use something from @tensorflow/tfjs-tflite. It doesn't really matter what gets used as long as you make sure webpack bundles it.

diff --git a/src/index.js b/src/index.js
index ef2edf8..7730a34 100644
--- a/src/index.js
+++ b/src/index.js
@@ -3,6 +3,9 @@ import ReactDOM from 'react-dom';
 import './index.css';
 import App from './App';
 import reportWebVitals from './reportWebVitals';
+import { loadTFLiteModel } from '@tensorflow/tfjs-tflite';
+
+loadTFLiteModel("https://raw.githubusercontent.com/hotg-ai/test-runes/master/audio/multispeech/model.tflite");

 ReactDOM.render(
   <React.StrictMode>

Opening the dev tools will show you that a bunch of exceptions coming from tflite_web_api_client.js because it seems to load files at runtime instead of importing them.

tflite_web_api_cc_simd.js:1 Uncaught SyntaxError: Unexpected token '<'
tflite_web_api_client.js:1973 Uncaught (in promise) TypeError: A[(d.tfliteWebApiName + "_ModuleFactory")] is not a function
    at push../node_modules/@tensorflow/tfjs-tflite/dist/tflite_web_api_client.js.$jscomp.generator.Engine_.program_ (tflite_web_api_client.js:1973)
    at push../node_modules/@tensorflow/tfjs-tflite/dist/tflite_web_api_client.js.$jscomp.generator.Engine_.nextStep_ (tflite_web_api_client.js:31)
    at push../node_modules/@tensorflow/tfjs-tflite/dist/tflite_web_api_client.js.$jscomp.generator.Engine_.next_ (tflite_web_api_client.js:27)
    at push../node_modules/@tensorflow/tfjs-tflite/dist/tflite_web_api_client.js.$jscomp.generator.Generator_.next (tflite_web_api_client.js:32)
    at g (tflite_web_api_client.js:1567)

I've found the best workaround is to use the public/ directory to make sure webpack copies the contents of @tensorflow/tfjs-tflite/dist/ into the output directory when it builds your app.

$ mkdir -p public/static && cd $_
$ ln -s ../../node_modules/@tensorflow/tfjs-tflite/dist ./js
$ cd ../..
$ yarn build

Any other info / logs

This is similar to #5532 in that @tensorflow/tfjs-tflite is hard to use with normal JavaScript packages, but that issue is about the code using browser-only APIs while this is about the NPM package being unusable without copying files out of your node_modules/ folder.

[jinjingforever]

Please see my comments in #6026. Thanks!

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No

[Michael-F-Bryan]

@jinjingforever thanks for your comment.

I understand why things work the way they do, but tflite.setWasmPath() isn't a very satisfying solution because it means every downstream user (including when @tensorflow/tfjs-tflite is a dependency of a dependency of a dependency) will need to set up their build system to copy node_modules/@tensorflow/tfjs-tflite/dist/tflite_web* into their app's static/ folder.

That means the package isn't usable with a bundler out of the box and adds a lot of friction, which is the issue this ticket is getting at.

Most bundlers already support a mechanism for code splitting and dynamic imports (e.g. see Webpack, Parcel), which would solve this issue while also simplifying @tensorflow/tfjs-tflite's code for selecting backends.

Can we reopen this issue for now?

[jinjingforever]

Sure! I agree that a better solution is needed. The main thing is that it is tricky to make the wasm binaries (.wasm files) work well with bundlers (e.g. can bundlers bundle wasm files)? From our experience with the tfjs wasm backend (where we also use setWasmPath), some extra steps are needed if you use npm.

We will investigate more on this.

[jackson-sandland]

Was this ever resolved? I've got a react app that I'd like to bundle tensorflowJS assets but webpack is unable to handle the tf file types.

[miguel-lorenzo]

any chance of solving this?