Ai provider by sroussey · Pull Request #179 · workglow-dev/libs

sroussey · 2026-02-16T20:24:08Z

Note

Medium Risk
Touches core provider registration and queue startup behavior across inline/worker modes, so regressions could break task execution or worker wiring. Changes are mostly API refactors with added tests, but impact is broad across providers, examples, and docs.

Overview
Introduces a new AiProvider base class (exported from @workglow/ai) that centralizes provider registration for inline vs worker execution, registers provider instances in AiProviderRegistry, and auto-creates an in-memory job queue via a new createDefaultQueue() helper (with options to configure concurrency or disable auto-creation).

Refactors the HuggingFace Transformers and TensorFlow MediaPipe implementations to provider classes (HuggingFaceTransformersProvider, TensorFlowMediaPipeProvider) with injected task maps (HFT_TASKS, TFMP_TASKS) and new worker entrypoints (HFT_WORKER_JOBRUN_REGISTER, TFMP_WORKER_JOBRUN_REGISTER), removing the older register_*_*JobFns helper modules.

Updates all docs, examples (CLI/web), and tests to use new ...Provider().register(...)/worker registration APIs, including safer ONNX env proxy setup in workers and revised guidance around queue creation.

^{Written by Cursor Bugbot for commit e5d23bf. This will update automatically on new commits. Configure here.}

…e and TensorFlow MediaPipe providers - Replaced `register_HFT_InlineJobFns` and `register_HFT_ClientJobFns` with `HuggingFaceTransformersProvider` and `TensorFlowMediaPipeProvider` for improved modularity and clarity. - Updated documentation and examples to reflect the new provider registration methods. - Enhanced worker registration for HuggingFace and TensorFlow MediaPipe tasks, streamlining the integration of AI models. - Removed deprecated registration functions to clean up the codebase.

- Replaced the manual queue setup process in the createQueue method with a call to createDefaultQueue for improved readability and maintainability. - This change reduces complexity by leveraging a dedicated function for queue creation, streamlining the overall implementation.

Copilot

Pull request overview

This PR introduces a new AiProvider abstraction in @workglow/ai and migrates @workglow/ai-provider integrations (HuggingFace Transformers + TensorFlow MediaPipe) from standalone register_* helper functions to provider classes with injected task maps.

Changes:

Added AiProvider base class + lifecycle/registry support (AiProviderRegistry.registerProvider/getProvider/getProviders) and a shared default queue creator.
Reworked ai-provider exports to provider classes (HuggingFaceTransformersProvider, TensorFlowMediaPipeProvider) and exported task maps (HFT_TASKS, TFMP_TASKS), removing the old inline/client/worker register modules.
Updated tests, examples, and docs to use the new provider registration approach.

Reviewed changes

Copilot reviewed 37 out of 37 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
packages/ai/src/provider/AiProvider.ts	New provider base class with inline/worker registration + queue auto-creation
packages/ai/src/provider/AiProviderRegistry.ts	Adds provider instance registry (for lifecycle/introspection)
packages/ai/src/queue/createDefaultQueue.ts	New shared helper to create/register default in-memory queues
packages/ai/src/common.ts	Exports `AiProvider` from `@workglow/ai`
packages/ai-provider/src/hf-transformers/*	Adds `HuggingFaceTransformersProvider`, `HFT_TASKS`, and new worker registration entrypoint
packages/ai-provider/src/tf-mediapipe/*	Adds `TensorFlowMediaPipeProvider`, `TFMP_TASKS`, and new worker registration entrypoint
packages/test/src/test/*/	Updates test suites to use provider classes + task maps
examples/web/src/*	Updates worker/main-thread registration to provider-based worker mode
examples/cli/src/*	Updates inline/worker registration to provider-based API
packages/ai/README.md	Updates docs to provider-based registration (but has inline examples missing task injection)
packages/ai-provider/README.md	Updates docs to provider-based registration (but has inline examples missing task injection)
docs/developers/01_getting_started.md	Updates getting started docs (but has inline example missing task injection)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-16T20:29:35Z


 // Register AI providers
-await register_HFT_InlineJobFns();
-await register_TFMP_InlineJobFns();
+await new HuggingFaceTransformersProvider().register({ mode: "inline" });
+await new TensorFlowMediaPipeProvider().register({ mode: "inline" });


This README snippet registers providers in inline mode without passing the task maps (HFT_TASKS / TFMP_TASKS). With the new provider API, inline mode requires tasks injected via the constructor, so this example will throw at runtime if copied. Update the snippet to pass the task maps (or switch it to worker mode).

Copilot · 2026-02-16T20:29:35Z

+    if (options.mode === "worker" && options.worker) {
+      const workerManager = globalServiceRegistry.get(WORKER_MANAGER);
+      workerManager.registerWorker(this.name, options.worker);
+      for (const taskType of this.taskTypes) {
+        registry.registerAsWorkerRunFn(this.name, taskType);


In register(), when mode is set to "worker" but options.worker is undefined, this code falls through to the inline-registration branch and throws the "tasks must be provided" error, which is misleading. Add an explicit validation/throw for mode === "worker" && !worker before this conditional so callers get a clear error and you don’t accidentally attempt inline registration.

Copilot · 2026-02-16T20:29:36Z

 async function main() {
  // 1. Register the AI provider
-  await register_HFT_InlineJobFns();
+  await new HuggingFaceTransformersProvider().register({ mode: "inline" });



This example registers HuggingFaceTransformersProvider in inline mode without injecting HFT_TASKS. With the new AiProvider contract, inline mode requires tasks via the constructor, so the snippet will throw. Update the example to pass HFT_TASKS (or demonstrate worker mode).

Copilot · 2026-02-16T20:29:36Z

+// Register with custom queue concurrency (provider creates queue with concurrency: 2)
+await new HuggingFaceTransformersProvider().register({
+  mode: "inline",
+  queue: { concurrency: 2 },
 });


In the "Custom Job Queue Configuration" section, this snippet calls new HuggingFaceTransformersProvider().register({ mode: "inline", ... }) without injecting HFT_TASKS. Inline mode will throw unless the tasks record is provided via the constructor. Update the snippet to pass HFT_TASKS (or make it a worker-mode example).

Copilot · 2026-02-16T20:29:36Z

+import { HF_TRANSFORMERS_ONNX, HuggingFaceTransformersProvider } from "@workglow/ai-provider";

 // Provider run functions on this thread
-await register_HFT_InlineJobFns();
+await new HuggingFaceTransformersProvider().register({ mode: "inline" });


This getting-started snippet registers HuggingFaceTransformersProvider in inline mode without passing the task map (HFT_TASKS). With the new provider API, inline mode requires tasks injected via the constructor. Update the snippet to pass HFT_TASKS (or change the doc to demonstrate worker-mode registration).

Copilot · 2026-02-16T20:29:36Z

+    const registry = getAiProviderRegistry();
+    registry.registerProvider(this);
+


registry.registerProvider(this) happens before validating that registration will succeed (e.g., tasks present for inline mode, worker present for worker mode). If registration throws afterward, the registry can retain a partially-registered provider instance. Consider deferring registerProvider() until after validation/registration completes (or rolling back on error).

Copilot · 2026-02-16T20:29:36Z

+  const server = new JobQueueServer(AiJob as any, {
+    storage,
+    queueName: providerName,
+    limiter: new ConcurrencyLimiter(concurrency, 100),
+  });


createDefaultQueue uses AiJob as any when constructing JobQueueServer, which drops type safety and can hide wiring errors. Consider typing the queue with AiJobInput<TaskInput> / TaskOutput and passing AiJob without as any (or using an explicit JobClass<...> cast) so this stays compile-time checked.

Copilot · 2026-02-16T20:29:37Z

+// 3. Register provider (inline mode, creates queue automatically)
+await new HuggingFaceTransformersProvider().register({ mode: "inline" });



This README example registers the provider in inline mode without passing the injected task map (e.g. HFT_TASKS). With the new AiProvider base class, inline mode requires tasks via the constructor and will throw at runtime. Update the snippet to either pass HFT_TASKS for inline mode or switch the example to mode: "worker".

Copilot · 2026-02-16T20:29:37Z

+import { HuggingFaceTransformersProvider } from "@workglow/ai-provider";

 // Registers run functions for all supported AI tasks on the current thread
-await register_HFT_InlineJobFns();
+await new HuggingFaceTransformersProvider().register({ mode: "inline" });
 ```


This later README snippet also uses new HuggingFaceTransformersProvider().register({ mode: "inline" }) without passing HFT_TASKS. Since inline mode now requires tasks via the constructor, the example will fail if copied. Adjust the snippet to provide HFT_TASKS (or demonstrate worker mode).

sroussey added 2 commits February 16, 2026 01:22

sroussey requested a review from Copilot February 16, 2026 20:24

sroussey self-assigned this Feb 16, 2026

Copilot started reviewing on behalf of sroussey February 16, 2026 20:24 View session

Copilot AI reviewed Feb 16, 2026

View reviewed changes

sroussey merged commit b0ee35f into main Feb 16, 2026
9 checks passed

sroussey deleted the ai-provider branch March 3, 2026 23:49

Navigation Menu

Uh oh!

Ai provider - #179

Ai provider#179
sroussey merged 2 commits into
mainfrom
ai-provider

sroussey commented Feb 16, 2026 •

edited by cursor Bot

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 16, 2026

Uh oh!

Copilot AI Feb 16, 2026

Uh oh!

Copilot AI Feb 16, 2026

Uh oh!

Copilot AI Feb 16, 2026

Uh oh!

Copilot AI Feb 16, 2026

Uh oh!

Copilot AI Feb 16, 2026

Uh oh!

Copilot AI Feb 16, 2026

Uh oh!

Copilot AI Feb 16, 2026

Uh oh!

Copilot AI Feb 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		const registry = getAiProviderRegistry();
		registry.registerProvider(this);

		// 3. Register provider (inline mode, creates queue automatically)
		await new HuggingFaceTransformersProvider().register({ mode: "inline" });

Uh oh!

Conversation

sroussey commented Feb 16, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sroussey commented Feb 16, 2026 •

edited by cursor Bot

Loading