feat(#3306): add boost-backend-module-llamastack provider module (#3572)

* feat(#3306): add boost-backend-module-llamastack provider module

Create the Llama Stack provider module as an independent createBackendModule
with ResponsesApiProvider, OpenAI Agent SDK orchestration, and all provider
caches backed by Backstage cacheService.

Implements:
- Task 2.1: Package scaffold with pluginId 'boost', moduleId 'llamastack'
- Task 2.2: ResponsesApiProvider and ResponsesApiProviderFactory
- Task 3.2: Model list cache (60s TTL via cacheService)
- Task 3.5: MCP auth token cache (dynamic TTL via cacheService)
- Task 3.8: Identity-keyed ClientManager cache (1h TTL)
- Task 3b.2: Llama Stack-specific types defined in module only
- Platform-ops 1.5: Provider session maps (24h TTL via cacheService)
- Platform-ops 1.6: ClientManager identity-keyed cacheService

All caches serialize to JSON strings for CacheService JsonValue compatibility.
No raw Map<> caches — all use Backstage coreServices.cache per Decision 3.

30 tests, all passing. Zero lint warnings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address review findings — streaming done-event, method naming, constant placement, null guard

- Fix streaming contract violation: return immediately after yielding done event
  instead of continuing to process remaining SSE events in the buffer
- Rename remove() to delete() in SessionMap and ClientManager for consistency
  with sibling classes (DocumentSyncService.deleteHash, cache.invalidate)
- Move TTL constants and KEY_PREFIX from module-level const to static readonly
  class members matching the DocumentSyncService pattern
- Add null-coalescing guard (result.output ?? []) in extractTextFromResponse
  to handle undefined output from the Responses API

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: flush decoder buffer after SSE stream ends

After the ReadableStream reader returns done, remaining buffer content
was silently discarded. Flush the TextDecoder and process any remaining
buffered SSE data before emitting the final done event.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add JSON.parse safety and sanitize error messages

Wrap JSON.parse in cache getters (ClientManager, SessionMap,
ModelListCache) with try/catch — on corrupted data, log warning,
delete the key, and return undefined.

Strip upstream API error body from thrown errors and yielded error
events to prevent leaking internal infrastructure details to clients.
Full error text is still logged server-side.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: wrap chatStream fetch in try/catch for network error handling

Wrap the fetch call in chatStream() with try/catch so network failures
(DNS, connection refused) yield a { type: 'error' } event instead of
propagating an unhandled exception. Full error logged server-side.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: destructure mcpAuthTokenCache and log skipped non-text inputs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: guard against empty input after non-text item filtering

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: fullsend-code <278716306+fullsend-ai-coder[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: gabemontero <gmontero@redhat.com>

Author: 3 people

workspaces/boost/plugins/boost-backend-module-llamastack/.eslintrc.js

@@ -0,0 +1 @@
+module.exports = require('@backstage/cli/config/eslint-factory')(__dirname);

workspaces/boost/plugins/boost-backend-module-llamastack/package.json

@@ -0,0 +1,64 @@
+{
+  "name": "@red-hat-developer-hub/backstage-plugin-boost-backend-module-llamastack",
+  "version": "0.1.0",
+  "license": "Apache-2.0",
+  "description": "Llama Stack provider module for the boost AI platform",
+  "main": "src/index.ts",
+  "types": "src/index.ts",
+  "publishConfig": {
+    "access": "public"
+  },
+  "backstage": {
+    "role": "backend-plugin-module",
+    "pluginId": "boost",
+    "pluginPackage": "@red-hat-developer-hub/backstage-plugin-boost-backend",
+    "pluginPackages": [
+      "@red-hat-developer-hub/backstage-plugin-boost-backend",
+      "@red-hat-developer-hub/backstage-plugin-boost-backend-module-llamastack",
+      "@red-hat-developer-hub/backstage-plugin-boost-common",
+      "@red-hat-developer-hub/backstage-plugin-boost-node"
+    ]
+  },
+  "exports": {
+    ".": "./src/index.ts",
+    "./package.json": "./package.json"
+  },
+  "typesVersions": {
+    "*": {
+      "package.json": [
+        "package.json"
+      ]
+    }
+  },
+  "dependencies": {
+    "@backstage/backend-plugin-api": "^1.9.1",
+    "@red-hat-developer-hub/backstage-plugin-boost-common": "workspace:^",
+    "@red-hat-developer-hub/backstage-plugin-boost-node": "workspace:^"
+  },
+  "devDependencies": {
+    "@backstage/cli": "^0.34.5"
+  },
+  "sideEffects": false,
+  "scripts": {
+    "build": "backstage-cli package build",
+    "clean": "backstage-cli package clean",
+    "lint": "backstage-cli package lint",
+    "lint:check": "backstage-cli package lint",
+    "lint:fix": "backstage-cli package lint --fix",
+    "test": "backstage-cli package test --passWithNoTests --coverage",
+    "tsc": "tsc",
+    "start": "backstage-cli package start",
+    "prepack": "backstage-cli package prepack",
+    "postpack": "backstage-cli package postpack"
+  },
+  "files": [
+    "dist"
+  ],
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/redhat-developer/rhdh-plugins.git",
+    "directory": "workspaces/boost/plugins/boost-backend-module-llamastack"
+  },
+  "homepage": "https://red.ht/rhdh",
+  "bugs": "https://github.com/redhat-developer/rhdh-plugins/issues"
+}

workspaces/boost/plugins/boost-backend-module-llamastack/report.api.md

@@ -0,0 +1,11 @@
+## API Report File for "@red-hat-developer-hub/backstage-plugin-boost-backend-module-llamastack"
+
+> Do not edit this file. It is a report generated by [API Extractor](https://api-extractor.com/).
+
+```ts
+import { BackendFeature } from '@backstage/backend-plugin-api';
+
+// @public
+const boostModuleLlamastack: BackendFeature;
+export default boostModuleLlamastack;
+```

workspaces/boost/plugins/boost-backend-module-llamastack/src/index.ts

@@ -0,0 +1,23 @@
+/*
+ * Copyright Red Hat, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * Llama Stack provider module for the boost AI platform.
+ *
+ * @packageDocumentation
+ */
+
+export { boostModuleLlamastack as default } from './module';

workspaces/boost/plugins/boost-backend-module-llamastack/src/module.ts

@@ -0,0 +1,95 @@
+/*
+ * Copyright Red Hat, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import {
+  coreServices,
+  createBackendModule,
+} from '@backstage/backend-plugin-api';
+import { boostProviderExtensionPoint } from '@red-hat-developer-hub/backstage-plugin-boost-node';
+import { ResponsesApiProviderFactory } from './provider/ResponsesApiProviderFactory';
+
+/**
+ * Llama Stack provider module for the boost plugin.
+ *
+ * Registers a ResponsesApiProvider via the boost extension point,
+ * enabling Llama Stack as an AI backend. All caches use Backstage
+ * cacheService (Decision 3).
+ *
+ * Install alongside the core boost plugin:
+ *
+ * ```ts
+ * // packages/backend/src/index.ts
+ * backend.add(import('@red-hat-developer-hub/backstage-plugin-boost-backend'));
+ * backend.add(import('@red-hat-developer-hub/backstage-plugin-boost-backend-module-llamastack'));
+ * ```
+ *
+ * @public
+ */
+export const boostModuleLlamastack = createBackendModule({
+  pluginId: 'boost',
+  moduleId: 'llamastack',
+  register(reg) {
+    reg.registerInit({
+      deps: {
+        providers: boostProviderExtensionPoint,
+        config: coreServices.rootConfig,
+        cache: coreServices.cache,
+        logger: coreServices.logger,
+      },
+      async init({ providers, config, cache, logger }) {
+        logger.info('Initializing Llama Stack provider module');
+
+        const factory = new ResponsesApiProviderFactory({
+          config,
+          cache,
+          logger,
+        });
+
+        const {
+          provider,
+          modelListCache,
+          mcpAuthTokenCache,
+          clientManager,
+          sessionMap,
+        } = factory.create();
+
+        // Register the provider with the boost plugin
+        providers.registerProvider(provider);
+
+        logger.info(
+          `Llama Stack provider registered (id: ${provider.descriptor.id})`,
+        );
+        logger.info(
+          `Caches initialized: modelList (60s TTL), mcpAuthToken (dynamic TTL), ` +
+            `clientManager (1h TTL), sessionMap (24h TTL)`,
+        );
+
+        // Log cache readiness (these are available for future route handlers)
+        logger.debug(
+          `ModelListCache: ready, ClientManager: ready, SessionMap: ready`,
+        );
+
+        // Suppress unused-variable warnings — these caches are initialized
+        // and available for provider-internal operations. Future iterations
+        // will wire them into route handlers and orchestration flows.
+        void modelListCache;
+        void mcpAuthTokenCache;
+        void clientManager;
+        void sessionMap;
+      },
+    });
+  },
+});

workspaces/boost/plugins/boost-backend-module-llamastack/src/provider/ClientManager.test.ts

@@ -0,0 +1,108 @@
+/*
+ * Copyright Red Hat, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import type {
+  CacheService,
+  LoggerService,
+} from '@backstage/backend-plugin-api';
+import { ClientManager } from './ClientManager';
+import type { ClientState } from '../types';
+
+function createMockLogger(): LoggerService {
+  return {
+    info: jest.fn(),
+    warn: jest.fn(),
+    error: jest.fn(),
+    debug: jest.fn(),
+    child: jest.fn().mockReturnThis(),
+  };
+}
+
+function createMockCache(): CacheService {
+  const store = new Map<string, unknown>();
+  const cache: CacheService = {
+    get: jest.fn(async (key: string) => store.get(key)) as CacheService['get'],
+    set: jest.fn(async (key: string, value: unknown) => {
+      store.set(key, value);
+    }),
+    delete: jest.fn(async (key: string) => {
+      store.delete(key);
+    }),
+    withOptions: jest.fn().mockReturnThis(),
+  };
+  return cache;
+}
+
+describe('ClientManager', () => {
+  let cache: CacheService;
+  let clientManager: ClientManager;
+
+  beforeEach(() => {
+    cache = createMockCache();
+    clientManager = new ClientManager({
+      cache,
+      logger: createMockLogger(),
+    });
+  });
+
+  it('stores and retrieves client state', async () => {
+    const state: ClientState = {
+      userRef: 'user:default/john',
+      lastActivity: '2025-01-01T00:00:00.000Z',
+      sessionCount: 1,
+    };
+
+    await clientManager.set('user:default/john', state);
+    const result = await clientManager.get('user:default/john');
+    expect(result).toEqual(state);
+  });
+
+  it('returns undefined for unknown user', async () => {
+    const result = await clientManager.get('user:default/unknown');
+    expect(result).toBeUndefined();
+  });
+
+  it('stores state with 1-hour TTL', async () => {
+    const state: ClientState = {
+      userRef: 'user:default/john',
+      lastActivity: '2025-01-01T00:00:00.000Z',
+      sessionCount: 1,
+    };
+
+    await clientManager.set('user:default/john', state);
+    expect(cache.set).toHaveBeenCalledWith(
+      'llamastack:client:user:default/john',
+      JSON.stringify(state),
+      { ttl: 3600000 },
+    );
+  });
+
+  it('records activity and increments session count', async () => {
+    const first = await clientManager.recordActivity('user:default/jane');
+    expect(first.sessionCount).toBe(1);
+    expect(first.userRef).toBe('user:default/jane');
+
+    const second = await clientManager.recordActivity('user:default/jane');
+    expect(second.sessionCount).toBe(2);
+  });
+
+  it('deletes client state', async () => {
+    await clientManager.recordActivity('user:default/john');
+    await clientManager.delete('user:default/john');
+    const result = await clientManager.get('user:default/john');
+    expect(result).toBeUndefined();
+  });
+});