Implement API endpoints for "OpenAI Assistants" client (#59)

* Update docs

* Add API endpoint for getting the OpenAI Functions specification

* Add "Resolve action from specs" API endpoint

* Add "Assistants API" docs and fix typos

* Update docs

Author: machulav

apps/docs/docs/clients/native/api.mdx

@@ -6,13 +6,13 @@ Use the Connery API to create your own client based on the Connery plugin infras
 
 Connery Runner exposes a dynamic OpenAPI schema that describes all the actions available on your runner.
 
-To get the OpenAPI schema, send a request to your runner's `GET /v1/actions/openapi` endpoint.
+To get the OpenAPI schema, send a request to your runner's `GET /v1/actions/specs/openapi` endpoint.
 The endpoint is private and requires an API Key for safety reasons.
 
 Use the following cURL to get the OpenAPI schema:
 
 ```
-curl -L '<PUBLIC_RUNNER_URL>/v1/actions/openapi' \
+curl -L '<PUBLIC_RUNNER_URL>/v1/actions/specs/openapi' \
 -H 'x-api-key: <CONNERY_RUNNER_API_KEY>'
 ```
 

apps/docs/docs/clients/native/index.mdx

@@ -16,7 +16,8 @@ Application Clients are applications in various platforms that allow running act
 
 - [Slack App](/docs/clients/native/slack)
 - [Make App](/docs/clients/native/make)
-- [OpenAI GPTs](/docs/clients/native/openai)
+- [OpenAI GPTs](/docs/clients/native/openai/gpt)
+- [OpenAI Assistants API](/docs/clients/native/openai/assistant)
 
 ## API Clients
 

apps/docs/docs/clients/native/openai/assistant.mdx

@@ -0,0 +1,128 @@
+# Assistants API
+
+Use Connery actions in Function Calling tool of OpenAI Assistants API.
+
+## Information
+
+The Assistants API allows you to build AI assistants within your own applications.
+An Assistant has instructions and can leverage models, tools, and knowledge to respond to user queries.
+The Assistants API currently supports three types of tools: Code Interpreter, Retrieval, and _Function Calling_.
+
+Function calling allows you to describe functions to the Assistants and have it intelligently return the functions that need to be called along with their arguments.
+The Assistants API will pause execution during a Run when it invokes functions, and you can supply the results of the function call back to continue the Run execution.
+
+Connery actions can easily be used in the Function Calling tool of OpenAI Assistants API.
+See below for instructions on configuring and using Connery actions in the Assistant.
+
+:::note Helpful resources
+
+- [OpenAI Assistants API](https://platform.openai.com/docs/assistants/overview/agents)
+- [Funtion Calling](https://platform.openai.com/docs/assistants/tools/function-calling)
+
+:::
+
+## Configuration
+
+### Prerequisites
+
+If you are new to Connery, we recommend you start with the [Quickstart](/docs/platform/quick-start/) guide.
+There, you will find all the information required to set up Connery and start using it.
+
+### Configure functions in the Assistant
+
+1. **Get OpenAI Functions specification for Connery actions installed on the runner.**
+
+```
+curl -L '<PUBLIC_RUNNER_URL>/v1/actions/specs/openai-functions' \
+-H 'x-api-key: <CONNERY_RUNNER_API_KEY>'
+```
+
+The `<PUBLIC_RUNNER_URL>` and `<CONNERY_RUNNER_API_KEY>` you should get during the runner setup in the [Quickstart](/docs/platform/quick-start/) guide.
+
+In response, you will get an OpenAI Function specification for every action installed on your Connery runner.
+
+2. **Use the specification to add functions to your assistant.**
+
+You can do it using Assistant API.
+See the [Defining functions](https://platform.openai.com/docs/assistants/tools/defining-functions) section of the Assistants AI documentation for more information.
+
+Also, you can do it on the [Assistants](https://platform.openai.com/assistants) or [Playground](https://platform.openai.com/playground) pages on the OpenAI developer platform.
+
+<p align="center">
+  <img
+    src="/img/openai/[email protected]"
+    alt="Functions configuration in Assistant"
+  />
+</p>
+
+:::info
+
+The UI does not allow adding an array of functions at once. So you need to add them one by one.
+
+:::
+
+Once you add the functions, you can use them as any other functions in the Function Calling tool of the Assistants API.
+
+## Usage
+
+1. **Get functions from the model that have to be called.**
+
+When you initiate a Run with a user Message that triggers the function, the model can provide multiple functions.
+See the [Reading the functions called by the Assistant](https://platform.openai.com/docs/assistants/tools/reading-the-functions-called-by-the-assistant) section of the Assistants AI documentation for more information.
+
+2. **Resolve the function name to the Connery action.**
+
+```
+curl -L '<PUBLIC_RUNNER_URL>/v1/actions/specs/resolve-action-from-specs?specActionKey=<FUNCTION_NAME>' \
+-H 'x-api-key: <CONNERY_RUNNER_API_KEY>'
+```
+
+Replace `<FUNCTION_NAME>` with the function name you got from the model.
+
+In the response you will get the Plugin ID and Action ID which you can then use to run the action in the runner.
+
+:::info
+
+Because of the limitation of the OpenAI Function schema, we can not put the Plugin ID and Action ID in the function name.
+So, we use a short hashed action key in the OpenIA Function schema.
+Then we resolve it to the Plugin ID and Action ID required to run the action on the runner.
+
+This is a temporary solution which will be adjusted in the future.
+
+:::
+
+3. **Run the action on the runner and get the result.**
+
+Use the following API endpoint to run the action:
+
+```
+curl -L '<PUBLIC_RUNNER_URL>/v1/plugins/<PLUGIN_ID>/actions/<ACTION_ID>/run' \
+-H 'Content-Type: application/json' \
+-H 'x-api-key: <CONNERY_RUNNER_API_KEY>' \
+-d '<ARGUMENTS>'
+```
+
+Replace `<PLUGIN_ID>` and `<ACTION_ID>` with the values you got from the previous step.
+
+Replace `<ARGUMENTS>` with the arguments you got from the model.
+It should be a flat JSON object like this:
+
+```
+{
+  "argument1": "value1",
+  "argument2": "value2"
+}
+```
+
+After the action is executed, you will get the result in the response.
+
+4. **Send the result back to the model.**
+
+See the [Submitting functions outputs](https://platform.openai.com/docs/assistants/tools/submitting-functions-outputs) section of the Assistants AI documentation for more information.
+
+:::tip One action - multiple opportunities
+
+Once the action is configured on Connery Runner, you can use it not only in the Assistants API but also on many other platforms.
+Check the list of [Clients](/docs/clients/native/) to see what platforms are supported.
+
+:::

apps/docs/docs/clients/native/openai.mdx renamed to apps/docs/docs/clients/native/openai/gpt.mdx

@@ -1,4 +1,4 @@
-# OpenAI GPTs
+# GPTs
 
 Use Connery actions in your OpenAI GPTs to make them more powerful.
 
@@ -68,13 +68,13 @@ Once you have a Connery runner up and running and a GPT created, you can add Con
 
 1. **Get OpenAPI schema** with all the actions installed on your runner
 
-To get the OpenAPI schema, you must send a request to your runner's `GET /v1/actions/openapi` endpoint.
+To get the OpenAPI schema, you must send a request to your runner's `GET /v1/actions/specs/openapi` endpoint.
 The endpoint is private and requires an API Key for safety reasons.
 
 Use the following cURL to get the OpenAPI schema:
 
 ```
-curl -L '<PUBLIC_RUNNER_URL>/v1/actions/openapi' \
+curl -L '<PUBLIC_RUNNER_URL>/v1/actions/specs/openapi' \
 -H 'x-api-key: <CONNERY_RUNNER_API_KEY>'
 ```
 

apps/docs/docs/clients/native/openai/index.mdx

@@ -0,0 +1,5 @@
+import DocCardList from "@theme/DocCardList";
+
+# OpenAI
+
+<DocCardList />

apps/docs/docs/platform/use-cases/index.mdx

@@ -32,7 +32,8 @@ We plan to add more clients for other AI Agent and Chatbot platforms.
 
 :::note Related Clients
 
-- [OpenAI GPTs](/docs/clients/native/openai)
+- [OpenAI GPTs](/docs/clients/native/openai/gpt)
+- [OpenAI Assistants API](/docs/clients/native/openai/assistant)
 - [LangChain Tool](/docs/clients/native/langchain)
 
 :::

apps/docs/docs/platform/use-cases/send-email-from-a-custom-openai-gpt-using-connery-actions.mdx

@@ -30,5 +30,5 @@ significantly boosting productivity and the overall utility of the AI tool.
 ## Related links to set up a similar solution
 
 - Plugin, its source code, and documentation for this use case: [connery-io/gmail](https://github.com/connery-io/gmail)
-- Client and its documentation for this use case: [OpenAI GPTs](/docs/clients/native/openai)
+- Client and its documentation for this use case: [OpenAI GPTs](/docs/clients/native/openai/gpt)
 - How to set up Connery: [Quickstart](/docs/platform/quick-start)

apps/docs/sidebars.js

@@ -69,9 +69,22 @@ const sidebars = {
       collapsible: false,
       collapsed: false,
       items: [
+        {
+          type: "category",
+          label: "OpenAI",
+          collapsible: false,
+          collapsed: false,
+          items: [
+            'clients/native/openai/gpt',
+            'clients/native/openai/assistant',
+          ],
+          link: {
+            type: 'doc',
+            id: 'clients/native/openai/index',
+          },
+        },
         'clients/native/slack',
         'clients/native/make',
-        'clients/native/openai',
         'clients/native/api',
         'clients/native/cli',
         'clients/native/langchain',

apps/docs/static/img/openai/[email protected]

@@ -1,8 +1,10 @@
-import { Body, Controller, Get, Inject, Post } from '@nestjs/common';
+import { Body, Controller, Get, HttpException, HttpStatus, Inject, Post, Query } from '@nestjs/common';
 import { ILlm } from ':src/shared/llm/llm.interface';
 import { ActionIdentifiedOutput, ActionNotIdentifiedOutput } from ':src/shared/llm/types';
 import { ObjectResponse } from ':src/shared/types';
 import { OpenApiForActions } from ':src/shared/openapi-for-actions';
+import { OpenAiFunctionsForActions } from ':src/shared/openai-functions-for-actions';
+import { IPluginCache } from ':src/shared/plugin-cache/plugin-cache.interface';
 
 type IdentifyActionBody = {
   prompt: string;
@@ -13,6 +15,8 @@ export class ActionsController {
   constructor(
     @Inject(ILlm) private llm: ILlm,
     @Inject(OpenApiForActions) private openApiForActions: OpenApiForActions,
+    @Inject(OpenAiFunctionsForActions) private openAiFunctionsForActions: OpenAiFunctionsForActions,
+    @Inject(IPluginCache) private pluginCache: IPluginCache,
   ) {}
 
   // This endpoint is deprecated and will be removed in the future
@@ -31,11 +35,41 @@ export class ActionsController {
     return this.identifyAction(body);
   }
 
-  @Get('/v1/actions/openapi')
+  @Get('/v1/actions/specs/openapi')
   async getActionsOpenApi(): Promise<any> {
     return this.openApiForActions.getOpenApiSchema();
   }
 
+  @Get('/v1/actions/specs/openai-functions')
+  async getOpenAiFunctionsSchemaForActions(): Promise<any> {
+    return this.openAiFunctionsForActions.getOpenAiFunctionsSchema();
+  }
+
+  // NOTE: This is a temporary endpoint.
+  // This endpoint resolves specActionKey (short hashed action key) to pluginKey and actionKey,
+  // so that the client can use the Connery API usign the pluginKey and actionKey.
+  // This is done becasue the OpenAI and OpenAPI specs have limitations for the action keys,
+  // so we can not store the full action key (pluginKey/actionKey) in the specs.
+  @Get('/v1/actions/specs/resolve-action-from-specs')
+  async resolveActionFromSpecs(
+    @Query('specActionKey') specActionKey: string,
+  ): Promise<{ pluginKey: string; actionKey: string }> {
+    if (!specActionKey) {
+      throw new HttpException(`The query param 'specActionKey' is required.`, HttpStatus.BAD_REQUEST);
+    }
+
+    // At the moment we still use the actionKey as the specActionKey.
+    // But this should be changed in the future as it's not scalable solution.
+    // Expecially when we have the same actionKey in multiple plugins.
+    // TODO: Change the specActionKey to be a short hashed action key.
+    const action = await this.pluginCache.getAction(specActionKey);
+
+    return {
+      pluginKey: action.plugin.key,
+      actionKey: action.definition.key,
+    };
+  }
+
   private async identifyAction(
     body: IdentifyActionBody,
   ): Promise<ObjectResponse<ActionIdentifiedOutput | ActionNotIdentifiedOutput>> {

apps/runner/src/clients-api/actions.controller.ts

@@ -0,0 +1,41 @@
+import { Inject, Injectable } from '@nestjs/common';
+import { IPluginCache } from './plugin-cache/plugin-cache.interface';
+import { OpenAiFunctionSchema } from './types';
+
+@Injectable()
+export class OpenAiFunctionsForActions {
+  constructor(@Inject(IPluginCache) private pluginCache: IPluginCache) {}
+
+  async getOpenAiFunctionsSchema(): Promise<OpenAiFunctionSchema[]> {
+    const actions = await this.pluginCache.getActions();
+
+    const openAiFunctions: OpenAiFunctionSchema[] = [];
+
+    for (const action of actions) {
+      const openAiFunction: OpenAiFunctionSchema = {
+        name: action.definition.key,
+        description: action.definition.description || '',
+        parameters: {
+          type: 'object',
+          properties: {},
+          required: [],
+        },
+      };
+
+      for (const inputParameter of action.definition.inputParameters) {
+        openAiFunction.parameters.properties[inputParameter.key] = {
+          type: 'string',
+          description: inputParameter.description || '',
+        };
+
+        if (inputParameter.validation?.required) {
+          openAiFunction.parameters.required.push(inputParameter.key);
+        }
+      }
+
+      openAiFunctions.push(openAiFunction);
+    }
+
+    return openAiFunctions;
+  }
+}

apps/runner/src/shared/openai-functions-for-actions.ts

@@ -1,6 +1,6 @@
 import { rmSync } from 'fs';
 import { PluginDownloader } from './plugin-downloader';
-import { Inject, Injectable } from '@nestjs/common';
+import { HttpException, HttpStatus, Inject, Injectable } from '@nestjs/common';
 import { find, filter } from 'lodash';
 import { IPluginCache } from './plugin-cache.interface';
 import { ActionRuntime, PluginRuntime } from 'lib';
@@ -33,7 +33,7 @@ export class MemoryCacheService implements IPluginCache {
     const plugin = find(this._plugins, (action: PluginRuntime) => action.key === pluginKey);
 
     if (!plugin) {
-      throw new Error(`The plugin '${pluginKey}' is not found on the runner.`);
+      throw new HttpException(`The plugin '${pluginKey}' is not found on the runner.`, HttpStatus.NOT_FOUND);
     }
 
     return plugin;
@@ -62,7 +62,7 @@ export class MemoryCacheService implements IPluginCache {
     ) as ActionRuntime[];
 
     if (actions.length === 0) {
-      throw new Error(`The action '${actionKey}' is not found on the runner.`);
+      throw new HttpException(`The action '${actionKey}' is not found on the runner.`, HttpStatus.NOT_FOUND);
     } else if (actions.length > 1) {
       // TODO: handle this case properly
       throw new Error(

apps/runner/src/shared/plugin-cache/memory-cache.service.ts

@@ -9,6 +9,7 @@ import { IConfig } from './config/config.interface';
 import { IPluginCache } from './plugin-cache/plugin-cache.interface';
 import { ILlm } from './llm/llm.interface';
 import { OpenApiForActions } from './openapi-for-actions';
+import { OpenAiFunctionsForActions } from './openai-functions-for-actions';
 
 @Module({
   imports: [ConfigModule],
@@ -30,7 +31,8 @@ import { OpenApiForActions } from './openapi-for-actions';
       useClass: OpenAiService,
     },
     OpenApiForActions,
+    OpenAiFunctionsForActions,
   ],
-  exports: [IConfig, IPluginCache, ILlm, OpenApiForActions],
+  exports: [IConfig, IPluginCache, ILlm, OpenApiForActions, OpenAiFunctionsForActions],
 })
 export class SharedModule {}

apps/runner/src/shared/shared.module.ts

@@ -18,3 +18,22 @@ export type ErrorResponse = {
     message: string;
   };
 };
+
+//
+// Other types
+//
+
+export type OpenAiFunctionSchema = {
+  name: string;
+  description: string;
+  parameters: {
+    type: 'object';
+    properties: {
+      [key: string]: {
+        type: 'string';
+        description: string;
+      };
+    };
+    required: string[];
+  };
+};