Skip to content

Commit

Permalink
✨ New feature (src/adapters/GoogleGeminiAdapter.ts): Support tools by…
Browse files Browse the repository at this point in the history 
… v1beta version of Google Gemini API

GoogleGeminiAdapter.ts uses the v1beta version of the Google Gemini API.
and added support for tools. This allows us to take advantage of new features in the API and provide more advanced functionality. We also improved the way responses are handled and implemented logic to format them to match OpenAI's format.
  • Loading branch information
@takuya-o
takuya-o committed Apr 19, 2024
1 parent 96370a7 commit c94eb9d
Show file tree
Hide file tree
Showing 12 changed files with 894 additions and 470 deletions.
2 changes: 1 addition & 1 deletion .env-sample
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ OPENAI_API_KEY=sk-234234234234234234
# AZURE_OPENAI_API_KEY=0123456789abcdefghijklmno
# AZURE_OPENAI_API_INSTANCE_NAME=example-name
# AZURE_OPENAI_API_DEPLOYMENT_NAME=gpt-35-turbo
# AZURE_OPENAI_API_VERSION=2023-12-01-preview
# AZURE_OPENAI_API_VERSION=2024-03-01-preview

# AZURE_OPENAI_IMAGE_API_KEY=0123456789abcdefghijklmno
# AZURE_OPENAI_IMAGE_API_INSTANCE_NAME=example-name
Expand Down
14 changes: 7 additions & 7 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## Enhanced from the [original yGuy/chatgpt-mattermost-bot](https://github.com/yGuy/chatgpt-mattermost-bot)

* Expimental support Cohere Command R+ and Google Gemini 1.5 Pro API. Text chat only.
* Experimental support Cohere Command R+ that input is text only and Google Gemini 1.5 Pro API that can input text and image
* Support GPT-4V Vision API
+ Realization of multimodal text and images in conjunction with image plugins. **Limitation**: The plugins are not possible in threads that have been image attached, Under the 1106 gpt-4-vision-vewview.
![The screenshot depicts a fictional chat in Mattermost where a user asks a bot to describe a zoo scene, and to create an image that evokes positive emotions; the bot responds with a colorful zoo illustration and a vivid sunset beach scene.](./mattermost-gpt4v.png)
Expand Down Expand Up @@ -46,24 +46,24 @@ or when [running the docker image](#using-the-ready-made-docker-image) or when c
| MATTERMOST_TOKEN | yes | `abababacdcdcd` | The authentication token from the logged in mattermost bot |
| OPENAI_API_KEY | yes | `sk-234234234234234234` | The OpenAI API key to authenticate with OpenAI |
| OPENAI_API_BASE | no | `http://example.com:8080/v1` | The address of an OpenAI compatible API. Overrides the default base path (`https://api.openai.com`) |
| OPENAI_MODEL_NAME | no | `gpt-3.5-turbo` | The OpenAI or Cohere language model to use, defaults to `gpt-3.5-turbo` |
| OPENAI_MODEL_NAME | no | `gpt-3.5-turbo` | The OpenAI or Cohere language model to use, defaults to `gpt-3.5-turbo` |
| OPENAI_MAX_TOKENS | no | `2000` | The maximum number of tokens to pass to the OpenAI API, defaults to 2000 |
| OPENAI_TEMPERATURE | no | `0.2` | The sampling temperature to use, between 0 and 2, defaults to 1. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. |
| MAX_PROMPT_TOKENS | no | `2096` | Maximum number of prompt tokens, default to 2000. For example on GPT-4 = 8196 - OPENAI_MAX_TOKENS. |
| OPENAI_VISION_MODEL_NAME | no | `gpt-4-vision-preview` | The OpenAI VISION model to use, defaults to use OPENAI_MODEL_NAME. Use this when you want to change the model when attaching an image. |
| OPENAI_IMAGE_MODEL_NAME | no | `dall-e-2` | The OpenAI IMAGE model to use, defaults to `dall-e-3` |
| AZURE_OPENAI_API_KEY | no | `0123456789abcdefghijklmno` | The Azure OpenAI Service API key to authoenticate. If OPENAI_API_KEY is also set, the original OpenAI is used for vision or image generation. |
| AZURE_OPENAI_API_KEY | no | `0123456789abcdefghijklmno` | The Azure OpenAI Service API key to authenticate. If OPENAI_API_KEY is also set, the original OpenAI is used for vision or image generation. |
| AZURE_OPENAI_API_INSTANCE_NAME | no | `example-name` | The instance name on the Azure OpenAI Service |
| AZURE_OPENAI_API_DEPLOYMENT_NAME | no | `gpt-35-turbo` | The name of the deployed model on the Azure OpenAI Service |
| AZURE_OPENAI_API_VERSION | no | `2023-12-01-preview` | The Azure OpenAI version |
| AZURE_OPENAI_API_VERSION | no | `2024-03-01-preview` | The Azure OpenAI version |
| AZURE_OPENAI_API_VISION_KEY | no | `0123456789abcdefghijklmno` | The Azure OpenAI Service API key to authoenticate. If this is set, Azure OpenAI is used for vision. |
| AZURE_OPENAI_API_VISION_INSTANCE_NAME | no | `example-name` | The instance name on the Azure OpenAI Service for Vision API if specially needed |
| AZURE_OPENAI_API_VISION_DEPLOYMENT_NAME | no | `gpt-4v` | The name of the deployed model on the Azure OpenAI Service for Vision API if specially needed |
| AZURE_OPENAI_API_IMAGE_KEY | no | `0123456789abcdefghijklmno` | The Azure OpenAI Service API key to authoenticate. If this is set, Azure OpenAI is used for image generation. |
| AZURE_OPENAI_API_IMAGE_INSTANCE_NAME | no | `example-name` | The instance name on the Azure OpenAI Service for Image API if specially needed |
| AZURE_OPENAI_API_IMAGE_DEPLOYMENT_NAME | no | `Dalle3` | The name of the deployed model on the Azure OpenAI Service for Iamge API if specially needed. DALL-E 3 require 2023-12-01-preview API Version at least |
| COHERE_API_KEY | no | `0123456789abcdefghijklmno` | The Cohere API key to authoenticate. If OPENAI_API_KEY is also set, the original OpenAI is used for vision or image generation. |
| GOOGLE_API_KEY | no | `0123456789abcdefghijklmno` | The Gemini API key to authoenticate. If OPENAI_API_KEY is also set, the original OpenAI is used for vision or image generation. Tested model is only 'gemini-1.5-pro-latest' |
| AZURE_OPENAI_API_IMAGE_DEPLOYMENT_NAME | no | `Dalle3` | The name of the deployed model on the Azure OpenAI Service for Image API if specially needed. DALL-E 3 require 2023-12-01-preview API Version at least |
| COHERE_API_KEY | no | `0123456789abcdefghijklmno` | The Cohere API key to authenticate. If OPENAI_API_KEY is also set, the original OpenAI is used for vision or image generation. |
| GOOGLE_API_KEY | no | `0123456789abcdefghijklmno` | The Gemini API key to authenticate. If OPENAI_API_KEY is also set, the original OpenAI is used for vision or image generation. Tested model is only 'gemini-1.5-pro-latest'' |
| YFILES_SERVER_URL | no | `http://localhost:3835` | The URL to the yFiles graph service for embedding auto-generated diagrams. |
| NODE_EXTRA_CA_CERTS | no | `/file/to/cert.crt` | a link to a certificate file to pass to node.js for authenticating self-signed certificates |
| MATTERMOST_BOTNAME | no | `"@chatgpt"` | the name of the bot user in Mattermost, defaults to '@chatgpt' |
Expand Down
6 changes: 3 additions & 3 deletions compose.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
services:
chatcpt:
image: ${CI_REGISTRY_IMAGE:-gitlab.example.com/user/chatgpt-mattermost-bot}:${CI_COMMIT_REF_NAME:+local}
image: ${CI_REGISTRY_IMAGE:-gitlab.example.com/user/chatgpt-mattermost-bot}:${CI_COMMIT_REF_NAME:-local}
build:
context: .
args:
Expand Down Expand Up @@ -29,7 +29,7 @@ services:
# DEBUG_COLORS: ${DEBUG_COLORS}

cohere:
image: ${CI_REGISTRY_IMAGE:-gitlab.example.com/user/chatgpt-mattermost-bot}:${CI_COMMIT_REF_NAME:+local}
image: ${CI_REGISTRY_IMAGE:-gitlab.example.com/user/chatgpt-mattermost-bot}:${CI_COMMIT_REF_NAME:-local}
build:
context: .
args:
Expand All @@ -38,7 +38,7 @@ services:
env_file: .env.cohere

gemini:
image: ${CI_REGISTRY_IMAGE:-gitlab.example.com/user/chatgpt-mattermost-bot}:${CI_COMMIT_REF_NAME:+local}
image: ${CI_REGISTRY_IMAGE:-gitlab.example.com/user/chatgpt-mattermost-bot}:${CI_COMMIT_REF_NAME:-local}
build:
context: .
args:
Expand Down
Loading

0 comments on commit c94eb9d

Please sign in to comment.