✨ feet(openai-wrapper.ts, .env-sample, README.md, MEMO.md): Separate …

…Azure OpenAI API keys and endpoints for text, vision, and image

    Separated Azure OpenAI API keys and endpoints into three: text, vision, and image. This allows for individual configuration of each API's version and deployment name, providing more flexible settings. Also improved error messages to make it easier to identify the cause when a problem occurs.

    ✨ feat(botservice.ts, ExitPlugin.ts, UnuseImagesPlugin.ts): added ability to ignore images
    Added a new plugin `UnuseImagesPlugin`. This plugin instructs the bot to ignore images when the user asks to "ignore images". Also, a new function `isUnuseImages` was added to `botservice.ts` to determine whether to ignore images or not. This allows the bot to ignore images according to the user's request.

    📝 fix(README.md, openai-wrapper.ts): Update documentation and default values
    Fixed some descriptions and default values in README.md that were incorrect. Specifically, corrected the default value for OPENAI_MODEL_NAME to be `gpt-3.5-turbo`, updated the version for AZURE_OPENAI_API_VERSION to `2023-12-01-preview`, and added to the description for AZURE_OPENAI_API_IMAGE_DEPLOYMENT_NAME that DALL-E 3 requires at least the 2023-12-01-preview API version.

    📝 docs(README.md): Add support for GPT-4V Vision API and update Azure OpenAI API version
    By adding support for the GPT-4V Vision API, we have enabled the realization of multimodal text and images. Updating the Azure OpenAI API version allows us to use the latest features. Additionally, we added a new screenshot to visually demonstrate how users can interact with the bot.

    🔧 refactor(botservice.ts): improve image fetching method and enhance error handling
    To improve the image fetching method, the getBase64Image function has been modified to include a token parameter. This makes the function more versatile and allows it to fetch images with different authentication tokens. Additionally, to enhance error handling, a check has been added to ensure the fetch response is successful. This ensures that the application functions correctly even if image fetching fails.

    🔧 refactor(openai-wrapper.ts): completed transition from function_call to tool_calls
    The OpenAI API has been updated to use tool_calls instead of function_call. This change makes the code compatible with the new API and prevents future incompatibility issues. Additionally, this change improves the ability to handle multiple tool calls simultaneously.

.env-sample

@@ -1,19 +1,29 @@
 MATTERMOST_URL=https://mattermost.server.example
 MATTERMOST_TOKEN=abababacdcdcd
+#MATTERMOST_BOTNAME=@chatgpt
+PLUGINS=image-plugin
+
 OPENAI_API_KEY=sk-234234234234234234
 # OPENAI_MODEL_NAME=gpt-3.5-turbo
+# OPENAI_VISION_MODEL_NAME=gpt-4-vision-preview
 # OPENAI_IMAGE_MODEL_NAME=dall-e-2
 
 #AZURE_OPENAI_API_KEY=0123456789abcdefghijklmno
 #AZURE_OPENAI_API_INSTANCE_NAME=example-name
 #AZURE_OPENAI_API_DEPLOYMENT_NAME=gpt-35-turbo
-#AZURE_OPENAI_API_VERSION=2023-05-15
-
-PLUGINS=image-plugin
+#AZURE_OPENAI_API_VERSION=2023-09-01-preview
 #MAX_PROMPT_TOKEN=3000
 #OPENAI_MAX_TOKENS=2000
 #OPENAI_TEMPERATURE=1
 
+#AZURE_OPENAI_VISION_API_KEY=0123456789abcdefghijklmno
+#AZURE_OPENAI_VISION_API_INSTANCE_NAME=example-name
+#AZURE_OPENAI_VISION_API_DEPLOYMENT_NAME=gpt-4-vision-prewview
+
+#AZURE_OPENAI_IMAGE_API_KEY=0123456789abcdefghijklmno
+#AZURE_OPENAI_IMAGE_API_INSTANCE_NAME=example-name
+#AZURE_OPENAI_IMAGE_API_DEPLOYMENT_NAME=dall-e-2
+
 #DEBUG_LEVEL=INFO
 #DEBUG_JSON=true
 #DEBUG_COLORS=true

README.md

@@ -1,13 +1,17 @@
 ## Enhanced from the [original yGuy/chatgpt-mattermost-bot](https://github.com/yGuy/chatgpt-mattermost-bot)
 
+* Support GPT-4V Vision API
+  + Realization of multimodal text and images in conjunction with image plugins. **Limitation**: The plugins are not possible in threads that have been image attached, Under the 1106 gpt-4-vision-vewview.
+  ![The screenshot depicts a fictional chat in Mattermost where a user asks a bot to describe a zoo scene, and to create an image that evokes positive emotions; the bot responds with a colorful zoo illustration and a vivid sunset beach scene.](./mattermost-gpt4v.png)
+* No mention required in Direct Message
 * Support Azure OpenAI API
   + Use the original OpenAI for image generation even when using Azure OpenAI API
 * Build enhancement (The original is now in TypeScript as well by version 2.0.)
   + Formatted by Prettier
   + Lint by eslint
   + Build by esbuild
   + SWC for debug
-* Token-count-based conversation thread management
+* Token-count-based conversation thread management (MessageCollectPlugin may interfere)
 * Splitting message that are too long
 * Support GitLab AutoDevOps by test dummy
 
@@ -44,10 +48,18 @@ or when [running the docker image](#using-the-ready-made-docker-image) or when c
 | OPENAI_MAX_TOKENS    | no       | `2000`                       | The maximum number of tokens to pass to the OpenAI API, defaults to 2000                                                                                                                           |
 | OPENAI_TEMPERATURE   | no       | `0.2`                        | The sampling temperature to use, between 0 and 2, defaults to 1. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. |
 | MAX_PROMPT_TOKENS    | no       | `2096`                       | Maximum number of prompt tokens, default to 2000. For example on GPT-4 = 8196 - OPENAI_MAX_TOKENS.                                                                                                 |
-| AZURE_OPENAI_API_KEY | no       | `0123456789abcdefghijklmno`  | The Azure OpenAI Service API key to authoenticate. If OPENAI_API_KEY is also set, the original OpenAI is used for image generation.                                                                |
+| OPENAI_VISION_MODEL_NAME | no   | `gpt-4-vision-preview`       | The OpenAI VISION model to use, defaults to use OPENAI_MODEL_NAME. Use this when you want to change the model when attaching an image.                                                             |
+| OPENAI_IMAGE_MODEL_NAME  | no   | `dall-e-2`                   | The OpenAI IMAGE model to use, defaults to `dall-e-3`                                                                                                                                              |
+| AZURE_OPENAI_API_KEY | no       | `0123456789abcdefghijklmno`  | The Azure OpenAI Service API key to authoenticate. If OPENAI_API_KEY is also set, the original OpenAI is used for vision or image generation.                                                      |
 | AZURE_OPENAI_API_INSTANCE_NAME   | no | `example-name`         | The instance name on the Azure OpenAI Service                                                                                                                                                      |
 | AZURE_OPENAI_API_DEPLOYMENT_NAME | no | `gpt-35-turbo`         | The name of the deployed model on the Azure OpenAI Service                                                                                                                                         |
-| AZURE_OPENAI_API_VERSION         | no | `2023-03-15-preview`   | The Azure OpenAI version                                                                                                                                                                           |
+| AZURE_OPENAI_API_VERSION         | no | `2023-12-01-preview`   | The Azure OpenAI version                                                                                                                                                                           |
+| AZURE_OPENAI_API_VISION_KEY             | no | `0123456789abcdefghijklmno`  | The Azure OpenAI Service API key to authoenticate. If this is set, Azure OpenAI is used for vision.                                                                                   |
+| AZURE_OPENAI_API_VISION_INSTANCE_NAME   | no | `example-name`  | The instance name on the Azure OpenAI Service for Vision API if specially needed                                                                                                                   |
+| AZURE_OPENAI_API_VISION_DEPLOYMENT_NAME | no | `gpt-4v`        | The name of the deployed model on the Azure OpenAI Service for Vision API if specially needed                                                                                                      |
+| AZURE_OPENAI_API_IMAGE_KEY             | no | `0123456789abcdefghijklmno`  | The Azure OpenAI Service API key to authoenticate. If this is set, Azure OpenAI is used for image generation.                                                                          |
+| AZURE_OPENAI_API_IMAGE_INSTANCE_NAME   | no | `example-name`   | The instance name on the Azure OpenAI Service for Image API if specially needed                                                                                                                    |
+| AZURE_OPENAI_API_IMAGE_DEPLOYMENT_NAME | no | `Dalle3`       | The name of the deployed model on the Azure OpenAI Service for Iamge API if specially needed. DALL-E 3 require 2023-12-01-preview API Version at least                                               |
 | YFILES_SERVER_URL    | no       | `http://localhost:3835`      | The URL to the yFiles graph service for embedding auto-generated diagrams.                                                                                                                         |
 | NODE_EXTRA_CA_CERTS  | no       | `/file/to/cert.crt`          | a link to a certificate file to pass to node.js for authenticating self-signed certificates                                                                                                        |
 | MATTERMOST_BOTNAME   | no       | `"@chatgpt"`                 | the name of the bot user in Mattermost, defaults to '@chatgpt'                                                                                                                                     |