Skip to content

Commit dd77693

Browse files
plutasnyyplutasnyy
authored
Wrap all create/update connectors with generic tools (#49)
1 parent 8a655d0 commit dd77693

21 files changed

+1807
-831
lines changed

‎.github/workflows/release.yml

+5-9
Original file line numberDiff line numberDiff line change
@@ -33,21 +33,17 @@ jobs:
3333
uv sync
3434
uv pip install twine build toml
3535
36-
- name: Get version from main
37-
id: get-main-version
38-
run: |
39-
source .venv/bin/activate
40-
git switch main
41-
MAIN_VERSION=$(python -c "import toml; print(toml.load('pyproject.toml')['project']['version'])")
42-
echo "MAIN_VERSION=$MAIN_VERSION" >> $GITHUB_ENV
43-
git switch -
44-
4536
- name: Compare versions
4637
id: compare-versions
4738
run: |
4839
source .venv/bin/activate
40+
4941
CUR_VERSION=$(python -c "import toml; print(toml.load('pyproject.toml')['project']['version'])")
5042
43+
git switch --detach HEAD^
44+
MAIN_VERSION=$(python -c "import toml; print(toml.load('pyproject.toml')['project']['version'])")
45+
git switch -
46+
5147
echo "Current version in pyproject.toml: $CUR_VERSION"
5248
echo "Version on main branch: $MAIN_VERSION"
5349

‎CHANGELOG.md

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
1-
## 0.1.3-dev
1+
## 0.1.3
22

33
### Enhancements
44

55
- Duplicated description of the custom workflow was removed from the tools, freeing up tokens from the LLM.
66
- **Delete Connector Tools Unification**: All delete tools require just the ID of the connector, so they were combined into one tool for sources and one for destinations.
7+
- **Tools unification** - Source and destination connectors are now unified into a generic `create/update_source/destination_connector` tools.
78

89
### Fixes
910

‎README.md

+52-52
Original file line numberDiff line numberDiff line change
@@ -4,65 +4,65 @@ An MCP server implementation for interacting with the Unstructured API. This ser
44

55
## Available Tools
66

7-
| Tool | Description |
8-
|------|-------------|
9-
| `list_sources` | Lists available sources from the Unstructured API. |
10-
| `get_source_info` | Get detailed information about a specific source connector. |
11-
| `create_[connector]_source` | Create a source connector. Currently, we have s3/google drive/azure connectors (more to come!) |
12-
| `update_[connector]_source` | Update an existing source connector by params. |
13-
| `delete_[connector]_source` | Delete a source connector by source id. |
14-
| `list_destinations` | Lists available destinations from the Unstructured API. |
15-
| `get_destination_info` | Get detailed info about a specific destination connector. Currently, we have s3/weaviate/astra/neo4j/mongo DB (more to come!) |
16-
| `create_[connector]_destination` | Create a destination connector by params. |
17-
| `update_[connector]_destination` | Update an existing destination connector by destination id. |
18-
| `delete_[connector]_destination` | Delete a destination connector by destination id. |
19-
| `list_workflows` | Lists workflows from the Unstructured API. |
20-
| `get_workflow_info` | Get detailed information about a specific workflow. |
21-
| `create_workflow` | Create a new workflow with source, destination id, etc. |
22-
| `run_workflow` | Run a specific workflow with workflow id |
23-
| `update_workflow` | Update an existing workflow by params. |
24-
| `delete_workflow` | Delete a specific workflow by id. |
25-
| `list_jobs` | Lists jobs for a specific workflow from the Unstructured API. |
26-
| `get_job_info` | Get detailed information about a specific job by job id. |
27-
| `cancel_job` |Delete a specific job by id. |
7+
| Tool | Description |
8+
|--------------------------------|---------------------------------------------------------------|
9+
| `list_sources` | Lists available sources from the Unstructured API. |
10+
| `get_source_info` | Get detailed information about a specific source connector. |
11+
| `create_source_connector` | Create a source connector.) |
12+
| `update_source_connector` | Update an existing source connector by params. |
13+
| `delete_source_connector` | Delete a source connector by source id. |
14+
| `list_destinations` | Lists available destinations from the Unstructured API. |
15+
| `get_destination_info` | Get detailed info about a specific destination connector |
16+
| `create_destination_connector` | Create a destination connector by params. |
17+
| `update_destination_connector` | Update an existing destination connector by destination id. |
18+
| `delete_destination_connector` | Delete a destination connector by destination id. |
19+
| `list_workflows` | Lists workflows from the Unstructured API. |
20+
| `get_workflow_info` | Get detailed information about a specific workflow. |
21+
| `create_workflow` | Create a new workflow with source, destination id, etc. |
22+
| `run_workflow` | Run a specific workflow with workflow id |
23+
| `update_workflow` | Update an existing workflow by params. |
24+
| `delete_workflow` | Delete a specific workflow by id. |
25+
| `list_jobs` | Lists jobs for a specific workflow from the Unstructured API. |
26+
| `get_job_info` | Get detailed information about a specific job by job id. |
27+
| `cancel_job` | Delete a specific job by id. |
2828

2929
Below is a list of connectors the `UNS-MCP` server currently supports, please see the full list of source connectors that Unstructured platform supports [here](https://docs.unstructured.io/api-reference/workflow/sources/overview) and destination list [here](https://docs.unstructured.io/api-reference/workflow/destinations/overview). We are planning on adding more!
3030

31-
| Source | Destination |
32-
|------|-------------|
33-
| S3 | S3 |
34-
| Azure | Weaviate |
35-
| Google Drive | Pinecone |
36-
| OneDrive | AstraDB |
37-
| Salesforce | MongoDB |
38-
| Sharepoint | Neo4j|
39-
| | Databricks Volumes|
40-
| | Databricks Volumes Delta Table |
31+
| Source | Destination |
32+
|--------------|--------------------------------|
33+
| S3 | S3 |
34+
| Azure | Weaviate |
35+
| Google Drive | Pinecone |
36+
| OneDrive | AstraDB |
37+
| Salesforce | MongoDB |
38+
| Sharepoint | Neo4j |
39+
| | Databricks Volumes |
40+
| | Databricks Volumes Delta Table |
4141

4242

4343
To use the tool that creates/updates/deletes a connector, the credentials for that specific connector must be defined in your .env file. Below is the list of `credentials` for the connectors we support:
4444

45-
| Credential Name | Description |
46-
|------|-------------|
47-
| `ANTHROPIC_API_KEY` | required to run the `minimal_client` to interact with our server. |
48-
| `AWS_KEY`, `AWS_SECRET`| required to create S3 connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/api-reference/workflow/sources/s3) and [here](https://docs.unstructured.io/api-reference/workflow/destinations/s3) |
49-
| `WEAVIATE_CLOUD_API_KEY` | required to create Weaviate vector db connector, see how in [documentation](https://docs.unstructured.io/api-reference/workflow/destinations/weaviate) |
50-
| `FIRECRAWL_API_KEY` | required to use Firecrawl tools in `external/firecrawl.py`, sign up on [Firecrawl](https://www.firecrawl.dev/) and get an API key. |
51-
| `ASTRA_DB_APPLICATION_TOKEN`, `ASTRA_DB_API_ENDPOINT` | required to create Astradb connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ui/destinations/astradb)|
52-
| `AZURE_CONNECTION_STRING`| required option 1 to create Azure connector via ``uns-mcp`` server, see how in [documentation](https://docs.unstructured.io/ui/sources/azure-blob-storage) |
53-
| `AZURE_ACCOUNT_NAME`+`AZURE_ACCOUNT_KEY`| required option 2 to create Azure connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ui/sources/azure-blob-storage)|
54-
| `AZURE_ACCOUNT_NAME`+`AZURE_SAS_TOKEN` | required option 3 to create Azure connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ui/sources/azure-blob-storage) |
55-
| `NEO4J_PASSWORD` | required to create Neo4j connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ui/destinations/neo4j) |
56-
| `MONGO_DB_CONNECTION_STRING` | required to create Mongodb connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ui/destinations/mongodb) |
57-
| `GOOGLEDRIVE_SERVICE_ACCOUNT_KEY` | a string value. The original server account key (follow [documentation](https://docs.unstructured.io/ui/sources/google-drive)) is in json file, run `cat /path/to/google_service_account_key.json | base64` in terminal to get the string value |
58-
| `DATABRICKS_CLIENT_ID`,`DATABRICKS_CLIENT_SECRET` | required to create Databricks volume/delta table connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ui/destinations/databricks-volumes) and [here](https://docs.unstructured.io/ui/destinations/databricks-delta-table) |
59-
| `ONEDRIVE_CLIENT_ID`, `ONEDRIVE_CLIENT_CRED`,`ONEDRIVE_TENANT_ID`| required to create One Drive connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ui/destinations/onedrive) |
60-
| `PINECONE_API_KEY` | required to create Pinecone vector DB connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ui/destinations/pinecone) |
61-
| `SALESFORCE_CONSUMER_KEY`,`SALESFORCE_PRIVATE_KEY` | required to create salesforce source connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ingestion/source-connectors/salesforce)|
62-
| `SHAREPOINT_CLIENT_ID`, `SHAREPOINT_CLIENT_CRED`,`SHAREPOINT_TENANT_ID`| required to create One Drive connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ui/sources/sharepoint) |
63-
| `LOG_LEVEL` | Used to set logging level for our `minimal_client`, e.g. set to ERROR to get everything |
64-
| `CONFIRM_TOOL_USE` | set to true so that `minimal_client` can confirm execution before each tool call |
65-
| `DEBUG_API_REQUESTS` | set to true so that `uns_mcp/server.py` can output request parameters for better debugging |
45+
| Credential Name | Description |
46+
|-------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
47+
| `ANTHROPIC_API_KEY` | required to run the `minimal_client` to interact with our server. |
48+
| `AWS_KEY`, `AWS_SECRET` | required to create S3 connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/api-reference/workflow/sources/s3) and [here](https://docs.unstructured.io/api-reference/workflow/destinations/s3) |
49+
| `WEAVIATE_CLOUD_API_KEY` | required to create Weaviate vector db connector, see how in [documentation](https://docs.unstructured.io/api-reference/workflow/destinations/weaviate) |
50+
| `FIRECRAWL_API_KEY` | required to use Firecrawl tools in `external/firecrawl.py`, sign up on [Firecrawl](https://www.firecrawl.dev/) and get an API key. |
51+
| `ASTRA_DB_APPLICATION_TOKEN`, `ASTRA_DB_API_ENDPOINT` | required to create Astradb connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ui/destinations/astradb) |
52+
| `AZURE_CONNECTION_STRING` | required option 1 to create Azure connector via ``uns-mcp`` server, see how in [documentation](https://docs.unstructured.io/ui/sources/azure-blob-storage) |
53+
| `AZURE_ACCOUNT_NAME`+`AZURE_ACCOUNT_KEY` | required option 2 to create Azure connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ui/sources/azure-blob-storage) |
54+
| `AZURE_ACCOUNT_NAME`+`AZURE_SAS_TOKEN` | required option 3 to create Azure connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ui/sources/azure-blob-storage) |
55+
| `NEO4J_PASSWORD` | required to create Neo4j connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ui/destinations/neo4j) |
56+
| `MONGO_DB_CONNECTION_STRING` | required to create Mongodb connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ui/destinations/mongodb) |
57+
| `GOOGLEDRIVE_SERVICE_ACCOUNT_KEY` | a string value. The original server account key (follow [documentation](https://docs.unstructured.io/ui/sources/google-drive)) is in json file, run `base64 < /path/to/google_service_account_key.json` in terminal to get the string value |
58+
| `DATABRICKS_CLIENT_ID`,`DATABRICKS_CLIENT_SECRET` | required to create Databricks volume/delta table connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ui/destinations/databricks-volumes) and [here](https://docs.unstructured.io/ui/destinations/databricks-delta-table) |
59+
| `ONEDRIVE_CLIENT_ID`, `ONEDRIVE_CLIENT_CRED`,`ONEDRIVE_TENANT_ID` | required to create One Drive connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ui/destinations/onedrive) |
60+
| `PINECONE_API_KEY` | required to create Pinecone vector DB connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ui/destinations/pinecone) |
61+
| `SALESFORCE_CONSUMER_KEY`,`SALESFORCE_PRIVATE_KEY` | required to create salesforce source connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ingestion/source-connectors/salesforce) |
62+
| `SHAREPOINT_CLIENT_ID`, `SHAREPOINT_CLIENT_CRED`,`SHAREPOINT_TENANT_ID` | required to create One Drive connector via `uns-mcp` server, see how in [documentation](https://docs.unstructured.io/ui/sources/sharepoint) |
63+
| `LOG_LEVEL` | Used to set logging level for our `minimal_client`, e.g. set to ERROR to get everything |
64+
| `CONFIRM_TOOL_USE` | set to true so that `minimal_client` can confirm execution before each tool call |
65+
| `DEBUG_API_REQUESTS` | set to true so that `uns_mcp/server.py` can output request parameters for better debugging |
6666

6767

6868
### Firecrawl Source

0 commit comments

Comments
 (0)