Skip to content

Commit 9cec405

Browse files
abrichrabrichr
committed
Fix OmniMCP deployment and add utility files
- Added an environment variable override for PROJECT_NAME - Added .env.example to show required AWS credentials - Updated README with clearer installation instructions - Added CLAUDE.md with important command notes - Added paramiko dependency for OmniParser deployment - Modified omnimcp.py to ensure PROJECT_NAME consistency - Simplified openadapt/adapters/__init__.py imports
1 parent 47da97a commit 9cec405

File tree

5 files changed

+104
-26
lines changed

5 files changed

+104
-26
lines changed

‎omnimcp/.env.example

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# OmniMCP AWS Configuration Example
2+
# Copy this file to .env and fill in your AWS credentials
3+
4+
# AWS credentials for OmniParser deployment
5+
AWS_ACCESS_KEY_ID=your_access_key_id
6+
AWS_SECRET_ACCESS_KEY=your_secret_access_key
7+
AWS_REGION=us-east-2

‎omnimcp/CLAUDE.md

+63
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# OmniMCP Development Notes
2+
3+
**FOCUS: GET THIS WORKING ASAP**
4+
5+
⚠️ **CRITICAL RULES** ⚠️
6+
- NEVER VIEW the contents of any .env file
7+
- NEVER ASK to see the contents of any .env file
8+
- NEVER SUGGEST viewing the contents of any .env file
9+
- These files contain sensitive credentials that must remain private
10+
- ALWAYS USE --auto-deploy-parser when running OmniMCP
11+
- NEVER USE --allow-no-parser under any circumstances
12+
13+
## Installation Commands
14+
15+
```bash
16+
# Install OmniMCP with minimal dependencies
17+
./install.sh
18+
19+
# Install additional dependencies for OmniParser deployment
20+
# For temporary use (doesn't modify pyproject.toml):
21+
uv pip install paramiko
22+
23+
# For permanent addition (modifies pyproject.toml):
24+
# uv add paramiko
25+
```
26+
27+
## AWS Configuration for OmniParser
28+
29+
OmniParser deployment requires AWS credentials. These need to be set in OpenAdapt's deploy module:
30+
31+
```bash
32+
# Copy the deploy example file to the actual .env file
33+
cp /Users/abrichr/oa/src/OpenAdapt/deploy/.env.example /Users/abrichr/oa/src/OpenAdapt/deploy/.env
34+
35+
# Edit the .env file to add your AWS credentials
36+
# AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION must be set
37+
```
38+
39+
**TODO:** Implement functionality to override the .env file location to allow keeping credentials in the omnimcp directory.
40+
41+
## Running OmniMCP
42+
43+
```bash
44+
# Run in debug mode with auto-deploy OmniParser (no confirmation)
45+
omnimcp debug --auto-deploy-parser --skip-confirmation
46+
47+
# Run in CLI mode with auto-deploy OmniParser (no confirmation)
48+
omnimcp cli --auto-deploy-parser --skip-confirmation
49+
50+
# Run as MCP server with auto-deploy OmniParser (no confirmation)
51+
omnimcp server --auto-deploy-parser --skip-confirmation
52+
53+
# Always use auto-deploy with skip-confirmation for best results
54+
# DO NOT use --allow-no-parser as it provides limited functionality
55+
```
56+
57+
## Managing OmniParser EC2 Instances
58+
59+
```bash
60+
# To stop an OmniParser EC2 instance (prevents additional AWS charges)
61+
cd /Users/abrichr/oa/src/OpenAdapt/deploy
62+
uv python deploy/models/omniparser/deploy.py stop
63+
```

‎omnimcp/README.md

+9-9
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,9 @@
22

33
OmniMCP is a UI automation system that enables Claude to control the computer through the Model Control Protocol (MCP). It combines OmniParser's visual understanding with Claude's natural language capabilities to automate UI interactions.
44

5-
## Installation
5+
## Standalone Installation (minimal dependencies)
6+
7+
This standalone package provides OmniMCP with minimal dependencies, letting you use the core functionality without installing all of OpenAdapt's dependencies. It's part of a larger refactoring effort to make components more modular and easier to use.
68

79
### Prerequisites
810

@@ -32,8 +34,7 @@ install.bat
3234
This installation method:
3335
1. Creates an isolated virtual environment using uv
3436
2. Only installs the dependencies needed for OmniMCP
35-
3. Sets up Python to find OpenAdapt modules without installing the full package
36-
4. Allows you to run OmniMCP commands directly without polluting your system Python
37+
3. Sets up Python to find the required OpenAdapt modules without installing the full package
3738

3839
## Usage
3940

@@ -75,8 +76,8 @@ omnimcp cli --server-url=https://your-omniparser-server.example.com
7576
# Deploy OmniParser automatically without confirming
7677
omnimcp cli --auto-deploy-parser --skip-confirmation
7778

78-
# Allow running even if OmniParser isn't available (limited functionality)
79-
omnimcp cli --allow-no-parser
79+
# IMPORTANT: Always use auto-deploy with skip-confirmation
80+
omnimcp cli --auto-deploy-parser --skip-confirmation
8081

8182
# Disable automatic OmniParser deployment attempt
8283
omnimcp cli --auto-deploy-parser=False
@@ -110,11 +111,10 @@ OmniMCP requires access to an OmniParser server for analyzing screenshots:
110111
- OmniMCP will try to connect to `http://localhost:8000` by default
111112
- This requires running an OmniParser server locally
112113

113-
4. **Run Without OmniParser** (Limited functionality)
114-
- Use the `--allow-no-parser` flag to run even without OmniParser
115-
- Claude will only see raw screenshots without UI element detection
114+
4. **IMPORTANT: Always Use Auto-Deploy with Skip-Confirmation**
115+
- For best results, always use these flags together:
116116
```bash
117-
omnimcp cli --allow-no-parser
117+
omnimcp cli --auto-deploy-parser --skip-confirmation
118118
```
119119

120120
### Future Direction: Anthropic ComputerUse Integration

‎omnimcp/pyproject.toml

+18-17
Original file line numberDiff line numberDiff line change
@@ -14,24 +14,25 @@ authors = [
1414
]
1515

1616
dependencies = [
17-
"pynput>=1.7.6", # Keyboard and mouse control
18-
"pillow>=10.0.0", # Image processing
19-
"fire>=0.4.0", # CLI functionality
20-
"anthropic>=0.42.0", # Claude API
21-
"loguru>=0.6.0", # Logging
22-
"mcp>=0.9.0", # Model Control Protocol
23-
"requests>=2.31.0", # HTTP requests for OmniParser
24-
"mss>=6.1.0", # Screen capture
25-
"jinja2>=3.0.0", # For templating
26-
"posthog>=2.0.0", # For analytics
17+
"pynput>=1.7.6", # Keyboard and mouse control
18+
"pillow>=10.0.0", # Image processing
19+
"fire>=0.4.0", # CLI functionality
20+
"anthropic>=0.42.0", # Claude API
21+
"loguru>=0.6.0", # Logging
22+
"mcp>=0.9.0", # Model Control Protocol
23+
"requests>=2.31.0", # HTTP requests for OmniParser
24+
"mss>=6.1.0", # Screen capture
25+
"jinja2>=3.0.0", # For templating
26+
"posthog>=2.0.0", # For analytics
2727
"multiprocessing-utils>=0.1.0", # For process-local storage
28-
"numpy>=1.21.0", # For array operations
29-
"orjson>=3.8.0", # For fast JSON handling
30-
"dictalchemy3>=1.0.0", # For SQLAlchemy dict utils
31-
"joblib>=1.2.0", # For caching
32-
"boto3>=1.26.0", # For AWS services
33-
"botocore>=1.29.0" # For AWS SDK
28+
"numpy>=1.21.0", # For array operations
29+
"orjson>=3.8.0", # For fast JSON handling
30+
"dictalchemy3>=1.0.0", # For SQLAlchemy dict utils
31+
"joblib>=1.2.0", # For caching
32+
"boto3>=1.26.0", # For AWS services # For AWS SDK
33+
"botocore>=1.29.0",
34+
"paramiko>=3.5.1",
3435
]
3536

3637
[project.scripts]
37-
omnimcp = "omnimcp.run_omnimcp:main"
38+
omnimcp = "omnimcp.run_omnimcp:main"

‎openadapt/omnimcp.py

+7
Original file line numberDiff line numberDiff line change
@@ -350,6 +350,13 @@ def __init__(
350350
# Attempt to deploy OmniParser if confirmed
351351
if deploy_confirmed:
352352
logger.info("Deploying OmniParser service...")
353+
354+
# TODO: This is a temporary fix to avoid key name conflicts
355+
# The proper fix would be to modify the deploy module to
356+
# properly respect the PROJECT_NAME from omnimcp/.env or deploy/.env
357+
import os
358+
os.environ["PROJECT_NAME"] = "omnimcp" # Using the omnimcp project name
359+
353360
deploy_success = self.omniparser.deploy()
354361
if deploy_success:
355362
logger.info("OmniParser deployed successfully.")

0 commit comments

Comments
 (0)