Browser AI Agent 🤖

Experimental Project: Automating Job Applications on the Web

This project began as an experiment to explore and automate the process of applying for jobs online using AI-powered browser automation. The goal is to streamline and test the end-to-end workflow of searching for jobs, navigating to job listings, filling out application forms, and submitting applications—all through natural language commands and intelligent automation.

Built with TypeScript, Puppeteer, and OpenAI, this tool is designed to help understand the challenges and possibilities of automating real-world job application processes on the web.

Features

• 🔍 Web Search: Search across multiple search engines (Google, Bing, DuckDuckGo, Yahoo)
• 📝 Form Filling: Automatically fill out web forms with AI-generated data
• 💼 Job Applications: Automate job application processes
• 🖱️ Browser Automation: Click, scroll, navigate, and interact with web pages
• 📸 Screenshots: Take screenshots of pages or specific elements
• 📊 Data Extraction: Extract specific data from web pages
• 🤖 AI-Powered: Uses OpenAI to understand and plan complex tasks
• 🎯 Interactive Mode: Chat with the AI agent to perform tasks

Installation

1. Clone the repository:

   git clone <repository-url>
   cd browser-ai-agent

2. Install dependencies:

   npm install

3. Set up environment variables:

   cp env.example .env

   Edit .env and add your OpenAI API key:

   OPENAI_API_KEY=your_openai_api_key_here
   OPENAI_MODEL=gpt-4
   HEADLESS=false
   BROWSER_TIMEOUT=30000

4. Build the project:

   npm run build

Usage

Command Line Interface

Search the web:

npm start search "software engineering jobs"
npm start search "best restaurants in NYC" --engine bing

Navigate to a URL:

npm start navigate "https://example.com"

Apply for a job:

npm start apply-job --url "https://company.com/careers" --position "Software Engineer" --company "Tech Corp"

Interactive mode:

npm start interactive

Programmatic Usage

import { BrowserAIAgent } from './src/BrowserAIAgent';
import { AIAgentConfig, BrowserConfig } from './src/types';

const aiConfig: AIAgentConfig = {
  apiKey: process.env.OPENAI_API_KEY!,
  model: 'gpt-4',
  maxTokens: 2000,
  temperature: 0.7
};

const browserConfig: BrowserConfig = {
  headless: false,
  timeout: 30000,
  userAgent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
  viewport: { width: 1920, height: 1080 }
};

const agent = new BrowserAIAgent(aiConfig, browserConfig);

// Initialize the agent
await agent.initialize();

// Perform a search
const results = await agent.searchAndExtract(
  "software engineering jobs",
  {
    title: 'h3, h2, a',
    url: 'a',
    description: 'div, p, span'
  }
);

// Apply for a job
const jobData = {
  applicationUrl: 'https://company.com/careers',
  position: 'Software Engineer',
  company: 'Tech Corp',
  firstName: 'John',
  lastName: 'Doe',
  email: 'john.doe@example.com'
};

await agent.applyForJob(jobData);

// Close the agent
await agent.close();

Examples

Example 1: Search and Extract Job Listings

npm start search "remote software engineer jobs" --engine google

This will:

1. Search Google for remote software engineer jobs
2. Extract job titles, URLs, and descriptions
3. Display the results in a formatted list

Example 2: Interactive Job Application

npm start interactive

Then type:

Apply for the Software Engineer position at Google. Fill out the application form with my information and upload my resume.

The AI agent will:

1. Navigate to Google's careers page
2. Search for Software Engineer positions
3. Fill out the application form
4. Upload your resume
5. Submit the application

Example 3: Form Filling

await agent.fillForm(
  'https://example.com/contact',
  {
    name: 'John Doe',
    email: 'john@example.com',
    message: 'Hello, I would like to learn more about your services.'
  }
);

Task Types

The AI agent can perform the following types of tasks:

• SEARCH: Perform web searches
• NAVIGATE: Navigate to specific URLs
• FILL_FORM: Fill out web forms
• CLICK: Click on page elements
• SCROLL: Scroll the page in various directions
• SCREENSHOT: Take screenshots
• EXTRACT_DATA: Extract specific data from pages
• JOB_APPLICATION: Automate job applications

Configuration

Environment Variables

Variable	Description	Default
OPENAI_API_KEY	Your OpenAI API key	Required
OPENAI_MODEL	OpenAI model to use	gpt-4
HEADLESS	Run browser in headless mode	false
BROWSER_TIMEOUT	Browser timeout in milliseconds	30000
USER_AGENT	Browser user agent string	Chrome default

Browser Configuration

You can customize the browser behavior:

const browserConfig: BrowserConfig = {
  headless: false,        // Show browser window
  timeout: 30000,         // 30 second timeout
  userAgent: '...',       // Custom user agent
  viewport: {             // Browser viewport
    width: 1920,
    height: 1080
  }
};

Project Structure

src/
├── agents/
│   └── AIAgent.ts           # AI agent for task planning
├── browser/
│   └── BrowserManager.ts    # Browser automation manager
├── tasks/
│   ├── SearchTask.ts        # Web search implementation
│   ├── NavigateTask.ts      # Navigation implementation
│   ├── FillFormTask.ts      # Form filling implementation
│   ├── ClickTask.ts         # Click implementation
│   ├── ScrollTask.ts        # Scroll implementation
│   ├── ScreenshotTask.ts    # Screenshot implementation
│   ├── ExtractDataTask.ts   # Data extraction implementation
│   └── JobApplicationTask.ts # Job application implementation
├── types/
│   └── index.ts             # TypeScript type definitions
├── BrowserAIAgent.ts        # Main orchestrator
└── index.ts                 # CLI entry point

Development

Building

npm run build

Development Mode

npm run dev

Testing

npm test

Linting

npm run lint

Formatting

npm run format

Screenshots

Screenshots are automatically saved to the screenshots/ directory after each task execution.

Error Handling

The agent includes comprehensive error handling:

• Task Validation: Validates task parameters before execution
• Retry Logic: Attempts to recover from common failures
• Detailed Logging: Provides detailed logs for debugging
• Graceful Degradation: Continues execution even if some steps fail

Security Considerations

• API Key Security: Never commit your OpenAI API key to version control
• Browser Security: The agent runs in a controlled browser environment
• Data Privacy: Be careful when automating tasks with sensitive data

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

License

MIT License - see LICENSE file for details.

Support

If you encounter any issues or have questions, please:

1. Check the existing issues
2. Create a new issue with detailed information
3. Include error logs and steps to reproduce

Roadmap

• Support for more search engines
• Advanced form detection
• Multi-page workflows
• Custom task definitions
• Webhook integrations
• Dashboard interface
• Scheduled tasks
• Team collaboration features