ByteBot MCP Server

Production-grade Model Context Protocol (MCP) server for ByteBot's dual-API architecture, providing intelligent hybrid workflow orchestration for autonomous task execution and desktop computer control.

Overview

This MCP server integrates ByteBot's Agent API (task management) and Desktop API (computer control) into a unified interface for AI assistants like Claude. It enables:

• Autonomous Task Execution: Create and manage tasks for ByteBot to execute independently
• Direct Computer Control: Mouse, keyboard, screen capture, and file operations
• Hybrid Workflows: Intelligent orchestration with automatic monitoring and intervention handling
• Real-time Updates: Optional WebSocket support for live task status notifications

Features

Agent API Tools (Task Management)

• bytebot_create_task - Create new tasks with priority levels
• bytebot_list_tasks - List and filter tasks by status/priority
• bytebot_get_task - Get detailed task information with message history
• bytebot_get_in_progress_task - Check currently running task
• bytebot_update_task - Update task status or priority
• bytebot_delete_task - Delete tasks

Desktop API Tools (Computer Control)

Mouse Operations:

• bytebot_move_mouse - Move cursor to coordinates
• bytebot_click - Click with left/right/middle button
• bytebot_drag - Drag from one position to another
• bytebot_scroll - Scroll in any direction

Keyboard Operations:

• bytebot_type_text - Type text strings
• bytebot_paste_text - Paste text (for special characters)
• bytebot_press_keys - Keyboard shortcuts (Ctrl+C, Alt+Tab, etc.)

Screen Operations:

• bytebot_screenshot - Capture screen as base64 PNG
• bytebot_cursor_position - Get current cursor position

File I/O:

• bytebot_read_file - Read file content (base64)
• bytebot_write_file - Write file content (base64)

System:

• bytebot_switch_application - Switch to application
• bytebot_wait - Wait for specified duration

Hybrid Orchestration Tools (Priority 1)

• bytebot_create_and_monitor_task - Create task and wait for completion
• bytebot_monitor_task - Monitor existing task until terminal state
• bytebot_intervene_in_task - Provide help when task needs intervention
• bytebot_execute_workflow - Multi-step workflow with automatic error recovery

Prerequisites

• Node.js: 20.x or higher
• ByteBot Instance: Running and accessible at configured endpoints
  • Agent API (default: http://localhost:9991)
  • Desktop API (default: http://localhost:9990)

Installation

# Clone or download this repository
cd bytebot-mcp-server

# Install dependencies
npm install

# Build TypeScript code
npm run build

Configuration

1. Create Environment File

Copy the example environment file and customize:

cp .env.example .env

2. Edit .env File

# ByteBot Agent API (Task Management)
BYTEBOT_AGENT_URL=http://localhost:9991

# ByteBot Desktop API (Computer Control)
BYTEBOT_DESKTOP_URL=http://localhost:9990

# WebSocket Configuration (Optional)
BYTEBOT_WS_URL=ws://localhost:9991
ENABLE_WEBSOCKET=false

# Server Configuration
MCP_SERVER_NAME=bytebot-mcp

# Timeouts (milliseconds)
REQUEST_TIMEOUT=30000
DESKTOP_ACTION_TIMEOUT=10000

# Retry Configuration
MAX_RETRIES=3
RETRY_DELAY=1000

# Monitoring Configuration
TASK_POLL_INTERVAL=2000
TASK_MONITOR_TIMEOUT=300000

# File Configuration
MAX_FILE_SIZE=10485760

# Logging
LOG_LEVEL=info

3. Remote ByteBot Configuration

If ByteBot is running on a remote server:

BYTEBOT_AGENT_URL=http://your-server.com:9991
BYTEBOT_DESKTOP_URL=http://your-server.com:9990
BYTEBOT_WS_URL=ws://your-server.com:9991

MCP Client Setup

Claude Desktop

Add to your Claude Desktop configuration file:

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "bytebot": {
      "command": "node",
      "args": ["/absolute/path/to/bytebot-mcp-server/dist/index.js"],
      "env": {
        "BYTEBOT_AGENT_URL": "http://localhost:9991",
        "BYTEBOT_DESKTOP_URL": "http://localhost:9990"
      }
    }
  }
}

Zed Editor

Add to your Zed settings:

{
  "context_servers": {
    "bytebot": {
      "command": {
        "path": "node",
        "args": ["/absolute/path/to/bytebot-mcp-server/dist/index.js"]
      },
      "env": {
        "BYTEBOT_AGENT_URL": "http://localhost:9991",
        "BYTEBOT_DESKTOP_URL": "http://localhost:9990"
      }
    }
  }
}

Continue.dev

Add to .continue/config.json:

{
  "mcpServers": [
    {
      "name": "bytebot",
      "command": "node",
      "args": ["/absolute/path/to/bytebot-mcp-server/dist/index.js"],
      "env": {
        "BYTEBOT_AGENT_URL": "http://localhost:9991",
        "BYTEBOT_DESKTOP_URL": "http://localhost:9990"
      }
    }
  ]
}

Usage Examples

Example 1: Basic Task Creation

User: Create a task for ByteBot to search Wikipedia for "quantum computing"

Claude uses: bytebot_create_task
{
  "description": "Go to wikipedia.org and search for 'quantum computing'",
  "priority": "MEDIUM"
}

Response:
{
  "id": "task-123",
  "status": "PENDING",
  "priority": "MEDIUM",
  "createdAt": "2024-01-15T10:30:00Z"
}

Example 2: Hybrid Workflow (Create Monitor Complete)

User: Create a task to log into example.com and wait for it to complete

Claude uses: bytebot_create_and_monitor_task
{
  "description": "Navigate to example.com and log in with credentials from keychain",
  "timeout": 60000,
  "pollInterval": 2000
}

Response:
{
  "taskId": "task-456",
  "finalStatus": "COMPLETED",
  "completedAt": "2024-01-15T10:31:45Z",
  "messagesCount": 12,
  "task": { ... full task details ... }
}

Example 3: Task Needs Intervention

User: Create a task to fill out a complex form

Claude uses: bytebot_create_and_monito…