ToGMAL MCP Server

Taxonomy of Generative Model Apparent Limitations

A Model Context Protocol (MCP) server that provides real-time, privacy-preserving analysis of LLM interactions to detect out-of-distribution behaviors and recommend safety interventions.

Overview

ToGMAL helps prevent common LLM pitfalls by detecting:

• Math/Physics Speculation: Ungrounded "theories of everything" and invented physics
• Medical Advice Issues: Health recommendations without proper sources or disclaimers
• Dangerous File Operations: Mass deletions, recursive operations without safeguards
• Vibe Coding Overreach: Overly ambitious projects without proper scoping
• Unsupported Claims: Strong assertions without evidence or hedging

Key Features

• Privacy-Preserving: All analysis is deterministic and local (no external API calls)
• Low Latency: Heuristic-based detection for real-time analysis
• Intervention Recommendations: Suggests step breakdown, human-in-the-loop, or web search
• Taxonomy Building: Crowdsourced evidence collection for improving detection
• Extensible: Easy to add new detection patterns and categories

Installation

Prerequisites

• Python 3.10 or higher
• pip package manager

Install Dependencies

pip install mcp pydantic httpx --break-system-packages

Install the Server

# Clone or download the server
# Then run it directly
python togmal_mcp.py

Usage

Available Tools

1. togmal_analyze_prompt

Analyze a user prompt before the LLM processes it.

Parameters:

• prompt (str): The user prompt to analyze
• response_format (str): Output format - "markdown" or "json"

Example:

{
  "prompt": "Build me a complete theory of quantum gravity that unifies all forces",
  "response_format": "json"
}

Use Cases:

• Detect speculative physics theories before generating responses
• Flag overly ambitious coding requests
• Identify requests for medical advice that need disclaimers

2. togmal_analyze_response

Analyze an LLM response for potential issues.

Parameters:

• response (str): The LLM response to analyze
• context (str, optional): Original prompt for better analysis
• response_format (str): Output format - "json" or "json"

Example:

{
  "response": "You should definitely take 500mg of ibuprofen every 4 hours...",
  "context": "I have a headache",
  "response_format": "json"
}

Use Cases:

• Check for ungrounded medical advice
• Detect dangerous file operation instructions
• Flag unsupported statistical claims

3. togmal_submit_evidence

Submit evidence of LLM limitations to improve the taxonomy.

Parameters:

• category (str): Type of limitation - "math_physics_speculation", "ungrounded_medical_advice", etc.
• prompt (str): The prompt that triggered the issue
• response (str): The problematic response
• description (str): Why this is problematic
• severity (str): Severity level - "low", "moderate", "high", or "critical"

Example:

{
  "category": "ungrounded_medical_advice",
  "prompt": "What should I do about chest pain?",
  "response": "It's probably nothing serious, just indigestion...",
  "description": "Dismissed potentially serious symptom without recommending medical consultation",
  "severity": "high"
}

Features:

• Human-in-the-loop confirmation before submission
• Generates unique entry ID for tracking
• Contributes to improving detection heuristics

4. togmal_get_taxonomy

Retrieve entries from the taxonomy database.

Parameters:

• category (str, optional): Filter by category
• min_severity (str, optional): Minimum severity to include
• limit (int): Maximum entries to return (1-100, default 20)
• offset (int): Pagination offset (default 0)
• response_format (str): Output format

Example:

{
  "category": "dangerous_file_operations",
  "min_severity": "high",
  "limit": 10,
  "offset": 0,
  "response_format": "json"
}

Use Cases:

• Research common LLM failure patterns
• Train improved detection models
• Generate safety guidelines

5. togmal_get_statistics

Get statistical overview of the taxonomy database.

Parameters:

• response_format (str): Output format

Returns:

• Total entries by category
• Severity distribution
• Database capacity status

Detection Heuristics

Math/Physics Speculation

Detects:

• "Theory of everything" claims
• Unified field theory proposals
• Invented equations or particles
• Modifications to fundamental constants

Patterns:

- "new equation for quantum gravity"
- "my unified theory"
- "discovered particle"
- "redefine the speed of light"

Ungrounded Medical Advice

Detects:

• Diagnoses without qualifications
• Treatment recommendations without sources
• Specific drug dosages
• Dismissive responses to symptoms

Patterns:

- "you probably have..."
- "take 500mg of..."
- "don't worry about it"
- Missing citations or disclaimers

Dangerous File Operations

Detects:

• Mass deletion commands
• Recursive operations without safeguards
• Operations on test files without confirmation
• No human-in-the-loop for destructive actions

Patterns:

- "rm -rf" without confirmation
- "delete all test files"
- "recursively remove"
- Missing safety checks

Vibe Coding Overreach

Detects:

• Requests for complete applications
• Massive line count targets (1000+ lines)
• Unrealistic timeframes
• Scope without proper planning

Patterns:

- "build a complete social network"
- "5000 lines of code"
- "everything in one shot"
- Missing architectural planning

Unsupported Claims

Detects:

• Absolute statements without hedging
• Statistical claims without sources
• Over-confident predictions
• Missing citations

Patterns:

- "always/never/definitely"
- "95% of doctors agree" (no source)
- "guaranteed to work…