# scorable-mcp
## 基本信息
- Slug: `root-signals-scorable-mcp`
- Source: modelscope
- Publisher: @root-signals/scorable-mcp
- Categories: testing-and-qa-tools / autonomous-agents / developer-tools
- Hosted: No
- License: Unknown
- Source URL: https://www.modelscope.cn/mcp/servers/@root-signals/scorable-mcp
## 简介
暂无描述。
## 安装提示

```bash
docker run -e SCORABLE_API_KEY=<your_key> -p 0.0.0.0:9090:9090 --name=rs-mcp -d ghcr.io/scorable/scorable-mcp:latest
```

## MCP Server 详情

<h1 align="center">
  <img width="600" alt="Scorable logo" src="https://scorable.ai/images/scorable-color.svg" loading="lazy">
</h1>

<p align="center" class="large-text">
  <i><strong>Measurement & Control for LLM Automations</strong></i>
</p>

<p align="center">
  <a href="https://huggingface.co/scorable">
    <img src="https://img.shields.io/badge/HuggingFace-FF9D00?style=for-the-badge&logo=huggingface&logoColor=white&scale=2" />
  </a>

  <a href="https://discord.gg/QbDAAmW9yz">
    <img src="https://img.shields.io/badge/Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white&scale=2" />
  </a>

  <a href="https://sdk.scorable.ai/en/latest/">
    <img src="https://img.shields.io/badge/Documentation-E53935?style=for-the-badge&logo=readthedocs&logoColor=white&scale=2" />
  </a>

  <a href="https://scorable.ai/demo-user">
    <img src="https://img.shields.io/badge/Temporary_API_Key-15a20b?style=for-the-badge&logo=keycdn&logoColor=white&scale=2" />
  </a>
</p>

# Scorable MCP Server

A [Model Context Protocol](https://modelcontextprotocol.io/introduction) (*MCP*) server that exposes **Scorable** evaluators as tools for AI assistants & agents.

## Overview

This project serves as a bridge between Scorable API and MCP client applications, allowing AI assistants and agents to evaluate responses against various quality criteria.

## Features

- Exposes Scorable evaluators as MCP tools
- Implements SSE for network deployment
- Compatible with various MCP clients such as [Cursor](https://docs.cursor.com/context/model-context-protocol)

## Tools

The server exposes the following tools:

1. `list_evaluators` - Lists all available evaluators on your Scorable account
2. `run_evaluation` - Runs a standard evaluation using a specified evaluator ID
3. `run_evaluation_by_name` - Runs a standard evaluation using a specified evaluator name
6. `run_coding_policy_adherence` - Runs a coding policy adherence evaluation using policy documents such as AI rules files
7. `list_judges` - Lists all available judges on your Scorable account. A judge is a collection of evaluators forming LLM-as-a-judge.
8. `run_judge` - Runs a judge using a specified judge ID


## How to use this server

#### 1. Get Your API Key
[Sign up & create a key](https://scorable.ai/settings/api-keys) or [generate a temporary key](https://scorable.ai/demo-user)

#### 2. Run the MCP Server

#### 4. with sse transport on docker (recommended)
```bash
docker run -e SCORABLE_API_KEY=<your_key> -p 0.0.0.0:9090:9090 --name=rs-mcp -d ghcr.io/scorable/scorable-mcp:latest
```

You should see some logs (note: `/mcp` is the new preferred endpoint; `/sse` is still available for backwardcompatibility)

```bash
docker logs rs-mcp
2025-03-25 12:03:24,167 - scorable_mcp.sse - INFO - Starting Scorable MCP Server v0.1.0
2025-03-25 12:03:24,167 - scorable_mcp.sse - INFO - Environment: development
2025-03-25 12:03:24,167 - scorable_mcp.sse - INFO - Transport: stdio
2025-03-25 12:03:24,167 - scorable_mcp.sse - INFO - Host: 0.0.0.0, Port: 9090
2025-03-25 12:03:24,168 - scorable_mcp.sse - INFO - Initializing MCP server...
2025-03-25 12:03:24,168 - scorable_mcp - INFO - Fetching evaluators from Scorable API...
2025-03-25 12:03:25,627 - scorable_mcp - INFO - Retrieved 100 evaluators from Scorable API
2025-03-25 12:03:25,627 - scorable_mcp.sse - INFO - MCP server initialized successfully
2025-03-25 12:03:25,628 - scorable_mcp.sse - INFO - SSE server listening on http://0.0.0.0:9090/sse
```

From all other clients that support SSE transport - add the server to your config, for example in Cursor:

```json
{
    "mcpServers": {
        "scorable": {
            "url": "http://localhost:9090/sse"
        }
    }
}
```


#### with stdio from your MCP host

In cursor / claude desktop etc:

```yaml
{
    "mcpServers": {
        "scorable": {
            "command": "uvx",
            "args": ["--from", "git+https://github.com/scorable/scorable-mcp.git", "stdio"],
            "env": {
                "SCORABLE_API_KEY": "<myAPIKey>"
            }
        }
    }
}
```

## Usage Examples

<details>
<summary style="font-size: 1.3em;"><b>1. Evaluate and improve Cursor Agent explanations</b></summary><br>

Let's say you want an explanation for a piece of code. You can simply instruct the agent to evaluate its response and improve it with Scorable evaluators:

<h1 align="center">
  <img width="750" alt="Use case example image 1" src="https://github.com/user-attachments/assets/bb457e05-038a-4862-aae3-db030aba8a7c" loading="lazy">
</h1>

After the regular LLM answer, the agent can automatically
- discover appropriate evaluators via Scorable MCP (`Conciseness` and `Relevance` in this case),
- execute them and
- provide a higher quality explanation based on the evaluator feedback:

<h1 align="center">
  <img width="750" alt="Use case example image 2" src="https://github.com/user-attachments/assets/2a83ddc3-9e46-4c2c-bf29-4feabc8c05c7" loading="lazy">
</h1>

It can then automatically evaluate the second attempt again to make sure the improved explanation is indeed higher quality:

<h1 align="center">
  <img width="750" alt="Use case example image 3" src="https://github.com/user-attachments/assets/440d62f6-9443-47c6-9d86-f0cf5a5217b9" loading="lazy">
</h1>

</details>

<details>
<summary style="font-size: 1.3em;"><b>2. Use the MCP reference client directly from code</b></summary><br>

```python
from scorable_mcp.client import ScorableMCPClient

async def main():
    mcp_client = ScorableMCPClient()
    
    try:
        await mcp_client.connect()
        
        evaluators = await mcp_client.list_evaluators()
        print(f"Found {len(evaluators)} evaluators")
        
        result = await mcp_client.run_evaluation(
            evaluator_id="eval-123456789",
            request="What is the capital of France?",
            response="The capital of France is Paris."
        )
        print(f"Evaluation score: {result['score']}")
        
        result = await mcp_client.run_…