PolarHub
  • Agents
  • MCP Servers
  • Skills
  • PolarBear
PolarHub © 2026
技能市场测试开发benchmark-optimization-loop
返回「测试开发」

benchmark-optimization-loop

by affaan-m·ecc

0下载测试开发

简介

Use when the user asks to make something faster, try many variants, run recursive optimization, benchmark latency/throughput/cost, or choose the best implementation by repeated measured tests.

SKILL.md 详细内容

来自 ECC · 3KB

Benchmark Optimization Loop

Use this skill to convert "make it 20x faster" or "try 50 recursive optimizations" into a bounded measured loop that can actually improve a system.

Required Baseline

Do not optimize until these exist:

  • the operation being optimized;
  • the correctness gate that must stay green;
  • the metric: wall time, p95 latency, rows/sec, cost/run, memory, error rate;
  • the current baseline;
  • the search budget: max variants, max time, max spend, max data impact.

If the user asks for an unrealistic target, keep the ambition but make the loop bounded and measurable.

Loop

  1. Measure the baseline.
  2. Identify bottlenecks from evidence.
  3. Generate variants that test one hypothesis each.
  4. Run variants with the same input shape.
  5. Reject variants that fail correctness, safety, or reproducibility.
  6. Promote the fastest safe variant.
  7. Codify the winning path in a script, command, test, config, or doc.
  8. Rerun the baseline and winner to confirm the delta.

Variant Table

Track variants like this:

Variant | Hypothesis | Command | Time | Correct? | Notes
baseline | current path | npm run job | 120s | yes | stable
batch-500 | fewer round trips | npm run job -- --batch 500 | 42s | yes | winner
parallel-8 | more workers | npm run job -- --workers 8 | 31s | no | rate limited

Recursive Search

For recursive or hyperparameter work:

  • persist every run to a ledger;
  • compare against the prior accepted winner, not only the previous run;
  • keep a holdout or replay check;
  • stop when improvement is within noise, correctness fails, cost exceeds the budget, or the search starts changing more variables than it can explain.

Use phrases like "best measured safe variant" instead of "global optimum" unless the search space was actually exhaustive.

Promotion Gate

A variant cannot become the new default until:

  • correctness tests pass;
  • the performance delta is repeated or explained;
  • rollback is obvious;
  • the change is encoded in source control or a durable runbook;
  • the final summary includes exact commands and measurements.

相关 Skills(来自「测试开发」)

测试用例生成 Skills v2.0

包含 15 个子 Skill 的测试用例生成流水线,从产品说明书读取、需求分析、测试策略、正负向用例编写、质量审查到 Excel 输出。

50

Simulink HIL Modeling Standard

通用 Simulink HIL 建模、重构、迁移、评审和发布规范。用于设计 Plant/Control/IO/Bus/Fault/Monitor 架构,定义 Subsystem、Bus、ValueType、MonBus 和 Variant 接口,治理数据字典、参数、采样时间、多核任务、引用组件和初始化流程,检查 .slx/.mdl/.sldd/MATLAB Project,或为实时 HIL 模型生成合规报告、迁移方案、测试计划与发布证据。

47

ICDConverter AVIAGE Template

航空电子系统 ICD 接口控制文件(AVIAGE SYSTEMS 格式)转标准 Template 格式转换专家,适用于 ICD 转换、接口控制文件格式统一和帧结构解析。

2

ICDConverter TIU

面向航空电子 ICD Excel 文件的标准 Template 格式转换 Skill,支持协议帧分析、颜色编码识别、字段元数据索引、帧长度校验和 Excel 输出。

2

ai-regression-testing

Regression testing strategies for AI-assisted development. Sandbox-mode API testing without database dependencies, automated bug-check workflows, and patterns to catch AI blind spots where the same model writes and reviews code.

0

angular-developer

Generates Angular code and provides architectural guidance. Trigger when creating projects, components, or services, or for best practices on reactivity (signals, linkedSignal, resource), forms, dependency injection, routing, SSR, accessibility (ARIA), animations, styling (component styles, Tailwind CSS), testing, or CLI tooling.

0

给 Agent 的安装提示词

复制下方脚本发给 AI,Agent 会自动完成下载和解压

Download and install a skill. Follow these steps EXACTLY. If any step fails, STOP and report the error.

SKILL_NAME="benchmark-optimization-loop"
DOWNLOAD_URL="/downloads/ecc-skills/benchmark-optimization-loop.zip"
INSTALL_CMD="Download benchmark-optimization-loop.zip and extract it into your Agent skills directory."

STEP 1 - FIND SKILL DIRECTORY
[ -d "$HOME/.agents/skills" ] && echo "USER: $HOME/.agents/skills"
[ -d "$PWD/.agents/skills" ] && echo "PROJECT: $PWD/.agents/skills"
for d in "$HOME"/.*/skills; do [ -d "$d" ] && echo "USER: $d"; done
for d in "$PWD"/.*/skills; do [ -d "$d" ] && echo "PROJECT: $d"; done

Decide based on output (deduplicate lines first):
a) Only USER lines -> INSTALL_DIR="<that dir>/$SKILL_NAME"
b) Only PROJECT lines -> INSTALL_DIR="<that dir>/$SKILL_NAME"
c) Both USER and PROJECT -> ask the user to pick global install or project install.
d) Nothing found -> mkdir -p "$HOME/.agents/skills"; INSTALL_DIR="$HOME/.agents/skills/$SKILL_NAME"

STEP 2 - PREPARE
[ -f "$INSTALL_DIR/SKILL.md" ] && echo "EXISTS" || echo "NEW"
# If EXISTS, ask user whether to overwrite. If no, STOP.
mkdir -p "$INSTALL_DIR"

STEP 3 - DOWNLOAD
TMPDIR=$(mktemp -d)
curl -fSL -o "$TMPDIR/skill.zip" "$DOWNLOAD_URL" 2>/dev/null || wget -q -O "$TMPDIR/skill.zip" "$DOWNLOAD_URL"
# If both fail, try INSTALL_CMD as fallback: $INSTALL_CMD
# If still fail, STOP.

STEP 4 - EXTRACT & INSTALL
unzip -o "$TMPDIR/skill.zip" -d "$TMPDIR/ext" 2>/dev/null || python3 -m zipfile -e "$TMPDIR/skill.zip" "$TMPDIR/ext"
# If both fail, this is not a valid ZIP. STOP.

SKILL_PATH=$(find "$TMPDIR/ext" -maxdepth 2 -name "SKILL.md" -print | head -1)
if [ -n "$SKILL_PATH" ]; then
  cp -R "$(dirname "$SKILL_PATH")"/. "$INSTALL_DIR"/
else
  echo "FATAL: SKILL.md not found"; find "$TMPDIR/ext" -type f
fi

STEP 5 - VERIFY
head -5 "$INSTALL_DIR/SKILL.md"

DONE
Read "$INSTALL_DIR/SKILL.md" and follow its instructions to finish the user's task.
粘贴到 Claude / Cursor / Codex 等 Agent 后,会按步骤自动安装到 ~/.agents/skills/

自动安装

通过 PolarBear 协议从 ZIP 地址导入 Skill

Skill 压缩包地址
/downloads/ecc-skills/benchmark-optimization-loop.zip

polarbear://skills/download?name=benchmark-optimization-loop&url=%2Fdownloads%2Fecc-skills%2Fbenchmark-optimization-loop.zip

手动安装

3 步完成

  1. 1

    下载 Skill 压缩包

    下载 ZIP
  2. 2

    将其解压到 Agent 的 Skills 目录

    ~/.agents/skills/benchmark-optimization-loop/
  3. 3

    重启或重新加载 Agent,让 Skill 可用

    重启 Agent 客户端,或刷新会话后即可调用此 Skill。

来源

平台
ecc
作者
affaan-m
Slug
benchmark-optimization-loop
在源站查看