SWEfficiency
SWEfficiency:
Evaluate performance improvements of human-written and LLM-generated patches on real-world tasks
Authors
GitHub→ Paper→ Dataset→

Our work

Mind map placeholder
Placeholder

Performance, Measured.

tbu
Tasks
tbu
Agents / Models
tbu
Best Mean Speedup
tbu
Best Pass Rate

Features

Criteria

Results

Analysis

Examples

Run SWEfficiency

placeholder
placeholder line 1
placeholder line 2
placeholder line 1
placeholder line 2
placeholder line 3

Platforms

Non‑LLM Benchmark

Provide an instance ID / GitHub PR URL / Docker image URL and a benchmark workload, then compare the performance of the human‑expert patch (Before/After). The page also integrates a local Helper for secure execution, a Docker availability check, and optional submission to the public dataset.

Go to Non‑LLM Bench

Install Local Helper

Detecting local Helper…
This page explains the Helper. We recommend reading it before installing and running the Helper and performing the benchmark.
Learn more about Helper
1) Download helper (macOS):
One‑line install and run the local Helper (HTTPS, 127.0.0.1:5050). Use the code below to install the helper when you never installed it before or uninstalled it.
/bin/bash -lc 'curl -fsSL https://lichanghengxjtu.github.io/SWEfficiency/nonllmplatform/downloadhelper/install_helper.sh | bash'
2) Restart helper:
Kickstart the LaunchAgent to restart the Helper immediately. Use the code below to restart the helper when you want to restart it or the helper is not running or need to be updated.
launchctl bootout gui/$(id -u)/com.swefficiency.helper || true; launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.swefficiency.helper.plist; launchctl enable gui/$(id -u)/com.swefficiency.helper; launchctl kickstart -k gui/$(id -u)/com.swefficiency.helper
3) Uninstall helper:
Stop the LaunchAgent and remove installed files and work dir. Use the code below to uninstall the helper when you want to uninstall it. Remember to run this code when you no longer need to benchmark on SWEfficiency.
launchctl bootout gui/$(id -u)/com.swefficiency.helper || true; rm -f ~/Library/LaunchAgents/com.swefficiency.helper.plist; rm -rf ~/.SWEfficiency ~/SWEfficiencyWork
After installation, go to Non‑LLM Bench to run Docker checks.