SWEfficiency

About the Helper

What is it?

  • A FastAPI server listening on 127.0.0.1:5050 (HTTPS preferred)
  • Installed under ~/.SWEfficiency/helper (code + Python venv)
  • Auto‑started at login via a LaunchAgent com.swefficiency.helper
  • Only accepts requests from allowed web origins (CORS)
  • Runs Docker jobs with restricted options (reduced privileges)

What does it help with?

  • Health detection for the Non‑LLM Bench page
  • Local Docker availability checks
  • Prepare and run benchmark jobs (Before/After) with your workload code
  • Optional upload to a public data repo (via GitHub Device Flow) only if you opt‑in

Quick install (macOS)

Run this in Terminal (macOS 11+):

/bin/bash -lc 'curl -fsSL https://lichanghengxjtu.github.io/SWEfficiency/nonllmplatform/downloadhelper/install_helper.sh | bash'

The installer will generate a localhost certificate, create/update a Python venv, install dependencies, write a LaunchAgent, and start the Helper.

Restart / Uninstall

Restart (stop then start):

launchctl bootout gui/$(id -u)/com.swefficiency.helper || true; launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.swefficiency.helper.plist; launchctl enable gui/$(id -u)/com.swefficiency.helper; launchctl kickstart -k gui/$(id -u)/com.swefficiency.helper

Uninstall:

launchctl bootout gui/$(id -u)/com.swefficiency.helper || true; rm -f ~/Library/LaunchAgents/com.swefficiency.helper.plist; rm -rf ~/.SWEfficiency ~/SWEfficiencyWork

Verify

Health endpoint:

curl -sk https://127.0.0.1:5050/api/health | cat

Expected JSON (example):

{"ok":true,"server":"swefficiency-helper","docker_sock":true}

Privacy & Security

  • Local‑only: all sensitive actions run on your machine. Listens on 127.0.0.1 with HTTPS.
  • HTTPS by default: a self‑signed certificate for localhost is generated and trusted locally.
  • CORS allowlist: by default only allows https://lichanghengxjtu.github.io and http://localhost:8000.
  • Docker runs with reduced privileges (no new privileges, limited CPU/memory/PIDs, no network where applicable).
  • No data is uploaded unless you explicitly opt‑in; only the benchmark record (workload text and metrics) is submitted, and only above a threshold.

HTTP API (for reference)

  • GET /api/health – helper health info
  • GET /api/docker/check – check Docker availability
  • POST /api/bench/prepare – create a job and write your workload code
  • POST /api/bench/run – run Before/After and parse Mean/Std
  • POST /api/submit – record a local submission (JSONL under ~/SWEfficiencyWork)
  • POST /api/upload_run – optional upload to the data repo via PR (if you opt‑in)
  • POST /api/upload/start – start GitHub Device Flow auth and get a user code
  • POST /api/upload/token – provide a personal token (fallback; not recommended)

Configuration

  • SWEF_ALLOWED_ORIGINS – comma‑separated CORS origins (default includes https://lichanghengxjtu.github.io)
  • SWEF_WORK_ROOT – sandbox root (default ~/SWEfficiencyWork)
  • SWEF_DATA_REPO – GitHub repo to push PRs to (default LichanghengXJTU/SWEf-data)
  • SWEF_DATA_PATH – path inside the repo (default Non_LLM_user_data)
  • SWEF_GH_CLIENT_ID / SWEF_GH_CLIENT_SECRET – GitHub Device Flow app creds (if not set, you may be asked to provide a token via /api/upload/token)

Logs

  • Service stdout: ~/Library/Logs/swefficiency-helper.log
  • Service stderr: ~/Library/Logs/swefficiency-helper.err

Follow logs:

tail -f ~/Library/Logs/swefficiency-helper.log ~/Library/Logs/swefficiency-helper.err

Requirements

  • macOS 11+
  • Python 3.12 preferred (the installer will create a venv)
  • Docker Desktop (for running benchmarks)

Troubleshooting

  • Browser says certificate is not trusted → Visit https://127.0.0.1:5050/api/health once and accept the local certificate.
  • CORS blocked → Ensure the page origin is in SWEF_ALLOWED_ORIGINS (default includes https://lichanghengxjtu.github.io). Restart the helper after changes.
  • GitHub upload requires token → Set SWEF_GH_CLIENT_ID/SECRET for Device Flow, or provide a PAT to /api/upload/token as a fallback.

FAQ

  • Does the website upload my local files? No. Only the workload code you paste, metadata, and metrics are used. Actual execution happens locally in Docker.
  • Can I disable uploads entirely? Yes. Do not check the “I agree to upload” option; the page will only record locally.
Back to Platforms