Eve library

@nous/ml-engineer-agent

Other

Revision 1

On this page

Agentic ML/SW engineer for the Phase-178 autonomous-research stack (TODOS/phase-178 §178.4).

Components#

Module Summary
sandbox.py SWE-ReX-style sandbox — runs agent code in an isolated subprocess with a per-run FS snapshot, CPU/address-space rlimits, a wall-clock timeout, a replay cache (deterministic resume), and a bounded parallel SandboxPool. (§178.4.1.2)
aide.py AIDE tree search — nodes (code, score, diagnostics), improve/debug/diversify operators chosen by node state, beam + best-child selection, and a SandboxEvaluator that really executes candidates and parses SCORE:. (§178.4.1.1)
swe_agent.py mini-swe-agent edit → test → observe loop; "resolved" is decided by actually running the task's tests in the sandbox. (§178.4.2.1)
benchmarks.py MLE-Bench (score → medal rate), RE-Bench (agent-vs-human frontier at 2h/8h/32h), SWE-Bench (pass-rate over Verified/Pro/rebench/Compass task sets), SWE-Lancer (economic value per dollar), SWE-EVO (multi-PR completion), and a continuous Leaderboard with regression alarms. (§178.4.1.3/.1.4, .2.2–.4)
kernel_agent.py GPU-kernel autotuner agent — edits kernel params, verifies numerical equivalence against a reference before accepting, minimises a roofline cost; ships real RMSNorm / rotary / attention / associative-scan reference kernels. (§178.4.1.5)

The sandbox executes real subprocesses and the SWE loop's resolution check is executor-grounded; the tests run AIDE end-to-end (debug a raising solution, then hill-climb to the optimum via real code execution). Reproducing the published MLE-Bench / SWE-Bench numbers additionally needs an LLM code/patch model and the benchmark datasets — those are the plug-in seams.

Testing#

bash
python3 -m pytest tests/ -q     # 21 tests (spawn real subprocesses)
python3 -m ruff check .