AlphaEvolve: A coding agent for scientific and algorithmic discovery

3 min read

AlphaEvolve improves programs by evolution, with LLMs writing the mutations. The user marks code blocks to evolve and supplies an evaluate function that returns scalar scores. A prompt sampler builds prompts from earlier programs and their scores, an ensemble of Gemini 2.0 Flash (many cheap proposals) and Gemini 2.0 Pro (fewer, better ones) writes diffs in a SEARCH/REPLACE format, and evaluators score the children, which go into a program database mixing MAP-Elites and island models. Evaluation can run as a cascade of harder tests, and LLM-graded properties such as simplicity can be added as extra scores.

Findings

  • Matrix multiplication: better best-known rank for 14 targets, including the first rank-48 algorithm for two 4×4 complex-valued matrices. Recursive Strassen gives 49, and the case had been open for 56 years.
  • On more than 50 open maths problems it matches the best known construction on about 75% and improves on about 20%, including Erdős’s minimum overlap problem and the kissing number in 11 dimensions.
  • Google’s Borg scheduler: an evolved scoring heuristic recovers on average 0.7% of fleet-wide compute that would otherwise be stranded, confirmed after fleet rollout.
  • A Gemini matmul tiling heuristic gives a 23% average kernel speed-up and 1% less Gemini training time. Editing compiler IR for FlashAttention gives a 32% kernel speed-up and 15% on pre and post-processing.
  • Ablations on tensor decomposition and kissing numbers: removing evolution, prompt context, meta-prompt evolution, full-file evolution or the large model each makes results worse.

Limits

  • It needs an automatic evaluator. The authors call this the main limitation and put problems needing manual experiments out of scope.
  • In the infrastructure cases correctness comes from the problem framing (the scheduler heuristic only ranks machines Borg already allows; tiling leaves the maths unchanged) or from human review. The evaluator is trusted, never tested.
  • A white paper by the system’s owners, with compute given loosely (evaluations “on the order of 100 compute-hours” per candidate are feasible).
  • The TPU circuit simplification was also caught independently by downstream synthesis tools.

Relevance

An evolutionary search is only as safe as its scorer, and the thing being scored will find the scorer’s gaps. For robots the scorer would be a simulator or a runtime monitor, so the open question for robust robotics and runtime-monitoring is this: how do you evolve a controller or coordination heuristic when the evaluator itself might be wrong or attacked? See also EmboCoach-Bench.