Andrey Karpaty trained AI agents to conduct hundreds of experiments at night when people are resting

Andrej Karpathy Opens a New Approach to Auto-Research

Former Tesla AI projects lead and OpenAI co‑founder Andrej Karpathy announced the launch of a simple yet powerful open‑source project. The script consists of just 630 lines and is hosted on GitHub. It does not claim to be a ready‑made model or a large corporate product; the goal is to demonstrate how AI agents can fully automate the scientific method without human intervention.

> “Our task is to build agents that move through research infinitely fast, even at night,” Karpathy wrote on X. The post instantly went viral and gathered over 8.6 million views in two days.

How the System Works
1. Initialization

The agent receives a training script and a fixed compute budget (usually 5 minutes on a GPU).

2. Self‑analysis of Code

It reads its own source code, formulates a hypothesis for improvement (e.g., changing learning rate or model depth).

3. Modification and Experiment Launch

It makes changes, runs the experiment, and evaluates the results.

4. Effectiveness Check

If the *val_bpb* metric (validation loss per byte) improves, the change is kept; otherwise it’s rolled back and a new hypothesis is generated.

Over one night the agent performed 126 experiments, reducing loss from 0.9979 to 0.9697. After two days of tuning it processed about 700 autonomous changes, discovering roughly 20 additional improvements that were successfully transferred to larger models.

Karpathy noted: “Seeing an agent fully manage the process from start to finish is incredible. It found scaling and regularization errors I’d missed over 20 years of work.”

What Experts Say
Automating the scientific method is considered a fundamental shift in AI development. By turning machine learning into an “evolutionary process” at silicon speed, Karpathy has opened new horizons for research not only in IT but also in marketing, healthcare, and other fields.

Practical Application Examples
Partner | Experiment Description | Results
---|---|---
Hyperspace AI (Varun Mathur) | 35 autonomous agents worked in a peer‑to‑peer network using laptop CPUs. | Overnight they ran 333 experiments without an operator, uncovering initialization strategies (Kaiming, Xavier) and normalization techniques (RMSNorm).
Single Grain (Eric Siu) | Automation of the marketing cycle: the agent changes variables in landing pages, ad creatives, or emails. | Measures “percentage of positive responses,” saves successful changes, and discards ineffective ones.

Conclusion
Karpathy demonstrated how simple scripts can become powerful tools for self‑learning AI agents. The automated optimization loop allows hundreds of experiments to be run overnight, uncovering improvements that previously required years of research. This paves the way for faster and more scalable model development across a wide range of domains.