Experiment Runner (proj-012)
Adaptive experiment runner for survival arena. Runs experiments one by one, analyzes results, commits. Between experiments -- human/AI decides what to change next.
Workflow (adaptive)
1. --status → view what's been done
2. --next → run the next pending experiment
3. Analysis → view analysis.json, understand the result
4. Decision → what to change? (only environment conditions, not behavior)
5. Edit YAML → modify the next experiment or add a new one
6. Repeat from #2
Rule: we only change conditions (pressure, resources, architecture). Never hardcode agent behavior.
When to use
- "run experiment" / "next experiment"
- "what is the experiment status"
- "run EXP-011c"
- "run the next 2 experiments"
- proj-012 experiment pipeline
Dependencies
- Python 3, PyYAML (
pip install pyyaml) - Claude CLI (for LLM queries in the arena)
- Git (for committing results)
Paths
| What | Path |
|---|---|
| Arena | $AGENTS_PATH/survival-arena/arena.py |
| Orchestrator | $AGENTS_PATH/survival-arena/run_experiments.py |
| Plan (YAML) | $AGENTS_PATH/survival-arena/experiments.yaml |
| Results | $AGENTS_PATH/survival-arena/experiment_results.json |
| Logs | $AGENTS_PATH/survival-arena/logs/experiments/ |
| Documentation | $PROJECT_ROOT/projects/docs/proj-012-agi-consciousness/ |
How to execute
Next experiment (main mode)
cd $AGENTS_PATH/survival-arena
python3 run_experiments.py --next
Next N experiments
python3 run_experiments.py --next 2
View status
python3 run_experiments.py --status
Run a specific experiment
python3 run_experiments.py --experiment EXP-011c
Run all pending (batch mode)
python3 run_experiments.py --resume
View commands without running
python3 run_experiments.py --dry-run --next 3
Check arena config
python3 arena.py --config-dump --upkeep-base 0 --architecture single
arena.py parameters
| Parameter | Description | Default |
|---|---|---|
--upkeep-base N | Pressure: maintenance cost per turn | 2 |
--regen-rate N | Resource regeneration rate per turn | 3 |
--num-nodes N | Number of resource nodes (distributed across clusters) | 12 |
--child-ratio F | Child token share | 0.35 |
--repro-threshold N | Reproduction threshold | 120 |
--repro-cost N | Reproduction cost | 70 |
--architecture TYPE | single / dual-same / dual-split / dual-kahneman | dual-kahneman |
--experiment-id ID | Identifier for logs | - |
--config-dump | Show config as JSON and exit | - |
--model MODEL | haiku / sonnet | sonnet |
--turns N | Number of turns | 50 |
--seed N | Random seed | - |
--parallel N | Parallel LLM calls | 4 |
run_experiments.py parameters
| Parameter | Description |
|---|---|
--next [N] | Run next N pending (default: 1) |
--status | Show status and exit |
--phase P2 | Run a specific phase |
--experiment EXP-011c | Run a single experiment |
--resume | Run all pending (batch) |
--dry-run | Show commands without executing |
--plan FILE | Path to experiments.yaml |
Phases (roadmap, adapts as we go)
| Phase | What we test | Initial experiments |
|---|---|---|
| P1 | Validation of v4.2c (map + clusters) | 3 |
| P2 | Yerkes-Dodson (pressure) | 5 |
| P3 | Architecture (phase transition) | 4 |
| P4 | Emergent parenting | 3 |
| P5 | Model phenotypes | 4 |
| P6 | Long evolution (200t) | 2 |
What the orchestrator does for each experiment
- Reads config from experiments.yaml (merge: defaults < phase < experiment)
- Builds CLI command for arena.py
- Runs subprocess, timeout 2 hours
- Analyzes JSONL
- Saves to
logs/experiments/EXP-XXX/(config.json, analysis.json, console.txt) - Updates experiment_results.json
- Commits to git
- On error -- retries 2 times, 30s backoff
Analysis results
For each experiment computes:
- Shannon entropy (action distribution diversity)
- Social action % (TRADE + COMMUNICATE + REPRODUCE)
- MOVE+GATHER % (survival focus)
- GATHER success rate (v4.2c: do agents understand the map)
- MOVE % (migration to clusters)
- Dual-system distribution (panic/normal/strategic %)
- Parent-child trades
- NAP detection (alliance/pact/peace keywords)
- Population dynamics (start/end/max/min)
- Reproductions count
- Max generation reached
Troubleshooting
| Problem | Solution |
|---|---|
pyyaml not found | pip3 install pyyaml |
| Timeout on 200t experiment | Increase timeout in run_experiments.py (7200 -> 14400) |
| Rate limit from API | Decrease --parallel (4 -> 2) |
| Cannot find log | Check that arena.py creates a file in logs/ |
| Git commit failed | Check that you're on main, no conflicts |
Related skills
git-workflow-- commit procedure