Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
1b4c179
Update README.md with arxiv
RobertTLange Sep 25, 2025
2fb7548
add google gemini embeding model
takeruhukushima Sep 25, 2025
27af71c
fix: Fix database summary when patch_name metadata is missing
dexhunter Sep 25, 2025
9586cdb
Update README.md
RobertTLange Sep 26, 2025
396c66a
Merge pull request #2 from dexhunter/fix/display
RobertTLange Sep 26, 2025
a60bc9e
docs: change repo name on the onboarding doc
Koki-Kazaore Sep 28, 2025
0003552
Update README
Aladoro Sep 28, 2025
be2e203
Added a doc explaining how to add suport for a local LLM and embeddin…
vicruz99 Oct 12, 2025
bf0c1d4
Add rust to supported languages
LiaCastaneda Oct 13, 2025
77d1819
Ensure setuptools discovers subpackages
iwiwi Oct 14, 2025
929f072
Mark shinka.webui as a package
iwiwi Oct 14, 2025
59a338c
Merge pull request #18 from SakanaAI/fix-packaging
RobertTLange Oct 15, 2025
23ace36
fix apply_full.py when the patch has incomplete (0,1) markers instead…
51616 Oct 24, 2025
06209a2
Merge pull request #21 from 51616/fix-full-patch-no-markers-bug
RobertTLange Oct 27, 2025
c9c468b
Merge pull request #12 from vicruz99/feature/local-models
RobertTLange Oct 27, 2025
c5b1abe
Update README.md
RobertTLange Oct 27, 2025
ccc1326
Merge branch 'main' into lia/add-support-for-rust
RobertTLange Oct 27, 2025
e8ef6de
Merge pull request #15 from LiaCastaneda/lia/add-support-for-rust
RobertTLange Oct 27, 2025
d2211b2
Merge pull request #7 from Koki-Kazaore/docs/change_repo_name
RobertTLange Oct 27, 2025
ded4576
Update inspirations.py - archive
RobertTLange Oct 27, 2025
7ceea8c
Merge pull request #1 from takeruhukushima/main
RobertTLange Oct 27, 2025
ee6e8a5
Update dependencies gemini embed
RobertTLange Oct 27, 2025
a759778
Update dbase.py path default
RobertTLange Oct 30, 2025
c097a88
Fix reasoning token sampling
RobertTLange Oct 30, 2025
6d5e208
Fix anthropic budget sampling
RobertTLange Oct 30, 2025
9b4d7c7
fix shinka_launch --help
RobertTLange Nov 2, 2025
d7a3f7e
fix wrap_eval catch
RobertTLange Nov 2, 2025
397e0fd
add documentation for resuming experiments
RobertTLange Nov 2, 2025
f6896dc
fix OAI dependency db for visualization
RobertTLange Nov 2, 2025
94a2805
Merge pull request #28 from SakanaAI/fix_minor
RobertTLange Nov 2, 2025
1d9d498
Fix init program island copying -> archive
RobertTLange Nov 2, 2025
2f01b3e
fix:GEMINI_API_KEY name error
takeruhukushima Nov 3, 2025
12738f2
Merge pull request #29 from takeruhukushima/rename_gemini_api
RobertTLange Nov 3, 2025
f5f7e68
use dependency-groups.dev
ifsheldon Nov 8, 2025
14739fc
Add support for Claude Sonnet 4.5 (claude-sonnet-4-5-20250929)
arun-pathiban-ddog Nov 8, 2025
7dd7245
Merge pull request #35 from ifsheldon/dev-group
RobertTLange Nov 8, 2025
4f0708b
Merge pull request #36 from arun-pathiban-ddog/add-claude-sonnet-4.5-…
RobertTLange Nov 8, 2025
ed9f51f
Add Swift language support
jeethu Nov 3, 2025
0437118
ignore warning for correct behavior when no improvement is detected, …
Aladoro Nov 11, 2025
831ddf6
Merge pull request #40 from SakanaAI/ignore-logsubtract-warning
RobertTLange Nov 11, 2025
259e786
Allow boolean flags for eval jobs
jm424 Nov 12, 2025
8a615a4
Merge pull request #41 from jm424/jai/allow_eval_job_bool_flags
RobertTLange Nov 13, 2025
3251a70
Add json support
jeremycochoy Nov 17, 2025
1ac33cc
Merge pull request #46 from jeremycochoy/feature/json_support
RobertTLange Nov 19, 2025
3fb579c
Merge branch 'main' into jeethu/swift
RobertTLange Nov 19, 2025
929090c
Merge pull request #37 from jeethu/jeethu/swift
RobertTLange Nov 19, 2025
ed8f1b4
llm: Add GPT-5.1 and Gemini 3 Pro models
jm424 Nov 19, 2025
70e485f
Merge pull request #48 from jm424/jai/add-newer-models
RobertTLange Nov 20, 2025
ecf762b
Update README.md
RobertTLange Nov 22, 2025
c686d7f
Update getting_started.md
RobertTLange Nov 22, 2025
8fcdf75
Extract shared runner helper
jeethu Nov 3, 2025
75cb6df
Fix type annotation
jeethu Nov 4, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,16 @@
<img src="https://img.shields.io/badge/python-%3E%3D3.10-blue" />
<a href="https://github.com/SakanaAI/ShinkaEvolve/blob/master/LICENSE.md"><img src="https://img.shields.io/badge/license-Apache2.0-blue.svg" /></a>
<a href="https://github.com/astral-sh/ruff"><img src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json" /></a>
<a href="http://arxiv.org/abs/2212.04180"><img src="http://img.shields.io/badge/paper-arxiv.2212.04180-B31B1B.svg" /></a>
<a href="http://arxiv.org/abs/2509.19349"><img src="http://img.shields.io/badge/paper-arxiv.2509.19349-B31B1B.svg" /></a>
<a href="https://colab.research.google.com/github/SakanaAI/ShinkaEvolve/blob/main/examples/shinka_tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" /></a>
</p>


`ShinkaEvolve` is a framework that combines Large Language Models (LLMs) with evolutionary algorithms to drive scientific discovery. By leveraging the creative capabilities of LLMs and the optimization power of evolutionary search, `ShinkaEvolve` enables automated exploration and improvement of scientific code. The system is inspired by the [AI Scientist](https://sakana.ai/ai-scientist/), [AlphaEvolve](https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/) and the [Darwin Goedel Machine](https://sakana.ai/dgm/): It maintains a population of programs that evolve over generations, with an ensemble of LLMs acting as intelligent mutation operators that suggest code improvements.
[`ShinkaEvolve`](https://arxiv.org/abs/2509.19349) is a framework that combines Large Language Models (LLMs) with evolutionary algorithms to drive scientific discovery. By leveraging the creative capabilities of LLMs and the optimization power of evolutionary search, `ShinkaEvolve` enables automated exploration and improvement of scientific code. The system is inspired by the [AI Scientist](https://sakana.ai/ai-scientist/), [AlphaEvolve](https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/) and the [Darwin Goedel Machine](https://sakana.ai/dgm/): It maintains a population of programs that evolve over generations, with an ensemble of LLMs acting as intelligent mutation operators that suggest code improvements.

The framework supports **parallel evaluation of candidates** locally or on a Slurm cluster. It maintains an archive of successful solutions, enabling knowledge transfer between different evolutionary islands. `ShinkaEvolve` is particularly well-suited for scientific tasks where there is a verifier available and the goal is to optimize performance metrics while maintaining code correctness and readability.

![](docs/conceptual.png)
![evolution](https://github.com/user-attachments/assets/22cf3468-17fe-4995-9e13-d602b490a54e)

## Documentation 📝

Expand All @@ -26,6 +26,7 @@ The framework supports **parallel evaluation of candidates** locally or on a Slu
| 📓 **[Tutorial Notebook](examples/shinka_tutorial.ipynb)** | Interactive walkthrough of Shinka features | Hands-on examples, configuration, best practices |
| ⚙️ **[Configuration](docs/configuration.md)** | Comprehensive configuration reference | All config options, optimization settings, advanced features |
| 🎨 **[WebUI](docs/webui.md)** | Interactive visualization and monitoring | Real-time tracking, result analysis, debugging tools |
|🕹️ **[Local LLM Support](https://github.com/SakanaAI/ShinkaEvolve/blob/main/docs/support_local_llm.md)**| Instructions for Local LLMs | How to setup local LLMs on your machine|

## Installation & Quick Start 🚀

Expand All @@ -52,9 +53,9 @@ For detailed installation instructions and usage examples, see the [Getting Star
| Example | Description | Environment Setup |
|---------|-------------|-------------------|
| ⭕ [Circle Packing](examples/circle_packing) | Optimize circle packing to maximize radii. | `LocalJobConfig` |
| 🤖 [Agent Design](examples/agent_design) | Design agent scaffolds for math tasks. | `LocalJobConfig` |
| 🤖 [Agent Design](examples/adas_aime) | Design agent scaffolds for math tasks. | `LocalJobConfig` |
| 🎯 [ALE-Bench](examples/ale_bench) | Code optimization for ALE-Bench tasks. | `LocalJobConfig` |
| ✨ [Novelty Generator](examples/novelty_generator_bck) | Generate creative, surprising outputs (e.g., ASCII art). | `LocalJobConfig` |
| ✨ [Novelty Generator](examples/novelty_generator) | Generate creative, surprising outputs (e.g., ASCII art). | `LocalJobConfig` |


## `shinka` Run with Python API 🐍
Expand Down Expand Up @@ -308,9 +309,9 @@ If you use `ShinkaEvolve` in your research, please cite it as follows:

```
@article{lange2025shinka,
title={ShinkaEvolve: Towards Open-Ended and Sample-Efficient Program Evolution},
title={ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution},
author={Lange, Robert Tjarko and Imajuku, Yuki and Cetin, Edoardo},
journal={arXiv preprint},
journal={arXiv preprint arXiv:2509.19349},
year={2025}
}
```
```
4 changes: 2 additions & 2 deletions configs/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ defaults:
- _self_
- database@_global_: island_small
- evolution@_global_: small_budget
- task@_global_: mad_tf
- task@_global_: circle_packing
- cluster@_global_: local
- variant@_global_: mad_tf_example
- variant@_global_: circle_packing_example

verbose: false
results_dir: results
Expand Down
85 changes: 78 additions & 7 deletions docs/getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

Shinka is a framework that combines Large Language Models (LLMs) with evolutionary algorithms to drive scientific discovery. This guide will help you get started with installing, configuring, and running your first evolutionary experiments.

![](../docs/conceptual.png)

## Table of Contents

1. [What is Shinka?](#what-is-shinka)
Expand Down Expand Up @@ -53,7 +55,7 @@ pip install uv

```bash
git clone <shinka-repository-url>
cd shinka
cd ShinkaEvolve

# Create virtual environment with Python 3.11
uv venv --python 3.11
Expand All @@ -79,7 +81,7 @@ conda activate shinka

```bash
git clone <shinka-repository-url>
cd shinka
cd ShinkaEvolve
pip install -e .
```

Expand Down Expand Up @@ -249,7 +251,7 @@ from shinka.core import run_shinka_eval

def main(program_path: str, results_dir: str):
"""Main evaluation function called by Shinka"""

metrics, correct, error_msg = run_shinka_eval(
program_path=program_path,
results_dir=results_dir,
Expand All @@ -268,22 +270,22 @@ def main(program_path: str, results_dir: str):
def validate_packing(run_output):
"""Returns (is_valid: bool, error_msg: str or None)"""
centers, radii, reported_sum = run_output

# Check constraints (bounds, overlaps, etc.)
if constraint_violated:
return False, "Specific error description"

return True, None # Valid solution
```

**2. Metrics Aggregation Function** - Computes fitness and organizes results:
```python
def aggregate_metrics(results, results_dir):
"""Returns metrics dictionary with required structure"""

# Extract data from results
centers, radii, reported_sum = results[0]

return {
"combined_score": float(reported_sum), # PRIMARY FITNESS (higher = better)
"public": { # Visible in WebUI/logs
Expand Down Expand Up @@ -331,6 +333,75 @@ The `run_shinka_eval` function returns three values:

## Advanced Usage

### Resuming Experiments

If you need to pause and resume an evolutionary run, or extend a completed run with more generations, Shinka supports seamless resumption from existing results.

#### How Resuming Works

When you specify an existing `results_dir` that contains a database, Shinka will:
- Detect the previous run automatically
- Restore the population database and all program history
- Resume meta-recommendations from the last checkpoint
- Continue from the last completed generation

#### Using the CLI (Hydra)

```bash
# Resume an existing run and extend to 50 generations
shinka_launch \
variant=circle_packing_example \
evo_config.results_dir=results_20250101_120000 \
evo_config.num_generations=50

# Or with a custom task
shinka_launch \
task=circle_packing \
database=island_small \
evolution=small_budget \
cluster=local \
evo_config.results_dir=path/to/previous/results \
evo_config.num_generations=100
```

#### Using the Python API

```python
from shinka.core import EvolutionRunner, EvolutionConfig
from shinka.database import DatabaseConfig
from shinka.launch import LocalJobConfig

# Point to existing results directory
evo_config = EvolutionConfig(
num_generations=50, # Extend to 50 total generations
results_dir="results_20250101_120000", # Existing results
# ... other config parameters ...
)

job_config = LocalJobConfig(
eval_program_path="examples/circle_packing/evaluate.py",
)

db_config = DatabaseConfig(
archive_size=20,
num_islands=2,
)

# Run will automatically detect and resume
runner = EvolutionRunner(
evo_config=evo_config,
job_config=job_config,
db_config=db_config,
)
runner.run()
```

**Important Notes:**
- The `num_generations` parameter should be set to the **total** number of generations you want (not additional generations)
- For example, if you completed 20 generations and want 30 more, set `num_generations=50`
- The database configuration (number of islands, archive size, etc.) should match the original run
- All previous progress, including the best solutions and meta-recommendations, will be preserved

### Environment Management for Local Jobs

When running jobs locally, you have several options for managing Python environments:
Expand Down
Loading