Skip to content

docs: disambiguate Level 0 package vs schema Level 0/1/2 #133

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
quint-mbbdigital wants to merge 1 commit into
base: main
Choose a base branch
from
Open

docs: disambiguate Level 0 package vs schema Level 0/1/2 #133

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,20 @@

This repository runs append-only, paired nanoGPT experiments comparing base optimizers with and without the repository's WW-PGD extension. It provides experiment infrastructure and descriptive analysis; it does **not** ship a scaling-law fit, acceleration conclusion, statistical-significance test, or alpha-generalization result.

## Level naming (read this once)

Three different “Level 0” objects appear in this repository. They are **not** interchangeable.

| Name | What it is | Typical size / entry |
|---|---|---|
| **Schema / runner Levels 0–2** | `wwgpt` pilot ladder (`--level N`, `configs/level{N}_*.yaml`, `run_one_pair.sh` / multiseed) | **Level 0:** 1 layer · 1 head · width 64 (see config). Levels 1–2 scale up. |
| **`level_0_baseline/`** | Separate **baseline-only** optimizer comparison package (SGD / AdamW / Muon) | **4 layers · 4 heads · width 128** · own scripts/notebooks — see [`level_0_baseline/README.md`](level_0_baseline/README.md) |
| **`level_0_wwpgd/`** | Sibling of the baseline package: same 4L/128 protocol **plus** stock WW-PGD event projection | See [`level_0_wwpgd/README.md`](level_0_wwpgd/README.md) |

There are also `level_1_baseline/` and `level_1_wwpgd/` packages for the next baseline rung. Those names still mean **package folders**, not `wwgpt --level 1`.

**Rule of thumb:** if the path is a top-level `level_*_baseline` / `level_*_wwpgd` directory, it is the **package** experiment. If the command is `wwgpt … --level N` or `configs/levelN_*.yaml`, it is the **schema** pilot ladder.

## Quick start on a local MacBook

Run the repository from its checked-out `main` branch and keep the prepared corpus and results under `/tmp`:
Expand Down
2 changes: 2 additions & 0 deletions level_0_baseline/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Level Zero nanoGPT optimizer baselines

> **Naming:** This package is **not** the same object as schema/runner `wwgpt --level 0` (1×1×64 pilots under `configs/level0_*.yaml`). See the root [README § Level naming](../README.md#level-naming-read-this-once).

This folder contains the canonical **baseline-only Level Zero experiment** for
`CalculatedContent/nanogpt-experiments`.

Expand Down
2 changes: 2 additions & 0 deletions level_0_wwpgd/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Isolated Level 0 nanoGPT + WWPGD

> **Naming:** This package pairs with `level_0_baseline` (4L/128). It is **not** schema/runner `wwgpt --level 0`. See the root [README § Level naming](../README.md#level-naming-read-this-once).

This sibling experiment deliberately reuses the exact FineWeb-Edu GPT-2-BPE model, AdamW optimizer, minibatch stream, learning-rate schedule, evaluation probes, checkpoint policy, and WeightWatcher cadence from `level_0_baseline`. The only scientific intervention is a fresh stock WWPGD event projection immediately after each successful AdamW update.

## Frozen paired protocol
Expand Down
Loading