RETEP: REal-Time Esports Prediction

RETEP predicts the eventual winner of a League of Legends match from visible game state. The tabular Riot API and temporal baselines live in retep/ and scripts/; the OCR branch converts broadcast screenshots or sampled video frames into the same kind of structured numeric features.

OCR Dependencies

Install Python dependencies:

pip install -r requirements.txt

The default OCR backend uses pytesseract, which also needs the system Tesseract executable:

• Windows: install Tesseract from the UB Mannheim build and add tesseract.exe to PATH.
• macOS: brew install tesseract
• Ubuntu/Debian: sudo apt-get install tesseract-ocr

easyocr is optionally supported with --backend easyocr if you install it separately.

OCR Workflow

1. Run OCR

python scripts/run_ocr.py \
  --input_dir data/screenshots_labeled \
  --output data/interim/ocr_features.csv \
  --regions configs/ocr_regions_default.json \
  --backend tesseract \
  --save_debug_crops data/interim/debug_ocr_crops

The script recursively processes .png, .jpg, .jpeg, and .webp files. Output can be .csv, .json, .jsonl, or .parquet.

Important output columns include image_path, match_id, frame_time_seconds, game_clock_seconds, blue_team_kills, red_team_kills, blue_gold, red_gold, objective/tower counts, per-field confidence columns, raw_<region> debug text, parse_status, and error.

2. Evaluate OCR

Create a small manual label CSV with image_path and any fields you want to check, for example:

image_path,game_clock_seconds,blue_team_kills,red_team_kills,blue_gold,red_gold
frame_001.png,120,0,0,3800,4000

Then run:

python scripts/evaluate_ocr.py \
  --pred data/interim/ocr_features.csv \
  --labels data/ocr_labels.csv

The evaluator reports exact match accuracy, mean absolute error, parse failure rate, and worst failures for numeric fields.

3. Build Clean Static Features

python scripts/build_ocr_features.py \
  --ocr data/interim/ocr_features.csv \
  --output data/interim/ocr_static_features.csv

If the screenshots are chronological frames from the same match, add temporal consistency checks:

python scripts/build_ocr_features.py \
  --ocr data/interim/ocr_features.csv \
  --output data/interim/ocr_static_features.csv \
  --temporal_consistency

Use global --temporal_consistency only when all rows are from the same match/sequence. For mixed screenshots, group by a sequence column instead:

python scripts/build_ocr_features.py \
  --ocr data/interim/ocr_features.csv \
  --output data/interim/ocr_static_features.csv \
  --sequence_col match_id

Optional outcome join:

python scripts/build_ocr_features.py \
  --ocr data/interim/ocr_features.csv \
  --outcomes data/ocr_outcomes.csv \
  --output data/interim/ocr_static_features.csv

The feature builder preserves raw OCR columns and appends cleaned fields such as game_clock_seconds_clean, blue_gold_clean, red_gold_clean, blue_team_kills_clean, red_team_kills_clean, gold_diff_clean, kill_diff_clean, missingness indicators, ocr_flags, and ocr_quality_score. Temporal consistency is useful for same-match frame sequences because team gold and kills should be nondecreasing, but it should not be applied globally to mixed matches.

Objective extraction is experimental. Void grub/objective numeric counts can be read with small digit OCR into optional columns such as blue_grubs_clean, red_grubs_clean, and grubs_diff_clean. Dragon counts are icon-based and less reliable than clock, gold, and kills; their optional blue_dragons_clean, red_dragons_clean, and dragons_diff_clean fields should be treated as debugging features until validated.

4. Train Optional Static OCR Baseline

python scripts/train_ocr_static_baseline.py \
  --features data/interim/ocr_static_features.csv \
  --label blue_win

This baseline uses cleaned clock, gold, kill, missingness, and OCR quality features with a simple imputed/scaled logistic regression. Tiny labeled datasets are treated as sanity checks only.

Tune Regions

Edit configs/ocr_regions_default.json. Region coordinates are normalized fractions of image width/height:

"game_clock": {"x1": 0.483, "y1": 0.047, "x2": 0.517, "y2": 0.075}

This makes the same config scale across 720p, 1080p, and similar 16:9 captures, but broadcast overlays vary. The currently validated core fields are game clock, blue/red gold, and blue/red kills. Towers, grubs/objectives, and dragon icons are experimental and should not be required for baseline training yet.

Save debug crops while tuning:

python scripts/run_ocr.py \
  --input_dir data/screenshots \
  --output data/interim/ocr_features.csv \
  --save_debug_crops data/interim/debug_ocr_crops

Each image gets raw and preprocessed crops per region.

ML Integration

OCR output is intentionally tabular. Use scripts/build_ocr_features.py for the static screenshot baseline table, or convert rows into temporal-style features with retep.vision.feature_adapter.ocr_rows_to_temporal_features. OCR features are noisier than Riot API features, but they demonstrate the screenshot-to-tabular bridge from visible broadcast HUD state.

For ML evaluation with multiple frames from the same game, split by match_id, not by screenshot/frame, to avoid leakage.

Limitations

• Broadcast HUD layouts vary by league, year, replay mode, and resolution, so ROI tuning is manual.
• Tiny icon counts and stylized gold text may need per-region preprocessing changes.
• OCR rows from pro broadcasts will not exactly match the high-Elo ranked Riot API distribution.
• This is a modular OCR MVP, not a trained object detector or custom OCR model.