Skip to content

task_description.md

Latest commit

 

History

History
160 lines (99 loc) · 7.16 KB

task_description.md

File metadata and controls

160 lines (99 loc) · 7.16 KB
Raw

The 2026 NeuroGolf Championship

URL: https://www.kaggle.com/competitions/neurogolf-2026 Host: The Neurosynthetic Research Institute Type: Research Prediction Competition (part of IJCAI-ECAI 2026 Competitions Track) Tagline: Design the smallest neural networks to solve ARC-AGI image transformations.

Overview

Design the smallest possible neural networks to solve ARC-AGI image transformations (all drawn from the ARC-AGI benchmark suite) and discover how many parameters those tasks actually require.

Solving a task is only the first step. Doing it efficiently is harder.

Today's AI systems perform well on familiar tasks but often struggle with new ones. This gap is highlighted by François Chollet's ARC-AGI benchmark suite (and subsequent ARC Prize competitions), in which each task is presented as a series of grids illustrating some specific transformation.

In this competition you work with tasks from the ARC-AGI public training set (v1) and build neural networks that reproduce each transformation. Your models must be:

  1. Correct — produce exact outputs across all tests.
  2. As small as possible — minimal parameter count and memory footprint.

You submit ONNX-formatted networks and aim to jointly minimize their size and parameter count. The objective is a network that solves each task with as few operations as possible.

Strong solutions could help define how many layers of computation these tasks actually require, serve as reference implementations, and support research into more adaptable AI systems.

Example (hypothetical task #000)

A .zip submission might include task000.onnx embodying a single-layer 3×3 convolutional network:

def weight(channel_out, channel_in, kernel_coord):
    if kernel_coord == ( 0, 0) and channel_in == channel_out: return 1.0
    if kernel_coord == ( 0, 0) and channel_in != 5 and channel_out == 0: return -1.0
    if kernel_coord == (-1, -1) and channel_in != 5 and channel_out == 0: return 1.0
    if kernel_coord == (-1, -1) and channel_in != 5 and channel_out == 5: return -1.0
    return 0.0

network = neurogolf_utils.single_layer_conv2d_network(weight, kernel_size=3)

Applied to a 30×30 image grid with channel depth ten, this network requires 900 parameters total.

Constraints

  • All tensors and parameters in each ONNX network file must have statically-defined shapes (so performance can be properly evaluated).
  • Disallowed ONNX operations: Loop, Scan, NonZero, Unique, Script, Function.
  • Each ONNX file is limited to at most 1.44 MB.
  • These constraints are checked automatically by an official network validator.

Evaluation

For any of the 400 tasks in the ARC-AGI public training v1 suite, your team earns a score of:

max(1, 25 - ln(cost))

for a functionally correct network, where cost is the sum of:

  • The total number of parameters in the network, plus
  • The total memory footprint of the network (in bytes).

Functional correctness is determined by validating the network against the original ARC-AGI benchmarks and a small private benchmark suite (to prevent overfitting). To earn points, your network must produce correct results across all tests.

Submission file

Submit a file named submission.zip containing at most one ONNX file per task:

task001.onnx
task002.onnx
...
task400.onnx

Note: if the metric needs adjustment — or additional ONNX operators must be banned — organizers will announce changes and rescore submissions.

Dataset Description

Create a suite of neural networks implementing transformations, each implicitly described by a series of image grids (e.g., rotation, cropping, magnification). Your network must achieve the desired result across all exemplars using the simplest possible architecture.

Task files

Each of the 400 tasks is stored in a named JSON file (e.g., task001.json). Each file is a dictionary with three fields:

  • train — input/output pairs originally in ARC-AGI-1 for training.
  • test — input/output pairs originally in ARC-AGI-1 for testing.
  • arc-gen — additional input/output pairs from the ARC-GEN-100K dataset.

A pair is a dictionary with "input" and "output" fields. A grid is a rectangular matrix (list of lists) of integers 0–9. Smallest grid is 1×1, largest is 30×30.

Input encoding

Before being passed to your networks, each input grid is converted to a tensor of size [BATCH_DIM=1, CHANNELS=10, HEIGHT=30, WIDTH=30], using:

  • A one-hot channel encoding for each colored pixel.
  • A zero-hot channel encoding for any "clear" pixels outside the original border.

Output construction

For all pairs in each example subset (train + test + arc-gen), your network must construct the correct output grid(s): fill each cell with a 1 for the correct channel and 0 for others (or 0 for all channels if the cell lies beyond the image border). Only exact solutions (all cells match) are correct.

The official scoring metric also uses a private dataset (fewer examples per task) when validating, to prevent overfitting.

Files

  • 401 files (json, py), 97.15 MB total, License Apache 2.0.
  • Includes a neurogolf_utils folder (helper code, e.g. single_layer_conv2d_network).
  • Example: task001.json contains train (5 pairs), test (1 pair), arc-gen (262 pairs).

Prizes — Total $50,000

  • First Prize: $12,000
  • Second Prize: $10,000
  • Third Prize: $10,000
  • Top Student Team: $8,000 (a "student team" has graduate/undergraduate students as ≥50% of membership).
  • Longest Leader: $10,000 — awarded to the team holding 1st place on the leaderboard for the longest period between May 6, 2026 00:00 UTC and July 15, 2026 23:59 UTC. (If the competition restarts, the Longest Leader window matches the new start/deadline.)

Timeline

  • April 15, 2026 — Start Date.
  • July 8, 2026 — Entry Deadline (accept rules before this date to compete).
  • July 8, 2026 — Team Merger Deadline (last day to join or merge teams).
  • July 15, 2026 — Final Submission Deadline.

All deadlines at 11:59 PM UTC unless noted. Organizers may update the timeline.

IJCAI-ECAI 2026

Part of the IJCAI-ECAI 2026 Competitions Track. Top submissions are invited to give talks at a special session at the conference in Bremen, Germany (attendance not required to participate; in-person attendees cover their own travel/expenses). Winning team members are invited to collaborate on a contest retrospective submitted to the IJCAI 2027 Demo Track.

Participation Stats (as of reading)

  • 5,940 entrants · 1,821 participants · 1,671 teams · 176,626 submissions.
  • Tags: Golf, Neural Networks, Artificial Intelligence, Optimization, Custom Metric.

Other Resources

  • arcprize.org — ARC Prize Foundation and its mission to accelerate AGI development.
  • onnx.ai — ONNX, an open format to represent machine learning models.

Citation

Michael D. Moffitt, Walter Reade, Ashley Oldacre, and Addison Howard. The 2026 NeuroGolf Championship. https://kaggle.com/competitions/neurogolf-2026, 2026. Kaggle.