repo-tokens

Repo Tokens

A GitHub Action that calculates the size of your codebase in terms of tokens and updates a badge in your README.

     

Usage

- uses: qwibitai/nanoclaw/repo-tokens@v1
  with:
    include: 'src/**/*.ts'
    exclude: 'src/**/*.test.ts'

This counts tokens using tiktoken and writes the result between HTML comment markers in your README:

The badge color reflects what percentage of an LLMs context window the codebase fills (context window size is configurable, defaults to 200k which is the size of Claude Opus). Green for under 30%, yellow-green for 30%-50%, yellow for 50%-70%, red for 70%+.

Why

Small codebases were always a good thing. With coding agents, there's now a huge advantage to having a codebase small enough that an agent can hold the full thing in context.

This badge gives some indication of how easy it will be to work with an agent on the codebase, and will hopefully be a visual reminder to avoid bloat.

Examples

Repos using repo-tokens:

Repo	Badge
NanoClaw

Full workflow example

name: Update token count

on:
  push:
    branches: [main]
    paths: ['src/**']

permissions:
  contents: write

jobs:
  update-tokens:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'

      - uses: qwibitai/nanoclaw/repo-tokens@v1
        id: tokens
        with:
          include: 'src/**/*.ts'
          exclude: 'src/**/*.test.ts'
          badge-path: '.github/badges/tokens.svg'

      - name: Commit if changed
        run: |
          git add README.md .github/badges/tokens.svg
          git diff --cached --quiet && exit 0
          git config user.name "github-actions[bot]"
          git config user.email "github-actions[bot]@users.noreply.github.com"
          git commit -m "docs: update token count to ${{ steps.tokens.outputs.badge }}"
          git push

README setup

Add markers where you want the token count text to appear:

<!-- token-count --><!-- /token-count -->

The action replaces everything between the markers with the token count.

Inputs

Input	Default	Description
include	required	Glob patterns for files to count (space-separated)
exclude	''	Glob patterns to exclude (space-separated)
context-window	200000	Context window size for percentage calculation
readme	README.md	Path to README file
encoding	cl100k_base	Tiktoken encoding name
marker	token-count	HTML comment marker name
badge-path	''	Path to write SVG badge (empty = no SVG)

Outputs

Output	Description
tokens	Total token count (e.g., 34940)
percentage	Percentage of context window (e.g., 17)
badge	The formatted text that was inserted (e.g., 34.9k tokens · 17% of context window)

How it works

Composite GitHub Action. Installs tiktoken, runs ~60 lines of inline Python. Takes about 10 seconds.

The action counts tokens and updates the README but does not commit. Your workflow decides the git strategy.