Add minimal GPT-2 chat example

[Dobiasd]

Summary

Stacks on top of #466. Adds the pieces to run a small GPT-2 chat REPL in C++ on top of frugally-deep.

• keras_export/save_gpt2_backbone_for_fdeep.py — rebuilds keras_hub's GPT2Backbone (or GPT2CausalLM with --with-lm-head) as a plain Keras Functional model using only fdeep-supported primitives. Weights copied over; output matches keras_hub within float32 noise.
• keras_export/save_gpt2_weights_bin.py — dumps weights to a flat binary file.
• include/fdeep/llm/gpt2_bpe.hpp — byte-level BPE tokenizer. ASCII-focused pre-tokenization; round-trip safe; token IDs match keras_hub on the test set.
• include/fdeep/llm/gpt2_generator.hpp — slow-but-correct generation loop on top of fdeep::model::predict (full re-encode each step). Useful as a reference path.
• include/fdeep/llm/gpt2_cached.hpp — stateful Eigen-based GPT-2 forward with a per-layer K/V cache, loaded from the binary file. prefill() seeds the cache; step() advances by one token in roughly constant time.
• examples/gpt2_chat/ — stdin REPL with greedy / temperature / top-k sampling, opt-in via -DFDEEP_BUILD_GPT2_CHAT=ON.

The cached engine bypasses fdeep::model::predict because frugally-deep's runtime is stateless and a per-step KV cache fits a custom forward path more cleanly. The slow generator remains available for anyone who'd rather stay on the generic runtime.

Numbers (gpt2_base_en, single CPU thread)

• Cached engine load: 0.18 s
• Prefill: ~18 ms / prompt token
• Decode: ~18 ms / token sustained (~55 tok/s)
• vs slow re-encode path with seq_len=64: ~966 ms/token, identical token sequence at temperature 0 — ~55× speedup.

Caveats

• BPE pre-tokenizer is ASCII-focused (no \p{L}\p{N} regex). Encode/decode is byte-safe; non-Latin scripts simply won't split exactly the way HuggingFace's reference does.
• No top-p / repetition penalty yet.
• Prefill is per-token (no batched prefill matmul).

Test plan

• BPE round-trips on assorted sample strings
• BPE token IDs match keras_hub on the test set
• Cached engine produces the same greedy output as the slow gpt2_generator reference path on "Once upon a time, in a small village"
• Chat REPL produces coherent text on a couple of prompts (greedy and temperature=0.8 / top_k=40)
• Run the example through CI on a Linux runner (would need to wire up the model build step; left for follow-up since the model file is too large to include)

🤖 Generated with Claude Code

[Dobiasd]

Closing — landed as an experiment, not pursuing for merge.