Keep the parse-expr fuzz harness from growing without bound (Covered by dco/Brian_Egge.md)

OSS-Fuzz testcase 4850385077207040 reports fuzz-parse-expr running out of
memory, with no minimized reproducer and a 51 byte unminimized one:

  '66666\xaa6f.r6766#66c'anotpx.t!=1 \x82.x 0 f+\x99 -A\0"ot?pS

Replayed once it is nothing -- 11ms, a syntax error. Replayed repeatedly it
is the whole story: every read of it leaves the process about 168KB larger
than before, for good. ClusterFuzz could not minimize it because no single
input causes the failure; it is the harness, accumulating.

What accumulates comes from regex literals. The harness's own comment says
"this harness only parses -- nothing is compiled or evaluated", and the
second half is true; the first is not quite. A regex literal is turned into a
matching function where it is read: makeRegexFn determinizes it and defines
the result in the compiler under a fresh name (".regex." + freshName()), and
the types that takes -- the capture buffer, the function's own -- are
interned in the process-wide type memo (tctorMaps in lang/type.C), which
holds a reference of its own to each. Nothing removes the definition, and
only compactMTypeMemory() lets go of the types. Measured per readExpr,
retained, unsanitized:

  1+2                           0KB
  1 +        (syntax error)     0KB
  'abc'                        31KB
  'a.b.c.d.e'                 108KB
  the testcase above          168KB

Nothing but regex literals retains anything, and what a regex retains scales
with its DFA. Where it goes, measured on the testcase in cycles of 100 reads:

  100 reads                              +17.1MB
    compactMTypeMemory()                  -8.9MB   (the memo's half)
    destroy the cc                       -15.2MB   (its half, and its own)
    build a new cc                       +10.1MB
  net per cycle, doing both               +0.5MB

So roughly half is the memo and half is the compiler, and neither release
alone is enough: with the compiler alive the memo's half goes but the
compiler's stays, and with the compiler replaced but the memo left alone the
memo's half stays -- which is what a first version of this change did, and
under ASan on Linux it grew almost as fast as no change at all (2512MB against
3169MB at 8000 reads). Doing both holds it.

So the harness now does both, on a count of inputs: compact the memo every
64, as the decoder harnesses do, and replace the compiler every 1024 -- a
fraction of a second each time, amortised over enough inputs not to show.
Counts are a coarse measure of what has accumulated but a portable and
predictable one; resident size is neither, since freed memory is not handed
back to the operating system, and a version that watched RSS rebuilt the
compiler on every input once it first went over.

One detail of the replacement matters. The first compiler is built in the
slot's initializer, not on first use: constructing a cc also constructs the
LLVM statics it depends on, and everything static is destroyed at exit in
reverse order of construction, so a slot registered empty and filled later
would be destroyed after those statics and the cc in it would tear down
against an LLVM context that was already gone. That shape crashed at exit in
LLVMContext::removeModule; this one exits cleanly, as the original did.

Under ASan on Linux, 8000 copies of the testcase, RSS at 1024/2048/4096/8000:

  before:  1081 / 1388 / 2001 / 3169 MB -- climbing
  after:    915 / 1071 / 1108 / 1146 MB -- levelling off

This bounds the harness. Both halves of the growth are the library's, and a
host that reads regex literals from many distinct sources has the same
exposure: the memo's half is what the decoder harness change (morganstanley#549) also
works around, and the compiler's half -- a compiled matcher per regex literal
read, never reclaimed -- is left as it is, since making them reclaimable is a
design change rather than a fix. Both are worth knowing about.

The checked-in corpus replays cleanly. A local five minute fuzz run of the
new harness found a pre-existing stack overflow in dfaState's recursion on a
regex within the DFA-state cap, which is unrelated to this change and is
being handled separately. README and the harness's header comment updated to
say what the harness actually does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016rGT4C394qeh2DBQhqy3Tb

Author: brianegge

fuzz/README.md

@@ -50,10 +50,13 @@ Notes per harness:
   `hog` or the `Storage` tests. Malformed page metadata can make the reader
   attempt large mappings, so keep the default `-rss_limit_mb` in place, and a
   malformed page table can make it spin, so keep `-timeout` in place too.
-* **fuzz-parse-expr** — seeds in `corpus/parse-expr/`. The compiler instance
-  is constructed once and reused; parsing allocates from arenas that are not
-  reclaimed per-iteration, so run with `-detect_leaks=0` and let
-  `-rss_limit_mb` restart the process as needed.
+* **fuzz-parse-expr** — seeds in `corpus/parse-expr/`. Reading a regex
+  literal compiles a matcher into the compiler and interns its types in the
+  process-wide type memo, and nothing releases either, so a run that reads
+  regexes grows without bound unless the harness intervenes: it compacts the
+  memo every 64 inputs and replaces the compiler every 1024 (see the harness
+  for the figures). Parsing also allocates from arenas that are not reclaimed
+  per-iteration, so run with `-detect_leaks=0`.
 
 Replaying a reproducer (works in both build modes):
 

fuzz/fuzz_parse_expr.C

@@ -2,23 +2,81 @@
 //
 // Evaluating Hobbes code is running trusted native code, but *reading* source
 // text sits on the near side of that trust boundary: lexing and parsing must
-// be safe on arbitrary bytes. This harness only parses -- nothing is
-// compiled or evaluated.
+// be safe on arbitrary bytes. Nothing here is evaluated.
+//
+// Reading is not entirely free of compilation, though, and the difference
+// shows up over a campaign rather than on any one input. A regex literal is
+// turned into a matching function where it is read: makeRegexFn determinizes
+// it and defines the result in the compiler under a fresh name, and the types
+// that takes are interned in the process-wide type memo (tctorMaps in
+// lang/type.C), which holds a reference of its own to each of them. Nothing
+// removes the definition, and only compactMTypeMemory() lets go of the types.
+// So a `cc` reused across inputs, with the memo left alone, grows by every
+// regex it has ever read: measured at about 168KB per read of the twenty
+// character regex in OSS-Fuzz testcase 4850385077207040 -- roughly half of
+// it in the memo and half in the compiler -- against no growth that can be
+// measured at all for an input with no regex in it. Left alone, a campaign
+// feeding regexes runs the process out of memory. That is what the testcase
+// reports: an out-of-memory that ClusterFuzz could not minimize, because no
+// single input causes it.
+//
+// So two things are done periodically, one for each half. The type memo is
+// compacted every few dozen inputs, as the decoder harnesses do; that gives
+// back the memo's half and costs about as much as a parse. The compiler is
+// replaced every thousand or so; that gives back its half, and costs the
+// fraction of a second it takes to build one, amortised over enough inputs
+// not to show. Neither alone is enough: each leaves the other half growing
+// without bound. Counts of inputs are a coarse stand-in for how much has
+// accumulated, but a portable and predictable one -- resident size is not,
+// because freeing memory does not hand it back to the operating system, so a
+// harness that watched RSS rebuilt the compiler on every input once it first
+// went over.
 
 #include <hobbes/hobbes.H>
 
 #include <cstddef>
 #include <cstdint>
 #include <exception>
+#include <memory>
 #include <string>
 
 namespace {
 
-hobbes::cc& compiler() {
-  static hobbes::cc c;
+const unsigned long inputsPerCompaction = 64;
+const unsigned long inputsPerCompiler   = 1024;
+
+// The first compiler is built in the initializer rather than on first use.
+// Constructing a cc also constructs the LLVM statics it depends on, and at
+// exit everything static is destroyed in reverse order of construction: a slot
+// that was registered empty and filled afterwards would be destroyed after
+// those statics, and the cc inside it would tear down against an LLVM context
+// that was already gone.
+std::unique_ptr<hobbes::cc>& compilerSlot() {
+  static std::unique_ptr<hobbes::cc> c(new hobbes::cc());
   return c;
 }
 
+hobbes::cc& compiler() {
+  std::unique_ptr<hobbes::cc>& c = compilerSlot();
+  if (!c) {
+    c = std::unique_ptr<hobbes::cc>(new hobbes::cc());
+  }
+  return *c;
+}
+
+void reclaimPeriodically() {
+  static unsigned long read = 0;
+  ++read;
+  if (read % inputsPerCompiler == 0) {
+    compilerSlot().reset();
+  }
+  if (read % inputsPerCompaction == 0) {
+    // after the compiler is let go, where that happened, so that the types
+    // only it was holding are released too
+    hobbes::compactMTypeMemory();
+  }
+}
+
 } // namespace
 
 extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) {
@@ -28,5 +86,6 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) {
   } catch (const std::exception&) {
     // rejecting malformed source is the expected behavior
   }
+  reclaimPeriodically();
   return 0;
 }