Stop a long-lived process growing by the types and regexes it has read (Covered by dco/Brian_Egge.md)

Follow-up to morganstanley#549 and morganstanley#550, which bounded the fuzz harnesses against two
kinds of growth and left the library's own behaviour alone. This is the
library half.

1. The type memo is compacted where untrusted type descriptions are decoded.

hobbes::decode interns every type it builds in the process-wide type memo
(tctorMaps in lang/type.C), which holds a reference of its own to each
entry and lets go only when compactMTypeMemory() is called -- which, until
now, happened in exactly one place (eval/search.C). The RPC server
(ipc/net.C) decodes peer-supplied type descriptions and serialized
expressions on every prepare request, and the machine REPL (ipc/prepl.C)
decodes the same from its driver, and neither compacted, so a server handed
many distinct type descriptions grew by every one -- about 380 bytes per
distinct type, for the life of the process.

Compact after each request that decodes: net.C's prepare cases (0 and 1,
including when they fail, since a rejected description is the one that is
only garbage) and prepl.C's meta commands. What the request needed is held
by the compiler and stays; what it only passed through goes. The hot paths
-- invoking a prepared expression, invoking a compiled thunk -- decode
nothing and are left alone. A compaction walks the memo under its lock and
costs on the order of a decode, against a request that compiles an
expression.

unique_refc_map::get now returns its entry by value rather than by reference
into the map. The reference was handed out after the lock was released, and
compact() can erase that entry from another thread the moment it is -- a
latent hazard that periodic compaction from a server thread would turn into
a real one. The one caller (MonoType::makeType) copied the result
immediately anyway.

2. A compiled regex matcher is reused for the same regexes.

A regex literal is compiled into a matching function where it is read:
makeRegexFn determinizes it and defines the result under a fresh name
(".regex." + freshName()), and nothing removes that. morganstanley#550 measured about
83KB retained in the compiler per read of a twenty character regex. A
process that reads the same regex literals repeatedly -- the same expression
text arriving over RPC, a REPL session -- compiled a new matcher each time.

makeRegexFn now keeps, per compiler, the matchers it has compiled, by an
encoding of the regexes each one matches, and a match on the same regexes
reuses the function and result mapping compiled for them: the function is
pure in its input and the mapping is a property of the regexes, so neither
depends on where the match that uses them is. The capture buffer expression
and binding names are remade per call, because they are cheap and carry a
source location. The encoding is injective by construction -- every node
tagged, every name length-prefixed -- where show() is not: it prints
characters raw, so 'a|b' (either) and 'a\|b' (three literals) print alike
and must not share a matcher. Distinct regexes still each get a matcher;
that is compiling code, and is not a leak.

Measured as morganstanley#550 did, one compiler kept alive, cycles of 100 reads of the
OSS-Fuzz testcase's regex with the memo compacted between cycles:

  before (morganstanley#550's measurement)     +17.1MB per 100 reads, +8.3MB net of compaction
  after                            +0.47MB per 100 reads, +0.45MB net of compaction

The remainder is the syntax-error path of the parser retaining its source
text, which is documented in read/parser.C and is not touched here.

The full test suite passes. test/Matching.C checks that a reused matcher
matches as the first did, that the two regexes above do not share one, and
that captured groups still bind through a reused matcher.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016rGT4C394qeh2DBQhqy3Tb

Author: brianegge

include/hobbes/eval/cc.H

@@ -5,6 +5,7 @@
 #include <hobbes/eval/jitcc.H>
 #include <hobbes/eval/search.H>
 #include <hobbes/lang/expr.H>
+#include <hobbes/lang/pat/regex.H>
 #include <hobbes/lang/preds/subtype/obj.H>
 #include <hobbes/lang/tylift.H>
 #include <hobbes/lang/type.H>
@@ -361,6 +362,16 @@ private:
   bool shouldThrowOnHugeRegexDFA = false;
   int dfaOverNfaMaxRatio = 4;
 
+  // the regex matchers compiled into this compiler so far, by an encoding of
+  // the regexes each one matches. A match on the same regexes reuses the
+  // matcher already defined for them rather than defining another: every
+  // regex literal read is determinized and compiled into a function here, and
+  // nothing removes those, so this is what keeps a process that keeps reading
+  // the same regexes -- the same expression text over RPC, say -- from
+  // growing by a matcher per read (see makeRegexFn)
+  friend CRegexes makeRegexFn(cc *, const Regexes &, const LexicalAnnotation &);
+  std::unordered_map<std::string, CRegexes> regexFnCache;
+
   // the bound root type-def environment
   using TypeAliasMap = std::map<std::string, PolyTypePtr>;
 

include/hobbes/util/ptr.H

@@ -18,14 +18,19 @@ template <typename T, typename ... Args>
   public:
     using object_type = T;
 
-    const std::shared_ptr<T>& get(const std::function<T*(Args...)>& mk, const Args&... args) {
+    // returned by value rather than by reference into the map: compact() can
+    // erase an entry from another thread as soon as the lock is released, and
+    // a reference handed out here would then be to storage that is gone. A
+    // copy is a reference of its own, which also keeps the entry from being
+    // the one compact() erases while the caller still holds it.
+    std::shared_ptr<T> get(const std::function<T*(Args...)>& mk, const Args&... args) {
       std::lock_guard<std::recursive_mutex> lock(mutex);
       auto k = std::tuple<Args...>(args...);
-      const auto& r = this->values[k];
-      if (r) return r;
-      
-      this->values[k] = std::shared_ptr<T>(mk(args...));
-      return this->values[k];
+      auto& r = this->values[k];
+      if (!r) {
+        r = std::shared_ptr<T>(mk(args...));
+      }
+      return r;
     }
 
     size_t compact() {

lib/hobbes/ipc/net.C

@@ -259,6 +259,7 @@ void evaluateNetREPLRequest(int c, void *d) {
         fdwrite(c, uint8_t(0));
         fdwrite(c, std::string(ex.what()));
       }
+      compactMTypeMemory();
       break;
     case 1:
       // prepare a serialized expression, also return its type
@@ -286,6 +287,7 @@ void evaluateNetREPLRequest(int c, void *d) {
         fdwrite(c, uint8_t(0));
         fdwrite(c, std::string(ex.what()));
       }
+      compactMTypeMemory();
       break;
     case 2:
       // invoke a prepared expression

lib/hobbes/ipc/prepl.C

@@ -508,6 +508,16 @@ void runMachineREPLStep(cc* c) {
       resetMemoryPool();
       break;
     }
+
+    // the meta commands above decode types and expressions sent by the peer,
+    // and every type that builds is interned in the process-wide type memo,
+    // which holds on to it until asked not to. Ask after each one: what the
+    // command needed is held by the compiler and stays, what it only passed
+    // through goes. Invoking a compiled function (the default case) decodes
+    // nothing and is the hot path, so it is left alone.
+    if (cmd < CMD_COUNT) {
+      compactMTypeMemory();
+    }
   } catch (std::exception& ex) {
     std::string exn = ex.what();
     dbglog("*** " + exn);

lib/hobbes/lang/pat/regex.C

@@ -1345,6 +1345,58 @@ ExprPtr makeRegexCaptureBuffer(const Regexes& regexes, const LexicalAnnotation&
 /**************************
  * make a function to determine which among the input regexes here a later string matches
  **************************/
+// an encoding of a regex that two regexes share only if they are the same
+// regex: every node is tagged, and every name is length-prefixed, so the
+// encodings of different trees cannot run together. (show() is not that: it
+// prints characters raw, so 'a|b' and the three literals a, |, b look alike.)
+struct regexKeyF : public switchRegex<UnitV> {
+  std::string* out;
+  explicit regexKeyF(std::string* out) : out(out) { }
+
+  UnitV with(const REps*) const override { *this->out += "e;"; return unitv; }
+  UnitV with(const RCharRange* x) const override {
+    *this->out += "r" + str::from(static_cast<int>(x->b)) + "," + str::from(static_cast<int>(x->e)) + ";";
+    return unitv;
+  }
+  UnitV with(const RStar* x) const override {
+    *this->out += "*(";
+    switchOf(x->v, *this);
+    *this->out += ")";
+    return unitv;
+  }
+  UnitV with(const REither* x) const override {
+    *this->out += "|(";
+    switchOf(x->lhs, *this);
+    *this->out += ",";
+    switchOf(x->rhs, *this);
+    *this->out += ")";
+    return unitv;
+  }
+  UnitV with(const RSeq* x) const override {
+    *this->out += ".(";
+    switchOf(x->lhs, *this);
+    *this->out += ",";
+    switchOf(x->rhs, *this);
+    *this->out += ")";
+    return unitv;
+  }
+  UnitV with(const RBind* x) const override {
+    *this->out += "b" + str::from(x->var.size()) + ":" + x->var + "(";
+    switchOf(x->def, *this);
+    *this->out += ")";
+    return unitv;
+  }
+};
+
+std::string regexFnKey(const Regexes& regexes) {
+  std::string k;
+  for (const auto& r : regexes) {
+    switchOf(r, regexKeyF(&k));
+    k += "\n";
+  }
+  return k;
+}
+
 CRegexes makeRegexFn(cc* c, const Regexes& regexes, const LexicalAnnotation& rootLA) {
   CRegexes result;
 
@@ -1354,6 +1406,20 @@ CRegexes makeRegexFn(cc* c, const Regexes& regexes, const LexicalAnnotation& roo
     result.captureVarsAt[i] = bindingNames(regexes[i]);
   }
 
+  // if these regexes have been compiled into this compiler before, the
+  // function and the result mapping made then serve now: the function is
+  // pure in its input and the mapping is a property of the regexes, so
+  // neither depends on where the match that uses them is. (The capture
+  // buffer expression above is remade with this match's annotation, and the
+  // binding names recomputed, because they are cheap and carry a location.)
+  const std::string key = regexFnKey(regexes);
+  auto cached = c->regexFnCache.find(key);
+  if (cached != c->regexFnCache.end()) {
+    result.fname   = cached->second.fname;
+    result.rstates = cached->second.rstates;
+    return result;
+  }
+
   // our NFA will non-deterministically jump to every possible start state
   NFA nfa;
   nfa.resize(1);
@@ -1379,6 +1445,13 @@ CRegexes makeRegexFn(cc* c, const Regexes& regexes, const LexicalAnnotation& roo
 
   // and that's the function that the outer match logic should use
   result.fname = fname;
+
+  // remember the function and the result mapping for the next match on these
+  // regexes; not the capture buffer expression, which carries this match's
+  // source location and would pin it for the life of the compiler
+  CRegexes& kept = c->regexFnCache[key];
+  kept.fname   = result.fname;
+  kept.rstates = result.rstates;
   return result;
 }
 

test/Matching.C

@@ -517,6 +517,36 @@ TEST(Matching, hugeRegexDFACompilesWithoutQuadraticBlowup) {
 #endif
 }
 
+// A regex literal is compiled into a matching function where it is read, and
+// a match on the same regexes now reuses the function compiled for them the
+// first time rather than defining another. Reuse has to be by the regex
+// itself, not by how it prints: 'a|b' (either) and 'a\|b' (three literals)
+// print alike but match differently, and must not share a matcher.
+TEST(Matching, regexMatchersAreReusedButOnlyForTheSameRegex) {
+  auto either1 = c().compileFn<int(const std::string &)>(
+      "s", "match s with | 'a|b' -> 1 | _ -> 0");
+  auto either2 = c().compileFn<int(const std::string &)>(
+      "s", "match s with | 'a|b' -> 2 | _ -> 0");
+  auto literal = c().compileFn<int(const std::string &)>(
+      "s", "match s with | 'a\\|b' -> 3 | _ -> 0");
+
+  EXPECT_EQ(either1("a"), 1);
+  EXPECT_EQ(either1("b"), 1);
+  EXPECT_EQ(either1("a|b"), 0);
+  EXPECT_EQ(either2("a"), 2);
+  EXPECT_EQ(either2("a|b"), 0);
+  EXPECT_EQ(literal("a|b"), 3);
+  EXPECT_EQ(literal("a"), 0);
+
+  // and captured groups still bind in a reused matcher
+  auto cap1 = c().compileFn<long(const std::string &)>(
+      "x", "match x with | '(?<pre>a+)b' -> length(pre) | _ -> 0L");
+  auto cap2 = c().compileFn<long(const std::string &)>(
+      "x", "match x with | '(?<pre>a+)b' -> length(pre) * 10L | _ -> 0L");
+  EXPECT_EQ(cap1("aaab"), 3L);
+  EXPECT_EQ(cap2("aaab"), 30L);
+}
+
 TEST(Matching, noRaceInterpMatch) {
   c().alwaysLowerPrimMatchTables(true);
   c().buildInterpretedMatches(true);