All notable changes to redb.Route will be documented in this file. This changelog covers the NuGet-published packages:
| Package | Description |
|---|---|
redb.Route |
Core engine: DSL, processors, expressions, telemetry |
redb.Route.Amqp |
AMQP 1.0 transport |
redb.Route.As2 |
AS2 (RFC 4130) B2B/EDI transport — signed/encrypted S/MIME over HTTP with MDN receipts |
redb.Route.AzureServiceBus |
Azure Service Bus transport |
redb.Route.Controllers |
Transport-agnostic controller dispatch |
redb.Route.Core |
Bridge to redb.Core props storage |
redb.Route.Elasticsearch |
Elasticsearch 8.x transport |
redb.Route.Exec |
Local process execution transport (exec: scheme) |
redb.Route.File |
File system transport |
redb.Route.Firebase |
Firebase (Firestore, Cloud Storage, FCM) transport |
redb.Route.Ftp |
FTP/FTPS transport |
redb.Route.GenericFile |
Base library for file-based transports |
redb.Route.Grpc |
gRPC transport |
redb.Route.Http |
HTTP/HTTPS transport |
redb.Route.Http.Hosting |
Shared Kestrel HTTP hosting (SharedHttpServerManager) reused by HTTP-based transports |
redb.Route.IbmMq |
IBM MQ transport |
redb.Route.Kafka |
Apache Kafka transport |
redb.Route.Ldap |
LDAP / Active Directory transport |
redb.Route.Llm |
LLM transport — universal OpenAI-compatible provider + native AnthropicProvider |
redb.Route.Llm.Abstractions |
LLM tool-capability contracts (ILlmToolDescriptor, LlmToolCapability, .AsLlmTool() DSL) |
redb.Route.Llm.Mcp |
MCP-client connector (mcp: scheme) — spawns external Model Context Protocol servers, auto-discovers tools via tools/list |
redb.Route.Llm.Tools |
Utility LLM tools — HttpFetch / JsonPath / XPath / MathEval / RegexExtract / Tavily web search |
redb.Route.Mail |
Email transport (SMTP, IMAP, POP3) |
redb.Route.MqttNet |
MQTT 5.0 transport |
redb.Route.Quartz |
Quartz.NET scheduling transport |
redb.Route.RabbitMQ |
RabbitMQ transport |
redb.Route.Redis |
Redis transport |
redb.Route.S3 |
AWS S3 / MinIO transport |
redb.Route.Sftp |
SFTP transport |
redb.Route.SignalR |
SignalR transport |
redb.Route.Sql |
SQL database transport |
redb.Route.Sqs |
Amazon SQS + SNS transport |
redb.Route.Tcp |
Raw TCP transport |
redb.Route.Telegram |
Telegram Bot transport |
redb.Route.Validation.Adapters |
FluentValidation + DataAnnotations adapters |
redb.Route.WebSocket |
WebSocket transport |
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
Note on version history: redb.Route has been running in production since version 1.0.0. Versions 1.0.0 – 1.0.3 were not published to NuGet (internal deployments only). The first public NuGet release is 1.0.4.
Why a minor.
.PropagateToolHeaders(...)/?propagateToolHeaders=is new public API, so this cannot be a patch. Everything else here is a fix, and existing routes are unchanged: the option is opt-in and empty by default. The ecosystem moves together —redbcore,redb.Tsakandredb.Identityship 3.6.0 alongside, which also puts the core back on the shared number after the 3.5.1 split.
- Fluent DSL builders — endpoint URIs now round-trip through the parser; space-bearing values no longer
corrupt (fixed a
Cron.Schedule(...)crash that took down the whole module). Every fluent builder wrote query values withHttpUtility.UrlEncode(application/x-www-form-urlencoded, where space →+), but the endpoint parser decodes withUri.UnescapeDataString(RFC 3986, where+stays+). The two are not inverse, so any value containing a space came back corrupted. Most visiblyCron.Schedule("job", "0 */5 * * * ?")— every cron expression has spaces — producedschedule=0+*/5+..., andCronEndpointOptions.Validate()then threw anArgumentExceptioninsidecontext.Start(), bringing down the entire module (all routes, not just the one job). The same latent mismatch affected SQL statements, LDAP filters, Exec arguments, LLM prompts and any other space-bearing DSL value. Fixed at the writer side across all builders (HttpUtility.UrlEncode→Uri.EscapeDataString, which writes%20and round-trips cleanly). The parser is deliberately untouched: hand-written endpoint URIs and existing routes are byte-for-byte unaffected, andSystem.Webis dropped as a dependency. Guarded by aBuild()→Parse()round-trip test so the invariant "write it the way the parser reads it" cannot silently regress again. redb.Route.Llm— conversation isolation no longer depends on tree-filter semantics (cross-conversation leak + tree corruption).RedbConversationStore.LoadPathAsyncdetected the conversation head withTreeQuery(root).WhereLeaves(), trusting the rooted query to scope the subtree. It did not: the core evaluated it as "freshest leaf of the whole schema" (fixed for Pro ind88ff9fe, still open for the Free PVT routing), so any turn of any conversation could load whoever wrote last globally — and the next message was then attached under that foreign node, corrupting_id_parentirreversibly. Head detection now goes through the conversation FK the connector already stamps (value_longon the indexed_objectsrow): the newest message of a conversation is always a leaf, soWhereRedb(o => o.ValueLong == rootId)ordered bydate_createis exactly "the freshest leaf" without depending on how a provider implements tree filters. The FK was always stamped correctly, so this also recovers already-corrupted trees on read. Three further defects in the same file, independent of the core:AppendAsync/LoadPathAsync(convId, leafId)resolved a message id across the whole schema, so an application that lets clients branch by message id could read — and write into — another conversation. The lookup is now scoped by the conversation FK and fails loudly.- The foreign conversation root slipped into the transcript as a role-less
MessageProps(the path was trimmed byid != rootId, which only recognises its own root), posting an empty role to the provider — a 400 that reads like a model problem. - The root-id cache was keyed by conversation id alone while the redb instance is chosen per
exchange (
?redb=), so with two named databases a root resolved in A was returned for B. Keyed by(instance, conversation)now.
redb.Route.Llm— tool routes now run on a child of the agent exchange, not on a naked one.AgentEngine.DispatchToolEndpointAsyncbuilt the tool's exchange from scratch (new Message(...)+IProducerTemplate.RequestBody), so a route mounted with.AsLlmTool(...)/[ExposeAsLlmTool]/ MCP received noProperties(includingLlmKeys.RedbName), noRouteIdand a fresh DI scope from the root container instead of the conversation's. Every scoped service the agent route had resolved — principal, tenant accessor, per-exchangeIRedbService— came back empty inside the tool. Only the ambient transaction survived, because it flows through the async context. Dispatch now usesparentExchange.CreateLinkedChild(msg)+IProducerTemplate.RequestAsync, which is whatdocs/LLM/PLAN.md §4.1specified and whatILlmToolDescriptor/RouteToolBridgedocumented. The child shares the parent's scope without owning it, so it releases only the__redb_scope:*entries the tool itself opened and the conversation keeps its scope after the call. Tool dispatch is sequential within an iteration, so the shared scope carries no concurrency risk.redb.Route.Llm— the run's principal and audit tags reach tools. The engine copied a hard-coded three-header allowlist (llm.conversation.id,X-Correlation-Id,CorrelationId) and nothing else, so a "who is asking" tool had no way to learn the subject except asking the model for it — an argument the model can be talked into forging.llm.user.idandllm.audit.*are now propagated as the values resolved for the run, not as raw headers:?user=${header.X-User-Id}and?audit=reach tools too, which a header copy would have missed.
-
redb.Route.Llm—.PropagateToolHeaders(...)/?propagateToolHeaders=. Opt-in list of extra header names forwarded from the agent exchange to every tool call; a trailing*makes an entry a prefix match. Also on the inline step asLlmCallBuilder.WithPropagatedToolHeaders(...)..To(Llm.Factory("claude") .Tools("profile_state,billing_check") .User("${header.X-User-Id}") .PropagateToolHeaders("x-tenant-id", "accept-language"))
Propagation stays default-deny — the inbound transport's header set is never forwarded implicitly, so an HTTP consumer's
Authorization/Cookiecannot ride into a tool by accident. The policy lives in the new publicToolHeaderPolicy, andAgentRequest.PropagateToolHeaderscarries it to the engine.Note the trust model this assumes:
llm.user.idandllm.audit.*are read off the inbound exchange byLlmProducer, so a route that forwards client headers verbatim lets the caller set them. That was already true for what the engine persists inMessageProps.UserId; a route whose tools make access decisions on the principal must strip or overwrite client-suppliedllm.*headers before thellm://hop.
Why a separate patch instead of shipping inside 3.5.0. The AS2 connector and the shared Kestrel host landed on
developafter 3.5.0 had already been published to nuget.org (2026-08-05, 61 packages). A published version cannot be replaced, so these two packages — and theredb.Route.Httpbuild that goes with them — ship as 3.5.1.
redb.Route.Httpis republished here on purpose.SharedHttpServerManagermoved out of it intoredb.Route.Http.Hosting, so the 3.5.0 build on nuget.org still carries its own private copy of the multiplexing server. Mixing that 3.5.0 withredb.Route.As2would put two Kestrel managers in one process — andredb.Route.As2does not referenceredb.Route.Http, so NuGet would never upgrade it for you. Takeredb.Route.Http3.5.1 whenever you use AS2 on a shared port; the public API of the package is unchanged either way.The whole
redb.Routeline moves to 3.5.1 together, so no combination of 3.5.x Route packages can mix the two hosting models.redbcore,redb.Tsakandredb.Identitystay on 3.5.0 — the shared-runtime compat gate compares the minor, and 3.5.0 ↔ 3.5.1 is patch drift, which it allows.
redb.Route.As2— new AS2 (RFC 4130) B2B/EDI connector. Exchange business documents with trading partners over HTTP(S) using signed and encrypted S/MIME messages and MDN receipts. Schemesas2/as2s. Both directions, synchronous and asynchronous MDN, signed MDN, and the standard signature / encryption algorithm matrix (sha-1/256/384/512×aes-128/192/256-cbc,3des, optional RFC 3274 compression). Crypto is MimeKit (Bouncy Castle) — the same foundation the AS2 industry interoperates on.services.AddRedbRouteAs2(); context.AddToRegistry("walmart", new As2ConnectionFactory { OurCertificate = ourPfx, PartnerCertificate = theirCer, As2From = "OUR-ID", As2To = "WALMART-ID", PartnerUrl = "https://partner/as2", Sign = true, Encrypt = true, SignedMdn = true, MdnMode = As2MdnMode.Sync }); From(As2.Receive("/inbound").Host("0.0.0.0").Port(4080).ConnectionFactory("walmart")) .To("direct://process-edi"); // receive server From("direct://out") .To(As2.Send("https://partner/as2").ConnectionFactory("walmart")); // send + verify MDN
- Partner config via
As2ConnectionFactoryin the registry — certificates, AS2 IDs and the agreed profile referenced by.ConnectionFactory("name"), never in the URI. - MDN outcome on
exchange.Outfor a producer:redbAs2.mdnDisposition,redbAs2.signatureValid,redbAs2.mdnMicMatch(the partner received exactly what we sent). Async MDN usesAs2.ReceiveMdn(path)with correlation byOriginal-Message-ID. - Received document on the consumer: business content type on
Message.ContentType(the S/MIME wrapper type is deliberately kept off the headers), AS2 headers verbatim, computed MIC underredbAs2.mic. - Full connector parity —
IEndpointStatistics/ health in Tsak,Consumer/Clienttelemetry spans,[Sensitive]secret redaction. - Interop validated against a live OpenAS2 v4.9.0 in both directions (
redb → OpenAS2andOpenAS2 → redb, signed + encrypted, positive MDN, MIC verified). Seesrc/redb.Route.As2/TESTING.md.
- Partner config via
redb.Route.Http.Hosting— new shared Kestrel hosting package.SharedHttpServerManager(the multiplexing HTTP server, one Kestrel perhost:port) was extracted fromredb.Route.Httpinto a standalone package so HTTP-based connectors (redb.Route.Http,redb.Route.As2, …) share one server without depending on each other. Register once withservices.AddRedbRouteHttpHosting()(idempotent); every connector resolves the same singleton. No breaking change — the types keep theredb.Route.Httpnamespace, soredb.Route.Httpsource and public API are unchanged.
Why a minor (3.4 → 3.5). This release adds public API: the Message History, XSLT and Routing Slip EIPs, and
{{key}}/{{key:default}}property placeholders in endpoint URIs. Backward-compatible — existing routes are unchanged, and every new feature is opt-in (Message History is off until.MessageHistory()orEnableMessageHistory). The ecosystem number moves together:redbcore,redb.Tsakandredb.Identityall ship 3.5.0 alongside — see the rootCHANGELOG.mdfor why the runtime cannot stay a minor behind the framework.The AS2 connector and the shared Kestrel host were written against this version but merged after it was published; they ship in 3.5.1 — see above.
- Message History EIP —
.MessageHistory()/RouteEngineOptions.EnableMessageHistory. Records the trail an exchange takes through a route: each node is timed and appended toexchange.Properties["CamelMessageHistory"]as aMessageHistoryEntry(route id, node id, label, elapsed ms), and the trail is dumped on failure (when retries are exhausted) so the failing step is visible. Apache Camel parity: disabled by default (slight overhead), enabled globally (EnableMessageHistory) or per route with.MessageHistory(bool)— a route override wins over the global setting. Read the trail withMessageHistory.GetEntries(exchange)/ render it withMessageHistory.Format(exchange). The elapsed time is recorded even when a node throws.From("direct://orders").MessageHistory() .Process(validate) .To("http://fulfil"); // on failure the log shows: routeId · to1 · to · 12.400 (etc.)
- XSLT transformation —
.Xslt(...)/.XsltContent(...)and thexslt:component. Transforms the exchange body through an XSLT stylesheet, Apache Camelxslt:parity. Modelled exactly on the built-in validator (leaf DSL and a component, engine behind an interface): the engine isIXsltEnginewith a built-inXslCompiledTransformEngine(BCLSystem.Xml.Xsl, XSLT 1.0, zero external dependencies — same as Camel's default JAXP engine; a Saxon-backed engine for 2.0/3.0 can be added later as anotherIXsltEngine, exactly as third-party validators live inredb.Route.Validation.Adapters). The stylesheet is compiled once when the route is built and reused.From("direct://orders").Xslt("styles/orders.xsl"); // from a file (imports resolve) From("direct://orders").XsltContent(inlineStylesheet, XsltOutput.Bytes); From("direct://orders").To("xslt:styles/orders.xsl?output=bytes"); // component, registered out of the box
- Parameters — all message headers and exchange properties are passed to the stylesheet as
xsl:param(a stylesheet sees the ones it declares), matching Camel. - Input body —
string,byte[],Stream,XmlReader,XDocument/XElement, orIXPathNavigable. output—string(default) /bytes/dom.failOnNullBody(default true).allowTemplateFromHeader— when set, a per-message stylesheet from theCamelXsltResourceUri/CamelXsltStylesheetheader overrides the compiled default (dynamic stylesheets are compiled once and cached).- The
xslt:component is registered out of the box —.To("xslt:...")needs no setup, and it composes as an endpoint (a Routing Slip / Recipient List can targetxslt:). - Security: script and
document()are disabled (BCLXsltSettings.Default) for stylesheets from outside the app. Content-Typeis left unchanged after the transform (matching Camel).output=fileis intentionally not provided — route the transformed body to the File connector (.To("file:...")) instead.
- Parameters — all message headers and exchange properties are passed to the stylesheet as
- Routing Slip EIP —
.RoutingSlip(...). Pipes the exchange through a list of endpoints that is computed once up front (contrast the Dynamic Router, which recomputes the next hop after every step). The slip may be a factory delegate, anIExpressionyielding a delimited URI string, or a${...}template — Apache Camel parity, including the default,delimiter andignoreInvalidEndpoints. The same exchange is threaded through the endpoints in sequence with Pipeline semantics (each hop'sOutbecomes the next hop'sIn; the finalOutis left for an InOut caller), and the current endpoint is exposed on the exchange propertyCamelSlipEndpoint. Producers are cached per URI.From("direct://orders") .RoutingSlip("${header.route}") // e.g. "direct://validate,direct://enrich,amqp://out" // or a factory: From("direct://orders") .RoutingSlip(ex => new[] { "direct://validate", "direct://enrich", "amqp://out" });
- Property placeholders in endpoint URIs —
{{key}}/{{key:default}}. Apache CamelPropertiesComponentparity: endpoint URIs (and any string routed throughGetEndpoint) externalise their configuration instead of hard-coding hosts, queue names, ports or paths. Placeholders are resolved once at route-compile time — values come fromIConfiguration(environment variables, appsettings, user-secrets — with the container's own precedence, so no Camel-styleenv:/sys:prefix functions are needed), then the context's own properties (SetProperty) as a container-free fallback. A placeholder with neither a value nor a default fails fast at compile time. Resolution is single-pass and a URI without{{is untouched, so existing routes are unaffected.Deliberately minimal (matching the redb.Route "no gold-plating" line): no nested placeholders, noFrom("{{orders.source}}") // e.g. amqp://broker/orders, from appsettings .To("http://api/{{tenant}}/pay"); From("amqp://broker:{{amqp.port:5672}}/orders"); // default when the key is not configured
{{?optional}}parameter removal, no encryption —IConfigurationalready covers the layered-source need.PropertyPlaceholderResolveris public for reuse. redb.Route.IbmMq— event-driven receive via XMSMessageListener, cutting delivery latency from ~250–500 ms to single-digit ms. The default consumer polls with a blocking MQGET-WAIT on the IBM.WMQ managed client, which carries an internal ~500 ms tick independent ofwaitInterval— so an idle/low-traffic queue delivered messages ~250–500 ms after they arrived. The managed IBM.WMQ client exposes no public async-consume API (there is noMQQueue.Cb/MQQueueManager.Ctlin any 9.4.x — verified by reflection), so the fix uses XMS .NET (IBM.XMS), the JMS-style client that does support event-drivenMessageListenerpush. New opt-in optionreceiveMode=listener(default stayspoll) activates it; measured publish→receive on a local broker dropped to ~2–6 ms.The listener runs the route synchronously on the XMS dispatch thread — one session = one in-flight message = natural back-pressure. The package reference changed fromr.From("wmq:ORDERS?queueManager=QM1&channel=DEV.APP.SVRCONN&receiveMode=listener")
IBMMQDotnetClientto its supersetIBMXMSDotnetClient(sameamqmdnetstd.dllfor IBM.WMQ +amqmxmsstd.dllfor IBM.XMS; same vendor and version), so the default poll path is byte-for-byte unchanged.- Transacted listener (
receiveMode=listener&transacted=true) uses a JMSSESSION_TRANSACTEDsession and commits/rolls back on the delivering session — a processing error rolls back and the broker redelivers, exactly like the poll path's connection syncpoint. AnIbmMqXmsAckActionis registered on the exchange so a route-level.Transaction()block settles it as part of the route unit-of-work (mirroring the poll path'sIbmMqAckAction); if the route has no transaction block the engine settles the session itself (commit on success, rollback on error). The action is idempotent, so the two paths never double-settle. concurrentConsumers=Ncreates N XMS sessions on the shared connection — N competing consumers processing in parallel (each session single-threaded, queue-manager load-balanced), one in-flight per session for back-pressure. A topic clamps to a single subscriber (parallel subscriptions would duplicate delivery), same rule as the poll path.- Request-reply, backout threshold and W3C trace propagation all work on the listener path, at
full parity with the poll path. A request carrying
JMSReplyTogets anInOutexchange and theOutbody is sent back correlated by message id (non-persistent, so temporary reply queues accept it); a message pastbackoutThresholdis copied tobackoutQueue; andtraceparent/tracestateon the message continue the distributed trace. RPC reply and the backout copy are issued on the delivering session, so undertransactedthey commit atomically with consuming the request. - RPC client leg is event-driven too. With
receiveMode=listenerthe producer's request-reply now receives the response through an XMSMessageListeneron the reply queue instead of the poll loop that carried the ~500 ms tick, so the whole round-trip is fast (measured ~16 ms warm vs the poll floor of ~250–500 ms). The request is still sent over IBM.WMQ; only reply reception moves to XMS. A dynamic reply queue uses an XMS temporary queue (owned and consumed by the producer's own connection); a configuredreplyToQueueis used as-is. - Header parity on the listener path. The XMS consumer and the event-driven RPC reply now carry
the same headers as the poll path: the
redbIbmMq.*MQMD metadata (Destination, QueueManager, MsgId, CorrelId, Priority, BackoutCount — mapped from the JMS/MQMD properties, gated bymqmdReadEnabledlike poll) and the application (user) headers. - Fluent
.Listener()/.ReceiveMode(...)on the builder mirror thereceiveModeURI option.
- Transacted listener (
redb.Route.IbmMq— user-header catalogue moved to the JMSusrfolder (interoperability fix). User headers are carried as a single JSON property (MQ property names forbid hyphens, soX-Custom-Idcan't be an individual property). That property was written under a dotted name (redbIbmMq.HeaderKeys), which MQ places in a custom MQRFH2 folder that JMS clients — including IBM.XMS and any other JMS consumer — do not surface as a user property, so headers were invisible off the IBM.WMQ path. It now uses an unqualified name (redbIbmMqHeaders) that lands in the standardusrfolder, readable by IBM.WMQ, IBM.XMS and any JMS client alike. Wire-format note: a producer on this version and a consumer on an older one (or vice-versa) will not exchange user headers across the change — deploy producer and consumer of the IBM MQ connector together. MQMD metadata and message bodies are unaffected.
- Internal: unified the scope-body pipeline builder (
NodePipeline). Every scope definition (Choice, Filter, CircuitBreaker, TryCatch, Loop, Metered, Replayable, Aggregate, IdempotentConsumer, Resequence, Threads, Traced, Transaction, Throttle, Debounce, Split) carried its own identical copy of the "0 → no-op / 1 → the child / N → a pipeline" logic; these now route through a singleNodePipeline.Body/Node. Behaviour is unchanged (verified by the full suite, incl. 70 transacted tests), and it provides the single compile-time seam that Message History (above) decorates each node through. No public API or DSL change.
redb.Route.Http— the fluentHttp.Listen("/path").Host(..).Port(..)dropped the first path segment. The builder emits host/port as query parameters (http:/api/honest/echo?host=..&port=..), so the whole leading-slash path is the route — butHttpEndpoint.ConsumerPathassumed any leading-slash path was the nonstandard/host:port/routeform and stripped its first segment, registering/honest/echoinstead of/api/honest/echo(and/webhookcollapsed to/). It now strips the first segment only when it is an actual embedded host (recognised by a:in that segment — route segments have none), soHttp.Listen("/api/honest/echo").Host("0.0.0.0").Port(5092)registers the full path. The equivalent URI DSL (http:0.0.0.0:5092/api/honest/echo) was already correct and is unchanged.
Why a minor bump (3.3 → 3.4). This release adds public API surface, not just fixes: replay checkpoints (
.Replayable,IExchange.Snapshot,IRouteContext.ReplayAsync/GetReplayMarkers), the reusable scope-nesting validation primitives (ICompositeScope/IDurableScope/IScopeNestingRule/IBranchingDefinition), namedConnectionFactoryon every connector, andProducerTemplateexchange-typed overloads — alongside the endpoint-URI secret-redaction security hardening. The whole ecosystem ships at 3.4.0. Backward-compatible: existing routes are unchanged.
- Endpoint-URI secrets no longer leak into logs, telemetry, health checks, or the Tsak
dashboard. Credentials carried in an endpoint URI — a query-parameter (
?password=,?bindPassword=,?sessionToken=,?saslPassword=,?connectionString=, …) or a userinfo password (amqp://user:pass@host) — were being written in cleartext at route-build and endpoint-start (Compiled route …,Endpoint … started), in the OpenTelemetryredb.route.endpointspan tag and metric label, in the inflight-exchange and health-check metadata, and in theCompiledRoute.FromUriDTO that the Tsak CLI/dashboard render. The masking that did exist was bypassed by these raw-string paths and, where it ran, (a) disclosed the first two characters of every secret and (b) used an exact-match deny-list of eight names — sobindPassword,sessionToken,sslKeyPassword,authToken,clientSecret,privateKeyPassphrase,sharedAccessKey, and userinfo passwords all slipped through. Fixes:- Full redaction to a constant
****— no more first-two-character disclosure. - Substring-based secret detection replacing the exact-match list (covers the names above and
any
*password*/*token*/*secret*/*apikey*/*credential*variant); benign params likeroutingKey/partitionKey/clientId/usernameare deliberately never masked. - Userinfo passwords are masked (
user:pass@host→user:****@host) — previously outside the masker entirely. - Every core log / telemetry / metric / health-check / DTO boundary is routed through a new
format-preserving
EndpointUri.Sanitize(string)(keeps scheme,://, path, param order, and non-secret values byte-for-byte). Unnamed routes now derive a sanitized route id, so a secret can no longer surface through{RouteId}log lines. redb.Route.Elasticsearch— theNodes=startup log now sanitizes each node URL (userinfo).redb.Route.Exec— the debugexec →line logs the executable and argument count only; argument values (which routinely carry secrets on the command line) are no longer emitted.- Producer/consumer start-stop lines that printed the raw endpoint key are sanitized —
GenericFileProducer(the base for FTP/SFTP, whose credentials live in query parameters, so?password=was reaching the log verbatim) plus the in-process Direct / SEDA / Mock / Log components. - New public API on
EndpointUri:Sanitize(string),IsSensitiveKey(string), andAddSensitiveKeys(params string[])for connectors to register non-standard secret parameter names (analogous to Camel'saddSanitizeKeywords). [Sensitive]on an endpoint option is now the source of truth for what gets redacted. Guessing a secret from its parameter name fails open — that is exactly howbindPassword,sessionTokenandsslKeyPasswordleaked: they were credentials nobody had put on the list. An option marked[Sensitive]is redacted because it was declared one:public string Server { get; set; } = "localhost"; // printed in logs [Sensitive] public string? BindPassword { get; set; } // always ****
EndpointOptions.BindFromUriharvests those declarations by reflection (once per options type) and feeds them intoEndpointUri.AddSensitiveKeys, so the keyword set is derived from the code, never hand-maintained and a newly added credential option cannot be forgotten. This mirrors Apache Camel, where@UriParam(secret = true)is the declaration and the runtime list (SensitiveUtils.SENSITIVE_KEYS) is generated from those annotations by a build plugin; the .NET version needs no build step, only reflection. All 37 credential options across 22 connectors are annotated. The name-keyword heuristic remains as a backstop for a URI rendered before any endpoint of that scheme has been created.- Display-only change: routing identity, endpoint cache keys (
NormalizedKey/BaseKey), and message flow are unaffected.CompiledRoute.FromUriis now a redacted display value and must not be re-parsed to recover credentials.
- Full redaction to a constant
redb.Route.Ldap—connectionFactoryis now actually resolved: newLdapConnectionFactory.LdapBuilder.ConnectionFactory()andLdapEndpointOptions.ConnectionFactoryexisted since 3.3.x but nothing ever read them —LdapEndpointtookBindDn/BindPasswordstraight off the URI, so a service-account password had to be written into the route and from there reached logs and the dashboard.LdapEndpointnow resolves the namedLdapConnectionFactoryfrom the route registry and fills in every connection/credential option the URI did not set (an explicit URI value still wins, so existing routes are unchanged; a missing factory logs a warning and falls back to URI parameters). A route can now carry no credentials at all:context.AddToRegistry("honest-ldap", new LdapConnectionFactory { Server = "ldap.corp.local", Port = 636, Ssl = true, BindDn = "cn=svc-reader,dc=corp,dc=local", BindPassword = Environment.GetEnvironmentVariable("LDAP_BIND_PASSWORD") }); r.From("ldap://SEARCH:dc=corp,dc=local?connectionFactory=honest-ldap&filter=(objectClass=user)")
- Secrets embedded in exception messages are redacted before logging.
OnExceptionProcessorlogsex.Messageon redelivery and retries-exhausted; a driver exception can carry a connection string (...;Password=…;…). Those two sites now run the message through the newEndpointUri.RedactSecrets(string), which maskskey=valuesecret assignments inside arbitrary text while preserving everything else. Note: whenLogStackTraceis enabled the exception object itself is handed to the logger and cannot be scrubbed in-process — useRedactSecretsin a logging-sink filter for that path.
- Replay checkpoints —
.Replayable("name")save-points. A named point in a route that snapshots the exchange as it passes, so the tail of the route (everything after the marker) can be re-run later from that frozen state — e.g. the platform replaying a failed exchange from the last successful step instead of from the mangled current state. The capturedRouteCheckpointlands inexchange.Properties["route.checkpoint"](last marker wins) and replay is a typed in-process callIRouteContext.ReplayAsync(routeId, markerName, snapshot). Full developer guide (incl. the Tsak-integration contract):docs/REPLAY_CHECKPOINTS_GUIDE.md.From("timer://poll?period=5000") .Process(chargeCard) .Replayable("after-charge") // save-point: card already charged .Process(sendReceipt) .To("http://receipts") .EndReplayable();
IExchange.Snapshot()/IMessage.Snapshot()— a deep, isolated copy distinct fromClone(): the body is deep-copied so the captured state is frozen against later in-place mutation (whereasClone()intentionally shares the body — relied upon by e.g. Splitter aggregation). v1 handles immutable /byte[]/ICloneablebodies and throws loudly otherwise (never a silent shallow share). A snapshot carries no DI scope (dormant data — no per-message leak). Also corrected the misleading "Deep copy" doc onClone().exposed: trueadditionally publishes the marker asdirect:__replay:{routeId}:{name}so other routes can.To(...)it; the default (exposed: false) is reachable only viaReplayAsync.IRouteContext.ReplayAsync/GetReplayMarkersandProducerTemplateexchange-typed overloads (Send/SendAsync/RequestAsync(..., IExchange, ct), caller-owned) round out the API.- A non-snapshot-able body degrades gracefully (warn, no capture) — checkpoints never break the
happy path. Routing identity, cache keys, and existing
Clone()behaviour are unchanged.
- Reusable scope-nesting validation. A general mechanism (not an ad-hoc per-type check) for
expressing where a definition may/may not nest:
ICompositeScope/IDurableScopescope-category markers,IScopeNestingRule(a node declaresAllowed/Warn/Forbidagainst an ancestor category), andIBranchingDefinition(definitions whose children live outsideOutputs— Choice When/Otherwise, TryCatch catch/finally — expose them for a generic tree-walk). The validator applies the rules generically:Forbid→ build error,Warn→ log. First use: a replay checkpoint may not cross a branching composite (build error) and warns inside a durable transaction (replay runs outside it). New structural constraints ship on the definition, never in the validator. - Named
ConnectionFactoryfor connectors that previously had no way to keep credentials out of the endpoint URI. Registered in the route registry and referenced by name (?connectionFactory=my-bot), so the secret never enters the URI at all — nothing to mask in logs, telemetry, or the dashboard. Each factory fills only the options the URI did not set explicitly, so an inline URI value always wins and existing routes are unchanged; a name that is not in the registry logs a warning and falls back to URI parameters. Rolled out to all ten connectors that previously had no such mechanism: Telegram (TelegramConnectionFactory— bot token; token-less DSL overloadsTg.Receive().ConnectionFactory("bot")), MqttNet (MqttConnectionFactory— broker address + username/password/TLS), Http (HttpConnectionFactory— Basic/Bearer credentials, TLS certificate password, timeout;AuthTokensupports${...}expressions exactly like the URI form; the request address deliberately stays in the URI path so a factory can never silently redirect a route), Mail (MailConnectionFactory— one factory shared by SMTP / IMAP / POP3: mailbox username/password, OAuth2 access token, auth mechanism, transport security and client certificate; a host taken from the URI path is never overridden by the factory), Ftp / Sftp (FtpConnectionFactory/SftpConnectionFactoryover a sharedRemoteFileConnectionFactorybase inredb.Route.GenericFile— host/port/username/password and timeouts in the base, FTPS settings for FTP, private-key path + passphrase, host-key checking and proxy credentials for SFTP), SignalR (SignalRConnectionFactory— hub access token, transport and TLS material), and Grpc / Tcp / WebSocket (GrpcConnectionFactory/TcpConnectionFactory/WsConnectionFactory— TLS certificate password and connect timeouts). For the connectors whose address lives in the endpoint path (Http, Mail, SignalR, Grpc, Tcp, WebSocket) the factory deliberately carries no host/port, so it can never silently redirect a route; thewssscheme likewise still forces TLS on regardless of the factory. A fluent.ConnectionFactory("name")was added to every builder that has one (Telegram also gains token-less mode overloads); SignalR and WebSocket are URI-only and unchanged in that respect.context.AddToRegistry("support-bot", new TelegramConnectionFactory { Token = Environment.GetEnvironmentVariable("TELEGRAM_TOKEN")! }); r.From("telegram://receive?connectionFactory=support-bot") // no token in the route
redb.Route.Telegram— reply target as a first-class producer option:replyToMessageId(expression-capable) with fluentReplyTo(long)/ReplyTo(IExpression)/.ReplyToIncoming(). Replying to the message that triggered the exchange previously required a manual.Processstep copyingtelegram.messageIdintotelegram.replyToMessageId. The option accepts a constant id or a${...}expression resolved per message;.ReplyToIncoming()is sugar forreplyToMessageId=${header.telegram.messageId}. An explicittelegram.replyToMessageIdheader still wins; an expression that resolves to nothing sends the message unthreaded. A constant that is not a message id fails validation at endpoint start. Applies tosend/document/photo.redb.Route.Telegram— edit/delete target as an option:messageId(expression-capable) with fluentMessageId(long)/MessageId(IExpression). Chaining send → edit previously required a manual copy oftelegram.sentMessageIdintotelegram.messageId; nowTg.Edit(token).MessageId(Header(TelegramHeaders.SentMessageId))does it. The header still wins.redb.Route.Telegram—answermode supportsshowAlert(URI option,.ShowAlert()fluent, per-messagetelegram.showAlertheader): the callback answer is shown as a modal alert instead of a toast.redb.Route.Telegram— per-messagetelegram.captionheader fordocument/photo, wins over thecaptionoption (consistent withparseMode/fileName).redb.Route.Telegram— Mini App payloads (WebApp.sendData) are now surfaced: headerstelegram.webAppData/telegram.webAppButtonText. Aweb_app_datamessage carries no text, so the consumer previously handed such an update to the route with an empty body and no way to reach the payload short of parsing the rawUpdate. The data now becomes the exchange body — same contract as text messages and callback queries — and is also exposed as a header. Applies to both the long-polling and webhook paths (sharedTelegramUpdateMapper);telegram.messageTypeis"WebAppData"for filtering.
redb.Route.Telegram—document/photomodes silently ignoredtelegram.replyToMessageIdandtelegram.replyMarkup. Both headers were documented in the producer-headers table but only wired intosend, so a photo with inline buttons or a document sent as a reply lost its markup / reply target. Both modes now pass them to the Bot API.
Why the bump. No functional changes to redb.Route — this is an ecosystem sync release, and the whole family is published at 3.3.3 (all 34 packages).
Two reasons:
- The family had drifted apart. Base sat at 3.3.1 while
SqlandSqswere at 3.3.2 from the partial release below — so a shared-layer install mixed3.3.1and3.3.2archives side by side, and "which versions go together" needed a table. From 3.3.3 every package in the ecosystem — redb core, redb.Route, redb.Tsak — ships one number.- It picks up
redb.Core3.3.3. Not every package depends on redb storage —redb.Routeitself and the transports (Kafka, RabbitMQ, …) do not. Butredb.Route.Coreandredb.Route.Llmdo, and at 3.3.1 both pinnedredb.Core3.3.0, whose embeddedredb_init.sqlfailed schema initialization under a non-superuser database owner (see the redb core changelog). Sinceredb.Route.Coreis the package a redb-backed route worker is built on, that broken init reached any deployment on a least-privilege database. All packages are rebuilt againstredb.Core3.3.3.The
Sql/Sqsfeatures listed under 3.3.2 below are not re-announced here — they shipped in 3.3.2 and are unchanged; those packages only change number.Also from this release, the shared assembly layer is distributed as one archive per OS (
redb-route-shared-3.3.3-<rid>.zip, all 32 connectors inside) instead of one archive per connector. Seepublish/ARCHITECTURE.md.
Partial release. Only these two connector packages are published at 3.3.2; the rest of the family — including
redb.Routeitself — stays at 3.3.1 and is unchanged. Both packages depend onredb.Route >= 3.3.1, so they drop into an existing 3.3.1 install without touching anything else.
redb.Route.Sql—mode=Procedurenever took the procedure name from the URI path, which made the fluentSql.Procedure(...)builder unusable.SqlBuilder.Build()only ever emits the name into the URI path, whileSqlEndpointOptions.Validate()demanded a separateprocedureName=parameter — so everySql.Procedure("sp_x")route threwArgumentException: ProcedureName is required for Procedure mode.at endpoint creation, and the string-URI form had to repeat the name twice (sql:sp_x?mode=Procedure&procedureName=sp_x).SqlComponent.CreateEndpointnow falls back to the URI path whenprocedureName=is absent, which is what theSqlEndpointxml-doc already promised ("The path part of the URI is the SQL query or stored procedure name"). An explicitprocedureName=still wins, so nothing that works today changes behaviour. The bug survived becauseSql.Procedureappeared in the docs but in no test and no route; an end-to-end test through the realEndpointUriParsernow covers it.
redb.Route.Sql—outputClassnow actually maps rows to a POCO. The option was bound from the URI and silently ignored:PocoRowMapper<T>existed but was never instantiated, and every result came back asDictionary<string, object?>. The newSqlRowMapperFactoryresolves the type name (assembly-qualified, full, or short against loaded assemblies), verifies a public parameterless constructor, and caches the mapper. Producer:SelectList→ a typedList<T>,SelectOne→T,StreamList→IAsyncEnumerable<T>. Poll consumer: the message body becomes the mapped POCO. The consumer still maps the raw row dictionary alongside the POCO, because header population and the@nameauto-bind inonSuccess/onFailurerun off the raw columns — a POCO would silently drop any column it has no property for. An unresolvable type name now fails loudly instead of being ignored. Unset (the default) → dictionaries, exactly as before.redb.Route.Sql—outputHeadernow delivers the result to a header instead of the body. Also previously bound and ignored (SqlProducer.SetResultcarried a "Check if result should go to header or body" comment and unconditionally wrote to the body). WithoutputHeader=nameset, the query result lands in that header and the incoming body is left intact — which is what makes a SQL lookup usable as an enrichment step rather than a payload-destroying one. Supported forSelectList,SelectOne,Scalar,StreamList, and formode=Procedure&asFunction=true. Unset → result replaces the body, exactly as before.redb.Route.Sqs— SNS raw message delivery on the SNS→SQS auto-subscription. NewrawMessageDeliveryoption (Sns.Topic(...).SubscribeSnsToSqs(arn).RawMessageDelivery(), orrawMessageDelivery=truein the URI). When the SNS publisher auto-subscribes an SQS queue (subscribeSnsToSqs=true), it now also sets the subscription'sRawMessageDelivery=true, so the queue receives the bare payload — and SNS message attributes map to SQS message attributes, restoring W3C trace continuity across the hop — instead of the default SNS JSON notification envelope ({"Type":"Notification","Message":...}) which the subscriber would otherwise have to unwrap. Default remainsfalse(AWS-compatible envelope).
redb.Route.Sql— README rewritten URI-first, and corrected where it contradicted the parser. It documented SQL placeholders as:id/:body, butSqlParameterParseronly ever recognised@name, and there is no implicit@body— a scalar body never binds itself into a parameter (useparam.msg=${body}), and an unmatched placeholder silently becomesDBNull. The single URI example also omitted the mandatorymode=Pollfor a consumer. The README now leads with string URIs for all three modes, documents the parameter-binding priority, and flags what remains unimplemented. Seedocs/SQL_PROCEDURE_MODE_REGRESSION.mdfor the full audit, including the options left alone on purpose (transactedis a no-op on producers, which always open a local transaction;batchSizeis an on/off flag, not a chunk size;SqlHeaders.GeneratedKeysis never set but is kept because removing a public constant would break consumers).
- RabbitMQ — full AMQP basic-property round-trip (regression fix). The producer forwarded only
CorrelationIdfrom headers — and by a bare name that didn't match what the consumer stamped — silently droppingReplyTo/MessageId/Priority/Expiration/Type/AppId/UserId/Timestamp/ContentEncoding/DeliveryModeon a consume→produce hop. The producer now maps every settable string/byteBasicPropertiesfield from headers via cached reflection (plus explicitTimestamp— anAmqpTimestamp, notIConvertible— andPersistent/DeliveryMode); the consumer stamps them symmetrically. Standard properties now use their bare well-known names (ReplyTo,Priority, …) so a round-trip carries them through with no docs needed;redbRmq.*is still accepted on the producer for back-compat and remains the prefix for delivery metadata (Exchange/RoutingKey/DeliveryTag/Redelivered/ConsumerTag). - AMQP 1.0 — property forwarding +
CorrelationIdround-trip. The producer now forwardsMessageId/CorrelationId/ReplyTo/Subject/GroupId/To/ContentEncoding/ReplyToGroupId/UserId/GroupSequence/CreationTime/AbsoluteExpiryTime/Durablefrom headers (header wins over option).CorrelationIdwas read by a bare name that didn't match the consumer'sredbAmqp.CorrelationId, breaking round-trip — now aligned. Standard AMQP 1.0 properties use bare names (transport metadata staysredbAmqp.*); the consumer additionally stampsContentEncoding/To/ReplyToGroupId/UserId. - Azure Service Bus — batch send now sets message properties.
SendBatchAsynccreated bareServiceBusMessages with no native/application properties; it now applies the same property mapping as single send (sharedApplyProperties), with a uniqueMessageIdper batched message. - Redis — stream-field header prefixed. The XADD field map was read by a bare
"StreamFields"key, inconsistent with every other Redis header; now the prefixedRedisHeaders.StreamFieldsconstant (bare name still accepted for back-compat).
- IBM MQ — MQMD header forwarding split into two tiers. The standard, JMS-equivalent MQMD fields
(
CorrelId/Priority/Expiry/Persistence/ReplyToQueue+ReplyToQueueManager) now forward from headers by default (header wins) — matching what a WMQ JMS client maps fromJMSCorrelationID/JMSPriority/JMSExpiration/JMSDeliveryMode/JMSReplyTo, so a naive consume→produce preserves them. The advanced/raw fields (MsgType/Format/GroupId/MsgSeqNumber) remain gated behindMqmdWriteEnabled(mirrors IBM MQ JMSWMQ_MQMD_WRITE_ENABLED) as they can alter message semantics.MqmdReadEnabledstaystrue.
- Fluent DSL —
stringoverloads on expression-first connector builders. Value methods that accepted onlyIExpression(so a bare string literal wouldn't compile) now have additivestringoverloads across RabbitMQ, Sftp, Ftp, File, MqttNet, Kafka, Redis and Http (Ldap already had them). Each wraps the value inStringExpression, so.Host("localhost")works as a constant and.RoutingKey("order.${header.type}")still interpolates — full parity with the URI form. Purely additive; existingIExpressionmethods are unchanged.
redb.Route.Llm— keyword knowledge retrieval (IKnowledgeStore.SearchTextAsync). A case-insensitive substring search over stored chunks, alongside the existing embedding path (SearchAsync) — for corpora without embeddings or callers with no query vector (e.g. a small structured rule-set where exact terms beat semantic similarity). Added as a default interface member (source-compatible for external implementers).RedbKnowledgeStorepushes a server-sideLIKEonto the indexed_objects.notecolumn (Contains(.., OrdinalIgnoreCase)→ComparisonOperator.ContainsIgnoreCase), returns only the matched rows, and ranks that small set in-process by occurrence count (dropping rows that matched only the metadata envelope);InMemoryKnowledgeStoredoes the same over its dictionary.redb.Route.Llm— prebuiltknowledge_searchtool (KnowledgeSearchTool). A ready.AsLlmToolroute overIKnowledgeStore.SearchTextAsyncso an agent can query the knowledge base itself: input{query, top_k?, collection?}→{results:[{id, collection, score, text}]}. The store is taken fromKnowledgeSearchOptions.Storeor resolved from the exchange'sIServiceProvider(picks upAddRedbLlmStorage()'s store).Collectioncan be pinned so the model'scollectionargument is ignored — scoping an agent to a single tenant / document set. Lives in the main package (notredb.Route.Llm.Tools) because it depends on the engine'sIKnowledgeStore; the utility-tools package stays Abstractions-only. WhenKnowledgeSearchOptions.EmbeddingProvideris set the tool runs semantic search (embeds the query → cosineSearchAsync); otherwise keyword (SearchTextAsync).redb.Route.Llm— embeddings transport (IEmbeddingProvider+OpenAiEmbeddingProvider). The retrieval half of RAG: turns text into vectors so the already-shippedKnowledgeChunkProps+IKnowledgeStore.SearchAsync(cosine) become usable. One OpenAI-compatible client (POST{baseUrl}/embeddings) over the sameLlmConnectionFactoryasOpenAiProvider— setModelIdto the embedding model (e.g.text-embedding-3-small) and it talks to OpenAI, Mistral, Together, DeepSeek, Gemini-compat, vLLM, llama.cpp, Ollama, LM Studio, … Preserves input order via the provider-reportedindex,EmbedOneAsyncconvenience for a single text.redb.Route.Llm—knowledge://ingest scheme (KnowledgeComponent). Producer-only:To("knowledge://<collection>?chunkChars=1000&overlap=100&embed=true")takes the exchange body as a document, chunks it (deterministic character windows with overlap), optionally embeds each chunk when anIEmbeddingProvideris registered, and upserts into theIKnowledgeStore. Chunk ids are{docId}#{index}(docId from?docId=or theknowledge.doc.idheader) so a re-ingest of the same document replaces its chunks in place. Turns document loading into a route:From("file://docs?include=*.md").To("knowledge://handbook").redb.Route.Llm—.Knowledge(collection, k)retrieval DSL. A route step that retrieves the top-K chunks for the current message and injects them into the system prompt (LlmHeaders.SystemPrompt), so a following.To("llm://…")answers grounded on them —From("kafka://questions").Knowledge("handbook", k: 5).To("llm://claude"). Semantic when anIEmbeddingProvideris available (embed query →SearchAsync), else keyword (SearchTextAsync); augments any existing system prompt; a no-op when no store is wired or nothing is retrieved (never breaks the pipeline). This completes the connector's RAG loop:knowledge://ingest → embeddings → keyword/semantic search →knowledge_searchtool /.Knowledge()injection.redb.Route.Llm—embed://scheme (EmbedComponent). Embedding as a first-class route step, symmetric withllm://:To("embed://<factory>")turns the exchange body (a text →float[], or a collection of texts →float[][], order preserved) into vectors onOut.Body. The URI host names anLlmConnectionFactory(setModelIdto the embedding model), so different routes pick different embedding models by name —From("kafka://texts").To("embed://openai").To("vector://sink").EmbedComponent.ProviderFactoryis overridable (defaults toOpenAiEmbeddingProvider.Create).- New connector:
redb.Route.Sqs— Amazon SQS + SNS (native AWS SDK for .NET v4). One package, two schemes:sqs://(queue consumer + producer) andsns://(topic publisher + SNS→SQS fan-out). The SQS consumer does long-polling,ConcurrentConsumers(N)competing loops, visibility timeout with an optional extend-while-processing heartbeat, at-least-once acknowledgement (delete on success), transacted ack via.Transacted(), and FIFO. The producer does single + batch send and FIFO group/dedup ids; the SNS publisher supports subject / message structure / FIFO and an SNS→SQS auto-subscription. LocalStack / ElasticMQ compatible viaserviceUrl=, full AWS credential chain, and W3C trace-context propagation across the hop. Seeredb.Route.Sqs/README.md. - New connector:
redb.Route.Telegram— Telegram Bot API (built onTelegram.Bot). Schemetelegram://: a long-polling consumer (receive, singlegetUpdatesstream per token, one bot client shared per token) and a producer withsend/document/photo/edit/delete/answermodes. Handles the 429retry_afterrate-limit contract (waits and retries), validatesparseMode(throws on a typo instead of silently sending raw markup), supports inline / reply keyboards (WithInlineKeyboard/WithReplyKeyboard), a webhook-unpack pipeline (UnpackTelegramUpdate), and a fluent DSL (Tg.Receive/Send/Document/...). Delivery is at-most-once (Telegram advances the update offset on dispatch — documented; no redelivery); parallel processing via the.Threads(N)EIP; consumer telemetry + producer spans. Seeredb.Route.Telegram/README.md. controller.Redb()— extension onRedbControllerthat resolves the per-request scopedIRedbService(its own connection) for the controller's current exchange. Use instead ofContext.GetRedbService(), which returns the shared captive singleton..Threads(N)concurrency EIP (coreredb.Route). A Camel-style processing-concurrency stage:From(...).Threads(N)…EndThreads()caps a route section's concurrency at N, so a strictly serial source (poll consumers, MQTT, a single request thread) can process up to N exchanges at once — the general-purpose alternative to.To("seda://x").ConcurrentConsumers(N)without a named endpoint. Adaptive by exchange pattern: InOnly is a fire-and-forget hand-off (clone + worker pool, a transaction boundary like.To("seda://")); InOut runs the body inline on the same exchange under aSemaphoreSlimgate, so the reply — onOutorIn— is preserved losslessly and request/reply (RPC) works across it (InOut is not a transaction boundary — the ambient transaction flows into the inline body). Options:.MaxQueueSize(n)and.EnqueueTimeout(TimeSpan)(default = wait for a free slot; on timeout throwsTimeoutException). Ordering not preserved at N > 1. Seeredb.Route/CONCURRENCY.md.
redb.Route.Llm— non-ASCII text mangled on the wire (\uXXXXescaping). Several JSON serializers used the default encoder, which escapes every non-ASCII char. On tool results this ~6×'d the tokens the model saw for Cyrillic / CJK content (and could surface literalМ…); in the knowledge store it buried chunk text as\uXXXXin thenotecolumn, making the new keywordLIKEunable to match a raw non-ASCII query. Switched toJavaScriptEncoder.UnsafeRelaxedJsonEscapinginAgentEngine(tool-reply + error serializers),OpenAiProvider(request body — matchesAnthropicProvider, which already did this),McpProtocol(MCP JSON-RPC), and theRedbKnowledgeStorechunk envelope.redb.Route.Llm—RedbKnowledgeStore.UpsertManyAsync(bulk ingest) was non-functional. Two latent bugs on the never-exercised bulk path: (1) the existing-key lookupkeys.Contains(o.ValueString)over astring[]threwNotSupportedExceptionfrom the redb query parser (C# 13 binds it toMemoryExtensions.Contains); (2) a Props-hash "skip if unchanged" pre-check silently dropped every re-upsert of the property-lessKnowledgeChunkProps(empty Props → constant hash). Fixed the parser (seeRedBase.Corechangelog) and removed the pre-check so bulk re-ingest actually updates chunk text/embeddings.- DI-scope / connection leaks on the exchange lifecycle. Several paths created a per-exchange DI
scope (which owns a redb DB connection) that could escape without being disposed:
ThreadsProcessor,SedaProducerandVmProducerleaked the clone when the hand-off enqueue failed (cancellation / queue completed);WireTapProcessorleaked the tap clone when a useronPrepare/newBodycallback threw before dispatch; the scheduledllm://andexec://consumers never disposed their per-tick exchange (a scope leaked on every fire). All now dispose infinallyon every path. In addition,Exchange.ReleaseScopes()is now resilient to a throwingDisposeAsync: each cached scope is released in isolation (a fault is logged and the loop continues), so one broken scope can no longer strand its siblings — which would otherwise re-introduce the very connection leak this guards against. - Lazy producer start-up is now race-safe (no more cold-start
NullReferenceExceptionunder concurrency). On a cold start, concurrent exchanges reaching a not-yet-started producer could observe it half-initialised:ConnectableProducer.Start()set itsIsStartedflag beforeConnectAsync()completed, andToProcessorhanded out the producer as soon as it was created — beforeStart()finished. A second thread then calledProcess()on it (e.g. a Redis producer whose_dbwas still null → NRE). The bug was masked while sources were serial; it surfaced onceredb.Route.RabbitMQ3.2.2 madeConcurrentConsumers(N)truly parallel. Both paths are now single-flight — concurrent callers await the same startup, and a producer is observable as started only afterConnectAsync()has fully completed (its resources are ready). The dynamic-endpoint path (toD) was already safe viaLazy<Task<IProducer>>. - The default (unnamed)
IRedbServiceis now resolved per exchange, not as a shared singleton.ProcessWithRedb(...),SetBodyFromRedb(...),SetHeaderFromRedb(...)andBeginRedbTransaction()previously fell back to oneIRedbServicecaptured from the root DI provider — a single, non-thread-safe DB connection (EF-DbContext model) shared across every exchange. Under real concurrency (Splitter with parallel processing, SEDA,ConcurrentConsumers(N), concurrent HTTP requests) two exchanges drove that one connection at once and the driver threw "A command is already in progress" / "connection is busy". Each exchange now gets its own DI scope → its own scopedIRedbService→ its own pooled connection, cached on the exchange and disposed with it. The unscoped singleton is used only when there is no exchange (single-threaded seed).
Targeted hotfix in
redb.Route.Amqpandredb.Route.IbmMqonly — each bumps 3.2.0 → 3.2.1; every otherredb.Route.*package is unchanged (redb.Route.RabbitMQstays at 3.2.2,redb.Route.Kafkaat 3.2.1, the rest at 3.2.0). Both depend onredb.Route3.2.0 (no core change). Same class of bug as the RabbitMQ 3.2.2 fix:ConcurrentConsumers(N)was a no-op.Behaviour change to be aware of. Before this release, an AMQP or IBM MQ consumer processed messages strictly one at a time regardless of
ConcurrentConsumers— the option only sized an internalSemaphoreSlimthat a serial receive loop never let engage. NowConcurrentConsumers(N)runs N real competing consumers, so a route withN > 1processes up to N messages concurrently: per-destination ordering is no longer preserved on that route and its pipeline must be thread-safe. Routes at the default 1 stay strictly serial, exactly as before.
The AMQP consumer ran a single receive loop that awaited Process inline before receiving the next
message, so only one message was ever in flight; the SemaphoreSlim(ConcurrentConsumers) was
acquired and released by that same loop and never saw a second holder — a dead gate. Credit (link
prefetch) only buffered deliveries; it never produced concurrent processing.
The fix runs ConcurrentConsumers(N) as N independent competing consumers, each with its own
AMQP Session + ReceiverLink + serial loop (AMQPNetLite sessions/links are not thread-safe, so
concurrency comes from N independent workers, never from sharing one link). RPC replies are sent on
the worker's own session; the processed-count is updated atomically. A new
AmqpEndpoint.CreateDedicatedReceiverAsync creates a receiver on a fresh dedicated session.
Tuning note: for even load-balancing across competing consumers, keep Credit low (e.g. 1) —
a high credit lets one worker's prefetch buffer hoard the queue and drain it serially.
The IBM MQ consumer ran a single MQGET loop that awaited Process inline (Get → Process → Get),
so processing was strictly serial; the SemaphoreSlim(ConcurrentConsumers) was a dead gate (the count
was even wired correctly from options — unlike RabbitMQ where it was pinned to 1 — but the inline
await made it moot).
The fix runs ConcurrentConsumers(N) as N independent competing consumers, each with its own
dedicated MQQueueManager connection + destination handle + serial loop. This is required, not
cosmetic: the MQ managed client serialises MQI calls per connection and its transacted syncpoint is
connection-scoped (Backout() rolls back the whole connection), so each worker must own its
connection — which also makes transacted mode correct at N > 1 (per-worker commit/rollback). Queues
are opened INPUT_SHARED, i.e. true competing consumers. Topics are clamped to a single subscriber
with a warning — N managed non-durable subscriptions would each receive a copy of every message
(duplicate delivery), not share the load; use a queue destination for competing consumers. RPC reply,
backout-queue routing and the transacted ack all use the worker's own connection; the processed-count
is atomic. (The unrelated ~500 ms managed-client MQGET poll-tick latency — a future MQCB rewrite — is
untouched.)
redb.Route.Tests.Amqp— newAmqpConcurrencyTests:ConcurrentConsumers_ProcessesInParallel(5 workers,credit=1, observed max-concurrency 5),ConcurrentConsumersOne_ProcessesSerially(max-concurrency 1), andConcurrentConsumers_AllMessagesProcessedExactlyOnce(competing consumers share the queue — no duplication/loss). Full package suite: 133 passing against ActiveMQ Artemis.redb.Route.Tests.IbmMq— newIbmMqConcurrencyTests: the same three shapes onINPUT_SHAREDqueues (observed max-concurrency 5 atConcurrentConsumers(5); 1 at the default; 24/24 delivered exactly once). Full package suite: 163 passing against IBM MQ Developer Edition.
Targeted hotfix in
redb.Route.RabbitMQonly — this package bumps to 3.2.2; every otherredb.Route.*package is unchanged (redb.Route.Kafkastays at 3.2.1, the rest at 3.2.0). RabbitMQ 3.2.2 still depends onredb.Route3.2.0 (unchanged, already on NuGet) — no core changes, no mass-republish. Three items: (1) a fix for consumer dispatch concurrency, which was silently pinned to 1 soConcurrentConsumers(N)never actually parallelised a queue; (2) a fix for a channel leak on per-route Stop/Start; (3) a new framework-levelAutoAckconsumer option (broker-side auto-acknowledge / at-most-once), the RabbitMQ analogue of Kafka'sEnableAutoCommit.Behaviour change to be aware of. Before 3.2.2 every RabbitMQ consumer processed one message at a time regardless of
ConcurrentConsumers(the real gate — the AMQP consumer dispatch concurrency — was stuck at 1). With 3.2.2 a route that setsConcurrentConsumers(N > 1)now genuinely processes up to N messages concurrently, so per-queue message ordering is no longer preserved on such routes and their processors must be thread/concurrency-safe. Routes that leaveConcurrentConsumersat its default of 1 are unaffected — they stay strictly serial, exactly as before.
The RabbitMQ consumer gains a typed AutoAck option (default false), bound from the URI
(?autoAck=true|false) like every other endpoint option, plus a matching fluent
RabbitBuilder.AutoAck(bool) method.
When enabled, the consumer subscribes with autoAck: true, so the broker settles every
delivery on hand-off (at-most-once): there is no manual BasicAck/BasicNack, and a failure
in the processor does not requeue the message. This is the mirror of the manual-ack default
(at-least-once: ack after a successful turn, nack-requeue on failure) and the RabbitMQ analogue
of the Kafka EnableAutoCommit option shipped in 3.2.1 — it removes the need for a SEDA
hand-off stage when a route wants fire-and-forget "ack on receive" semantics (e.g. WSO2-style
autoAck=true).
AutoAck cannot be combined with Transacted — an auto-acked delivery is settled by the
broker on hand-off and cannot be transactionally committed or rolled back — so
RabbitMQEndpointOptions.Validate() now rejects that combination.
redb.Route.RabbitMQ — consumer dispatch concurrency was pinned to 1 (ConcurrentConsumers had no effect)
RabbitMQEndpoint.CreateChannelAsync built its channel with the three-argument
CreateChannelOptions(...) constructor, leaving the fourth parameter
(consumerDispatchConcurrency) at its compile-time default. In RabbitMQ.Client 7.2.1 that
default is Constants.DefaultConsumerDispatchConcurrency = 1, not null — verified against
the shipped assembly. A non-null per-channel value overrides the connection-level setting, so
every channel the library created was clamped to serial dispatch, and the value set on the
ConnectionFactory / URI (consumerDispatchConcurrency=N) was silently discarded — in both
named-factory and inline-connection modes.
The upshot: a single AsyncEventingBasicConsumer on a single channel received deliveries strictly
one at a time (inFlight = 1), and ConcurrentConsumers(N) — which only ever sized an internal
SemaphoreSlim — could not deliver any parallelism because the dispatcher never handed the
consumer more than one message at once. On production this showed up as a consumer that never kept
up: unacked climbed to the prefetch limit while messages were still processed serially.
The fix makes ConcurrentConsumers the single knob for consumer-side parallelism:
CreateChannelAsync now always passes consumerDispatchConcurrency explicitly, and the consumer
opens its consume channel with the dispatch concurrency set to ConcurrentConsumers (which also
sizes the semaphore). ConcurrentConsumers(N) therefore now processes up to N messages in
parallel. ConsumerDispatchConcurrency remains available as the connection-level default for other
channels and is no longer clobbered. A live-broker regression test
(Consumer_ConcurrentConsumers_ProcessesInParallel) publishes 20 messages, runs with
ConcurrentConsumers(5), and asserts the observed maximum concurrency exceeds 1; a companion test
asserts ConcurrentConsumers(1) stays strictly serial.
RabbitMQConsumer.Stop() cancelled its subscription (BasicCancelAsync), drained in-flight work,
and closed its dedicated RPC reply channel — but left its main consume channel open and still
registered in the endpoint's channel list. Closing that list is done only by
RabbitMQEndpoint.Stop(), which the engine invokes only on full context teardown, never on a
per-route StopRoute. Since StartRoute reuses the same consumer instance and opens a fresh
channel, each Stop/Start cycle of an individual route leaked one channel — a cancelled, idle
channel (no deliveries, unacked = 0, but still open with the prefetch showing) that accumulated on
the pooled connection until the whole context was disposed (tpkg hot-reload / container restart).
The consumer now owns and releases its channel: Stop() closes, disposes, and unregisters the
consume channel via a new RabbitMQEndpoint.ReleaseChannelAsync, and the Start-failure cleanup path
uses the same method (so a partial start no longer leaves a stale handle in the list either). Double
release is safe — endpoint.Stop() still guards on IChannel.IsOpen, and the list removal is
idempotent. This is a RabbitMQ-package-local fix; the core RouteContext.StopRoute is unchanged.
A regression test (Consumer_StopStartCycles_DoNotLeakChannels) runs five Stop/Start cycles on a
live broker and asserts the endpoint's tracked-channel count returns to 0 after every Stop.
redb.Route.Tests.RabbitMQ— new live-broker suiteRabbitMQConcurrencyLeakAutoAckTests:Consumer_ConcurrentConsumers_ProcessesInParallelandConsumer_ConcurrentConsumersOne_ProcessesSerially(dispatch concurrency),Consumer_StopStartCycles_DoNotLeakChannels(channel leak), andAutoAck_DeliversMessage/AutoAck_ProcessorThrows_MessageNotRequeued/ManualAck_ProcessorThrows_MessageRequeued(AutoAck vs manual-ack requeue). Plus unit tests for theAutoAckbuilder param, its default-off, and theAutoAck+Transactedvalidation guard. Full package suite: 112 passing against RabbitMQ 4.x.
The code changes in this release land in
redb.Route.Kafkaandredb.Route.RabbitMQ— and only these two packages are bumped to 3.2.1. Every otherredb.Route.*package stays at 3.2.0 (no changes); a targeted hotfix, not a mass-republish. Kafka/RabbitMQ 3.2.1 depend onredb.Route3.2.0. Three items: (1) a new framework-levelEnableAutoCommitoption on the Kafka consumer (defaulttrue) that brings Kafka offset-settle into parity with the RabbitMQ consumer's post-process ack; (2) a fix for a doubleBasicAckin the RabbitMQ consumer when a route-level.Transacted()wraps a non-transacted consumer; (3) a fix for the Kafka transacted producer, which threwLocal: Erroneous stateon the deferred send.Behaviour change to be aware of. Before 3.2.1 a Kafka consumer route committed its offset only at a transactional boundary (
.Transacted()/.CommitTransaction()); a plainFrom("kafka:...")processed messages but never advanced the committed offset (the offset only moved on a graceful stop via the partitions-revoked commit). With 3.2.1 the defaultEnableAutoCommit=truecommits the offset inline after a successful turn, so a plain consumer settles at-least-once exactly like the RabbitMQ consumer already did. Set?enableAutoCommit=falseto restore the old "commit only at a transaction boundary" behaviour. Inside a transactional route the option is effectively ignored — the transaction owns the commit (see below).
The Kafka consumer gains a typed EnableAutoCommit option (default true),
bound from the URI (?enableAutoCommit=true|false) like every other endpoint
option, plus a matching fluent KafkaBuilder.EnableAutoCommit(bool) method.
This is a framework-level setting, deliberately not librdkafka's own
enable.auto.commit: the underlying client stays at manual commit
(EnableAutoCommit = false in the built ConsumerConfig) always, so the
library never commits un-processed offsets on a background timer. Instead the
redb.Route consumer commits after it knows the turn succeeded:
- After a successful
Processor.Process(...), the consumer commits the offset inline (IConsumer.Commit) — mirroring the RabbitMQ consumer'sBasicAck-after-Process. Single-message mode commits that message's offset; batch mode commits the last offset of the batch. - A transactional route takes precedence.
KafkaCommitActionnow carries aCommittedflag (idempotentInterlockedguard). When a.Transacted()/.CommitTransaction()boundary commits the deferredKafkaCommitActionduring the turn,Committedis already set by the timeProcessreturns, so the consumer skips the inline commit. The transaction owns the offset andEnableAutoCommitis effectively ignored — it only matters on non-transactional routes. - On a failed turn (
Processthrows) the inline commit is skipped and the message is re-delivered — at-least-once, unchanged.
Net effect: Kafka and RabbitMQ consumers now share the same default mental model
("settle after a successful turn"). The previous behaviour — where a plain Kafka
consumer silently never advanced its committed offset — is opt-out via
?enableAutoCommit=false.
A Kafka producer with transacted=true set config.TransactionalId and called
_producer.InitTransactions(30s) on connect. That puts librdkafka into
transactional mode, where every Produce must be wrapped in
BeginTransaction … CommitTransaction. The connector never calls
BeginTransaction / CommitTransaction / AbortTransaction /
SendOffsetsToTransaction, so the deferred ProduceAsync — run when a
.Transacted() / .CommitTransaction() boundary commits the KafkaSendAction —
threw:
Confluent.Kafka.ProduceException: Local: Erroneous state
Confirmed end-to-end against a live 3-node KRaft cluster. The earlier tests missed it because they only asserted the deferred action was registered, never committed.
The fix drops transactional.id + InitTransactions from the transacted
producer path, leaving EnableIdempotence = true + Acks = All. The deferred
send is now a plain idempotent Produce deferred to the route boundary — which
delivers, and matches the connector's documented intent ("idempotent producer +
deferred commit, not EOS"). Real Kafka exactly-once
(BeginTransaction / SendOffsetsToTransaction / CommitTransaction) is scoped
in docs/KAFKA_TRANSACTIONS_TODO.md. A regression test
(TransactedProducer_DeferredCommit_Delivers) commits the deferred action and
asserts the message is actually delivered.
When a route-level .Transacted() segment wrapped a RabbitMQ consumer whose
endpoint was not ?transacted=true, the delivery was settled twice:
- The
TransactedProcessorcommitted the deferredRabbitMQAckActionat the.Transacted()boundary →BasicAck#1. - The consumer's own post-process branch (
if (!_options.Transacted)) then issuedBasicAck#2 on the same delivery tag.
RabbitMQ rejects the duplicate settle with
PRECONDITION_FAILED — unknown delivery tag and tears down the whole
channel, so every subsequent message on that channel is silently dropped. The
error path carried the symmetric double-BasicNack hazard.
RabbitMQAckAction now carries a Settled flag (idempotent Interlocked guard
on both Commit and Rollback — first settle wins). The consumer routes its
inline ack/nack through the same RabbitMQAckAction and skips it when
Settled is already set, so the broker sees exactly one settle per delivery. The
guard covers the success path and both the inner and outer catch blocks.
redb.Route.Tests.Kafka—Consumer_AutoCommitDefault_CommitsOffsetInline_BeforeStopandConsumer_AutoCommitDisabled_NoTransaction_DoesNotCommitInline. Both inspect the committed offset at the group coordinator (via a non-subscribing probe consumer, so it never joins the group / triggers a rebalance) while the consumer is still running — i.e. before any graceful stop, so the partitions-revoked commit cannot mask the result. The first asserts the offset is committed inline with the default option; the second asserts it stays uncommitted with?enableAutoCommit=falseand no transaction.redb.Route.Tests.RabbitMQ—Consumer_RouteTransactionAcksDuringProcess_NoDoubleAck. A processor commitsTRANSACT_ACTIONmid-Process(simulating the.Transacted()boundary), and the test asserts both published messages are processed — proving the channel survived the first ack instead of being torn down by a double settle.
All
redb.Route.*packages are versioned together at 3.2.0; the code changes in this release land inredb.Route,redb.Route.Llm,redb.Route.Llm.Tools,redb.Route.Llm.Mcp,redb.Route.Http,redb.Route.WebSocketandredb.Route.Exec(the other connectors are version-aligned, no code changes). The release bundles four areas of work: (1) end-to-end token-by-token streaming on the wire (IAsyncEnumerable<string>response bodies → SSE / chunked text over HTTP, one text frame per token over WebSocket); (2) REDB-backed stores for the remaining state surfaces (IBatchStore,IEvalRunStore,IKnowledgeStore,IPromptTemplateRegistry,IToolCacheStore); (3) async-batch callback plumbing (LlmCallbackProcessor+ newllm.batch.*headers); (4) a thin DSL/tool split across the homeless tools inredb.Route.Llm.Tools. Plus a named-redb-per-exchange hint (?redb=<name>), the newredb.Route.Llm.McpMCP-client connector that brings the community ecosystem of Model Context Protocol servers into the agent toolset, and two targeted bug fixes (LLM agent loop orphan tool_use recovery, Exec OEM codepage on Windows). No public API was removed or renamed. The store-interface additions are optional parameters with defaults — existing implementations and call sites compile unchanged.
Both throttle processors gain an opt-in rejectOnOverflow constructor flag
(default false for backward compatibility) plus a matching fluent
.RejectOnOverflow() method on ThrottleDefinition and
KeyedThrottleDefinition. When set, overflow exchanges are short-circuited
with HTTP 429 Too Many Requests + a Retry-After header carrying
the current rate-limit period in delta-seconds (RFC 7231 §7.1.3), and a
small structured JSON body:
{
"error": "rate_limit_exceeded",
"error_description": "Rate limit exceeded. Retry after 1 second(s).",
"retry_after": 1
}The behaviour is now selectable per-route in the DSL:
// Legacy: silent semaphore-wait until a slot frees (still the default).
.Throttle(maxPerPeriod: 10)
// RFC 6585 — fast-fail with 429 + Retry-After. Recommended for HTTP-facing
// routes; the silent-wait variant looks like a hung server to clients.
.Throttle(maxPerPeriod: 10).RejectOnOverflow()
// Same option on the per-key (per-IP / per-client_id) variant.
.Throttle(e => e.GetClientId(), maxPerPeriod: 10, period: TimeSpan.FromSeconds(1))
.RejectOnOverflow()The non-blocking SemaphoreSlim.Wait(0) probe replaces the unconditional
await WaitAsync(ct) so a slot is only acquired when one is actually
available — the legacy mode still falls back to WaitAsync to preserve
the exact old timing for callers that didn't opt in.
Implementation lives in two places: the per-processor flag, and a shared
ThrottleRejection.Apply(exchange, period) helper that writes the 429
response (redbHttp.ResponseCode, Retry-After, JSON body) and calls
exchange.Stop() so no downstream processor (WireTap, tx commit,
idempotency capture) runs against the rejected exchange. Apache Camel's
Throttler has the same axis (rejectExecution=true/false) — this
brings the redb.Route EIP into parity.
The default stays false so every existing route continues to behave
exactly as it did in 3.1.0 and earlier; opting in is a per-route choice.
redb.Route.Llm / redb.Route.Http / redb.Route.WebSocket — end-to-end streaming wire contract for LLM token deltas
LlmProducer.ProcessStreamingAsync already emits an IAsyncEnumerable<string>
of provider token deltas into exchange.Out.Body when ?stream=true (or the
llm.streaming header) is set on the LLM endpoint. As of 3.1.0 only the
producer surface existed; downstream transports buffered the enumerable into
a single response. 3.1.1 wires the contract end-to-end so a route like
From("http://+:8080/chat")
.To("llm://claude?stream=true")
// Out.Body is IAsyncEnumerable<string> here
// HttpConsumer flushes each yield as one SSE 'data:' framestreams token-by-token to the browser, and the equivalent WebSocket route
From("ws://+:9001/chat")
.To("llm://claude?stream=true")
// Each yield → one WebSocketMessageType.Text frame, endOfMessage=truestreams token-by-token to the WebSocket client. No new types, no new options
— transports inspect Out.Body and pick the right wire shape.
Producer-side contract — LlmProducer now sets two response markers
alongside the streaming body:
Out.ContentType ??= "text/event-stream"when not already set, so the HTTP transport defaults to SSE framing.Out.Headers[LlmHeaders.Streaming] = true("llm.streaming") — a stable signal any downstream component can branch on. Visible inWireTap/Multicast/ audit routes.
Late-bound summary headers (llm.tokens.in, llm.tokens.out,
llm.stop_reason, llm.tool.iterations) are written after the
IAsyncEnumerable completes; llm.provider.id and llm.model.id are
written up-front. Transports collect them post-enumeration and surface them
in a transport-appropriate way (see HTTP event: done trailer below).
Scope. The streaming path calls
ILlmProvider.StreamAsyncdirectly and bypassesAgentEngine— so tools (?tools=) are not dispatched,AddRedbLlmStorage()stores are not invoked, and governance hooks do not fire on a streamed turn. Use streaming for user-facing rendering of a single assistant turn; keep the non-streaming path when you need tools, persistence, approvals or budgets.
HttpConsumer (redb.Route.Http). Detects Out.Body is IAsyncEnumerable<string> and picks one of two writers based on
Out.ContentType:
text/event-stream→ SSE: per-linedata:prefix, blank-line terminator per yield, response flushed per chunk. The stream ends withevent: done\ndata: {…json…}\n\nwhose payload is built opportunistically from whicheverllm.*summary headers are present on the message at end-of-stream (llm.tokens.in,llm.tokens.out,llm.cost.usd,llm.stop_reason,llm.tool.iterations,llm.model.id,llm.provider.id) — missing headers are omitted from the JSON, custom ones (e.g. a pricing-table-derivedllm.cost.usd) ride along for free.- anything else → chunked plain text: one yield = one chunk on the Transfer-Encoding stream, no SSE framing, no trailer.
Both writers set the standard "do-not-buffer-me" envelope:
Cache-Control: no-cache, no-transform, X-Accel-Buffering: no, and
IHttpResponseBodyFeature.DisableBuffering(). This neutralises nginx and
similar reverse-proxy buffers and is what makes SSE actually progressive on
the wire (without X-Accel-Buffering: no nginx by default holds the whole
response until the upstream closes). Empty / null chunks are skipped — LLM
providers periodically emit empty SSE keep-alives that must not turn into
empty wire chunks. Client cancellation (HttpClient aborts the request)
propagates into the server-side await foreach via HttpContext.RequestAborted,
so the upstream provider stream is torn down promptly — no pinned upstream
sockets.
WsConsumer (redb.Route.WebSocket). Detects the same body type in the
InOut branch of HandleWebSocket and yields one
WebSocketMessageType.Text frame per yield with endOfMessage=true. Order
is preserved (the per-connection receive loop awaits each SendAsync before
reading the next inbound frame, so writes are naturally serial per socket);
empty chunks are skipped. The cancellation token is the consumer's
drain-safe _drain.ProcessingToken, so an in-flight stream completes during
a graceful stop. The non-streaming ResolveResponseBody path is unchanged
for non-IAsyncEnumerable bodies.
Tests. Two transport-level test suites pin the wire contract without needing any LLM provider:
redb.Route.Tests.Http/HttpStreamingTests—Sse_PerChunkFlush_AndDoneTrailer,ChunkedPlain_NoSseFraming_NoTrailer,ChunksArriveProgressively_NotBuffered, andClientCancel_PropagatesToEnumerator. Verifies SSE line framing, theevent: doneJSON payload, progressive arrival (first byte well before last yield), and that aborting theHttpClientrequest surfaces on the server-side enumerator within seconds.redb.Route.Tests.WebSocket/WsStreamingTests—Streaming_OneFramePerYield_OrderPreservedandStreaming_EmptyChunksSkipped. Verifies one-frame-per-yield, ordered delivery, and that null / empty yields do not produce wire frames.
A new env-gated suite — redb.Route.Tests.Llm/LiveStreamingTests — exercises
ILlmProvider.StreamAsync end-to-end against real free-tier providers
(Anthropic Claude Haiku 4.5 via AnthropicProvider native SSE, plus Groq /
Cerebras / Gemini / Mistral / OpenRouter via OpenAiProvider). Each test
asserts more-than-one chunk on the wire (proves real streaming), at least one
text delta, the expected substring in the accumulated answer, and a non-null
terminal StopReason. Auto-skips when the corresponding key env var is
missing, same as LiveProviderTests.
The agent loop ships in-memory defaults for every governance surface; 3.1.0
shipped REDB-backed Conversation, Approval, CostBudget,
ToolIdempotency and AuditObserver stores. 3.1.1 lands the rest:
RedbBatchStore(IBatchStore) — tracks async-batch jobs submitted to Anthropic Message Batches / OpenAI Batch / vLLM batch endpoints; the callback webhook correlates back to the originating conversation through this store. Backed by the newLlmBatchPropsschema.RedbEvalRunStore(IEvalRunStore) — persists evaluation runs by scenario / fingerprint for leaderboard queries.RedbKnowledgeStore(IKnowledgeStore) — RAG retrieval over theKnowledgeChunkPropsschema.RedbPromptTemplateRegistry(IPromptTemplateRegistry) — versioned prompt store (the previous default was in-memory only).RedbToolResultCache(IToolCacheStore) — deterministic-tool result cache with TTL.
All five are opt-in through the same AddRedbLlmStorage() extension
(ServiceCollectionExtensions grew the appropriate TryAddSingleton
wiring) and ride on the existing IRedbService resolution path. A new
ToolIdempotencyProps schema replaces the ad-hoc storage shape used in
3.1.0 — see Changed below.
LlmCallbackProcessor— a vanillaIProcessorthat consumes inbound webhook callbacks from async-batch LLM APIs. Wired into any HTTP route (no new URI scheme): resolves the batch id from header / query / JSON body, deduplicates viaIToolIdempotencyStore(keyed"batch:<id>"), populates conversation / provider / model headers from the original submission stored inIBatchStore, and marks the batch completed. A duplicate callback setsLlmHeaders.BatchDuplicate=trueso a downstreamChoice().When(...).Stop()can drop it cleanly.- New
LlmHeadersconstants:BatchId(llm.batch.id),BatchStatus(llm.batch.status),BatchDuplicate(llm.batch.duplicate),ConversationMessageId(llm.conversation.message.id).
The LLM connector now lets a route pin which named IRedbService instance
its persistence stores write to. Useful when one Tsak host runs multiple
LLM products against different DBs.
- New URI option
?redb=<name>parsed intoLlmEndpointOptions.Redb. - New property key
LlmKeys.RedbName(llm.redb.name) stamped ontoIExchange.PropertiesbyLlmProducer; storage implementations resolve the redb instance viaIRouteContext.GetRedbService(name, exchange). - Every
I*Storemethod gained an optionalIExchange? exchange = nullparameter so REDB-backed implementations can read this hint without changing call sites — in-memory implementations ignore it. Source- compatible: every interface change is an optional parameter with a default; existing implementations and call sites compile unchanged. - Default
unnamedIRedbServicefrom the route context is used when no hint is set, matching the 3.1.0 behaviour exactly.
Each homeless tool was reshaped into a thin IProcessor (*Tool.cs) plus
a fluent route-DSL extension (*Dsl.cs) that mounts the processor with
typed options. Affects HttpFetchTool, JsonPathTool, MathEvalTool,
RegexExtractTool, TavilyWebSearchTool, XPathTool. The user-visible
DSL shape is:
From("direct:fetch-weather")
.AsLlmTool("get_weather")
.Description("Fetches weather for a URL.")
.Input("""{"type":"object","properties":{"url":{"type":"string"}},"required":["url"]}""")
.Then()
.HttpFetch(new HttpFetchOptions { HostAllowlist = ["api.weather.gov"] });A new shared helper LlmToolJson centralises the small JSON-payload
parsing / writing that every tool was duplicating. The split keeps the
agent-engine surface unchanged — tool descriptors and registry stay the
same; only the way you wire a tool into a route moves to a one-line DSL
call.
LlmMetricsexposes one more counter for stream chunks alongside the existing call / iteration / token meters.LlmConsumerhonours the same?redb=hint when scheduling aFrom("llm://...")agent run, so scheduled agents persist into the same named DB as inbound producer calls.Engine/PromptRef,Engine/Eval/LlmEvalRunnerupdated for the new store signatures.
OpenAiProvider.ResolveDefaultBaseUrl gains a "grok" / "xai" alias
that resolves to https://api.x.ai/v1/. No other changes: tool calls,
streaming, budget enforcement and conversation memory work identically to
every other OpenAI-compatible provider.
new LlmConnectionFactory("grok")
{
Provider = "grok",
ModelId = "grok-3-mini",
ApiKey = Environment.GetEnvironmentVariable("REDB_LLM_GROK_KEY")
}LiveProviderTests extended with five Grok scenarios (Smoke / NonAscii /
ToolUse / Usage / StopReason), gated on REDB_LLM_GROK_KEY.
Every assistant turn now persists the full set of inputs the provider call was made under. This closes the audit gap that previously forced auditors to trust that "the system prompt and sampling settings were the same as the ones currently in config" — now they're stamped on the row that produced the answer.
MessageProps (and its mirror ConversationMessageMeta) gain seven
nullable columns:
| Field | Set on | Purpose |
|---|---|---|
Temperature, MaxTokens, TopP |
assistant rows | effective sampling values after merging request + factory defaults |
PromptTemplateName, PromptTemplateVersion |
every row in the run | FK pair into PromptTemplateProps — pins the exact prompt text |
ToolSetHash |
assistant rows | SHA-256 of the canonical (name + description + InputSchema) of the tool set exposed on this call; detects tool-surface drift across runs |
ProviderSystemFingerprint |
assistant rows | OpenAI's system_fingerprint (and any echoing OpenAI-compatible provider — xAI, Together); null on Anthropic / Gemini-compat / Ollama |
Wiring:
AgentRequestgainsPromptTemplateName+PromptTemplateVersion. Callers that resolve a managed prompt template viaIPromptTemplateRegistryset the pair so the engine can stamp it on every persisted message of the run.AgentEnginecomputesToolSetHashonce per run (canonical sort by name, rawInputSchemafolded in verbatim — schema string changes show up as hash drift, which is exactly the auditor signal) and pipes it alongside the effectiveTemperature/MaxTokens/TopPintoPersistMessageAsync.OpenAiProvider.CompleteAsyncreadssystem_fingerprintfrom the response root and surfaces it onLlmResponse.ProviderSystemFingerprint; the engine forwards it to the assistant message row.RedbConversationStorewrites the seven fields intoMessagePropson append and rehydrates them on load; nothing else in the persist / materialise path changes.
Because every new column is nullable on both MessageProps and
ConversationMessageMeta, existing rows and existing call sites compile
and load unchanged. No migrations required — REDB picks up the new
props automatically.
What this still cannot solve. Closed-source provider drift where the backend does not surface a fingerprint (Anthropic, most Gemini-compat endpoints): when the provider silently re-releases a model under the same id, no per-message capture on our side can detect it. For compliance-bound deployments the only honest answer remains self-hosted (
ollama,lmstudio, vLLM viahuggingface) — the alias surface for those is unchanged.
The 3.1.1 audit-fields work above pinned the machine side of the call (model
id, prompt hash, tool-set hash, sampling settings). This follow-up extends the
same MessageProps row with the human / governance side — who issued
the call and under what business labels — so a single row answers the
auditor's full question without joining to anything external.
MessageProps (and the mirror ConversationMessageMeta) gain two more
nullable columns:
| Field | Type | Purpose |
|---|---|---|
UserId |
string? |
Principal id stamped on every row of the run — pulled from the producer's ?user= URI option (literal or ${header.X} expression) or, falling back, the llm.user.id header. |
AuditTags |
Dictionary<string,string>? |
Free-form key → value audit labels stamped on every row. Sources merged at producer time: the ?audit=key=val,key=val URI CSV (each side URL-encoded so commas/equals in literal values are safe) ⊕ inbound llm.audit.<name> headers; headers win on collision so per-call dimensions can override per-route defaults. |
AuditTags is a real REDB Pro Dictionary<string,string> — not JSON. That
means the column is queryable through native LINQ-to-SQL (see
redb.Examples/E060_DictContainsKey, E061_DictIndexer,
E062_DictNestedClass):
// Pull every row that came from a specific tenant, server-side, no client scan:
var rows = await redb.Query<MessageProps>()
.Where(m => m.AuditTags!["tenant"] == "acme-prod"
&& m.UserId == "alice@acme.com")
.ToListAsync();DSL surface — three new fluent methods on LlmBuilder:
.To(LlmDsl.Factory("haiku")
.User("${header.X-User-Id}") // principal — literal or ${header.X}
.Audit("tenant", "${header.X-Tenant}") // repeatable, dynamic
.Audit("env", "prod") // repeatable, literal
.PromptTemplate("triage", "v1") // (name, version) pinned per row
.AsUri())Wiring (additive, no breaking changes — every new field is nullable on both DTOs and every public method keeps its existing signature):
LlmHeadersgainsUserId = "llm.user.id"andAuditTagPrefix = "llm.audit.".LlmEndpointOptionsgainsUser,Audit(CSV),PromptTemplateName,PromptTemplateVersion. Bound from URI by reflection like the rest of the options — no parser change.LlmProducerresolves${header.X}/${property.X}/ literal expressions pre-call against the inbound exchange, merges the?audit=CSV with anyllm.audit.<name>headers (header wins on collision), and pipes the resolved values throughAgentRequest.AgentEngine.PersistMessageAsyncreadsrequest.UserId/request.AuditTagsand stamps both onto everyConversationMessageMetait creates — same row cardinality (system / user / tool-result / assistant), no extra writes.RedbConversationStorewrites both fields on append and rehydrates them on load.Dictionary<string,string>materialises through the framework's native dict serialiser; no custom JSON path on either side.
Demo: see demos/Llm.AuditShell/ — single-file
HTTP shell that exposes both option-side defaults and header-side overrides,
plus the swap comment for RedbConversationStore and the LINQ-by-AuditTags
query above.
redb.Route.Llm — operator-side audit fields: factory alias, base URL, provider response id, latency, key fingerprint, retry count
Building on the UserId + AuditTags row above, this slice closes the
"which connection actually answered, and how long did it take?" gap on the
operator side — the dimensions a host already knows pre-call but had to
reconstruct from logs after the fact. Six more nullable columns on
MessageProps / ConversationMessageMeta:
| Field | Type | Stamped on | Source |
|---|---|---|---|
FactoryName |
string? |
every row | LlmConnectionFactory.Name (the operator-chosen profile alias, e.g. "haiku", "gpt-mini") |
BaseUrl |
string? |
every row | LlmConnectionFactory.BaseUrl (null → provider default endpoint was used) |
ProviderResponseId |
string? |
assistant rows | provider response top-level id (OpenAI / xAI / Together / Anthropic) |
LatencyMs |
long? |
assistant rows | wall-clock around ILlmProvider.CompleteAsync |
ApiKeyFingerprint |
string? |
every row | SHA-256 of LlmConnectionFactory.ApiKey, first 16 hex chars (non-secret) |
RetryCount |
int? |
every row | route-framework retry counter on the inbound exchange (see fallback chain below) |
Why this matters in audit: when a host registers multiple connections to the
same provider — different keys per tenant, separate quotas per environment,
old/new key during rotation — the (provider, model) pair is no longer enough
to answer "which configured connection served this row?". FactoryName +
BaseUrl + ApiKeyFingerprint make the answer one column lookup.
ProviderResponseId cross-references the provider's own usage / billing
logs; LatencyMs lets a compliance pull profile p99 per (tenant, model)
without touching application telemetry.
LINQ-to-SQL example — slow-tail rows for a specific configured connection:
var slowGoldRows = await redb.Query<MessageProps>()
.Where(m => m.FactoryName == "haiku"
&& m.AuditTags!["tier"] == "gold"
&& m.LatencyMs > 5000)
.OrderByDescending(m => m.LatencyMs)
.ToListAsync();Wiring (additive, every field nullable on both DTOs):
LlmResponsegainsProviderResponseId. Both providers —OpenAiProviderandAnthropicProvider— parse the top-levelidfrom the response JSON.AgentEngine.PersistMessageAsyncaccepts two new optional parameters (providerResponseId,latencyMs). The assistant-persist site passeslast.ProviderResponseIdand the stopwatch elapsed around the provider call; non-assistant sites passnull, null.FactoryName/BaseUrl/ApiKeyFingerprintare computed insidePersistMessageAsyncfromrequest.Factoryso every row of the run carries them with no per-call-site plumbing change.ApiKeyFingerprintis computed viaSHA256.HashDataof the UTF-8 key bytes, thenConvert.ToHexString(hash[..8]).ToLowerInvariant()→ 16 hex chars. Empty / null key → null fingerprint. The key itself never reaches the conversation store.RetryCountis read once at the top ofRunAsyncfrom the inbound exchange and stamped identically on every row of the turn. Source order:exchange.Properties["RetryAttempt"](set byRetryProcessorfor the per-step.Retry(...)DSL), thenexchange.In.Headers["CamelRedeliveryCounter"](set byOnExceptionProcessor), thenexchange.In.Headers["CamelDeadLetterRedeliveryCount"](set byDeadLetterProcessor). Null when none are present (first / only delivery).RedbConversationStorewrites and rehydrates all six fields. No schema migration needed — REDB stores added props on existing rows transparently.
redb.Route.Llm.Mcp is a producer-only NuGet that lets the agent consume
the community ecosystem of Model Context Protocol servers (filesystem,
git, fetch, github, sqlite, Serena, …) without writing a C# adapter per
server. The package adds the mcp:// URI scheme — mcp://serverName/toolName
invokes tools/call on the named MCP server with the exchange body as JSON
arguments — and wires a hosted service that, on host startup, spawns each
registered server, performs the initialize + tools/list handshake, and
projects every remote tool into the existing IToolDescriptorRegistry as
an McpToolDescriptor : ILlmToolDescriptor. The agent picks them up via
DI like any native tool.
Because every MCP tool becomes a regular LlmToolCapability, the existing
audit (ToolSetHash), governance (Safety overrides per (server, tool)
regex), observability (OnToolInvokedAsync) and approval pipeline apply
verbatim — no parallel code paths.
Transports. McpTransport.Stdio(command, args, env, workDir) spawns an
external process and exchanges newline-delimited UTF-8 JSON-RPC frames over
stdin/stdout (stdin writes serialised through a SemaphoreSlim, stderr
drained to the logger at trace, stdout pump skips non-JSON lines). The
encoding is BOM-less UTF-8 (UTF8Encoding(false)) — the static
Encoding.UTF8 emits a BOM on first WriteLine and many MCP servers
(Serena, Anthropic reference) reject the BOM-prefixed first frame as
invalid JSON. McpTransport.Http(baseUrl, apiKey) POSTs JSON-RPC to the
base URL and opens an SSE channel for server-initiated frames
(notifications/tools/list_changed triggers a registry rebuild).
Cancellation. IProducerTemplate.RequestBody(uri, body, ct) (the
CT-aware overload) threads the cancellation token through IProducer.Process
and into IMcpClient.CallToolAsync(ct). On cancel the client emits a
JSON-RPC notifications/cancelled for the pending request id and removes
the TCS so callers stop waiting.
Tool name budget. Provider tool-name caps (Anthropic / OpenAI) max at 64
chars. McpToolDescriptor.BuildModelFacingName(server, tool) sanitises both
parts to [a-zA-Z0-9_], truncates the server prefix to 24 chars and the
tool to 36, and joins with __ (e.g. serena__get_symbols_overview).
Duplicates after truncation are logged and skipped.
Wiring.
services.AddRedbRoute()
.AddRedbRouteLlm()
.AddRedbRouteMcp()
.AddMcpServer("serena", McpTransport.Stdio(
"uvx",
["--from", "git+https://github.com/oraios/serena",
"serena", "start-mcp-server",
"--context", "ide",
"--project", projectPath]));The hosted service registers before RouteHostedService, so descriptors
are in the registry by the time routes compile.
Status / liveness. IMcpClient.Status exposes
Idle / Connecting / Healthy / Restarting / Dead; the producer
short-circuits with McpException when a registered client is Dead (no
silent hangs on a torn-down transport).
IProducerTemplate gained a third overload that accepts a
CancellationToken. Existing call sites that use the two-argument form
compile unchanged (the no-CT overload remains as a ct: CancellationToken.None
shim). AgentEngine.DispatchToolEndpointAsync now threads its run-level CT
through to the producer, so an aborted agent iteration cancels the in-flight
tool RPC at the transport layer instead of waiting for it to finish before
unwinding.
redb.Route.LlmI*Store contracts. Every store interface inredb.Route.Llm/Engine/Storage/*(IApprovalStore,IConversationStore,ICostBudgetStore,IEvalRunStore,IKnowledgeStore,IPromptTemplateRegistry,IToolCacheStore,IToolIdempotencyStore) gained an optionalIExchange? exchange = nullparameter to thread the named-redb hint through. Source-compatible: optional with default, existing implementations / call sites compile unchanged.ToolIdempotencyPropsschema — the per-tool-call idempotency rows moved from the genericToolCachePropsshape to a dedicatedToolIdempotencyPropsschema with explicit lifecycle fields. The two surfaces previously shared one table; splitting them lets the cache TTL and the idempotency receipt evolve independently. No data migration shipped — early-3.1.x adopters runningAddRedbLlmStorage()against populated data should treat this as fresh state (the wider rollout happens with thePhase 2story, where stores get their migration helpers).
IProducerTemplate.SendAsync(endpointUri, …) resolved an endpoint via
Context.GetEndpoint(uri) → cached an IProducer via endpoint.CreateProducer() →
called producer.Process(exchange) directly. For DirectVm / Direct /
Seda producers this was fine because they don't extend ConnectableProducer,
but for every other transport (HttpProducer, KafkaProducer, AmqpProducer,
AzureServiceBusProducer, MqttNetProducer, RabbitMqProducer, RedisProducer,
SmtpProducer, LdapProducer, WmqProducer, …) EnsureStarted() threw
InvalidOperationException: <name> has not been started. Call Start() first.
because ConnectableProducer.Process requires Start() to flip the started
flag and call ConnectAsync first. The cached producer was created but
never started, so SendAsync was effectively broken for every connection-
based transport — direct-vm-only scenarios masked the gap.
ProducerTemplate.SendAsync(IEndpoint, IMessage),
ProducerTemplate.SendAsync(IEndpoint, object),
ProducerTemplate.RequestBody(IEndpoint, object, ct), and
ProducerTemplate.RequestBody(IEndpoint, IMessage, ct) now call
await producer.Start(ct).ConfigureAwait(false) between
GetOrCreateProducer and the first Process call. ConnectableProducer.Start
short-circuits via Interlocked.CompareExchange on the started flag, so the
extra call is a one-time setup per producer / process-lifetime and a no-op
on every subsequent send.
This is the seam that unblocked outbound HTTP webhook delivery in
redb.Identity (W1 / outbound webhook subscriptions) — the identity events
route hands the message to ProducerTemplate.SendAsync(subscription.Url, …)
and the URL scheme (https://…, kafka://…, amqp://…) resolves to the
right transport without the Identity codebase touching IHttpClientFactory
or any transport-specific surface.
redb.Route.Controllers — HttpControllerDispatcher.WriteResult clears Out.Body when the controller returns null
When an HTTP controller returns null (intended: no response body → 204
No Content), HttpControllerDispatcher.WriteResult initialised the
response via:
exchange.Out ??= exchange.In.Clone();
var defaultCode = result is null ? 204 : 200;
if (result is not null) { exchange.Out.Body = result; }The Out ??= In.Clone() carried In.Body across. For HTTP DELETE / HEAD
requests with Content-Length: 0 In.Body is Array.Empty<byte>() —
non-null byte[]. With result is null the if-branch was skipped and
Out.Body stayed as that empty byte[]. Downstream the HTTP consumer
matched body is byte[] and called Response.Body.WriteAsync(...),
which on Kestrel hard-throws for 204 per RFC 7230 §3.3.3 / RFC 9112 §6.1
("Writing to the response body is invalid for responses with status
code 204" from HttpProtocol.FirstWriteAsyncInternal — fires even for
zero-length writes). Earlier pipeline side-effects (database mutations,
audit events) had already committed, so clients saw a torn TCP response
instead of a clean 204.
The dispatcher now explicitly nulls exchange.Out.Body in the result is null
branch. The request body is input; it must not echo into the response.
Symptom observed on SCIM DELETE /Users/{id} (RFC 7644 §3.6 mandates
204) but the bug is generic to any controller that signals 204 by
returning null.
Defense-in-depth companion to the dispatcher fix above. RFC 7230 §3.3.3 /
RFC 9112 §6.1 require that 1xx, 204, and 304 responses MUST NOT contain a
message body, and Kestrel hard-throws on Response.Body.WriteAsync for
those status codes — even on zero-length writes. HttpConsumer.WriteResponse
called WriteAsync unconditionally when body is byte[] and tore the
TCP response if any upstream layer set a body for those statuses.
The consumer now resolves the response status code before reaching the
body-write branches and short-circuits with an intentional no-op when
the status is 204, 304, or any 1xx. The header copy above the body
block still propagates Location / ETag / Set-Cookie, which is the
only legitimate payload for these status families. Silently dropping a
non-empty body for these codes is safer than letting Kestrel kill the
response mid-flight — a producer with a bug to fix is a less acute
symptom than a torn TCP connection visible to clients.
When a parallel Splitter (.Split(...).Parallel()) or Multicast
(MulticastProcessor, parallel by default) runs inside a .Transacted(...)
segment, every branch was dispatched with Task.Run — which flows the caller's
ExecutionContext. Because the route's TransactionScope uses
TransactionScopeAsyncFlowOption.Enabled, all branches observed and could
concurrently enlist resources in the same Transaction.Current.
System.Transactions forbids concurrent use of a single transaction across
threads: a second concurrent enlistment of a resource that participates in the
ambient transaction (SQL / ADO.NET / redb DB work) either promotes to MSDTC or
throws "transaction context in use by another thread".
Each parallel branch now runs under its own
Transaction.Current.DependentClone(DependentCloneOption.BlockCommitUntilComplete)
(new internal DependentTransactionBranch helper): the dependent clone is a
private ambient transaction for that branch's thread, and the parent's commit
blocks until every branch signals completion, so a branch's writes are never
committed half-finished. No-op when there is no ambient transaction (the common
non-transacted path runs with zero overhead).
Broker transports that defer via Properties["TRANSACT_ACTION"]
(Kafka / RabbitMQ / Redis / Azure Service Bus / AMQP) were never affected:
they do not enlist in System.Transactions, and each registers under a
per-message-unique key (kafka-send-{guid}, rabbitmq-ack-{deliveryTag},
asb-ack-{sequence}, …), so concurrent fan-out branches to the same endpoint
accumulate distinct entries in the thread-safe dictionary and commit/roll back
atomically. The stale TransactedProcessor doc comment that claimed keys were
"typically the endpoint URI" has been corrected.
redb.Route — detached branches (WireTap, Debounce) no longer leak the caller's transaction/trace context
WireTapProcessor (fire-and-forget Task.Run) and DebounceProcessor's
quiet-period flush (Task.Delay(...).ContinueWith(...)) both dispatch
downstream work on a thread-pool continuation that, by default, inherits
the caller's ExecutionContext. Two ambient values flowed across that
boundary and were unsafe once the originating route call had unwound:
System.Transactions.Transaction.Current— routes wrap segments in aTransactionScopecreated withTransactionScopeAsyncFlowOption.Enabled(TransactionPolicy.CreateScope), so the ambient transaction flows. A detached branch starting after the scope completed/disposed still saw the leakedTransaction.Current; any producer/DB code that auto-enlists threw "the current TransactionScope is already complete". A WireTap or Debounce nested inside a.Transacted(...)segment is the reproducer.System.Diagnostics.Activity.Current— the originating span, likewise already stopped, so telemetry the branch emitted was parented to an ended span (a child whose start time post-dates its parent — a "tail" hanging off a request that already returned).
Both branches now route through a new internal DetachedDispatch helper.
Capture() snapshots the trace context + transaction presence on the
originating thread; Enter() runs at the top of the branch body and
(a) opens a TransactionScope(Suppress) so the branch runs with no
ambient transaction (only when one actually leaked — zero cost otherwise),
and (b) re-roots telemetry as a fresh root span linked (ActivityLink)
to the originating trace — correlation is preserved without the broken
parent lifecycle. Non-transaction AsyncLocal state (user/auth context) is
deliberately left flowing, since a detached audit branch usually needs it.
Additionally, WireTapProcessor now strips the deferred-transport-action
dictionary (Properties["TRANSACT_ACTION"]) from its clone: Exchange.Clone()
copies Properties shallowly, so the tap clone previously shared the same
ConcurrentDictionary that the owning TransactedProcessor commits/rolls
back, and a tap mutating it could race the main commit. The tap runs with the
transaction suppressed, so it has no part in that set.
ThrottleProcessor / KeyedThrottleProcessor were audited and are not
affected — their detached ContinueWith only releases a rate-limit semaphore
slot; the downstream Process is awaited inline. The concurrent-enlistment
behaviour of parallel Splitter / Multicast branches (all sharing one
ambient transaction under Task.WhenAll) is a distinct concern tracked
separately and intentionally out of scope here.
SharedHttpServerManager matched routes in pure registration order and
returned the first whose template matched. A catch-all (/{**path})
therefore swallowed every route registered after it on the same listener —
a concrete path such as /api/echo could never win once a {**path}
dispatcher was already registered (acute when the catch-all auto-starts at
boot and the specific route registers later, e.g. an AutoStart(false)
route started by hand). ServerEntry.GetCompiled() now orders the match
table by specificity — literal-heavy templates first, route parameters
before more parameters, catch-all ({**…}) last — with registration order
kept as a stable tie-breaker so equal-specificity routes preserve their
previous first-registered-wins behaviour. Both MatchRoute and the
CORS-dispatch MatchByPath consume the ordered table, so a specific path
and a {**path} fallback can coexist on one port, matching ASP.NET-style
routing precedence.
AgentEngine.RunAsync now sanitises the loaded conversation path: if the
last persisted message is an assistant turn that has tool_use blocks
without a matching tool_result user turn after it (the previous run was
cancelled, timed out, or threw between persisting the assistant message
and dispatching the tool — see AgentEngine.cs lines 195/222), a
synthetic tool_result(error: "orphaned_tool_use_recovered") user message
is appended and persisted before the new user prompt is added.
Without this, any provider that strictly enforces tool_use/tool_result
pairing — notably Anthropic's Messages API
(400 invalid_request_error: tool_use ids were found without tool_result blocks immediately after) — 400's forever on every subsequent request,
poisoning the conversation permanently. RedeliveryPolicy then multiplies
the failure across retries.
The recovery is logged at warning level (Recovered {N} orphaned tool_use block(s) in conversation {Conv} on load.) so production occurrences are
visible. Applies uniformly to InMemoryConversationStore and
RedbConversationStore — recovery happens after LoadPathAsync,
provider-agnostic.
ExecProducer now sets ProcessStartInfo.StandardOutputEncoding /
StandardErrorEncoding to the host console's active codepage (cp437,
cp932, cp936, cp949, …) on Windows, falling back to UTF-8 on Linux/macOS.
Without this, .NET defaulted to UTF-8 when reading the redirected
streams while cmd.exe / fsutil / wmic / net emit OEM bytes — the
mismatch surfaced as U+FFFD replacement characters in redbExec.Stdout
and the downstream JSON tool body, breaking LLM agents on
Japanese / Chinese / Korean / Greek / Turkish-locale Windows hosts (any
non-Latin OEM codepage).
Adds a dependency on System.Text.Encoding.CodePages 9.0.0 — the BCL
only ships ASCII/UTF-8/UTF-16/UTF-32 encodings on .NET; cp932/cp936/cp949
require CodePagesEncodingProvider.
McpClientBase.OnTransportFailed now sets Status = McpClientStatus.Dead
in addition to failing pending requests. Previously, when an stdio child
process exited unexpectedly (or the read pump tripped), the client failed
in-flight requests but kept reporting Healthy, so subsequent
tools/call requests went through the producer and silently hung waiting
on a defunct stdin. The producer's if (Status is Dead) throw short
circuit was unreachable. The fix makes process death immediately
observable both at the registry level and at the producer level.
redb.Route — OnException declared inside a nested scope is now hoisted to route level (Camel parity)
RouteDefinition.CreateProcessor only scanned the top-level route
outputs for inline OnException blocks. An OnException declared inside a
nested scope — Transacted(), Traced(), Metered(), Throttle(),
Filter(), … — was never hoisted. Worse, the orphaned definition was then
compiled by the enclosing scope's pipeline builder via its silent
CreateProcessor fallback, which emitted the handler chain as an inline
pipeline step: the exception handler body executed on every exchange
with Exception == null, corrupting healthy requests (e.g. overwriting the
request body with an error response). OnWhen / Handled / redelivery
settings were silently ignored.
Two changes:
- Recursive hoisting. The route compiler now collects
OnExceptiondefinitions from the entire definition tree (depth-first, declaration order) and wraps the full route body with the handler envelopes — Apache Camel parity:onExceptionis route-scoped regardless of where it appears textually. Wrapping order is unchanged: last declared = outermost. - Fail-fast instead of silent fallback.
OnExceptionDefinition.CreateProcessorno longer compiles the handler chain as a standalone pipeline. A hoisted definition compiles to a no-op at its declaration site; a definition the compiler cannot hoist (e.g. declared inside aCatch/Finallyblock or another exception-handler pipeline) now throwsInvalidOperationExceptionatStart()with placement guidance, instead of corrupting traffic at runtime.
Found in redb.Identity: the /connect/token route wraps its body in
Transacted(...), so its OnException<InvalidOperationException> OAuth
error mapper ran inline on every token request and replaced the form
parameters with an error body before the OpenIddict extract step —
unsupported_grant_type/HTTP 400 on perfectly valid requests.
Why a minor bump (3.0.x → 3.1.0). This release ships four new NuGet packages (
redb.Route.Llm,redb.Route.Llm.Abstractions,redb.Route.Llm.Tools,redb.Route.Exec), one new URI scheme (exec:), a second LLM provider (nativeAnthropicProvider), and a new persistence extension (AddRedbLlmStorage) that brings five stores and nine REDB schemas. All additions are backwards-compatible — no public API on existing packages was removed or renamed — but the surface area added is too large to bury under a patch bump.
The Camel-style LLM connector becomes a published package. From("…")/
To("llm://…"), fluent builder (Llm.Factory("haiku") …), Camel-style
agent loop with tool dispatch, headers/URI options for system prompt,
conversation id, max tokens, temperature, max iterations, etc. See
redb.Route.Llm/README.md and doc/USER-GUIDE.md for the full surface.
OpenAiProvider— one provider class covering 14 OpenAI-compatible APIs throughLlmConnectionFactory.Build()aliases:openai,anthropic(OpenAI-compat endpoint),groq,cerebras,openrouter,gemini(OpenAI-compat endpoint),github-models,mistral,together,huggingface,deepseek,ollama,lmstudio, pluscustomfor any self-hosted gateway. The provider id only switches the default base URL and a couple of provider-specific headers (e.g. OpenRouter'sHTTP-Referer+X-Title).AnthropicProvider— native Messages API transport (POST /v1/messages), separate from the OpenAI-compat path. MapsLlmRequestto Anthropic'smessages/tools/tool_use/tool_resultcontent-block model and reassemblesLlmResponsefrom the standard envelope. Streaming is true SSE —content_block_start/content_block_delta/content_block_stopevents are reassembled per block; tool-use blocks accumulateinput_json_deltapartial JSON and surface as a single completeLlmToolUseBlockat end-of-block. Error mapping: HTTP 429 →LlmRateLimitException(honoursretry-after); HTTP 529 ("overloaded") and 5xx →LlmTransientException. JSON serialisation usesJavaScriptEncoder.UnsafeRelaxedJsonEscapingso Cyrillic / emoji /&/>/<are emitted as UTF-8, not\uXXXX— fixes unicode-escaped tool input in[*-TOOL] ▶ in=…route logs.- Reasoning-model fallback in
OpenAiProvider: when the responsemessage.contentis empty, the provider falls back tomessage.reasoningso models like Cerebrasgpt-oss-120borzai-glm-4.7still surface a textual answer throughLlmResponse. AddRedbLlmStorage()extension onIServiceCollection— wires the LLM connector to a named REDB instance through the Tsak registry key"redb-factory:{name}". Ships five stores backed byIServiceScopeFactoryper named instance:IConversationStore—RedbConversationStore: persistent multi-turn memory across runs and processes;AgentEngine.LoadPathAsyncresumes a transcript by id.IApprovalStore—RedbApprovalStore:IApprovalGatedecisions survive restarts, supports human-in-the-loop tools.ICostBudgetStore—RedbCostBudgetStore: per-tenant spend tracking, drivesIBudgetEnforcerhard cut-offs.IToolIdempotencyStore—RedbToolIdempotencyStore: dedup of expensive tool calls across retries.IAgentObserver—RedbAgentObserver: full audit of every iteration / tool invocation / approval into the store.- Nine REDB schemas (
[RedbScheme]POCOs):Conversation,Message,Approval,CostBudget,ToolCache,ToolAudit,KnowledgeChunk,PromptTemplate,EvalRun. Seeredb.Route.Llm/doc/STORAGE.mdfor the recipe catalogue (multi-turn chat, approval gates, hard budget, idempotent retries, audit, branching, scheduled agents).
#-registry prompts —LlmConnectionFactoryresolves system prompts from a registry by#nameso prompt text lives in the host's Tsak config layer, not inline in route code.- Live integration test infrastructure under
tests/redb.Route.Tests.Llm:LiveProviderTests— 5 scenarios (Smoke / NonAscii / ToolUse / Usage / StopReason) × free-tier providers (GitHub Models, Groq, Cerebras, OpenRouter, Gemini, Mistral, native Anthropic). Live end-to-end coverage with auto-skip when API keys are absent.LiveEndToEndTests— exercises the fullLlmComponent → LlmEndpoint → LlmProducer → AgentEnginepath against a real provider, including a tool-loop drivingIToolRegistry.LiveDslRouteTests— Apache Camel-style end-to-end routes:From("direct://...") → Process → To(Llm.Factory(...)) → Process → To("mock://..."), a two-LLM judge chain, and cross-context RPC overdirect-vm://llm-serviceusing a sharedSharedVmRegistry.ExecShellToolTests— agent +exec:shell tool against live Anthropic / OpenAI-compat providers.UtilityToolTests— agent +redb.Route.Llm.Tools(HttpFetch / JsonPath / XPath / MathEval / RegexExtract / Tavily) against live providers.
[EnvFact("VAR")]xUnit attribute — auto-skips a fact when the named environment variable is missing, so contributors without API keys keep a green build while CI with the right secrets runs the full live matrix.[Collection("LiveLlmSerial")]— shared collection across all live LLM tests so xUnit parallelism does not multiply free-tier rate limits.
A small, dependency-light contract package. Exists separately from
redb.Route.Llm so any of the 23 transports can expose itself as an
LLM tool by implementing .AsLlmTool(name) on the From(uri) route —
zero connector version bumps, zero transitive dependency on the LLM
provider implementation.
ILlmToolDescriptor— descriptor contract: capability metadata + endpoint URI the agent dispatches to.LlmToolCapability—Name,Description,InputSchema(JSON Schema string),LlmToolSafety(SideEffect,Caching,Cost,RequiresApproval,RequiredClaims).IToolDescriptorRegistry— global registry; populated by.AsLlmTool(...)at route-build time, queried byAgentEngineat dispatch time.RouteToolBridge— bridges anyFrom(uri)endpoint into the LLM tool surface. Forwards the model's JSON input through the host's producer template, inheriting the parent agent route's transaction scope, headers, principal and DI scope.[ExposeAsLlmTool]attribute — alternative to the fluent DSL: decorate a handler class and the bootstrapper turns it into a registered descriptor..AsLlmTool(name)DSL aspect inLlmToolDsl— Apache-Camel-style metadata aspect placed immediately after.From(uri). Closes with.End()or.Then(). Example:Works with any transport:From("direct:order-lookup") .AsLlmTool("get_order") .Description("Returns order details by id.") .Input("""{"type":"object","properties":{"orderId":{"type":"string"}},"required":["orderId"]}""") .SideEffect(ToolSideEffect.ReadOnly) .Cost(ToolCostClass.Cheap) .Then() .Bean<IOrderService>((svc, ex) => svc.HandleAsync(ex));
Direct,Http,Grpc,Sql,Sftp,File,Redis,Exec, etc. for request-response tools;Kafka,MQTT,Mail,SignalRfor fire-and-forget action tools.
Six ready-to-use utility tools that live as ordinary RouteBuilder
classes registered by .AsLlmTool(...), so they participate in the
same transaction scope, telemetry, error handling and DI as any other
route. All optional — depend only on what the agent needs.
| Tool | Purpose |
|---|---|
HttpFetchTool |
GET <url> with size cap and host allowlist; returns body + status + headers. Built on redb.Route.Http. |
JsonPathTool |
Evaluate a JSONPath expression against a JSON document. Built on the core compiled-JPath engine. |
XPathTool |
Evaluate an XPath expression against an XML document. Built on the core compiled-XPath engine. |
MathEvalTool |
Safe arithmetic evaluator (integers / decimals / + - * / % ^, parentheses, common functions). |
RegexExtractTool |
Apply a regex to a string; return all matches and named groups. |
TavilyWebSearchTool |
Tavily Search API (https://api.tavily.com/search); returns top-N results with snippets. API key via TAVILY_API_KEY. |
Local-process execution transport. New URI scheme exec: with two
operations: exec://run (one-shot producer) and a scheduled consumer
that runs commands on a cron: / qtimer: trigger.
AllowedCommands(params string[])— explicit allowlist. Every invocation whose command is not on the list is rejected before a process starts; this is the security envelope for LLM-driven shells.WorkingDirectory(string)— pinned CWD; relative paths in tool arguments resolve there, files written in one call survive to the next. Without this, processes inherit the worker's CWD (e.g. the source tree underdotnet run).TimeoutMs— hard kill on timeout.MaxStdoutBytes/MaxStderrBytes— cap captured output.- Output headers —
redbExec.ExitCode,redbExec.StdoutBytes,redbExec.StderrBytes, plusredbExec.TimedOut. exec:request schema —{"command":"<name>","args":["..."]};redbExecheaders,stdout,stderr,exitCodereturned. Designed to drop straight into.AsLlmTool("shell")for an LLM-driven shell; the demoredb.Route.DemoHTTP showcase wires it into a Claude agent. Seeredb.Route.Exec/README.md.
Two endpoints in LlmHttpRoutes modelled as the simplest possible
Camel-readable round-trip:
POST /api/llm/ask— body is the user prompt, six-step route asks Claude Haiku, logs token usage + stop reason, returns the model's reply as plain text. Conversation memory viaX-Chat-Idheader.POST /api/llm/shell— same shape but the agent has ashelltool wired throughExecComponentwith a pinned scratch directory underPath.GetTempPath()/redb-llm-shell/, allowlist{cmd, pwsh, powershell}on Windows /{sh, bash}on Linux, 5 s timeout, 8 KiB stdout/stderr caps.
OpenAiProvider.BuildRequestBody—LlmToolResultBlocknow always emitsrole: "tool"regardless of the originalLlmMessage.Roleof the block it lives on. Previously, whenAgentEngineproduced an Anthropic-stylerole: "user"message that carried a tool-result block, strict OpenAI-compatible gateways (Groq) rejected the request with400 messages.X : for role:user content not nullable. The provider now partitions blocks per message and emits a separaterole: toolentry per tool-result block.AnthropicProvider.JsonOpts— switchedEncodertoJavaScriptEncoder.UnsafeRelaxedJsonEscaping. Without it,block["input"]?.ToJsonString(JsonOpts)(response-parse path, surfaced asLlmToolUseBlock.InputJson) escaped Cyrillic / emoji /&/>/<to\uXXXXand made tool-input route logs unreadable. MatchesJsonMessageSerializer.DefaultOptions— safe for HTTP API responses; only unsafe inside HTML / inline JS, which this code path never produces.AgentEngine— now invokesIConversationStore.LoadPathAsyncwhen aConversationIdis present on the request. The persisted transcript was being written but not read back, so resumed runs saw an empty history. With the fix, multi-turn chat works end-to-end viaRedbConversationStore(AddRedbLlmStorage()).
redb.Route(DSL) — added a new universalEnd()extension method onIRouteDefinitionand a full set of typedEnd*()extension methods (EndFilter,EndChoice,EndWhen,EndOtherwise,EndSplit,EndMulticast,EndAggregate,EndCircuitBreaker,EndThrottle,EndDebounce,EndLoop,EndTryCatch,EndOnException,EndTransaction,EndLog,EndResequence,EndTraced,EndMetered,EndIdempotentConsumer,EndSaga). Each typedEnd*()walks theParentchain looking for a scope of the requested type and returns its parent route. This means a single.EndChoice()call from deep insideChoice → When → Split → Loglands directly at the route root — semantically identical to chaining.EndLog().EndSplit().EndChoice()but more concise when the intermediate scopes do not need extra steps. Each helper throws a preciseInvalidOperationExceptionwhen called outside a matching scope.redb.Route(DSL) — addedWhen(...)andOtherwise()as extension methods onIRouteDefinition. They walk theParentchain to find the enclosingChoiceDefinitionand dispatch to its instance method, so a sibling branch can be opened immediately after a sub-scope closes — for example.Choice().When(p).Split(...).EndSplit().When(p2).Process(...).EndChoice()now compiles and behaves the same as the equivalent nested-lambda form. Instance methods onChoiceDefinition/WhenDefinition/OtherwiseDefinitionkeep precedence over the extensions, so existing call sites are unaffected.redb.Route(DSL) — added a focused test fixture (DeepNestedDslTests, five scenarios) coveringChoice/When/Otherwise/Split/RichLogcomposition,TryCatchwith rich logging insideDoCatch<T>, mixed typed and universalEnd*()closers, cascadingEndChoice()from deep inside, and the diagnosticInvalidOperationExceptionraised whenEnd*()is called outside any matching scope.
redb.Route(DSL) — removed the legacyRouteStep/RouteStepProjectionAST and theRouteDefinition.Stepsprojection. TheProcessorDefinitiontree built by the fluent DSL is now the single source of truth for route construction; everything that used to readSteps(Normalizer, Saga, integration tests) now usesCreateProcessordirectly. The legacy files have been moved out of the shipping assembly intotmp/oldRoute/for reference only.
redb.Route(DSL) — the leaf DSL (To,Process,ProcessAsync,SetBody,SetHeader,SetProperty,RemoveHeader,RemoveProperty,Transform,Validate,Marshal/Unmarshal,ConvertBody,Stop,Delay,Sample,BeginTransaction/Commit/Rollback,SetPattern,Respond,Bean,StreamCaching,Throw*,Log*, plus every scope-opener:Filter,Choice,Split,Multicast,Loop,Aggregate,IdempotentConsumer,Throttle/Debounce/KeyedThrottle,Metered,Traced,Resequence,Transaction,Saga,OnException,OfType<T>,CircuitBreaker,TryCatch, etc.) is now defined exactly once in a new generic CRTP base,RouteDefinitionBase<TSelf>, instead of being duplicated across 27 scope-definition classes. Each typed leaf method returnsTSelf, so chaining always preserves the current scope's concrete type — e.g..Filter(p).To("a").SetHeader("k","v")keeps you onFilterDefinition,.Choice().When(p).To("a")keeps you onWhenDefinition, and only the explicitEnd*()/End()step exits the scope. There is no behavioural change for end users; the public DSL surface and route AST shape are identical to 3.0.0.redb.Route(DSL) —RouteDefinitionis now a thinRouteDefinitionBase<RouteDefinition>subclass that retains only route-level concerns:RouteId,From,AutoStart,Cluster,ProcessingTimeout,RoutePolicy,OnExceptionhoisting, andCreateProcessor. All other behaviour is inherited.redb.Route(DSL) — every pipeline-scope class (FilterDefinition,ChoiceDefinition/WhenDefinition/OtherwiseDefinition,CircuitBreakerDefinition/FallbackDefinition,LoopDefinition,SplitDefinition/MulticastDefinition,TryCatchDefinition/CatchDefinition/FinallyDefinition,IdempotentConsumerDefinition,OnExceptionDefinition,TransactionDefinition,SagaDefinition,MeteredDefinition,TracedDefinition,ResequenceDefinition,ThrottleDefinition/DebounceDefinition/KeyedThrottleDefinition,AggregateDefinition,OfTypeDefinition<T>,OfTypeFilterDefinition<T>) now inherits fromRouteDefinitionBase<TSelf>and contains only its own scope-specific configuration (options, branch openers,End*()navigation,CreateProcessoroverride). Per-class duplicates of the leaf DSL have been removed.redb.Route(DSL) —IRouteDefinitionremains the canonical cross-version contract;RouteDefinitionBase<TSelf>provides explicit interface implementations for every leaf method (split into a partial file,RouteDefinitionBase.IRouteDefinition.cs), so existing extension methods, test mocks, andAction<IRouteDefinition>configurators continue to bind unchanged.redb.Route(DSL) — non-pipeline definitions (LoadBalancerDefinition,ScatterGatherDefinition,NormalizerDefinition,RichLogScopeDefinition) intentionally remain onProcessorDefinition: they have no childOutputspipeline and no leaf DSL — they are configuration builders, and inheriting the CRTP base would have inflated their public surface with methods (To,Process, …) that are semantically invalid in those scopes.
redb.Route(DSL) —IRouteDefinition.GetContext()now correctly returns the owningIRouteContextwhen called on any nested scope (WhenDefinition,LoopDefinition,TracedDefinition,CatchDefinition, etc.). Previously it relied onself as RouteDefinition, which only matched the route root; after the CRTP refactor scope classes inherit fromRouteDefinitionBase<TSelf>(not fromRouteDefinition), and the cast silently returnednullinside any scope. The accessor now walks theParentchain up to the owningRouteDefinitionand returns itsContext. This restoresContext_IsAvailable_In{Choice,Loop,Traced,DoTry}Scopesemantics for extension methods that read context at DSL build time.redb.Route(DSL) —SagaDefinition.SetParentis no longer required: the parent link is now established uniformly throughAddOutput, which matches every other scope and removes a small inconsistency in the AST build path. Existing user code is unaffected.
redb.Route(DSL) — Package A "enterprise EIP closure": the parallel v2 type tree (IRouteDefinition2,RouteBuilder2,BlockStack,ExceptionRouteDefinition, the v1OldRouteCompiler, the v1 typedAbstractions/Typed/IRouteDefinition.cs, etc.) has been collapsed into a single canonical surface —IRouteDefinition/RouteDefinition/RouteBuilder. The route AST is now exclusively built fromIProcessorDefinitionnodes, each of which compiles itself viaCreateProcessor(IRouteContext); there is no separate compiler class. The previous "v2 DSL → bridge → legacy compiler" indirection has been removed.redb.Route(DSL) —IRouteContextis now propagated down the definition tree via aParentchain, so any nested*Definitioncan reach the owning context (logger factory, services, idempotent repositories, policy factories) without explicit threading.redb.Route(DSL) —RouteStepProjection: a read-only canonical projection of theIProcessorDefinitiontree intoRouteSteprecords, exposed asRouteDefinition.Steps. Intended for diagnostics, validation, and tooling (e.g. route visualisers); it is not used by the runtime compiler.FromStep,ToStep,FilterStep(with optionalSubStepsbody),ChoiceStep,SagaRouteStep, etc. all flow through this projection.redb.Route(DSL) —RouteBuilder.DefinitionsandRouteBuilder.ExceptionDefinitionsare nowpublic(previouslyinternal). This unblocks downstream test fixtures and tooling that need to introspect the route AST afterBuild().redb.Route(DSL) —OnExceptionDefinitiongained the fluent settersLogStackTrace(bool)andLogExhausted(bool)to match the rest of the CamelonException(...)builder surface.
redb.Route(DSL) —DynamicEndpointResolver: per-instance producer cache keyed by the URI resolved at runtime. Three constructors accept a string template (${header.xxx}/${property.yyy}/${body}placeholders), anIExpressioninstance, or a rawFunc<IExchange, string>. Producers are tracked viaRouteContext.TrackProducer(...)for graceful shutdown.redb.Route(DSL) —ToDynamicProcessor+ToDynamicDefinitionimplement Camel'stoD(...)—IRouteDefinition.ToD(string|IExpression|Func).redb.Route(DSL) —WireTapDynamicDefinition,EnrichDynamicDefinition,PollEnrichDynamicDefinitionand matchingIRouteDefinition.WireTap(...)/Enrich(...)/PollEnrich(...)overloads that accept a dynamic URI.EnrichProcessorandPollEnrichProcessorgained an alternate constructor taking aDynamicEndpointResolver; theirProcesschooses between the resolver and the cached producer at run time.redb.Route(DSL) — string-template expression DSL:SetBodyExpression(...),SetHeaderExpression(...),SetPropertyExpression(...)onIRouteDefinition.redb.Route(DSL) —LogDefinition.LogStaticDefinitionauto-upgrades toTemplateLogProcessorwhen the configured message contains a${...}placeholder, so users get template-interpolation without a separate API.redb.Route(Core) —RouteContextnow registers the currentILoggerFactoryinto its service collection so processors built from.Log(...)/ template expressions can resolve their logger without extra plumbing.
redb.Route(Tests) — new DSL reference suites that pin Camel semantics with extensive scenario coverage:Reference/DslChoiceReferenceTests.cs(~767 lines),Reference/DslDoTryReferenceTests.cs(~441 lines),Reference/DslFilterReferenceTests.cs(extended). These are the authoritative compatibility specs for Choice/When/Otherwise, TryCatchFinally and Filter scope semantics.redb.Route.Tests.Core— twelve tests (RedbRouteExtensionsTests,RedbTransactedActionTests) were rewritten on top of the realRouteDefinition+Exchangepipeline, removing the previousIRouteDefinitionmock-based scaffolding.
redb.Route.IbmMq— diagnostic timing aroundMQGET. The consumer emits aDebug-levelMQGET blocked for {N}mslog entry for any blocking get longer than ~50 ms. This was originally raised atInformationwhile diagnosing a ~500 ms producer→consumer latency in production; it has been lowered toDebugso it stays silent under default verbosity and only lights up when ops explicitly enable IBM MQ diagnostics.IbmMqProducer/IbmMqMessageHelper/IbmMqEndpoint/IbmMqComponentreceived the supporting plumbing.
-
redb.Route.IbmMq— ~500 ms minimum end-to-end latency on the managed client. The managed IBM MQ .NET client (amqmdnetstd.dll) used by this package is not event-driven onMQGETwithMQGMO_WAIT. It carries an internal polling tick of ~500 ms that is independent of theWaitIntervalsupplied inMQGMO:WaitIntervalonly governs the upper timeout, not the lower delivery-granularity bound. As a result the typical producer→consumer latency on this transport is ~500 ms even after channel reconfiguration (we have validatedSHARECNV(1)onDEV.APP.SVRCONN— it does not change the floor). The native (unmanaged) client is event-driven but requires the IBM MQ Client redistributable to be installed on the host, which is not viable for self-contained .NET deployments and is therefore not used here.Planned fix: rewrite
IbmMqConsumer.ReceiveLoopAsyncto use the managed async-consume API (MQQueue.Cb(...)+MQQueueManager.Ctl(MQOP_START, ...)). With the callback path the broker pushes messages and per-message latency drops to ~0. Tracked for a future release; the change is non-trivial because the loop becomes callback-driven (different cancellation, back-pressure and lifecycle model than the current poll loop). See the in-sourceKNOWN ISSUEblock inIbmMqConsumer.csfor details.Field diagnosis recipe. Enable
Debugonredb.Route.IbmMq.IbmMqConsumerand inspect theMQGET blocked for {N}mslog line:N ≈ 500 msconsistently → managed-client polling tick; the MQCB rewrite above is required.N < 50 mswhile end-to-end latency is still ~500 ms → the bottleneck is on the producer side (PUT missing a flush or an extra round-trip), not the consumer.
redb.Route(Telemetry) — shared telemetry identity. BothMeterandActivitySourcenow use a single canonical nameredb.Route, exposed via theRouteActivitySource.TelemetryNameconstant (also surfaced asRouteActivitySource.SourceNameandRouteMetrics.MeterName). OTel collectors can subscribe once and get both signals.redb.Route(Telemetry) —RouteTelemetryExtensions.StartTransportSpan(...)helper that opens a transport span with the conventional OpenTelemetry semantic attributes (messaging.system/db.system/http.method/rpc.system/network.transport, plusredb.route.endpoint,messaging.destination.name,messaging.operation). Returnsnullwhen no listener is registered (zero overhead).redb.Route(Telemetry) —ProcessorMetricsgained 16 new instruments covering the previously-unmeasured EIP processors:- WireTap:
redb.route.wiretap.dispatched,redb.route.wiretap.failed - Multicast:
redb.route.multicast.branches,redb.route.multicast.failed_branches - Recipient List:
redb.route.recipientlist.recipients - Aggregator:
redb.route.aggregator.completed,redb.route.aggregator.timed_out,redb.route.aggregator.inflight_groups - Idempotent Consumer:
redb.route.idempotent.duplicate,redb.route.idempotent.passed - Retry:
redb.route.retry.attempts,redb.route.retry.success,redb.route.retry.exhausted - Saga:
redb.route.saga.completed,redb.route.saga.compensated,redb.route.saga.failed - Dead Letter:
redb.route.deadletter.sent
- WireTap:
redb.Route(Telemetry) —MeteredProcessornow enriches every metric point with the new tagsredb.route.endpoint(canonical endpoint URI) andredb.route.scheme(transport scheme such ashttp,kafka,postgres) in addition to the existingredb.route.id.- Transport spans — 16 producers now open a transport span via the new
helper, producing OpenTelemetry-compliant span trees from the route pipeline
down to the wire:
Http,Sql,Sql(procedure),Grpc,MqttNet,AzureServiceBus,Redis,Elasticsearch,Tcp,S3,GenericFile(covers File / Sftp / Ftp),Firebase.Storage,Firebase.Firestore,Firebase.Fcm,WebSocket,SignalR. The five previously-instrumented transports (Kafka,RabbitMQ,IbmMq,Amqp,Mail,Ldap) keep their existing spans unchanged.
redb.Route(DSL) —IOldRouteDefinitionrenamed toIRouteDefinitionand all consumer projects (redb.Route.Controllers,redb.Route.Core,redb.Route.Validation.Adapters,redb.Route.Tests.Core) realigned. The Camel-style canonical name is now the single name across the public API.redb.Route—MeteredProcessorconstructor signature gained two optional parametersendpointUriandendpointScheme. Existing call sites that only pass(inner, routeId)continue to work;RouteContextnow wires the endpoint URI and scheme so dashboards can slice metrics per endpoint.redb.Route—InstrumentedProcessor.ActivityExtensions.RecordExceptionusesActivity.AddException(...)on NET9+ and falls back to a manualActivityEvent("exception", ...)withexception.type/exception.message/exception.stacktracetags on NET8, matching the OpenTelemetry exception-recording convention on both target frameworks.
redb.Route(Legacy) — the entire v1 compiler stack has been removed:OldRouteCompiler(~907 lines),OldRouteDefinition(~1500 lines partial),OldRouteDefinition<TIn>,OldRouteBuilder/OldInlineRouteBuilder,OldCompiledRoute,BlockStack,ExceptionRouteDefinition,IOldRouteDefinition, theLegacy/Abstractions/Typed/IRouteDefinition.cs,Legacy/Extensions/*, the v2→v1 bridges (RouteBuilder2BatchBridge,RouteDefinition2BridgeBuilder,ProcessorDefinitionWrapperStep), and theIRouteDefinition2/RouteBuilder2parallel surface. TheLegacy/folder no longer exists.RouteContext._builders/RouteContext._routesare nowList<RouteBuilder>/List<CompiledRoute>directly, with no intermediate adapter.redb.Route— five stale code comments still referencingOldRouteCompiler/OldRouteDefinition(inRouteStep,NormalizerDefinition,SagaDefinition,AggregatorProcessor,IdempotentConsumerProcessor) were rewritten in terms of the current type names; explanatory intent preserved.
- Pipeline EIP semantics.
PipelineProcessornow strictly follows the Camel Pipeline contract: between steps, anOutproduced by stepiis merged intoInand cleared before stepi+1runs; on the final stepOutis left as-is and is not synthesised fromIn. InOut callers should therefore consume the reply asexchange.Out ?? exchange.In. This was previously documented inline inPipelineProcessor.cs; recording it here as the authoritative engine contract. Downstream conventions (e.g. the Identity layer's "business processors write toIn.Body, do not pre-createOut") sit on top of this contract without changing it.
redb.Route.Tests— newTelemetry/InMemoryTelemetryTests.csusing the OpenTelemetry SDK in-memory exporters (OpenTelemetry.Exporter.InMemory) to verify: shared meter/activity-source name, WireTap dispatched/failed, Multicast branches/failed-branches, Idempotent passed/duplicate, Retry attempts/success/exhausted, transport-span semantic tags,MeteredProcessorendpoint/scheme tag enrichment, andActivity.AddExceptionevent emission.- Per-transport telemetry smoke tests — added
*TelemetrySmokeTests.csfiles (and one Firebase pair appended toFirebaseIntegrationTests) covering all P1 transport spans: Http, Tcp, WebSocket, Grpc, Sql, SqlProcedure, GenericFile, MqttNet, Redis, S3, Elasticsearch, SignalR, AzureServiceBus, Firestore, Firebase Storage. Each test builds a real endpoint, runs the producer throughOpenTelemetry.Sdk.CreateTracerProviderBuilder() .AddSource(RouteActivitySource.SourceName).AddInMemoryExporter(...), and asserts the conventional semantic attributes (http.method/network.transport/db.system/messaging.system/rpc.system/redb.system, plusredb.route.endpointandmessaging.destination.name). Docker-dependent tests are tagged[Trait("Category","Integration")].
- (none — completed below; see
### Testsfor the per-transport smoke sweep.)
redb.Route.Ldap(tests) —LdapEndpointOptionsTests.Validate_ZeroPageSize_*andLdapComponentTests.CreateEndpoint_InvalidPageSize_Throwswere updated to match the (already-shipped) behaviour wherePageSize=0legitimately disables the paged-results control. The tests now assert thatPageSize=0is accepted and that onlyPageSize < 0throws.redb.Route.Firebase(tests) —FirestoreEndpointOptionsTests.Validate_NoCredential_NoEnvVar_Throwsnow captures and restores theGOOGLE_APPLICATION_CREDENTIALSandFIRESTORE_EMULATOR_HOSTenvironment variables in atry/finallyto avoid racing withFirebaseIntegrationTests.InitializeAsync, which setsFIRESTORE_EMULATOR_HOSTfor the whole test host.redb.Route.Firebase(tests) — xUnit collection-level race fixed.try/finallyalone was not enough: by default xUnit runs test classes in different collections concurrently within an assembly, so option-validation classes that mutateFIRESTORE_EMULATOR_HOST/GOOGLE_APPLICATION_CREDENTIALScould still overlap with the live-emulator integration suite that reads them. IntroducedFirebaseEnvSensitiveCollection([CollectionDefinition("FirebaseEnvSensitive", DisableParallelization = true)]) and applied[Collection("FirebaseEnvSensitive")]to all four env-sensitive classes (FirestoreEndpointOptionsTests,FirebaseStorageEndpointOptionsTests,FcmEndpointOptionsTests,FirebaseIntegrationTests). Result: 149/149 PASS, no intermittentEmulator environment variable 'FIRESTORE_EMULATOR_HOST' is not setfailures.redb.Route(dev/test infra) —docker-compose.tests.yml: the Azure Service Bus emulator (servicebus) hadSQL_SERVER: azuriteconfigured, but Azurite is blob/queue/table storage and does not speak TDS. The emulator host therefore crash-looped on startup (initial run created MDFs in the container's writable layer, subsequent restarts failed with Cannot create file '/var/opt/mssql/data/SbGatewayDatabase.mdf' because it already exists), killing the AMQP listener mid-suite and producing AMQP transport failed to open because the inner transport tcpNN is closed on the consumer side. Added a dedicatedsqledgeservice (mcr.microsoft.com/azure-sql-edge:latest) withACCEPT_EULA=Y/MSSQL_SA_PASSWORD, changedservicebus.environment.SQL_SERVERtosqledge, declared the dependency, and bumpedstart_periodto60sto cover SQL Edge warm-up. This is a test-infra change only; published packages are not affected.
redb.Route—WireTapProcessorno longer propagates the caller'sCancellationTokeninto the fire-and-forget tap branch. Previously, when the main pipeline was cancelled (e.g. an HTTP request was aborted by the client), an in-flight audit/notification tap could be killed mid-write — typically surfacing as a failedExecuteNonQuery/Commiton the audit store. The tap branch now runs withCancellationToken.Noneand is only torn down on host shutdown, which matches the EIP "InOnly, detached" semantics of WireTap.
redb.Route.Core— bumpedredb.Coredependency to2.0.2.redb.Core 2.0.2renamesEavSaveStrategy→PropsSaveStrategy; no API changes inredb.Route.Coreitself.
redb.Route.Http—HttpConsumer.WriteResponseno longer echoes request headers back into the response. The original request header names are remembered on the exchange (redbHttp.RequestHeaderNamesproperty) and skipped when copying headers fromexchange.In(which acts as the fallback response message whenOutis not set).redb.Route.Http— invalid header values (control characters and non-ASCII bytes that Kestrel would reject) are now filtered out instead of crashing the response pipeline.redb.Route.Http— internal framework headers (redb*,Camel*) are stripped from outgoing responses.redb.Route.Http— body-less InOut responses (HTTP 302 redirects, 204 No Content, Set-Cookie-only replies) continue to propagateLocation/Set-Cookie/ etc. correctly; header copying remains unconditional and only the body write is gated onBody is not null.redb.Route.Ldap— service-account authenticated endpoints (bindDnis set) no longer reuse pooled connections. Active Directory could report "successful bind must be completed" on a pooled socket that was TCP-connected but no longer bound server-side. Such connections are now created per-operation and disposed on release.redb.Route.Ldap—PageSize=0is now a valid value that disables the RFC 2696 paged-results control entirely, for LDAP servers that do not support it. Validation acceptsPageSize >= 0.redb.Route.Ldap—LdapReferralExceptionraised during a search withfollowReferrals=falseis now logged at Debug and the result iteration breaks cleanly instead of bubbling up.
redb.Route.Http—HttpConsumer.HandleRequestwrapsWriteResponsein a try/catch that logs the failing method, path, route id and whether the response had already started, to aid diagnosing "response already started" errors.redb.Route.Ldap— service-accountBindswitched from the 4-argument overload (with explicit protocol version) to the 2-argumentBindAsync(dn, password, ct). TheprotocolVersionoption is no longer forwarded to the bind call (LDAPv3 default of the underlying client applies).
- License re-stated as Apache-2.0 as part of the RedBase 2.0 release
alignment. Previous public release (
1.0.4) carried the same license text inLICENSEbut was tagged as MIT in some README badges; all metadata is now consistent (Apache-2.0in csproj, README badges, and CONTRIBUTING). - Every nupkg now ships
LICENSE+NOTICEfiles (Apache 2.0 § 4). - Contributions are accepted under Apache-2.0; see
CONTRIBUTING.md. - Version bumped to
2.0.0to align with the RedBase 2.0 release train (root packages also moved 1.3.0 → 2.0.0). No source-level API changes vs 1.0.4.
First public NuGet release. The library has been production-tested since 1.0.0.
Core engine (redb.Route)
- Fluent DSL:
From → Process → Topipeline definition viaIRouteDefinition RouteBuilderbase class for encapsulating route logic in dedicated classes- Two-phase architecture: define (record
RouteSteplist) → compile (RouteCompilerbuilds processor chain) - 24 EIP pattern processors: Filter, Choice, Split, Aggregate, WireTap, Multicast, RecipientList, DynamicRouter, Loop, Delay, Resequencer, Enrich, PollEnrich, IdempotentConsumer, Throttle, CircuitBreaker, Retry, DeadLetterChannel, DoTry/DoCatch/DoFinally, Transacted, Respond
- Expression engine:
Body,Header,Property,Constant,JPath,XPath,StringExpression(Expr),Exchange - 17 predicate methods:
isEqualTo,isNotEqualTo,isGreaterThan,isLessThan,isGreaterThanOrEqualTo,isLessThanOrEqualTo,isBetween,contains,startsWith,endsWith,regex,In,isNull,isNotNull,Handled,ExceptionHandled - String expression templates:
${header.name},${body},${property.key} - Built-in components:
Direct,SEDA,Timer,Log,Mock - Validation: JSON Schema (
JsonSchemaValidator), XSD (XsdValidator), predicate (PredicateValidator) - Serialization: JSON and XML marshal/unmarshal
- Error handling:
OnException<T>with max redeliveries, exponential backoff, dead-letter routing - OpenTelemetry: distributed tracing (
Traced) and metrics (Metered) per route and per step - Structured logging DSL:
.Log(LogLevel).Message().Header().ShowRouteId() InOutexchange pattern supportRouteIdfor route identification and introspectionRouteEngineOptionsfor telemetry and metrics configuration- Multi-target:
net8.0,net9.0,net10.0
Transports
redb.Route.Kafka— consumer/producer, consumer groups, SASL/SSL, transactions, Confluent.Kafka 7.xredb.Route.RabbitMQ— queues, exchanges, DLX, priority, TTL, quorum queues, RabbitMQ.Client 7.xredb.Route.Redis— Pub/Sub, Streams (consumer groups), KV, Lists, Sorted Sets, Geo, StackExchange.Redisredb.Route.Sql— ADO.NET polling consumer, query/batch producer, stored procedures, provider-agnosticredb.Route.Http— HttpClient producer, Kestrel consumer, CORS, auth, TLS, named URL parametersredb.Route.Grpc— GrpcChannel client, Kestrel server, binary message exchangeredb.Route.File— polling consumer with glob, read locking, idempotency; atomic producer with temp-fileredb.Route.Sftp— SSH.NET, key/password auth, proxy, glob, chmod, recursive traversalredb.Route.MqttNet— MQTT 5.0, QoS 0/1/2, shared subscriptions, retained, TLS, MQTTnetredb.Route.Amqp— AMQP 1.0 (Artemis, Azure SB, Amazon MQ, Qpid), AMQPNetLiteredb.Route.Mail— SMTP producer, IMAP/POP3 consumers with IDLE push, attachments, OAuth, MailKitredb.Route.Tcp— text-line, length-prefixed, raw framing, TLS, InOut request-replyredb.Route.WebSocket— ClientWebSocket producer, Kestrel server consumer, ping/pong, subprotocolredb.Route.Quartz— Cron expressions, interval timers, Quartz.NET thread poolredb.Route.AzureServiceBus— queues, topics, sessions (FIFO), PeekLock/ReceiveAndDelete, batch sendredb.Route.Elasticsearch— 9 producer operations (index, update, delete, bulk, etc.), polling consumer, Elasticsearch 8.xredb.Route.Firebase— Firestore (CRUD, queries, batch), Cloud Storage, FCM; shared credential providerredb.Route.Ftp— FluentFTP, passive/active, FTPS/TLS, jail-path protection, idempotencyredb.Route.IbmMq— IBM MQI, queues, topics, transactions, RPC, message groups, W3C telemetryredb.Route.Ldap— LDAP/AD search, CRUD, authentication, change tracking, Novell.Directory.Ldapredb.Route.S3— AWS S3 + MinIO, multipart upload, SSE (S3/KMS/C), presigned URLs, versioning, Glacier restoreredb.Route.SignalR— Hub consumer (server), client producer (HubConnection), broadcast producer (IHubContext)
Integration & adapters
redb.Route.Core—RedbIdempotentRepositorybacked by redb.Core props storage;IRedbServiceaccess from routesredb.Route.Controllers—RedbController, attribute routing, parameter binding, 4 dispatchers (generic, HTTP, SignalR, gRPC)redb.Route.GenericFile— shared base for File, FTP, SFTP (abstract consumer/producer, options, file-ops interfaces)redb.Route.Validation.Adapters—FluentValidationMessageValidator<T>,DataAnnotationsValidator, DSL extensions