Enterprise Integration Patterns in .NET, the deep-dive series — Part 1: the four in-memory channels (and the Exchange they carry)

The earlier posts were the tour: what redb.Route is, an Apache Camel-style ESB for .NET — a fluent C# integration DSL with 22 connector projects (~30 URI schemes once you count the https/wss/es variants), ~30 Enterprise Integration Patterns implemented natively across 41 processors, 8 in-process components, a compiled expression engine, and a pluggable marshal/unmarshal layer. Enough touring.

This starts a deep-dive series — one piece at a time, the actual DSL, the actual semantics, the gotchas that bite you the first time. Not a feature list. A working manual. The series runs on four parallel tracks; you can follow whichever maps to the problem in front of you.

Track A — the in-process foundation (8 components)

The wires and the message model everything else stands on. Small surface, deep behavior.

1. 
   The four channels + the Exchange — direct, direct-vm, seda, vm, and the object they carry. (this post — the foundation)
   


2. 
   Scheduling & test components — timer, plus quartz/cron built on the same idea, and the log / mock / validator components you'll actually lean on in tests.

Track B — the Enterprise Integration Patterns (~30, across 41 processors)

Grouped the way Hohpe & Woolf's Enterprise Integration Patterns groups them, one article per cluster, each dissected against the shipped processor. (The codebase has 41 processors total; a handful — To, Log, Delegate, RoutePolicy — are plumbing rather than named patterns, so the honest count of distinct EIP is around thirty plus the 6 load-balancer strategies.)

• 
  Message Routing — Choice (Content-Based Router), Filter, Splitter (+ streaming splitter), Aggregator, Resequencer, Multicast, RecipientList, ScatterGather, DynamicRouter, LoadBalance (Round-Robin / Random / Weighted / Sticky / Failover).


• 
  Message Transformation — Marshal/Unmarshal and the data-format registry, ConvertBody, Enrich/PollEnrich (Content Enricher), ClaimCheck, StreamCaching, Normalize, and Transform/SetBody/SetHeader over the expression engine.


• 
  Messaging Endpoints — IdempotentConsumer, WireTap, competing consumers (seda/vm concurrentConsumers), Pipeline, request/reply (InOut), Bean<T>.


• 
  System Management & Reliability — OnException / TryCatch / DeadLetterChannel, CircuitBreaker, Throttle / KeyedThrottle, Delay / Debounce / Sampling, Timeout, Loop, Saga (+ compensation), RoutePolicy, Transaction/Transacted, and Validate (JSON Schema / XSD).

Track C — the expression & predicate engine (its own article, and it earns it)

The one piece every other track quietly depends on. I almost wrote it off as string interpolation in the roadmap — then I read the source. It's a genuine compiled little language: Tokenizer → Parser → an AST (BinaryOperationNode, UnaryOperationNode, FunctionCallNode, TernaryNode, PostfixOperationNode, property/index access), lowered to System.Linq.Expressions and .Compile()d to real IL — not tree-walked at runtime. There's a dedicated article because there's a dedicated language:

• 
  The ${...} Simple-style template isn't formatting — it's the entry point to the whole grammar: arithmetic (+ - * /), comparison, AND/OR/XOR/NOT, the ternary ?:, even postfix ++/--.


• 
  ~22 built-in functions, mined from the AST's FunctionCallNode: string (concat, upper, lower, trim, substring, replace, length, contains, startswith, endswith), numeric (abs, round, min, max), aggregates (sum, avg, count), date math (now, dateformat, dateadd), and the two that bridge into the structured world: jpath(...) and xpath(...).


• 
  JSONPath and XPath as first-class expression types — typed (TypedJsonPathExpression, TypedXPathExpression) and precompiled (CompiledJPathExpression, CompiledXPathExpression) when the path itself is static.


• 
  The predicate builders — the fluent half, for .Filter/.Choice/.Validate: isEqualTo, isNotEqualTo, isGreaterThan/isLessThan (…OrEqualTo), isBetween, contains, startsWith, endsWith, regex, In(...), isNull/isNotNull, composed with and/or/not. Seventeen of them, each a real IPredicate, not a closure.


• 
  Four separate ConcurrentDictionary compile-and-cache pools (template / property-resolver / logical / value) so every one of the above compiles once per distinct expression string and runs as a cached delegate forever after.

That's why it's its own installment, and honestly why it might be two: the template grammar and the predicate DSL are different front-ends onto the same compiler.

Track D — the 22 connectors, one article each

One focused article per connector, every example lifted from real production routes, not toy snippets:

• 
  Brokers & messaging — kafka, rabbitmq, amqp, asb (Azure Service Bus), wmq (IBM MQ), mqtt.


• 
  RPC & web — http/https, grpc, signalr, ws/wss, tcp.


• 
  Data & storage — sql, redis, elasticsearch, s3.


• 
  Files & transfer — file, ftp, sftp.


• 
  Enterprise & misc — smtp/pop3/imap (mail), ldap, fcm/fstore/fbstorage (Firebase), qtimer/cron (Quartz).

We start with Track A on purpose. Every pattern in Track B, every expression in Track C, and every connector in Track D is built on two things: a channel that carries a message from one route segment to the next, and the Exchange that is the message. Get these two right and the rest of the series is just composition. Get them wrong and you'll spend an afternoon wondering why your transaction silently didn't roll back.

A pipeline in redb.Route is From → [processors] → To. Let's look at what actually flows through it.

The Exchange — the heart, and it isn't simple

Everything that moves through a route is an IExchange. Not a byte[], not your DTO — an Exchange, a small object with more going on inside it than the name suggests. Here's the shape that matters:

public interface IExchange : IAsyncDisposable

{

    IMessage  In  { get; set; }     // the primary message — always present

    IMessage? Out { get; set; }     // the reply — lazy, null until you need it

    ExchangePattern Pattern { get; set; }   // InOnly (default), InOut, or OutOnly



    IDictionary<string, object?> Properties { get; }   // route-level metadata

    Exception? Exception { get; set; }                 // error state, in-band

    bool ExceptionHandled { get; set; }

    string ExchangeId { get; }                         // identity, stable across clones



    IExchange Clone();                                 // deep-ish copy + NEW DI scope

    IServiceProvider? ServiceProvider { get; }         // per-exchange DI scope

}

And the message it carries:

public interface IMessage

{

    object? Body { get; set; }                        // your payload — any object, or null

    string? ContentType { get; set; }

    IDictionary<string, object?> Headers { get; }     // metadata that DOES travel to brokers

    T? GetHeader<T>(string key);

    IMessage Clone();

}

Five things about this object decide how every channel and every pattern behaves. None of them are obvious from the type signature.

1. In vs Out, and the Pattern

In is the message coming through. Out is the reply, and it's lazy — for the default InOnly pattern it stays null and is never allocated. It only appears when a processor explicitly sets it (request/reply, or .Respond()).

There are three patterns, not two — this is the Apache Camel 2.x model, ported wholesale:

public enum ExchangePattern

{

    InOnly  = 0,   // fire-and-forget. Producer result is written to In; Out stays null. (default)

    InOut   = 1,   // request/reply. Original preserved in In, response written to Out.

    OutOnly = 2,   // explicit response via .Respond(); the RPC reply is taken from Out.

}

Here's the part the type signature hides, and the one place people get it wrong: HasOut does not tell you where the answer is. Even on an InOut exchange, a processor isn't obligated to populate Out — if it just mutates In.Body, the result lives in In. So the framework never trusts HasOut to find a reply. It reads Out ?? In, every time:

// ProducerTemplate.RequestBody — the canonical reply-extraction rule

exchange.Pattern = ExchangePattern.InOut;

await producer.Process(exchange);

return exchange.Out?.Body ?? exchange.In.Body;   // Out if present, otherwise In

This is verbatim how Camel's ProducerTemplate extracts a result (getResultMessage: "has Out → use Out, else In"). Copy that rule into your own code — exchange.Out ?? exchange.In — and you'll never chase a reply that quietly stayed in In. HasOut is a fact about allocation, not about where the data is; don't use it to route on the answer.

One honesty note for the JVM crowd: the live Out message and the OutOnly pattern are Camel 2.x semantics. Camel 3+ deprecated getOut()/setOut() and collapsed the pattern set toward InOnly/InOut, precisely because a separate Out message copied headers and bred subtle bugs. redb.Route keeps the fuller 2.x model on purpose — but if you're coming from modern Camel, that's the difference you'll notice first.

2. Properties vs Headers — the distinction that leaks bugs

Both are IDictionary<string, object?>. They are not interchangeable:

• 
  In.Headers travel with the message to the broker. Put a correlationId here and Kafka/RabbitMQ carry it downstream.


• 
  exchange.Properties are route-level metadata — RouteId, transaction markers, your own scratch state. They do not leave the process (the interface XML doc says exactly that: "Does NOT travel to brokers — use In.Headers for that"). Stash a DbContext handle or a retry counter here.

Put a value in the wrong dictionary and it either fails to reach the consumer (you used Properties) or leaks internal state onto the wire (you used Headers). The compiler won't catch it; both are just string-keyed dictionaries. Knowing which is which is half of using the framework correctly.

Read either with the typed accessors instead of casting by hand:

var attempt = exchange.GetProperty<int>("retryCount");     // route-level, stays in-process

var corr    = exchange.In.GetHeader<string>("correlationId"); // travels to the broker

The well-known keys the framework itself writes

This is the part that makes the section concrete, and it's the answer to "what's actually in Properties?" The pipeline and the processors populate a set of reserved keys as your exchange flows. There is no single ExchangeProperties constants file — they live next to the processor that owns each one — but here is the real registry, mined from source.

exchange.Properties — route-level, never leave the process:

Plus RouteId is promoted to a first-class property on the exchange (exchange.RouteId), not just a dictionary entry — that's what the logger prints as [rId:…].

In.Headers — travel with the message:

The Camel-compatible message headers come from the same world Camel users expect. The Splitter, for example, stamps every part:

// SplitterProcessor — each split message carries its coordinates

splitMessage.Headers["CamelSplitIndex"]    = index;          // 0-based position

splitMessage.Headers["CamelSplitSize"]     = total;          // batch size

splitMessage.Headers["CamelSplitComplete"] = index == total - 1; // last one?

and every transport contributes its own namespaced header constants — KafkaHeaders (redbKafka.Topic, redbKafka.Partition, redbKafka.Offset, …), SqlHeaders (redbSql.rowCount, redbSql.generatedKeys, …), SignalRHeaders (redbSignalR.ConnectionId, …), TcpHeaders, WsHeaders, ElasticsearchHeaders. Each is a static class of public const string so you bind against KafkaHeaders.Offset, not a stringly-typed "redbKafka.Offset" you might misspell. The rule of thumb: anything prefixed Camel* or redb<Transport>.* is a header (on the wire); anything in Properties is yours and the process's alone.

3. The exception travels in-band

When a processor throws, the exception doesn't just unwind the stack — it's captured onto exchange.Exception, with an ExceptionHandled flag beside it. That's what makes a dead-letter route able to branch on why something failed (when e.Exception is TimeoutException → …) instead of just that it failed. The error becomes data you can route on. We lean on this hard in the error-handling installment.

4. ExchangeId is stable across clones

Each exchange gets a Guid-based ExchangeId at creation. The non-obvious part: Clone() preserves it. A split into 500 parts, or a seda hop that clones the exchange, keeps the same id — so your logs and traces stitch the whole flow back to one origin. Identity survives copying; that's deliberate.

5. The DI scope — and four ways to copy an exchange

This is the part that's genuinely not simple, and it's the reason the channels behave the way they do.

An Exchange can own a DI scope — a per-message IServiceScope. Processors resolve scoped services (DbContext, IRedbService, …) from exchange.ServiceProvider, and they get the same instances for the lifetime of that exchange. A TransactionScope lives in exactly that scope. So the question "are these two exchanges in the same transaction?" reduces to "do they share a DI scope?"

There are four ways an exchange gets copied, and they differ only in what they do with that scope:

// from Exchange.Clone() — the scope-creating branch

if (_scopeFactory != null)

{

    clone._scopeFactory = _scopeFactory;

    clone._scope = _scopeFactory.CreateScope();   // <-- a brand-new scope

}

// from Exchange.CloneLinked() — the scope-sharing branch

_ownsScope = false,

_scope = _scope,            // <-- the SAME scope, and we won't dispose it

_scopeFactory = _scopeFactory

Hold onto that table. The entire transaction story of seda vs direct, and of Multicast vs a broker hop, is just which row got used. Clone() (new scope) means a new transaction; CloneLinked() (shared scope) means the same one.

There's also ReleaseScopes() — it disposes the DI scopes without touching the Body, so an aggregator can free database connections early while still holding the message data it's accumulating. And DisposeAsync() cleans up both the body (streams, stream caches) and the scopes. The object is IAsyncDisposable for a reason.

The gotcha: Clone() does NOT deep-copy the Body

Read Message.Clone() literally:

public IMessage Clone()

{

    var clone = new Message(Body) { ContentType = ContentType };  // Body reference copied

    foreach (var kvp in _headers)

        clone._headers[kvp.Key] = kvp.Value;

    return clone;

}

Headers get a fresh dictionary. Body is copied by reference. After a seda hop the producer's exchange and the worker's clone have independent headers, properties, and DI scopes — but they point at the same body object. If that body is a mutable List<T> or a POCO and both sides write to it, you have a data race the cloning looks like it prevented. The XML doc says "deep copy"; the honest truth is "deep copy of everything except the payload." Treat the body as immutable once it's in flight, or clone it yourself.

Two APIs on purpose

One last thing you'll notice in IntelliSense: every member has a C# idiomatic form (In, Body, GetHeader<T>) and a Java-style alias (getIn(), setBody(), getHeader<T>()). They're default interface methods over the same state, kept so the model reads the same as Apache Camel for anyone coming from the JVM. Use whichever; they're the same object.

Now — the wires that carry this thing.

The four channels — two axes

direct, direct-vm, seda, vm are how route segments talk to each other inside a process. Choosing between them is the single most common thing newcomers get wrong, and now you have the vocabulary for why: it comes down to threading and scope. They split on two axes — sync vs async, and one context vs across contexts:

direct is the contrast. seda is where the real work — and the real footguns — live, so that's where we'll spend the page.

direct:// — a method call wearing a URI

direct is not a queue. No thread, no buffer. A producer sending to a direct endpoint invokes the consumer's processor synchronously, on the same thread:

// the whole of DirectProducer.Process

var processor = _endpoint.ConsumerProcessor

    ?? throw new InvalidOperationException("No consumer registered for direct endpoint ...");

await processor.Process(exchange, ct);

The exchange is not cloned. Same object, same thread, same DI scope — straight to the consumer. Three consequences follow:

• 
  Exceptions propagate back to the caller. A throw in the direct consumer surfaces in the producer's route, where OnException/DoTry can catch it. (Remember §3 — it also lands on exchange.Exception.)


• 
  It's the same transaction. Same scope from the table above, so a direct hop inside a .Transaction() block commits and rolls back with everything around it.


• 
  The consumer must be started first, or the send throws. direct decouples your route definitions into named sub-routes — not your threads.

That's the entire personality of direct: a zero-cost, in-transaction call you can give a URI and reuse. It has no parameters because it has no machinery. Use it to break a big route into readable, reusable pieces.

seda:// — the async queue, in detail

seda (Staged Event-Driven Architecture) is the opposite of direct in every cell of the table. It's a real in-memory queue built on System.Threading.Channels. The producer enqueues and returns immediately; one or more background workers drain the queue on their own threads.

// SedaProducer.Process — the whole thing

var copy = exchange.Clone();                       // §5: new scope. the trap lives here.

await _endpoint.Queue.Writer.WriteAsync(copy, ct);

Two facts are baked into those two lines, and everything else about seda follows from them: it clones (so the worker and producer never share scope — that Clone() is row 1 of the table, a new scope), and it returns before the work is done (so the producer's thread, and its transaction, move on).

The parameters

seda takes three, all on the URI: seda://name?concurrentConsumers=4&size=1000&timeout=30000.

concurrentConsumers — throughput, at the cost of order

One worker is the default and keeps strict FIFO. Raising it spins up N independent loops:

// SedaConsumer.RunAsync

_workers = new Task[_options.ConcurrentConsumers];

for (var i = 0; i < _options.ConcurrentConsumers; i++)

    _workers[i] = WorkerLoop(pollCt, processingCt);

// each worker

await foreach (var exchange in _endpoint.Queue.Reader.ReadAllAsync(pollCt))

{

    await ProcessWithTracking(exchange, processingCt);

    Interlocked.Increment(ref _processedCount);

}

Two consequences worth stating plainly:

• 
  You trade ordering for throughput. With concurrentConsumers=1 the channel runs in SingleReader mode (a real optimization in System.Threading.Channels) and messages come out in order. With N>1, N workers pull concurrently and strict FIFO is gone — message 2 can finish before message 1. Only raise it when out-of-order processing is acceptable.


• It's per-endpoint backpressure relief: a slow downstream stops blocking the upstream producer, because the producer only ever writes to the queue and leaves.

size — bounded vs unbounded, and why you almost always want bounded

This is the parameter people skip and regret. The endpoint picks the channel implementation off size:

Queue = options.Size > 0

    ? Channel.CreateBounded<IExchange>(new BoundedChannelOptions(options.Size)

      {

          FullMode = BoundedChannelFullMode.Wait,    // producer awaits a free slot

          SingleReader = options.ConcurrentConsumers == 1,

          SingleWriter = false

      })

    : Channel.CreateUnbounded<IExchange>(/* ... */); // grows until you run out of memory

• 
  size=0 (default, unbounded): the queue grows as fast as producers write. If the consumer can't keep up, that's an unbounded memory leak with extra steps. Fine for bursty, bounded-volume work; dangerous for a firehose.


• 
  size>0 (bounded): FullMode = Wait means a full queue makes the producer await a free slot — backpressure that pushes the slowdown upstream instead of into your heap. This is what you want in production.
  

// bounded SEDA: 4 workers, at most 1000 queued, producer waits when full

From("seda://enrich?concurrentConsumers=4&size=1000")

    .Process(async (ex, ct) => await Enrich(ex, ct))

    .To("rabbitmq://enriched");

timeout — an honesty note

The options object documents timeout (default 30000 ms) as the enqueue wait for a bounded queue. Being straight with you: in the current code the producer enqueues with WriteAsync(copy, ct) and does not apply that timeout — a full bounded queue makes the producer wait on the channel until a slot frees or the CancellationToken fires, not until 30 seconds elapse. So today, plan around size and the cancellation token; treat timeout as declared-but-not-yet-wired and don't build a deadline assumption on it. (Flagging it here because guessing at framework behavior from a doc-comment is exactly how you ship a bug.)

Graceful shutdown — seda drains, it doesn't drop

When a route stops, seda doesn't throw away what's already queued:

// SedaConsumer.OnStopAccepting

_endpoint.Queue.Writer.TryComplete();   // stop accepting new; let readers finish

Completing the writer makes the workers' ReadAllAsync loop finish the remaining items and then exit cleanly (SedaConsumer is a DrainableConsumer). On a graceful stop, in-flight queued exchanges are processed, not lost.

The durability caveat

seda is in-memory and non-durable. A graceful stop drains; a crash or a hard kill does not — whatever was sitting in the channel is gone. seda is at-most-once across a process restart. When you need the queue to survive a restart, that's a broker (rabbitmq, kafka), not seda. seda is for decoupling within a process, not for durability.

The transaction trap, stated once and for all

Now §5 pays off. Because seda calls Clone() — row 1, a new DI scope on a different thread — anything past a seda:// hop is not in the caller's transaction.

From("kafka://orders")

    .Transaction()

        .To(Sql.Execute("INSERT …").Transacted())

        .To("seda://post-process")     // runs in a NEW scope, on another thread,

    .EndTransaction();                 //    OUTSIDE this transaction

If post-process throws, the INSERT above has already committed — the seda hop left the transaction the moment it cloned. This is the one mistake everyone makes exactly once. The fix is the table: if the hop must share the transaction, use direct (no clone, same scope); if you genuinely want to hand the work off and move on, seda is correct and you accept the new boundary. The DSL is the same; the scope is everything.

Mental model: direct = function call, seda = mailbox. One preserves your thread and your transaction; the other trades both for throughput and isolation.

direct-vm:// and vm:// — the same two, across module boundaries

In a multi-module host (this is how redb.Tsak runs several modules in one process) each module is its own RouteContext. Plain direct and seda are scoped to a single context — a producer in module A can't see a direct consumer in module B. The -vm variants lift exactly that wall by sharing the processor registry, and for vm the channel, through a SharedVmRegistry DI singleton:

• 
  direct-vm:// — synchronous, cross-context, no clone. A consumer in the billing module exposes direct-vm://charge; a producer in orders calls it like a local in-transaction method.


• 
  vm:// — asynchronous, cross-context, cloned-and-queued. The cross-module twin of seda, with the same concurrentConsumers and size parameters (and the same Clone(), so the same transaction boundary and the same shared-Body caveat).

The rule transfers cleanly: direct-vm for a synchronous cross-module call that shares the caller's transaction; vm for hand-it-off-and-move-on across modules. Same semantics as their in-context twins — just a wider blast radius.

Picking a channel

The whole decision in one table:

And the two facts that drive every row: does it clone (new scope = new transaction), and does it return before the work is done. Everything else is detail.

What's next

That's the foundation. You now know what flows through a route (Exchange — In/Out, Properties vs Headers, in-band exceptions, and the four scope-aware clone variants) and the four wires that carry it (direct/direct-vm sync-and-in-transaction, seda/vm async-and-isolated), down to the parameter that decides whether your queue applies backpressure or eats your heap.

Next in the series — Part 2: Splitter + Aggregator. We fan one message into many, process them with bounded parallelism, and re-assemble — and the Clone() vs CloneLinked() distinction from §5 turns out to be the whole story of whether the split shares the parent's transaction. Plus the aggregation-strategy contract where the first call hands you a null accumulator. Subscribe to the series if you want it when it lands.

If anything here fought you — especially the seda transaction boundary or the shared-Body clone — say so in the comments. That feedback is exactly what an early OSS release is for.

Links

All Apache 2.0. Questions in the comments are welcome — especially "does channel X do Y?", because that's how the docs get written.