One client, five languages: how the C# SDK reuses the Rust client

A client for a replicated log has more to get right than it looks. It finds the broker that leads a shard, follows a redirect when leadership moves, reconnects and resumes a subscription where it left off, and decides which errors are worth retrying. Write that once and it is hard enough. Write it five times and the five copies disagree, quietly, in the cases nobody tests.

So Felix writes it once. The Rust cluster client does the work. Python and TypeScript bind to it, the C ABI (libfelix and its header felix.h) exposes it to anything that can call C, and the C# SDK, Felix.Client, is a thin layer over that ABI. Rust, C, Python, TypeScript and C#: one set of routing, retry and reconnect rules.

What C# gets for free

Felix.Client calls libfelix with P/Invoke. When a C# publish lands on a shard that has moved, the client that follows the redirect is the Rust one, so a .NET program sees the same behaviour as a Python one, including the same error classes. There is no C# reimplementation of the protocol to fall behind.

Sharing code doesn't prove the wrapper is right, so every client is also checked against a shared specification. scenarios.toml is a catalogue of the semantics a client must implement: a round trip, a redirect, a resubscribe after a broker is lost, a typed error. Each language's tests name the scenarios they cover, and felix-conformance verify fails the build if a required scenario has no passing result. The C# scenarios run against a broker fixture on Linux x64. Every other platform (Linux, macOS and Windows, on x64 and arm64) loads the native library and calls into it, which is where a packaging mistake would show.

Crossing the boundary

The C ABI is asynchronous. A call returns at once, and the result arrives later as a C callback, on a Rust thread, with a user_data pointer the caller handed in. A .NET Task is the natural shape for that, and most of the SDK's care goes into making the translation safe.

Each call gets a small state object holding a TaskCompletionSource. The SDK allocates a GCHandle for it and passes the handle's address as user_data, so the object stays alive while the native side holds a pointer to it. The task is created with RunContinuationsAsynchronously, so the code awaiting it never runs on the Rust thread that invoked the callback.

The rule that matters is that the handle is freed exactly once. If the native call refuses the request, no callback will ever run, and the caller frees the handle. If the call returns OK, the callback owns the handle from then on and the caller must not touch it. A double free is worse than a leak here: handle slots are reused, so freeing one twice can release whatever call has been given the slot since, and that call's callback then finds nothing. The free sits behind an atomic exchange, so whichever path gets there first wins and the other does nothing. A test counts the handles freed against the calls made.

The last rule is that no exception crosses into native code. A managed exception that unwinds through a native frame ends the process. Every callback body is wrapped, and a failure inside it faults the task instead of propagating.

Tokens that renew themselves

A Felix token lasts 15 minutes by default, and the broker checks it each time a stream authenticates, including the streams a reconnect opens. A process that outlives one token needs a way to get the next. In C# that is a delegate on the connect options:

TokenProvider = async ct => await myIdentity.FreshFelixTokenAsync(ct)

The type is Func<CancellationToken, ValueTask<string>>. The timing is not C#'s business. The Rust core calls the provider lazily: when a stream is about to authenticate and the client holds no token, the held one has used two thirds of its life, or the broker has just refused it. An idle client makes no calls and runs no timer. Only one call is in flight at a time, so streams that open together wait on it and share the token. That is what makes it safe to rotate a refresh token inside the provider, which is how Felix's control plane issues them: every refresh spends the token you sent and returns a replacement.

A call that takes longer than 30 seconds fails the operations waiting on it. A provider that fails is final for that operation. It is not retried, because each attempt can rotate a refresh token, and a retry could spend one the first attempt had already replaced. The next operation asks again.

What the C ABI can't express is cancellation, so C# adds it. The CancellationToken the provider receives is cancelled at the 30 second timeout, when the connect fails, and once the client and everything made from it has been disposed. That last condition is why the token source is reference-counted. A subscription can outlive the client that created it, and both share the same connections and the same token. If disposing the client cancelled a fetch that was in flight, the control plane might already have rotated the refresh token while the new access token was lost on its way back. The old refresh token would be spent and the new one gone. So the fetch is cancelled only when the last handle that could need it is gone.

A library that wouldn't load

The odd bug in this work was on macOS. A release build of libfelix.dylib was refused by dlopen with mis-aligned LINKEDIT string pool. Nothing in the Rust or C# code was involved.

The cause was cargo's default release profile. On Apple targets rustc runs the system strip over every linked artifact, and on macOS 27 that strip left the shared library's LINKEDIT string pool 4-byte aligned, which dyld rejects. The fix is in Cargo.toml: the felix-capi release profile sets strip = "none". The symbols stay in the file, which a shared library exports anyway.

A library that builds but can't load passes every compile-time check, so CI loads the release build and calls into it on all six platform and architecture combinations.

Try it

dotnet add package Felix.Client

This subscribes before it publishes, so the record isn't missed, and bounds the whole exchange with one cancellation token:

using System.Text;
using Felix;

using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(30));

try
{
    await using var client = await FelixClient.ConnectAsync(new FelixClientOptions
    {
        Addrs = ["127.0.0.1:5000"],
        TenantId = "t1",
        Token = Environment.GetEnvironmentVariable("FELIX_TOKEN")
            ?? throw new InvalidOperationException("set FELIX_TOKEN"),
    }, cts.Token);

    // Subscribe first so the publish below is not missed.
    await using var sub = await client.SubscribeAsync(
        "t1", "default", "events", shard: 0, StartPosition.Latest, cts.Token);

    await client.PublishAsync(
        "t1", "default", "events", Encoding.UTF8.GetBytes("hello"), cancellationToken: cts.Token);

    var ev = await sub.ReadAsync(TimeSpan.FromSeconds(5), cts.Token);
    Console.WriteLine(ev is null ? "nothing arrived" : $"{ev.Offset}: {Encoding.UTF8.GetString(ev.Payload.Span)}");
}
catch (FelixException e) when (e.IsRetryable)
{
    Console.Error.WriteLine($"retryable: {e.Status}: {e.Message}");
}
catch (OperationCanceledException)
{
    Console.Error.WriteLine("gave up after 30 seconds");
}

Both handles are await using, so the subscription and the client are released on every path, including the exceptions. A broker that is down or a shard that is moving surfaces as a FelixException with IsRetryable set; anything else propagates. The C# client page covers sharded subscriptions, consumer groups, caches, counters, conditional publishes and the token provider in full, and the C ABI page describes the layer underneath, for a language that doesn't have an SDK yet.