In this post I describe a simple way to remove some byte[] allocations, no matter which version of .NET you're targeting, including .NET Framework. This will likely already be familiar to you if you write performance sensitive code with modern .NET, but I recently realised that this can be applied to older runtimes as well, like .NET Framework.

This post looks at the changes to your C# code to reduce the allocations, how the compiler implements the change behind the scenes, and some of the caveats and sharp edges to watch out for.

Span<T> and ReadOnlySpan<T> are a performance mainstay for .NET

ReadOnlySpan<T> and Span<T> were introduced into .NET a long time ago now, but they have had a significant impact on the code you can (and arguably should) write, particularly when it comes to performance sensitive code. These provide a "window" or "view" over existing data, without creating copies of that data.

The classic example is when you're manipulating string objects; instead of using SubString(), and creating additional copies of segments of the string, you can use AsSpan() to create ReadOnlySpan<char>() segments that can be manipulated almost as though they are separate string instances, but without all the copying.

This is probably the most common use of Span<T> in application code, but fundamentally the use of Span<T> to provide a view over any piece of memory means it's useful in many other situations. The fact that the backing of a Span<T> can be almost anything, means you can keep the same "public" API which potentially "swapping out" the backend.

Another common example of this is if you have some parsing (or similar) code and you need a buffer to store the temporary results. Prior to Span<T>, you would almost certainly have allocated a normal array on the heap for this, but with Span<T>, "stack allocating" using stackalloc becomes just as easy, and reduces pressure on the garbage collector:

Span<byte> buffer = requiredSize <= 256                  // If the required buffer size is small 
                        ? stackalloc byte[requiredSize]  // enough, then allocate on the stack.
                        : new byte[requiredSize];        // Fallback to a normal heap allocation

Virtually all new .NET runtime APIs are added with Span<T> or ReadOnlySpan<T> support, and you can even use them in old runtimes like .NET Framework via the System.Memory NuGet package (though you don't get all the same perf benefits that you do with .NET Core).

The ability to easily and safely (without needing directly falling back to unsafe and pointers) work with blocks of memory regardless of where they're from has really made Span<T> vital for any code that cares about performance. But this ability to provide an "arbitrary" view over memory also provides a way for the compiler to perform additional optimizations, as we'll see in the next section.

Removing byte[] allocations with ReadOnlySpan<byte>

The ability for the compiler to provide a view over arbitrary memory is what drives the optimization I'm going to talk about for the rest of this post.

Let's imagine you have some byte[] that you need for something. Some kind of processing requires it. You know the data it needs to contain upfront, so you store the array in a static readonly field, so that the data is only created once:

public static class MyStaticData
{
    private static readonly byte[] ByteField = new byte[] { 1, 2, 3, 4 };
}

This works absolutely fine, but it means when you first access that data, the runtime needs to create an instance of the array, fill it with the data, and store it in the field. After that, accessing the field is cheap, but the initial creation adds a small delay to the first use of that type.

However, starting with C# 8.0, and as long as that you only need a readonly view of the data, you can use a slightly different pattern, by exposing a ReadOnlySpan<byte> property instead of a field:

public static class MyStaticData
{
    // Before
    private static readonly byte[] ByteField = new byte[] { 1, 2, 3, 4 };

    // After
    private static ReadOnlySpan<byte> ReadOnlySpanProp => new byte[] { 1, 2, 3, 4 };
}

Now, normally, that's the sort of code that should be setting off performance alarm bells. It looks like it will be creating a new byte[] every time you access the property😱 But that's not what's happening.

We'll take a detailed look at the generated IL code shortly, for now we'll just talk at a high level.

When the compiler sees the pattern above, it does the following:

• Embed the byte[] data into the final assembly's metadata
• When ReadOnlySpanProp is invoked, instead of creating a byte[], create a ReadOnlySpan<byte> that points directly to the data in the assembly

So the returned ReadOnlySpan<byte> isn't pointing to data that exists on the heap or even on the stack; it's pointing to data that's embedded directly in the assembly. That means there's no allocation at all, which removes that startup overhead and means there's no pressure at all on the garbage collector 🎉

It's worth noting as well that this is a compiler feature, which means that as long as a System.ReadOnlySpan<T> type is available, you can use it. So as long as you add the System.Memory NuGet package to your .NET Framework app, you too can benefit from this zero-allocation technique!

Also, this doesn't just apply to converting static readonly byte[] fields to static ReadOnlySpan<byte> properties; it also applies to local variables too. Which means things like the following, which look like they allocate an array, actually don't:

public static void TestData()
{
    // This looks like it allocates, but it doesn't
    ReadOnlySpan<byte> arr = new byte[] { 0, 1, 2 };
}

Another minor thing to point out is that this also works with UTF-8 Strings Literals, which are logically represented as a byte[] by the type system. So this is also zero allocation:

public static class MyStaticData
{
    private static ReadOnlySpan<byte> ReadOnlySpanUtf8 => "Hello world"u8;
}

That's all great, but when I first used the byte[] approach, I was a little concerned. After all, it looked like it would be allocating and terribly inefficient, so I wanted to be sure. And what better way than checking the IL code the compiler generates.

What's happening behind the scenes?

There are multiple ways to check the generated code that the compiler generates. If you just want to check a "snippet" of code, then sharplab.io is a quick and easy option. Alternatively, there's ILSpy, or the JetBrains tools like dotPeek and Rider, and I'm sure Visual Studio has plugins for it.

To comfort myself, I first created a new .NET project using dotnet new classlib, and then I tweaked it to use .NET Framework. To be clear, the techniques shown so far work on all target frameworks, but I wanted to specifically test with .NET Framework, to prove that it's not just "new" frameworks this works with. I tweaked the project file as shown below:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <!-- Change the target framework to .NET Framework👇 -->
    <TargetFramework>net48</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <!-- Use the latest C# version (C# 14 with .NET 10 SDK)👇 -->
    <LangVersion>latest</LangVersion>
  </PropertyGroup>
  
  <!-- Reference System.Memory so we can use ReadOnlySpan<T>👇 -->
  <ItemGroup>
    <PackageReference Include="System.Memory" Version="4.6.3" />
  </ItemGroup>
</Project>

I then created the very simple class below, compiled, and used Rider to view the generated IL:

public static class MyStaticData
{
    private static ReadOnlySpan<byte> ReadOnlySpanProp => new byte[] { 1, 2, 3, 4 };
    private static ReadOnlySpan<byte> ReadOnlySpanUtf8 => "Hello world"u8;
}

I've commented the IL below to describe what it's doing, but the important thing is that we don't see any calls to newarr, InitializeArray(), or ToArray(), or other problematic calls. Instead, we see IL code that loads an address which points to data embedded in the PE image (i.e. the assembly), loads the length of the data (4 bytes), and then passes the pointer and length to the new ReadOnlySpan<T>() constructor and returns it. No copying, no new arrays, just a wrapper around bytes that are already loaded into memory 🎉

.class public abstract sealed auto ansi beforefieldinit
  MyStaticData
    extends [mscorlib]System.Object
{

  .field private static initonly unsigned int8 One

  .method private hidebysig static specialname valuetype [System.Memory]System.ReadOnlySpan`1<unsigned int8>
    get_ReadOnlySpanProp() cil managed
  {
    .maxstack 8

    // 👇 Push the address of the static field that contains the array data as a blob onto the stack
    IL_0000: ldsflda      int32 '<PrivateImplementationDetails>'::'9F64A747E1B97F131FABB6B447296C9B6F0201E79FB3C5356E6C77E89B6A806A'
    // 👇 Push the value '4' onto the stack
    IL_0005: ldc.i4.4
    // 👇 Create a new ReadOnlySpan<byte>
    IL_0006: newobj       instance void valuetype [System.Memory]System.ReadOnlySpan`1<unsigned int8>::.ctor(void*, int32)
    IL_000b: ret // Return the span

  } // end of method MyStaticData::get_ReadOnlySpanProp

  .method private hidebysig static specialname valuetype [System.Memory]System.ReadOnlySpan`1<unsigned int8>
    get_ReadOnlySpanUtf8() cil managed
  {
    .maxstack 8

    // 👇 Push the address of the static field that contains the UTF-8 data as a blob onto the stack
    IL_0000: ldsflda      valuetype '<PrivateImplementationDetails>'/'__StaticArrayInitTypeSize=12' '<PrivateImplementationDetails>'::'27518BA9683011F6B396072C05F6656D04F5FBC3787CF92490EC606E5092E326'
    // 👇 Push the value '11' onto the stack (the length of "Hello world" in UTF-8)
    IL_0005: ldc.i4.s     11 // 0x0b
    // 👇 Create a new ReadOnlySpan<byte>
    IL_0007: newobj       instance void valuetype [System.Memory]System.ReadOnlySpan`1<unsigned int8>::.ctor(void*, int32)
    IL_000c: ret // Return the span

  } // end of method MyStaticData::get_ReadOnlySpanUtf8
}

Great, we can see that it's clearly working as we expected and this is .NET Framework, so it is just a compiler feature and has no runtime requirements, so we can use it everywhere.

But we need to be careful… I showed that it works for byte[], but it doesn't work for everything…

Be careful, things can go wrong…

If you've read this far, you might be thinking "great, I'll use this for all my static array data", but I'm going to stop you there. Here-be dragons. The pattern above is only safe to use:

• If you have a byte[], sbyte[], or bool[].
• If all the values in the array are constants
• If the array is immutable (i.e. you return a ReadOnlySpan<T> not a Span<T>).

Breaking any of these rules may be disastrous for performance, so we'll examine each in turn.

Only byte, sbyte, and bool are allowed

The compiler optimizations shown so far can only be applied to byte-sized primitives, i.e. byte, sbyte, and bool. That's because the constant data would be stored in a little endian format, and needs to be translated to the runtime endian format, e.g. if the application is run on hardware which utilizes big endian numbers.

That means, that if you do the following (using int instead of byte), then the code compiles just fine, but unfortunately it doesn't generate the "zero allocation" code that you might expect:

public static class MyStaticData
{
    // ⚠️ Using `int` instead of `byte` _does_ cause an array 
    // to be allocated (on .NET Framework and < .NET 7)
    private static ReadOnlySpan<int> ReadOnlySpanPropInt => new int[] { 1, 2, 3, 4 };
}

If we check the generated IL for a .NET Framework app with the above, we can see the problematic newarr and InitializeArray calls. The compiler actually does some work to avoid the really problematic pattern which would create an array every time, by creating the array once, caching it in a static field, and then using that cached data for subsequent calls, but it still has a startup cost, and does more work than the optimized byte[] approach:

.method private hidebysig static specialname valuetype [System.Memory]System.ReadOnlySpan`1<int32>
  get_ReadOnlySpanPropInt() cil managed
{
  .maxstack 8

  // 👇Try to load the cached int[] data from the static 'cache' field
  IL_0000: ldsfld       int32[] '<PrivateImplementationDetails>'::CF97ADEEDB59E05BFD73A2B4C2A8885708C4F4F70C84C64B27120E72AB733B72_A6
  IL_0005: dup                                  // Duplicate the variable
  IL_0006: brtrue.s     IL_0020                 // If the data isn't null, we have it cached, so jump to the end
  IL_0008: pop                                  // The value was null, remove the duplicate
  IL_0009: ldc.i4.4                             // Load the length of the data (4)
  IL_000a: newarr       [mscorlib]System.Int32  // Allocate a new array on the heap
  IL_000f: dup                                  // Keep a copy of the array variable
  // 👇 Load the address of the int[] data embedded in the assembly
  IL_0010: ldtoken      field valuetype '<PrivateImplementationDetails>'/'__StaticArrayInitTypeSize=16' '<PrivateImplementationDetails>'::CF97ADEEDB59E05BFD73A2B4C2A8885708C4F4F70C84C64B27120E72AB733B72
  // 👇 Initialize the new array with the int[] data
  IL_0015: call         void [mscorlib]System.Runtime.CompilerServices.RuntimeHelpers::InitializeArray(class [mscorlib]System.Array, valuetype [mscorlib]System.RuntimeFieldHandle)
  IL_001a: dup                                  // Duplicate the variable
  // 👇 Store the now-populated array into the static 'cache' field
  IL_001b: stsfld       int32[] '<PrivateImplementationDetails>'::CF97ADEEDB59E05BFD73A2B4C2A8885708C4F4F70C84C64B27120E72AB733B72_A6
  // 👇 Create the `ReadOnlySpan<int>` wrapping the array
  IL_0020: newobj       instance void valuetype [System.Memory]System.ReadOnlySpan`1<int32>::.ctor(!0/*int32*/[])
  IL_0025: ret

} // end of method MyStaticData::get_ReadOnlySpanPropInt

So the "good" news is that this isn't much different to just using a static readonly int[], but it's still not ideal, and definitely isn't the zero-allocation version that you get with byte[].

Additionally, if you're on .NET 7+, a new API was added which actually does support this pattern. So if we change the target framework (to .NET 10 in this case), and recompile, then the IL is back to the zero allocation version, thanks to the call to RuntimeHelpers::CreateSpan, which handles fixing-up any endianness issues:

.method private hidebysig static specialname valuetype [System.Runtime]System.ReadOnlySpan`1<int32>
    get_ReadOnlySpanPropInt() cil managed
  {
    .maxstack 8

    // 👇 Load the address of the data
    IL_0000: ldtoken      field valuetype '<PrivateImplementationDetails>'/'__StaticArrayInitTypeSize=16_Align=4' '<PrivateImplementationDetails>'::CF97ADEEDB59E05BFD73A2B4C2A8885708C4F4F70C84C64B27120E72AB733B724
    // 👇 Call RuntimeHelpers::CreateSpan and return
    IL_0005: call         valuetype [System.Runtime]System.ReadOnlySpan`1<!!0/*int32*/> [System.Runtime]System.Runtime.CompilerServices.RuntimeHelpers::CreateSpan<int32>(valuetype [System.Runtime]System.RuntimeFieldHandle)
    IL_000a: ret

  } // end of method MyStaticData::get_ReadOnlySpanPropInt

So in summary, your mileage will vary here, and you don't really gain anything unless you're on .NET 7+. If you need to target older frameworks, then you're potentially better off just sticking to a good old static readonly int[] field instead.

All values must be constants

The next issue is that the whole approach shown in this post only works if all the values in the collection are constants. For example, the following example which uses a static readonly value inside the array compiles just fine:

public static class MyStaticData
{
    private static readonly byte One = 1;
    private static ReadOnlySpan<byte> ReadOnlySpanPropNonConstant => new byte[] { One, 2, 3, 4 };
}

but even on .NET 7+, this won't do the zero-allocation approach that you might be expecting. Instead, you get some really nasty "allocate a new array every time" behaviour 😱:

.method private hidebysig static specialname valuetype [System.Runtime]System.ReadOnlySpan`1<unsigned int8>
  get_ReadOnlySpanPropNonConstant() cil managed
{
  .maxstack 8

  IL_0000: ldc.i4.4                                  // Laod the length of the array
  IL_0001: newarr       [System.Runtime]System.Byte  // Create a new array
  IL_0006: dup                                       // Duplicate the variable reference
  // 👇 Get a reference to the data, and initialize the array
  IL_0007: ldtoken      field int32 '<PrivateImplementationDetails>'::'1E6175315920374CAA0A86B45D862DEE3DDAA28257652189FC1DFBE07479436A'
  IL_000c: call         void [System.Runtime]System.Runtime.CompilerServices.RuntimeHelpers::InitializeArray(class [System.Runtime]System.Array, valuetype [System.Runtime]System.RuntimeFieldHandle)
  IL_0011: dup
  IL_0012: ldc.i4.0                                      // Load the index to change '0'
  IL_0013: ldsfld       unsigned int8 MyStaticData::One  // Load the static field `One`
  IL_0018: stelem.i1                                     // Set array[0] = One
  // 👇 return the ReadOnlySpan<byte> around the new array
  IL_0019: call         valuetype [System.Runtime]System.ReadOnlySpan`1<!0/*unsigned int8*/> valuetype [System.Runtime]System.ReadOnlySpan`1<unsigned int8>::op_Implicit(!0/*unsigned int8*/[])
  IL_001e: ret
} // end of method MyStaticData::get_ReadOnlySpanPropNonConstant

That's…bad 😬 And it does it on every property access. Definitely watch out for that one, on all target frameworks.

Only use ReadOnlySpan<T>, not Span<T>

You have a similar "dangerous" scenario if you use Span<T> instead of ReadOnlySpan<T>:

public static class MyStaticData
{
    private static Span<byte> SpanProp => new byte[] { 1, 2, 3, 4 };
}

In this case, because you're returning mutable data (Span<T> instead of ReadOnlySpan<T>), the compiler can't use any of its fancy tricks, because the data needs to be mutable. All it can do is create a new array, initialize it with the correct initial values, and then hand it back wrapped in a mutable Span<T>:

.method private hidebysig static specialname valuetype [System.Runtime]System.Span`1<unsigned int8>
  get_SpanProp() cil managed
{
  .maxstack 8

  // [32 43 - 32 68]
  IL_0000: ldc.i4.4
  IL_0001: newarr       [System.Runtime]System.Byte
  IL_0006: dup
  IL_0007: ldtoken      field int32 '<PrivateImplementationDetails>'::'9F64A747E1B97F131FABB6B447296C9B6F0201E79FB3C5356E6C77E89B6A806A'
  IL_000c: call         void [System.Runtime]System.Runtime.CompilerServices.RuntimeHelpers::InitializeArray(class [System.Runtime]System.Array, valuetype [System.Runtime]System.RuntimeFieldHandle)
  IL_0011: call         valuetype [System.Runtime]System.Span`1<!0/*unsigned int8*/> valuetype [System.Runtime]System.Span`1<unsigned int8>::op_Implicit(!0/*unsigned int8*/[])
  IL_0016: ret

} // end of method MyStaticData::get_SpanProp

The failure path here is understandable, because there's really no way to do a safe zero-allocation approach when the data needs to be mutable. The big problem is that it's not obvious that it's a super-allocatey property instead of a zero-allocation version. If you accidentally fat-finger and write Span<T> instead of ReadOnlySpan<T>, or, you know, Claude does, then it's really not obvious from simply reviewing the code…

The only good news is that if you use modern features, namely collection expressions, you might catch the issue!

Reducing the risk of errors with collection expressions

So how do collection expressions help here? Well, those last two points, where one of the values isn't a constant, or where the variable is Span<T> instead of ReadOnlySpan<T> simply won't compile if you use the static property pattern with collection expressions:

public static class MyStaticData
{
    // Doesn't compile (That's good!)
    private static ReadOnlySpan<byte> ReadOnlySpanPruopNonConstantCollectionExpression => [One, 2, 3, 4];

    // Doesn't compile (That's good!)
    private static Span<byte> SpanPropCollectionExpression => [1, 2, 3, 4];
}

Attempting to compile this gives CS9203 errors:

Error CS9203 : A collection expression of type 'ReadOnlySpan<byte>' cannot be used in this context because it may be exposed outside of the current scope.
Error CS9203 : A collection expression of type 'Span<byte>' cannot be used in this context because it may be exposed outside of the current scope.

This gives you something of a safety-net. As long as you always use collection expressions for this scenario, you're blocked from making the most egregious errors. The case where you are using int is allowed, but as already flagged, that's not as bad, because it's actually supported on .NET 7+, and you still only create a single instance of the array and cache it in <.NET 7.

Unfortunately, collection expressions only save you in the static property case. If you are creating local variables, then collection expressions don't save you on .NET Framework (or on any .NET versions <.NET 8)

public static class MyStaticData
{
    private static readonly byte One = 1;

    public static void TestData()
    {
        // Oh no, these all allocate on .NET Framework!
        ReadOnlySpan<int> intArray = [1, 2, 3, 4]; // .NET 7+ doesn't allocate for this one

        ReadOnlySpan<byte> nonConstantArray = [One, 2, 3, 4]; // But you need .NET 8+ to avoid
        Span<byte> spanArray = [1, 2, 3, 4];                  // allocations for these two!
    }
}

If we take a look at the IL generated for .NET Framework for this method, we can see that the int[] case uses the "create a static array and cache it" approach, while the non-constant and Span<T> cases create a new array every time, the same as happens with a static property:

    .method public hidebysig static void
    TestData() cil managed
  {
    .maxstack 5
    .locals init (
      [0] valuetype [System.Memory]System.ReadOnlySpan`1<int32> intArray,
      [1] valuetype [System.Memory]System.ReadOnlySpan`1<unsigned int8> nonConstantArray,
      [2] valuetype [System.Memory]System.Span`1<unsigned int8> spanArray
    )

    // [10 5 - 10 6]
    IL_0000: nop

    // Load or initialize the static int[] field data
    IL_0001: ldsfld       int32[] '<PrivateImplementationDetails>'::CF97ADEEDB59E05BFD73A2B4C2A8885708C4F4F70C84C64B27120E72AB733B72_A6
    IL_0006: dup
    IL_0007: brtrue.s     IL_0021
    IL_0009: pop
    IL_000a: ldc.i4.4
    IL_000b: newarr       [mscorlib]System.Int32
    IL_0010: dup
    IL_0011: ldtoken      field valuetype '<PrivateImplementationDetails>'/'__StaticArrayInitTypeSize=16' '<PrivateImplementationDetails>'::CF97ADEEDB59E05BFD73A2B4C2A8885708C4F4F70C84C64B27120E72AB733B72
    IL_0016: call         void [mscorlib]System.Runtime.CompilerServices.RuntimeHelpers::InitializeArray(class [mscorlib]System.Array, valuetype [mscorlib]System.RuntimeFieldHandle)
    IL_001b: dup
    IL_001c: stsfld       int32[] '<PrivateImplementationDetails>'::CF97ADEEDB59E05BFD73A2B4C2A8885708C4F4F70C84C64B27120E72AB733B72_A6
    IL_0021: newobj       instance void valuetype [System.Memory]System.ReadOnlySpan`1<int32>::.ctor(!0/*int32*/[])
    IL_0026: stloc.0      // intArray

    // For the non-constant array, a new array is created each time
    IL_0027: ldloca.s     nonConstantArray
    IL_0029: ldc.i4.4
    IL_002a: newarr       [mscorlib]System.Byte
    IL_002f: dup
    IL_0030: ldtoken      field int32 '<PrivateImplementationDetails>'::'1E6175315920374CAA0A86B45D862DEE3DDAA28257652189FC1DFBE07479436A'
    IL_0035: call         void [mscorlib]System.Runtime.CompilerServices.RuntimeHelpers::InitializeArray(class [mscorlib]System.Array, valuetype [mscorlib]System.RuntimeFieldHandle)
    IL_003a: dup
    IL_003b: ldc.i4.0
    IL_003c: ldsfld       unsigned int8 MyStaticData::One
    IL_0041: stelem.i1
    IL_0042: call         instance void valuetype [System.Memory]System.ReadOnlySpan`1<unsigned int8>::.ctor(!0/*unsigned int8*/[])

    // For the `Spam<byte>` array, a new array is created each time
    IL_0047: ldloca.s     spanArray
    IL_0049: ldc.i4.4
    IL_004a: newarr       [mscorlib]System.Byte
    IL_004f: dup
    IL_0050: ldtoken      field int32 '<PrivateImplementationDetails>'::'9F64A747E1B97F131FABB6B447296C9B6F0201E79FB3C5356E6C77E89B6A806A'
    IL_0055: call         void [mscorlib]System.Runtime.CompilerServices.RuntimeHelpers::InitializeArray(class [mscorlib]System.Array, valuetype [mscorlib]System.RuntimeFieldHandle)
    IL_005a: call         instance void valuetype [System.Memory]System.Span`1<unsigned int8>::.ctor(!0/*unsigned int8*/[])

    // [15 5 - 15 6]
    IL_005f: ret

  } // end of method MyStaticData::TestData

So unfortunately, collection expressions don't save you here. Of course, you likely can (and should) be using stackalloc here for these small arrays, so this isn't necessarily a big deal. But you do need to know how to do this.

So what should we make of all this?

Conclusion

The good news is that if you use the right patterns, using static ReadOnlySpan<byte> properties to replace existing static readonly byte[] fields that contain read-only data can give a zero-allocation and essentially zero-startup cost improvement, even on .NET Framework.

However, if the field that you're "converting" is not byte[], bool[] or sbyte[], then you should think carefully about whether to convert it. int[] and other types are supported for similar optimizations on .NET 7+, but this requires runtime support, so if you're also targeting .NET Framework, .NET Standard, or .NET 6 and below, then I would seriously consider whether it's worth making the change.

You likely will see perf benefits on .NET 7+, but as far as I can tell, you're talking about a ~15% speed improvement for the initial creation of the array. But if you're calling RuntimeHelpers.CreateSpan() with every access, versus just loading a field, does that actually improve steady state performance? I don't know, I haven't checked, I'm just wondering😄

Where you really need to be careful is to only use constant values in your arrays (no static readonly values, please) and only use ReadOnlySpan<T>, not Span<T>. Luckily, you'll catch these automatically in your static properties if you're using collection expressions, as they simply won't compile. Which just another reason you should use collection expressions everywhere you can!😃

Replacing static byte[] fields with static ReadOnlySpan<byte> is probably the most common scenario you'll find, but you can also apply this to local variables. However, I suspect that scenario is going to be less common, simply because that's so clearly very allocating, it means that presumably you "don't care about performance" here, in which case there's no point making the ReadOnlySpan<byte> change.

There's another reason for not touching local definitions, which is that the collection expression "solution" described above doesn't cause compilation failures with local variables, so there isn't the same easy guardrails there.

If you're anything like me, then the fact that there are so many edge cases where you fall off a performance cliff is somewhat surprising. Generally the .NET team try quite hard to avoid these cliffs, or at least add analyzers to help steer you in the right direction. There seems to be little here to stop you doing the "wrong" thing.

Looking through the various issues and discussions, that's something that's come up multiple times, but it seems like the difficulty is generally "the problematic code patterns are actually valid sometimes". There's also the "well, you should be using stackalloc anyway" argument, as well as "collection expressions partially protect you":

• [Analyzer]: Unexpected allocation when converting arrays to spans
• [Proposal]: ReadOnlySpan initialization from static data
• compile-time readonly arrays, const T[]
• "ReadOnlySpan initialization from static data" language design notes
• [API Proposal]: ReadOnlySpan CreateSpan(RuntimeFieldHandle)

So all-in-all, this approach seems to be "use at your own risk". I still think it might be nice to have optional analyzers at least to try to protect you (and maybe someone's already written those). Nevertheless, the ability to reduce initialization costs to 0 if you have a bunch of static data is definitely a win; just make sure you only use it in a safe way!

This post follows on directly from my previous post, in which I describe how to run AI agents safely using the docker sandbox tool, sbx. In this post I describe how to create custom templates, so that your sandboxes start with additional tools. I show both how to add tools to the default template, and how to start with a different docker image and layer-on the docker sandbox tooling later.

An initial caveat to this post: I've been somewhat struggling to get my personal projects working in docker sandboxes, with .NET just hanging indefinitely during builds. This seems to be specific to my projects, as a hello world build is fine, but a word of caution: your mileage may vary.

Running agents safely in a docker sandbox

As I described in my previous post, working with AI agents in their default mode can mean an infuriating number of tool calls, that interrupt your flow, and generally slow you down:

However, ignoring these tool calls using the "bypass permissions" mode (AKA YOLO/dangerous mode) can be, well, dangerous. There's plenty of examples of AI agents going rogue; do you want to risk it? Docker Sandboxes provide one solution.

Docker sandboxes run in microVMs, which are isolated from the host machine. The only folder the sandbox can access is the working directory you give access to, and all network traffic goes through a network proxy, which can either block traffic, or it can inject credentials such that the coding agent never sees them directly.

I've only used docker sandboxes for a short while, but I've found they work relatively well for my purposes. However, one limitation is that some of the projects I'm working on have a bunch of requirements for tooling, which always needs to be installed in the sandbox. Doing that every time is a bit of a pain. Luckily, there's a solution: custom templates.

Creating a custom Claude Code template

The Docker sandbox documentation describes how to create a custom template, based on one of the default templates. I'm going to use the Claud Code examples in this post, but there's different templates for each of the supported agents. For each supported image there are also 2 variants: one that includes a Docker Engine, and one that doesn't. e.g.

• claude-code—includes a variety of dev tools.
• claude-code-docker—includes the same as above, but also has Docker Engine.

There's also a claude-code-minimal template which is similar to claude-code, but includes fewer tools, so you don't have npm, python, or golang, for example.

To create a custom template, you need to have Docker Desktop installed as you're basically building an OCI image (effectively a docker image, kinda, sorta). That's despite the fact that docker sandboxes don't run as docker containers, but rather as microVMs.

The following example, based on the documentation shows how to start from the default template, how to install package manager dependencies, and how to install other tools, using dotnet as an example:

FROM docker/sandbox-templates:claude-code-docker

# Switch to root to run package manager installs (.NET dependencies)
USER root
RUN apt-get update \
    && apt-get install -y --no-install-recommends \
    ca-certificates \
    libc6 \
    libgcc-s1 \
    libgssapi-krb5-2 \
    libicu76 \
    libssl3t64 \
    libstdc++6 \
    tzdata \
    zlib1g

# Most tools should be installed at user-level, using the agent user
USER agent
RUN curl -sSL https://dot.net/v1/dotnet-install.sh | bash /dev/stdin --channel 10.0 --no-path
ENV DOTNET_ROOT=/home/agent/.dotnet \
    PATH=$PATH:/home/agent/.dotnet:/home/agent/.dotnet/tools

This shows several important things:

• The base docker/sandbox-templates images are based on Ubuntu, so use apt-get for managing packages.
• The base images include two users, root and agent.
  • System-level package installations must be made using the root user.
  • Tools that install into the home directory must be installed using the agent user.

You can build the package using familiar docker build commands, but you must push it straight to an OCI registry (Docker Hub works!). You can't just build it locally as the docker sandbox doesn't share the image store with your local Docker host.

docker build -t my-org/my-template:v1 --push .

Once you've pushed the image to an OCI registry you can use it locally in a sandbox by using the --template or -t argument when calling sbx run:

sbx run -t docker.io/my-org/my-template:v1 claude

This will pull (and cache) the template you specify, and you'll have the extra tools immediately available in your sandbox. Note that you must include the docker.io (Docker Hub) or other prefix when specifying the template (which differs from when you're running "normal" docker commands).

I've created some sandboxes for .NET, similar to the above, and pushed them to dockerhub. You can see the definition of the images here. Feel free to use them if you wish!

Basing your custom templates on the standard default templates works well when you just want to make some extra tools available to your sandbox, but what if you fundamentally want to use a different base image? That's a bit trickier…

What if you need to change the base image?

The "supported" approach to these custom templates is shown in the previous section: you start with the docker/sandbox-templates and then install the extra tools on that base image. Currently, those images are based on ubuntu 25.10, which is a nice current base image. But what if you need to use an older image for running tests. This is the case for the Datadog .NET SDK where we build using old distro versions to ensure we can support customers running with early glibc versions.

This proves a little tricky, as it's not officially supported. On the one hand, to emulate the work the base images do, mostly there's just a few crucial configurations you need to add, such as setting NO_PROXY, creating an agent user, and installing the claude CLI. However, the docker/sandbox-templates images contain a lot more than that. Unfortunately, the contents of these images aren't readily available on GitHub, for example.

Luckily, you can see the contents of each layer on Docker Hub. It's a little bit messed up due to how buildkit renders it, but it is understandable. Based on each of those layers, I was able to effectively reverse-engineer the layering the docker/sandbox-templates:claude-code-docker image on top of a different base image.

This is all very hacky, could change at any time, and comes with no guarantees that it works for you 😅

The following shows a dockerfile that aims to perform all the steps the default docker/sandbox-templates to, but based on an arbitrary base image. There's quite a lot in here, but in summary:

• It configures various environment variables.
• Installs various basic tools (curl, certificates) and sets up various keyrings.
• Configures the agent user.
• Sets up a CLAUDE_ENV_FILE temporary session file.
• Installs a variety of tools (npm, golang, python, make etc).
• Installs Claude Code.

All in all, it looks a bit like this:

FROM dd-trace-dotnet/debian-tester AS base

# Grab stuff from the original sandbox
ENV NPM_CONFIG_PREFIX=/usr/local/share/npm-global
ENV PATH=/home/agent/.local/bin:/usr/local/share/npm-global/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ENV NO_PROXY=localhost,127.0.0.1,::1,172.17.0.0/16
ENV no_proxy=localhost,127.0.0.1,::1,172.17.0.0/16

WORKDIR /home/agent/workspace
RUN apt-get update \
    && apt-get install -yy --no-install-recommends \
    ca-certificates \
    curl \
    gnupg \
    && install -m 0755 -d /etc/apt/keyrings \
    && curl -fsSL https://download.docker.com/linux/debian/gpg | \
    gpg --dearmor -o /etc/apt/keyrings/docker.gpg \
    && chmod a+r /etc/apt/keyrings/docker.gpg \
    && echo \
    "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \
    $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
    tee /etc/apt/sources.list.d/docker.list > /dev/null

# Remove base image user
# Create non-root user
# Configure sudoers
# Create sandbox config
# Set up npm global package folder under /usr/local/share
RUN userdel ubuntu || true \
    && useradd --create-home --uid 1000 --shell /bin/bash agent \
    && groupadd -f docker \
    && usermod -aG sudo agent \
    && usermod -aG docker agent \
    && mkdir /etc/sudoers.d \
    && chmod 0755 /etc/sudoers.d \
    && echo "agent ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/agent \
    && echo "Defaults:%sudo env_keep += \"http_proxy https_proxy no_proxy HTTP_PROXY HTTPS_PROXY NO_PROXY SSL_CERT_FILE NODE_EXTRA_CA_CERTS REQUESTS_CA_BUNDLE JAVA_TOOL_OPTIONS\"" > /etc/sudoers.d/proxyconfig \
    && mkdir -p /home/agent/.docker/sandbox/locks \
    && chown -R agent:agent /home/agent \
    && mkdir -p /usr/local/share/npm-global \
    && chown -R agent:agent /usr/local/share/npm-global

RUN touch /etc/sandbox-persistent.sh && chmod 644 /etc/sandbox-persistent.sh && chown agent:agent /etc/sandbox-persistent.sh
ENV BASH_ENV=/etc/sandbox-persistent.sh

# Source the sandbox persistent environment file
# Export BASH_ENV so non-interactive child shells also source the persistent env
RUN echo 'if [ -f /etc/sandbox-persistent.sh ]; then . /etc/sandbox-persistent.sh; fi; export BASH_ENV=/etc/sandbox-persistent.sh' \
    | tee /etc/profile.d/sandbox-persistent.sh /tmp/sandbox-bashrc-prepend /home/agent/.bashrc > /dev/null \
    && chmod 644 /etc/profile.d/sandbox-persistent.sh \
    && cat /tmp/sandbox-bashrc-prepend /etc/bash.bashrc > /tmp/new-bashrc \
    && mv /tmp/new-bashrc /etc/bash.bashrc \
    && chmod 644 /etc/bash.bashrc \
    && rm /tmp/sandbox-bashrc-prepend
    && chmod 644 /home/agent/.bashrc \
    && chown agent:agent /home/agent/.bashrc

USER root

# Setup Github keys
RUN curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg \
    | tee /etc/apt/keyrings/githubcli-archive-keyring.gpg > /dev/null \
    && chmod a+r /etc/apt/keyrings/githubcli-archive-keyring.gpg \
    && echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" \
    | tee /etc/apt/sources.list.d/github-cli.list > /dev/null

# Install all the tools available in the claude-code-docker image
RUN apt-get update \
    && apt-get install -yy --no-install-recommends \
    dnsutils \
    docker-buildx-plugin \
    docker-ce-cli \
    docker-compose-plugin \
    git \
    jq \
    less \
    lsof \
    make \
    procps \
    psmisc \
    ripgrep \
    rsync \
    socat \
    sudo \
    unzip \
    gh \
    bc \
    default-jdk-headless \
    golang \
    man-db \
    nodejs \
    npm \
    python3 \
    python3-pip \
    containerd.io docker-ce \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

LABEL com.docker.sandboxes.start-docker=true

USER agent

FROM base AS claude

# Install Claude Code
RUN curl -fsSL https://claude.ai/install.sh | bash

ENV CLAUDE_ENV_FILE=/etc/sandbox-persistent.sh
CMD ["claude", "--dangerously-skip-permissions"]

If you don't want all the extra tools like npm, python and golang, you can instead base it on the claude-code-minimal image instead. In that case, the final tool install step looks a bit like this instead:

RUN apt-get update \
    && apt-get install -yy --no-install-recommends \
        bubblewrap \
        dnsutils \
        docker-buildx-plugin \
        docker-ce-cli \
        docker-compose-plugin \
        git \
        gh \
        jq \
        less \
        lsof \
        make \
        procps \
        psmisc \
        ripgrep \
        rsync \
        socat \
        sudo \
        unzip \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

Or, you know, install a mix of those tools. That's the advantage of this approach at least, you can install more of fewer tools, whatever you want! Whichever approach you like, you can again build and push the image to an OCI registry:

docker build --tag dd-trace-dotnet/sandbox --push .

You can then use the image in your sbx sandbox, just as before, but this time you'll be running in a base image that has all of your prerequisites installed.

Updating the version of Claude Code only

You might notice in the above Dockerfile that I put the Claude Code image in its own section of the multi-stage build:

FROM base AS claude

# Install Claude Code
RUN curl -fsSL https://claude.ai/install.sh | bash

ENV CLAUDE_ENV_FILE=/etc/sandbox-persistent.sh
CMD ["claude", "--dangerously-skip-permissions"]

That's not necessary, but I did it for a subtle reason. Claude Code updates a lot, but I didn't really want to update the entire image repeatedly for performance reasons. By moving the Claude Code install to its own final stage, I could rebuild just that stage, without having to rebuild the entire image, by using --no-cache-filter:

docker build --tag dd-trace-dotnet/sandbox --push --no-cache-filter claude .

It's just a minor thing, but it means updating to the latest Claude Code version is a much quicker process.

I still need to test this image out properly, but I tried it out with a previous version and it was working pretty well for me. I'd be interested to know if anyone else has tried something similar, or if you have a better solution (short of just yolo/dangerous direct on the host!).

Summary

In this post I described how to create custom templates for Docker Sandboxes. First I showed the official approach, which layers tools on top of yhe default sandbox templates in docker/sandbox-templates. This is the easiest approach, and works well if the specific base image doesn't matter too much to you. Then I showed how I reverse-engineered the sandbox templates to allow completely swapping out the base image. This was necessary for a project I was working on, where I specifically wanted to run agents in the same base image we use to build the project. this approach isn't supported, and I'm not 100% it's quite right, but it seems to do the job well enough!

In this post I describe one way to run coding agents locally safely while still using "YOLO" or "dangerous" mode, by using Docker Sandboxes.

Powerful agents, but they need a lot of hand holding

It's pretty safe to assume that if you're reading this, you're probably using some sort of coding agent these days, whether that's Claude Code, Codex, Copilot, or something else. I have a whole bunch of ethical, environmental, and sustainability concerns about the technology, but the fact is that in 2026, they've got so much better than they were even 6 months ago.

I am massively conflicted about the role of AI in software engineering (let alone other areas of life), but I'm not going to address that in this post. For better or worse, it feels like working with coding agents is practically becoming a job requirement, so learning how to do it safely seems important..

I've been very impressed with how effective Claude Code can be at adding new features, maintenance, and problem solving, but there's one thing that's infuriating… the dreaded tool-call permissions.

The real problem is that Claude asks this for endless things. Want to use grep? Confirm permission. Want to use sed? Confirm permission. Want to use cd because Claude Code doesn't understand Windows? Confirm permission.

This is an absolute killer for productivity. Using the tools like this becomes exhausting, constantly switching between terminal windows to find the agent that's managed to run into a wall again 🙄

In this post I'm focusing on Claude Code as that's what I have the most experience with, but everything pretty much works the same for other agents as I understand it too.

Live dangerously, if you dare

Of course, there is a solution, but it's not for the feint-hearted. Claude Code has the flag --allow-dangerously-skip-permissions, which adds a "bypass permissions" mode to the standard "plan" and "accept edit" modes.

This flag means "bypass permissions" mode is available, but it doesn't start in that mode. If you want to start in that mode, you can use --permission-mode bypassPermissions or --dangerously-skip-permissions instead.

If you start claude code using:

claude --allow-dangerously-skip-permissions

Then you'll get a warning:

and you can select bypass permissions on demand by cycling through modes with Shift+Tab

The trouble is, then you won't get any permission tool requests. If Claude decides to run something stupid that deletes your User folder then sorry, that's you hosed. It's right there in the warning…this is dangerous😅

And yet…

The experience in bypass permissions mode is just so much better when you want the agent to just go and do something (or even if you just want it to create a plan). It doesn't bother you about every little thing, it just does the task. That's a hard experience to give up, but there are options that can get you pretty close to this, safely.

Live safely, in a sandbox

Docker recently released Docker Sandboxes. It might surprise you that this actually isn't built on containers, but rather on isolated microVM sandboxes. This has some security advantages:

• Unlike containers, which share the host kernel, each sandbox has its own kernel
• The microVM runs a separate docker engine inside so you can build and run containers without having to mount your host docker socket
• The network in the microVM is isolated from the host. A network proxy runs on the host side, intercepting traffic, blocking access to the host's localhost, and automatically injecting authentication headers (so that the sandbox doesn't have access to them).

With all this isolation, the idea is that you can just let your agents run amok, without having to babysit them. Sounds ideal, right?

Docker sandboxes are experimental. Until recently, it was shipping with Docker Desktop and you ran commands like docker sandbox run. However, they recently switched to shipping a dedicated sbx tool that doesn't require docker desktop.

For the rest of this post I'll discuss the basics of getting started with sandboxes, and my brief experience with them.

Getting started with sbx sandbox

Due to the experimental nature of the sbx tool and Docker Sandboxes in general, this post is going to be relatively light on details and will focus on the basics, as I expect it will go out of date rapidly. Instead, I recommend you check the docs for more advanced usages.

Installing the sbx tool

To get started, you'll first need to install the sbx tool. Only macOS (arm64) and Windows (x86_64, Windows 11) are currently supported, and I'm going to provide the instructions for Windows seeing as that's what I'm using.

First, you'll need to make sure you have the HypervisorPlatform feature enabled. This is different (but related to) the full Hyper-V feature. It's used by WSL2, so it's very likely you already have this enabled anyway, but just in case, run the following in an administrator PowerShell prompt:

Enable-WindowsOptionalFeature -Online -FeatureName HypervisorPlatform -All

Next, install the sbx tool using WinGet, or by downloading the MSI from the Github releases page:

winget install -h Docker.sbx

You'll need to open a new terminal window to make sure the tool is available.

Signing in and configuring the defaults

Once you've installed (and opened a new terminal window), you have to sign-in to use docker sandboxes, so start by running sbx login (or just sbx, it'll do the same thing):

❯ sbx
You are not authenticated to Docker. Starting the sign-in flow...

Your one-time device confirmation code is: FXDG-FKTF
Open this URL to sign in: https://login.docker.com/activate?user_code=FXDG-FKTF

By logging in, you agree to our Subscription Service Agreement. For more details, see https://www.docker.com/legal/docker-subscription-service-agreement/

Waiting for authentication...

This pops up a login window, where you first confirm, and then login to docker:

You're then provided with a choice of how to configure your network:

Signed in as andrewlock.
Daemon started (PID: 52268, socket: \\.\pipe\docker_kaname_sandboxd)
Logs: C:\Users\sock\AppData\Local\DockerSandboxes\sandboxes\state\sandboxd\daemon.log

Select a default network policy for your sandboxes:

     1. Open         — All network traffic allowed, no restrictions.
     2. Balanced     — Default deny, with common dev sites allowed.
     3. Locked Down  — All network traffic blocked unless you allow it.

  Use ↑/↓ or 1–3 to navigate, Enter to confirm, Esc to cancel.

The descriptions here are relatively self-evident, and it depends on how locked down you want your sandbox to be. All communication from the sandbox goes through a proxy, so it's really this proxy you're configuring.

The network policies are a new feature since I started working with the sandboxes, so I haven't experimented with these myself yet. I tried out Balanced, and used sbx policy ls to describe the policies, and it configured the following as allowed domains; all other network requests will be blocked:

As you can see, this includes pretty much everything you might need for building applications, but it's notably missing things like documentation sites, so if your agent needs to go out to Microsoft learn (for example), it's going to be stuck. I think that could be a big gap in the balanced mode, so I switched to "open" mode instead by running sbx policy reset and choosing again.

Creating a sandbox

Once you've chosen your network policy, you can create your first sandbox. Navigate to your project folder, and run sbx run claude:

cd .\NetEscapades.EnumGenerators
sbx run claude

This downloads a docker image for the selected agent, and creates a sandbox named after the current working directory. Once downloaded, sbx uses the image to spin up a microVM and runs your agent of choice in YOLO/dangerously skip permissions mode:

Creating new sandbox 'claude-NetEscapades.EnumGenerators'...
aeacf85cf4c8: Download complete
4f33085e2ac1: Download complete
6b4ac13f7bd1: Download complete
Digest: sha256:aeacf85cf4c8e40f5d1a3709ed7f2a7f466f78787e56780ec321f0db6bc1a53a
Status: Downloaded newer image for docker/sandbox-templates:claude-code
✓ Created sandbox 'claude-NetEscapades.EnumGenerators'
  Workspace: D:\repos\oss\NetEscapades.EnumGenerators (direct mount)
  Agent: claude

To connect to this sandbox, run:
  sbx run claude-NetEscapades.EnumGenerators

Starting claude agent in sandbox 'claude-NetEscapades.EnumGenerators'...
Workspace: D:\repos\oss\NetEscapades.EnumGenerators
 ▐▛███▜▌   Claude Code v2.1.90
▝▜█████▛▘  Sonnet 4.6 · API Usage Billing
  ▘▘ ▝▝    /d/repos/oss/NetEscapades.EnumGenerators

  ↑ Opus now defaults to 1M context · 5x more room, same pricing

───────────────────────────────────────────────────────────────────────────────────────────
❯ 
───────────────────────────────────────────────────────────────────────────────────────────
  ⏵⏵ bypass permissions on (shift+tab to cycle)

And you're off to the races! You can hack away and know that the sandbox only has access to your working directory, so yes, it could delete your git repo (and there are ways to avoid that too), but that's basically the extent of the damage it can do.

Docker sandboxes currently have support for Claude Code, Codex, Copilot, Gemini, Kiro, OpenCode, and Docker Agent.

You can now set Claude to work, and it runs in --dangerously-skip-permissions mode, without needing to prompt you everytime it needs to use a tool. So at this point, you probably need to review and push/reject those changes. So it's worth thinking about how git works with sbx.

Committing changes to a git repository

There's basically 2 ways you can use sbx sandboxes:

• Direct mode
• Branch mode

In direct mode, the agent just edits files in your working directory, and commits directly to the git repository in that directory. This is the easiest to use and understand, but be aware that it has access to the whole git history, so technically the agent could end up breaking your git repo. I've never seen it, but it's important to be aware it could happen😅

In branch mode, the sbx sandbox creates a git worktree in a .sbx/ sub-folder in your root directory, and starts the agent in that sub-folder. The agent still has access to the root directory, but it means you can continue to work in the "main" working directory, or you could start additional agents working in other worktrees.

To start a sandbox in branch mode, pass the --branch flag. For example:

# agent creates a worktree at .sbx/<sandbox-name>-worktrees/my-feature
sbx run claude --branch my-feature

# agent generates its own name for the branch + worktree
sbx run claude --branch auto

Now, it's important to note that this creates the folder inside your git working directory:

├── .sbx/
│   └── claude-NetEscapades.EnumGenerators-worktrees/
│       └── my-feature/
│           ├── build/
│           ├── docs/
│           ├── samples/
│           ├── src/
│           │   ├── NetEscapades.EnumGenerators/
│           │   ├── NetEscapades.EnumGenerators.Attributes/
│           │   ├── NetEscapades.EnumGenerators.Generators/
│           │   ├── NetEscapades.EnumGenerators.Interceptors/
│           │   ├── NetEscapades.EnumGenerators.Interceptors.Attributes/
│           │   └── NetEscapades.EnumGenerators.RuntimeDependencies/
│           ├── tests/
│           │   ├── NetEscapades.EnumGenerators.Tests/
│           │   ├── NetEscapades.EnumGenerators.IntegrationTests/
│           │   ├── NetEscapades.EnumGenerators.Interceptors.IntegrationTests/
│           │   ├── NetEscapades.EnumGenerators.Benchmarks/
│           └── NetEscapades.EnumGenerators.sln
├── build/
├── docs/
├── samples/
├── src/
│   ├── NetEscapades.EnumGenerators/
│   ├── NetEscapades.EnumGenerators.Attributes/
│   ├── NetEscapades.EnumGenerators.Generators/
│   ├── NetEscapades.EnumGenerators.Interceptors/
│   ├── NetEscapades.EnumGenerators.Interceptors.Attributes/
│   └── NetEscapades.EnumGenerators.RuntimeDependencies/
├── tests/
│   ├── NetEscapades.EnumGenerators.Tests/
│   ├── NetEscapades.EnumGenerators.IntegrationTests/
│   ├── NetEscapades.EnumGenerators.Interceptors.IntegrationTests/
│   ├── NetEscapades.EnumGenerators.Benchmarks/
└── NetEscapades.EnumGenerators.sln

That's a bit of a pain in general, because that whole working directory shows up in the git diff:

❯ git status
On branch my-feature
Untracked files:
  (use "git add <file>..." to include in what will be committed)
        .sbx/

nothing added to commit but untracked files present (use "git add" to track)

That means you need to add this directory to your project's .gitignore file. Or, a neater way, is to add the folder to the gitignore globally on your machine. The following PowerShell script reads the core.excludesFile setting (if it's set) and either adds the .sbx/ folder to this file, or adds it to the default location at $HOME/.config/git/ignore.

# Get the path to the default ignore file
$path = git config --global core.excludesFile
if (-not $path) { $path = "$HOME/.config/git/ignore" }

# Create the parent directory
New-Item -ItemType Directory -Force -Path (Split-Path $path) | Out-Null

# Add .sbx/ to the file
Add-Content -Path $path -Value ".sbx/"

This seems to work pretty well, but again, be aware that the agent could still screw up your git directory, because it fundamentally has access to it. So make sure you have a backup (e.g. you've pushed to a remote repository), just in case. Or alternatively, work on an entirely separate clone of the repo.

⚠️ One git workflow that won't work is creating a worktree yourself, and then running a sandbox directly in this folder. In this scenario, the agent doesn't have access to the "parent" git repository, so it won't be able to commit any changes, which is a great way to confuse both it and you 😅.

I mentioned earlier that Docker Sandboxes don't just run docker containers, they run in microVMs. However, that also means you can't get an overview of your sandboxes using docker or docker desktop. So how do you know what's going on with your sandboxes?

Getting an overview with a TUI dashboard

sbx ships with several commands for viewing and managing sandboxes:

❯ sbx --help
Docker Sandboxes creates isolated sandbox environments for AI agents, powered by Docker.

Run without a command to launch interactive mode, or pass a command for CLI usage.

Usage:
  sbx.exe
  sbx.exe [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  create      Create a sandbox for an agent
  exec        Execute a command inside a sandbox
  help        Help about any command
  login       Sign in to Docker
  logout      Sign out of Docker
  ls          List sandboxes
  policy      Manage sandbox policies
  ports       Manage sandbox port publishing
  reset       Reset all sandboxes and clean up state
  rm          Remove one or more sandboxes
  run         Run an agent in a sandbox
  save        Save a snapshot of the sandbox as a template
  secret      Manage stored secrets
  stop        Stop one or more sandboxes without removing them
  version     Show Docker Sandboxes version information

but there's also a neat "dashboard" view, which you can start by running sbx without any arguments (once you login for the first time):

This dashboard shows each of your running sandboxes, the resources they're using, the network requests they're making, and the global network rules. It's a neat little TUI you can use to get an overview of your sandboxes!

With that, you should have most of the basics ready for working safely with agents in a sandbox! In the next post I'll look at how you can run custom templates in your sandbox instead of the default template, but before we leave, it's worth highlighting some of the limitations.

So what's the catch?

Before the release of sbx, there was a Docker Desktop based version of Docker Sandbox that worked pretty much the same way as sbx in many ways. But it had a massive limitation; it was limited to using a maximum of 4GB of memory, and was not configurable. For large projects, this proved to be a big issue, making it virtually unusable for me. Luckily, that's not the case with sbx, which has a --memory option to control this, and defaults to 50% of host memory.

One thing I haven't figured out yet is how to get commit signing work. I use 1Password to sign my commits, which runs an ssh-agent.exe for commit signing. But I haven't worked out how to share that into the sandbox. As a workaround, I've settled for letting the sandbox create unsigned commits. Then, once it's all finished, on the host side I do a simple rebase, which then signs all the commits. It's a bit annoying, but not the end of the world. If you know of a workaround, I'd love to hear about it!

Another tricky point is the network policies. It looks like a nice way to limit the blast radius of a rogue agent, but I feel like I'd always be running into limitations, trying to curate the policies. Seems like a useful "organisation policy" level control, but frankly I'm probably just going to run it in open mode. The sandbox ensures the agent can't mess up my system, and as it doesn't have access to any of my keys or private data, I'm not too worried about what sites it tries to access.

The final issue is performance, which has, unfortunately, been the deal breaker for me in many cases. Even for simple projects, I've found that the performance hit from running in a sandbox can be crippling. I only recently ran into this issue (I swear it wasn't so much of an issue a couple of weeks ago), so I'm hoping it's something that will be addressed soon😬

Summary

In this post I described how to use the docker sandbox tool sbx to run AI coding agents in a sandbox. Using a sandbox means you can run the tools in yolo or --dangerously-skip-permissions mode, so you don't have to babysit it constantly. I've found this greatly increases velocity, and running in a sandbox removes the sense of uneasiness that I get whenever I choose to live dangerously on my machine! This post describes how to set up the sbx tool, discusses the network policy architecture, and how to commit to git. In the next post I'll describe how to create custom templates, which can be useful if you have specific tools you need installed in the sandbox for the agent to work with.

In this post I take a brief look at the Microsoft.Extensions.Options.Contextual package that I came across the other day. I try to understand the purpose of the package, look at how to install and configure it, and finally consider whether it's something people should consider using themselves.

What is Microsoft.Extensions.Options.Contextual?

I was browsing around in the dotnet repositories for something the other day, when I spotted this library: Microsoft.Extensions.Options.Contextual. I was somewhat intrigued considering I hadn't heard this library mentioned before, or had seen it used. What's more, a little probing seemed to suggest it's not used by any other first party .NET libraries either. So what is it for, and how does it work?

According to the library itself, the library provides:

APIs for dynamically configuring options based on a given context

That's pretty vague 😅 What are "options" and what is "context" here? It's likely easiest to understand with an example, even if it's a simple one. The example below is based on the documentation in the package, using the classic "weather forecast" scenario.

Configuring options based on another type

First of all, let's imagine we have some configuration options:

internal class WeatherForecastOptions
{
    public string TemperatureScale { get; set; } = "Celsius"; // Celsius or Fahrenheit
    public int ForecastDays { get; set; }
}

This is pretty standard stuff—which unit to use for temperature display, and how many days to include in the forecast.

The interesting thing is how these options are configured. If you're doing global configuration for your app then you might have something like this (using Configure() and/or AddOptions()):

var builder = WebApplication.CreateBuilder(args);

builder.Services
    .AddOptions<WeatherForecastOptions>() // Register the options
    .Configure(x => x.ForecastDays = 7) // Configure in Code
    .BindConfiguration(builder.Configuration["Weather"]); // Bind to configuration

That's all well and good, but you're fundamentally "globally" configuring the WeatherForecastOptions instances (whether you're using singleton, transient, or scoped instances).

I'm not going to get into the general IOptions<> abstraction debate here. Many people rail against it, and I sympathize. That said, if you don't like those abstractions, I don't know that you'll like the ones we're introducing shortly 😉.

What if, instead, you want to configure an options object arbitrarily based on some context object? For example, imagine you want to change the TemperatureScale based on the country associated with the current user. This "context" might be encapsulated in some Appcontext object:

internal class AppContext
{
    public Guid UserId { get; set; }
    public string? Country { get; set; }
}

Note that this is a different use case to named options where you have a distinct named set of "global" options. For contextual options we explicitly configure the app based on the provided context.

For example, at the call site, contextual options would look something like this:

// Get an IContextualOptions<TOptions, TContext> from DI
IContextualOptions<WeatherForecastOptions, AppContext> _contextualOptions;

AppContext context; // Get an instance of the context object from somewhere

// Get an instance of WeatherForecastOptions using the AppContext instance
WeatherForecastOptions options = await _contextualOptions.GetAsync(context, cancellationToken: default);

The IContextualOptions<> instance is available in DI when you set up contextual options, and the AppContext is whatever you want to use to control how your options object is created. You then pass one to the other, and voila, you have a configured contextual options object, with properties set based on AppContext.

Of course, there's a lot more to it behind the scenes, so we'll look at how to get started with this in the next section.

Adding the Microsoft.Extensions.Options.Contextual package

To get started with contextual option, you need to add the Microsoft.Extensions.Options.Contextual package to your project. Interestingly, it appears that this package has never made it out of preview:

This means you must use the --prerelease flag when adding the package to your project with the .NET CLI:

dotnet add package Microsoft.Extensions.Options.Contextual --prerelease

This adds the latest version to your project file:

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>

  <ItemGroup>
    <!-- Add this 👇 -->
    <PackageReference Include="Microsoft.Extensions.Options.Contextual" Version="10.4.0-preview.1.26160.2" />
  </ItemGroup>
</Project>

The latest version of the package supports .NET Framework and .NET 8+.

Once you have the package installed, you can start configuring your options and context.

Configuring the contextual options

Once we have the package installed, we need to configure all the moving parts:

• Add [OptionsContext] to your context object, to drive a source generator
• Define an IOptionsContextReceiver object for extracting value from the context
• Define how to take the extracted context values and apply them to your options object

It likely isn't clear why you need all these components at this points, but just go with it for now. It'll become clearer how they all fit together later.

Adding the ContextualOptionsGenerator

The first step is to mark your context object partial and add the [OptionsContext] attribute:

[OptionsContext] // 👈 Add attribute, and make the type partial
internal partial class AppContext
{
    public Guid UserId { get; set; }
    public string? Country { get; set; }
}

I've used a mutable class for AppContext in the example above, but you could also make it a struct, and (ideally) make it readonly too.

The [OptionsContext] object drives a source generator, the ContextualOptionsGenerator, which generates a separate partial class, which looks a little like this:

// <auto-generated/>
using Microsoft.Extensions.Options.Contextual;

partial class MyAppContext : IOptionsContext
{
    void IOptionsContext.PopulateReceiver<T>(T receiver)
    {
        receiver.Receive(nameof(Country), Country);
        receiver.Receive(nameof(UserId), UserId);
    }
}

This makes your context type implement IOptionsContext:

public interface IOptionsContext
{
    void PopulateReceiver<T>(T receiver)
        where T : IOptionsContextReceiver;
}

which interacts with:

public interface IOptionsContextReceiver
{
    void Receive<T>(string key, T value);
}

O…K… so what has that got us? 😅 Effectively the source generator provides a way to extract properties from a type by their name, and pass them to a "receiver" type. So now we'll implement that receiver.

Implementing an IOptionsContextReceiver

Implementing an IOptionsContextReceiver for your target is pretty simple; just create a type that implements the interface, and which can receive any type of T value. as you saw above, this receiver will be invoked with each of the properties of the "context" object, so the idea is to simply extract the keys that you care about in your context.

In the following example, we only care about the "Country" key, so that's the value we extract. I've done a little further manipulation of the value in this example, converting the "Country" into an assumed default temperature unit:

// Implement IOptionsContextReceiver to receive the context values
internal class CountryTemperatureReceiver : IOptionsContextReceiver
{
    // This property exposes the "extracted" values to others
    public string? DefaultUnit { get; private set; }

    // The receive implementation could receive a T of any type
    public void Receive<T>(string key, T value)
    {
        // When you receive the key you're looking for...
        if (key == "Country")
        {
            // ... extract the value and store it
            DefaultUnit = value is "USA" ? "Fahrenheit" : "Celsius";
        }
    }
}

Ok, we're almost there, we have:

• Our "context" object
• A source generated IOptionsContext which populates a "receiver" based on the context object
• Our "receiver" object
• Our "target" options that we want to configure

Now it's time to put it all together

Configuring the WeatherForecastOptions

We've almost got all the basic pieces, all that remains is to set up the configuration of our options object, WeatherForecastOptions. We can do this partially using "normal" options configuration, by providing lambdas or binding to configuration, and partially using our new "contextual" configuration options which serves as a contextual "loader":

// Create a standard .NET app builder (in this case a web app)
var builder = WebApplication.CreateBuilder(args);

builder.Services
    .Configure<WeatherForecastOptions>(x => x.ForecastDays = 7) // Configure in Code
    .Configure((IOptionsContext ctx, WeatherForecastOptions opts) => // Configure contextual options
    {
        // 1. Create an instance of the receiver 
        var receiver = new CountryTemperatureReceiver();

        // 2. Populate the receiver based on the context
        ctx.PopulateReceiver(receiver);

        // 3. Update the options based on the receiver's values
        if (receiver.DefaultUnit is not null)
        {
            opts.TemperatureScale = receiver.DefaultUnit;
        }
    });

With the code above we've now configured an WeatherForecastOptions object, based on some unknown IOptionsContext, via the CountryTemperatureReceiver. This code is invoked as needs be, using the provided IOptionsContext instance, by the IContextualOptions implementation.

Retrieving a contextual WeatherForecastOptions object

Finally, we have all the pieces in place. All that remains is to use the IContextualOptions object, as we saw earlier:

// Get an IContextualOptions<TOptions, TContext> from DI
IContextualOptions<WeatherForecastOptions, AppContext> _contextualOptions;

AppContext context; // Get an instance of the context object from somewhere

// Get an instance of WeatherForecastOptions using the AppContext instance
WeatherForecastOptions options = await _contextualOptions.GetAsync(context, cancellationToken: default);

When invoked, the IContextualOptions<> object loads a "globally" configured IOptions<WeatherForecastOptions> instance, and then calls our "loader" function. This uses the IOptionsContext to pass values to our IOptionsReceiver object, and then we use those values to configure the WeatherForecastOptions. Finally, the configured object is returned, configured both via the standard IOptions system and by contextual options.

So, what's this for again?

So, if you're anything like me, you might reading this and thinking…Why? Why go to all that length?

On first blush it looks a little bit like "loose coupling". Sure, you could set the default WeatherForecastOptions based on the properties of the AppContext directly, but by using the receiver, you're now "loosely coupled".

Except, you're not, really, are you?

If you had code like this, then it would be very clear that you're directly coupled:

WeatherForecastOptions opts = new();
AppContext ctx = new();

// Configure the options directly based on the AppContext
opts.TemperatureScale = ctx.Country is "USA" ? "Fahrenheit" : "Celsius";

But to my eyes, introducing the receiver doesn't really reduce that coupling. It just means that you're now coupled indirectly via the magic string "Country". If you rename the Country property, then suddenly this breaks.

All of which makes me wonder, why bother? Is all this infrastructure really necessary to provide another way to configure an IOptions object? What was the thinking here?

Feature flags, theoretically

Luckily, after playing around for a while, I found an API proposal for introducing these contextual options. But the funny thing was, the API proposal was closed as "not planned" with the API "needs work" 😅 So… what happened here?!

The proposed API and API usage given in the issue are pretty much exactly what is shipped and currently available, and is pretty much the only example around, i.e. the classic WeatherForecast example.

What is revealing is one question raised in the API review meeting:

• How many IOptionsContextReceiver implementations are there in practice?
  • There's an internal one for Azure, but we don't know of an open-source version of a > service that provides contextual options.
  • LaunchDarkly is a commercial service that could provide contextual options.

The fact that they mention LaunchDarkly here is telling. It shows that they were clearly thinking about this as a "feature flags" solution.

Except, there's already a feature flags solution from Microsoft, that I wrote about extensively 7 years ago 😅 That's also based on the .NET configuration system. Now, I haven't really looked at that solution in a loong time, but seeing as Microsoft.FeatureManagement has about 140 million downloads, I think it's safe to say some people think it does the job. So introducing a sideways approach to do something very similar feels a little strange to me.

Right now, I can only see one package that uses the Microsoft.Extensions.Options.Contextual and that is a package for doing feature flagging/experimentation: https://github.com/excos-platform/config-client

In Microsoft's defence, Microsoft.Extensions.Options.Contextual is very much marked as experimental, such that you really have to put some effort in if you actually want to use it.

Definitely experimental

To start with, there's no stable version of the Microsoft.Extensions.Options.Contextual package. All versions of the package are marked preview, so you'll need to bear that in mind when you install it.

Additionally, all the APIs are also decorated with the [Experimental] attribute, which generate the EXTEXP0018 compile-time error. This is similar to the [Obsolete] attribute, but it's applied to new APIs that might change, as opposed to old ones that are deprecated. If you want to use any of the APIs you have to explicitly opt in by using #pragma or other methods to disable the warnings:

#pragma warning disable EXTEXP0018

A particularly irritating aspect is the fact that even the source generated code has this error, and seeing as you can't add #pragma to these APIs, you effectively must disable the warning globally (which kind of defeats the purpose of the attribute a bit in my opinion). The easiest way to disable it is to add it to the NoWarn variable in your .csproj file:

<PropertyGroup>
  <!-- Suppresses "For evaluation purposes only and is subject to change or removal in future updates" -->
  <NoWarn>$(NoWarn);EXTEXP0018</NoWarn>
</PropertyGroup>

So given all that, is there any value to these APIs, and should you use them?

In short, I don't think so. If you're completely bought into the IOptions life, and you specifically have a need for something like that, then, maybe, I guess. But with none of the package versions ever reaching 1000 downloads, it doesn't seem like many people consider it worth it.

And if what you really want is feature flagging and/or experimentation, then maybe consider taking a look at OpenFeature instead (something that I might do a post on soon)!

Summary

In this post I took an introductory look at the Microsoft.Extensions.Options.Contextual package. This package builds on the IOptions abstractions common to .NET. I show the basics of how to use the package, focusing on the partial decoupling it provides between a "context" object and your options object. Ultimately, I'm not convinced that this "decoupling" is worth this extra complexity, so if you have any experience with the package yourself, I'd be interested in your experiences in the comments!

In this post I describe some of the significant restructuring to my source generator NuGet package NetEscapades.EnumGenerators which you can use to add fast methods for working with enums. I start by describing why the package exists and what you can use it for, then I describe what motivated the restructuring, and finally what the changes are and a call to action.

The tl;dr; is that there are now three different packages, and exactly which one is best for you depends on what you're trying to do. Check the section below or the project's README for details!

As an aside, I really want to give this package a stable 1.0.0 release shortly, as I think we've solved most of the corner cases that were bugging me. Which means if the new package structure doesn't work for you, now is the time to raise an issue, before it gets set in stone!

Why should you use an enum source generator?

NetEscapades.EnumGenerators was one of the first source generators I created using the incremental generator support introduced in .NET 6. I chose to create this package to work around an annoying characteristic of working with enums: some operations are surprisingly slow.

Note that while this has historically been true, this fact won't necessarily remain true forever. In fact, .NET 8+ provided a bunch of improvements to enum handling in the runtime.

As an example, let's say you have the following enum:

public enum Colour
{
    Red = 0,
    Blue = 1,
}

At some point, you want to print the name of a Color variable, so you create this helper method:

public void PrintColour(Colour colour)
{
    Console.WriteLine("You chose "+ colour.ToString()); // You chose Red
}

While this looks like it should be fast, it's really not. NetEscapades.EnumGenerators works by automatically generating an implementation that is fast. It generates a ToStringFast() method that looks something like this:

public static class ColourExtensions
{
    public string ToStringFast(this Colour colour)
        => colour switch
        {
            Colour.Red => nameof(Colour.Red),
            Colour.Blue => nameof(Colour.Blue),
            _ => colour.ToString(),
        }
    }
}

This simple switch statement checks for each of the known values of Colour and uses nameof to return the textual representation of the enum. If it's an unknown value, then it falls back to the built-in ToString() implementation to ensure correct handling of unknown values (for example this is valid C#: PrintColour((Colour)123)).

If we compare these two implementations using BenchmarkDotNet for a known colour, you can see how much faster ToStringFast() implementation is:

Even though recent versions of .NET are way faster, the overall pattern hasn't changed: .NET is way faster than .NET Framework, and the ToStringFast() implementation is way faster than the built-in ToString(). Obviously your mileage may vary and the results will depend on the specific enum you're using, but in general, using the source generator should give you a free performance boost.

That's the basics of the package, for more details see the project's README. In the next section I describe how adding some new features managed to break users, and what we did in response.

Adding new features by adding to the marker attribute dll

In version 1.0.0-beta19 of NetEscapades.EnumGenerators I introduced a bunch of new features that had been long standing requests:

• Support for disabling number parsing.
• Support for automatically calling ToLowerInvariant() or ToUpperInvariant() on the serialized enum.

There wasn't really a technical reason I took so long to add these features. The problem was that I didn't want to add dozens of different overloads of Enum.Parse() or ToString() to accommodate all the different possible options. Similarly, I didn't want to add these all as different (IMO, ugly) additional extension methods.

I solved this issue in what I thought was a neat way. When you referenced the NetEscapades.EnumGenerators package, your library references the NetEscapades.EnumGenerators.Attributes.dll file that is shipped in the package:

This is just one way to add "marker" attributes to a target application, and until recently it was the most reliable way. My epiphany was to realise that I could put other types in this dll too; including types that are part of the target app's "public" API.

So I added EnumParseOptions and SerializationOptions types to NetEscapades.EnumGenerators.Attributes.dll, and used these types to add "general, extensible" overloads for the generated Parse and ToString() methods, something like this:

public static partial class ColourExtensions
{
    // EnumParseOptions controls case sensitivity, number parsing, matching metadata attributes 
    public static Colour Parse(string? name, in EnumParseOptions options);

    // SerializationOptions controls case transforms and whether to use metadata attributes
    public static string ToString(Color valu, in SerializationOptions options)
}

This all seemed very neat. If we want to add more features, we can just extend the options objects, no need for new methods or anything. However, I inadvertently managed to break a bunch of users 🤦‍♂️

When new features break users…

Shortly after publishing the new version of the package, I received reports of users seeing the following error:

Error CS0012: The type 'EnumParseOptions' is defined in an assembly that is not referenced. You must add a reference to assembly 'NetEscapades.EnumGenerators.Attributes, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'. (184, 11)

The issue seemed pretty clear: the NetEscapades.EnumGenerators.Attributes.dll that contains the new options objects used in the API wasn't being referenced in the final application. As I said in my initial response:

I'm guessing that you're excluding assets where you reference the package, e.g. something like this?

<PackageReference Include="NetEscapades.EnumGenerators" Version="1.0.0-beta19" > PrivateAssets="all" />

or like this:

<PackageReference Include="NetEscapades.EnumGenerators" Version="1.0.0-beta19" ExcludeAssets="all" />

you can't do that, you need just a normal reference:

<PackageReference Include="NetEscapades.EnumGenerators" Version="1.0.0-beta19" />

But what I totally overlooked is that many people intentionally add references to source generators using this pattern:

For all our code generators, we list them in the Directory.Build.props at the solution level as such:

<PackageReference Include="NetEscapades.EnumGenerators"
                  ExcludeAssets="runtime"
                  PrivateAssets="all"
                  TreatAsUsed="true"/>

We do this so that every project has access to code generation, yet doesn't pass on the generator assemblies themselves as transitive dependencies. Our expectation is such that when we distribute our packages, we don't impose upon our consumers any unnecessary dependencies through transitive references (i.e. available in everything, pass it onto nothing).

Unfortunately for me, that makes total sense 😅

For some reason, I thought that source generators (and analyzers) didn't flow transitively to downstream projects. But that's not true, they do flow transitively. To make that clearer, imagine you have two projects, MyProject.Lib and MyProject.Web. MyProject.Web has a reference to MyProject.Lib, and MyProject.Lib adds a reference to NetEscapades.EnumGenerators. By default, MyProject.Web also gets a transitive reference to NetEscapades.EnumGenerators.

     [MyProject.Lib]          ←              [MyProject.Web]
            ↓                                      (↓)  
[NetEscapades.EnumGenerators]           [NetEscapades.EnumGenerators]

This is how "normal" project/package references work, but for some reason I thought source generators/analyzers were special. But it turns out no.

So why does this matter? Well it means that adding PrivateAssets="all" and ExcludeAssets="runtime" actually makes sense in a lot of cases, particularly if you're creating reusable libraries. If you're using a generator to generate code in a single package, and you don't want to force downstream consumers of your package to automatically be opted in to source generator, then these attributes can help, though they serve slightly different reasons:

• PrivateAssets="all" ensures that any projects referencing this project don't get the dependency as a transitive dependency. In this example above, if MyProject.Lib set PrivateAssets="all", then MyProject.Web wouldn't get a transitive reference to NetEscapades.EnumGenerators.
• ExcludeAssets="runtime" ensures that any runtime assets included in the package are not copied to the build output. In our case, that means the NetEscapades.EnumGenerators.Attributes.dll would not be copied to the build output.

So on that basis, it's clear that adding ExcludeAssets="runtime" was causing the missing reference error at run time. Previously the attribute only contained enum attributes, and those were elided by default anyway, so the "runtime dependencies" were actually just a benign spandrel that could be excluded without issue. But when I added the EnumParseOptions and SerializationOptions, suddenly there was a hard dependency and things went boom 💥

The solution: more packages

Now, the "easy" fix would be to just tell users they're holding it wrong, and to just not use ExcludeAssets="runtime" but there were some very reasonable explanations that users had for not wanting to do this.

One of the most compelling reasons was that the NetEscapades.EnumGenerators source generator is entirely an "implementation detail". Forcing additional downstream runtime dependencies on consumers just to be "allowed" to use the source generator doesn't feel right. Especially if you're not even using the EnumParseOptions or SerializationOptions overloads!

After some various discussion and back and forth, we settled on two key changes:

• Move the "runtime dependencies" assembly (which currently only contains EnumParseOptions and SerializationOptions) into a separate package, NetEscapades.EnumGenerators.RuntimeDependencies.
• If the EnumParseOptions types are available in the compilation (because you added a reference to the runtime dependencies package) then the source generator uses the EnumParseOptions and SerializationOptions types in its public API. However, if they're not available, it emits enum-specific versions of these types instead, which avoids the need to add the runtime dependencies package. More on this shortly!

In addition, to make onboarding easier, the actual source generator was split into a completely separate package, NetEscapades.EnumGenerators.Generator, and _NetEscapades.EnumGenerators becomes a metapackage instead:

NetEscapades.EnumGenerators
  |____NetEscapades.EnumGenerators.Generators
  |____NetEscapades.EnumGenerators.RuntimeDependencies

As discussed, these packages provide the following:

• NetEscapades.EnumGenerators is a meta package for easy install.
• NetEscapades.EnumGenerators.Generators contains the source generator itself.
• NetEscapades.EnumGenerators.RuntimeDependencies contains dependencies that need to be referenced at runtime by the generated code.

The default approach is to reference the meta-package in your project. The runtime dependencies and generator packages will then flow transitively to any project that references yours, and the generator will run in those projects by default. But by splitting these packages up, we get more flexibility.

Avoiding runtime dependencies

As already discussed, in some cases you may not want these dependencies to flow to other projects, such as when you're using NetEscapades.EnumGenerators internally in your own library. In this scenario, you can take the following approach:

• Reference NetEscapades.EnumGenerators.Generators directly, and set PrivateAssets=All (and optionally ExcludeAssets="runtime")
• Optionally reference NetEscapades.EnumGenerators.RuntimeDependencies directly.

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
  </PropertyGroup>

  <!-- 👇 Add the generator package with PrivateAssets -->
  <PackageReference Include="NetEscapades.EnumGenerators.Generators" Version="1.0.0-beta21" PrivateAssets="All"/>

  <!-- Optionally add the runtime dependencies package -->
  <PackageReference Include="NetEscapades.EnumGenerators.RuntimeDependencies" Version="1.0.0-beta21" />
</Project>

The NetEscapades.EnumGenerators.RuntimeDependencies package is a "normal" dependency that contains the EnumParseOptions, SerializationOptions, and SerializationTransform types:

namespace NetEscapades.EnumGenerators;

/// <summary>
/// Defines the options use when parsing enums using members provided by NetEscapades.EnumGenerator.
/// </summary>
public readonly struct EnumParseOptions { }

/// <summary>
/// Options to apply when calling <c>ToStringFast</c> on an enum.
/// </summary>
public readonly struct SerializationOptions

/// <summary>
/// Transform to apply when calling <c>ToStringFast</c>
/// </summary>
public enum SerializationTransform

However, if you don't add a reference to the NetEscapades.EnumGenerators.RuntimeDependencies package, the source generator creates "nested" versions of the dependencies in each generated extension method instead of using the "global" versions:

namespace SomeNameSpace;

public static partial class MyEnumExtensions
{
    // ... generated members

    // The runtime dependencies are generated as nested types instead
    public readonly struct EnumParseOptions { }
    public readonly struct SerializationOptions
    public enum SerializationTransform
}

Generating the runtime dependencies as nested types like this has both upsides and downsides:

• It avoids placing downstream dependency requirements on consumers of your library.
• You can still use these additional overloads internally, even without having additional runtime dependencies
• It makes consuming the APIs that use the runtime dependencies more verbose.

To make that last point concrete, if you add a reference to the RuntimeDependencies package, your code might look something like this:

// parsing
string serialized = "Red"
Color value = ColourExtensions.Parse(
    serialized, new EnumParseOptions(enableNumberParsing: false));

// serialization
var result = value.ToStringFast(
    new SerializationOptions(transform: SerializationTransform.LowerInvariant));

In contrast, if you omit the runtime dependencies, you have to use the full (nested) type names:

// parsing
string serialized = "Red"
Color value = ColourExtensions.Parse(
    serialized, new ColourExtensions.EnumParseOptions(enableNumberParsing: false));

// serialization
var result = value.ToStringFast(
    new ColourExtensions.SerializationOptions(transform: ColourExtensions.SerializationTransform.LowerInvariant));

which, you know, is pretty ugly. But it does avoid those runtime dependencies, and solves the users problems, so it's what is available today!

Choosing the correct packages for your scenario

So where does that leave us?

In general, for simplicity, if you're creating an app of some sort I recommend just using a "normal" package reference to NetEscapades.EnumGenerators (and thereby implicitly using NetEscapades.EnumGenerators.RuntimeDependencies). This particularly makes sense when you are the primary consumer of the extension methods, or where you don't mind if consumers end up referencing the generator package:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
  </PropertyGroup>

  <PackageReference Include="NetEscapades.EnumGenerators" Version="1.0.0-beta21" />
</Project>

This "default" scenario also gets the best experience in terms of the optional "usage analyzers", which flag places that you should consider calling ToStringFast().

In contrast, if you are producing a reusable library and don't want any runtime dependencies to be exposed to consumers, I recommend using NetEscapades.EnumGenerators.Generators and setting PrivateAssets=All and ExcludeAssets="runtime".

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
  </PropertyGroup>

  <PackageReference Include="NetEscapades.EnumGenerators.Generators" Version="1.0.0-beta21" PrivateAssets="All" ExcludeAssets="runtime" />
</Project>

The final option is to reference NetEscapades.EnumGenerators.Generators and set PrivateAssets=All and ExcludeAssets="runtime" (to avoid it being referenced transitively), but then also reference NetEscapades.EnumGenerators.RuntimeDependencies, to produce easier-to consume APIs.

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
  </PropertyGroup>

  <PackageReference Include="NetEscapades.EnumGenerators.Generators" Version="1.0.0-beta21" PrivateAssets="All" ExcludeAssets="runtime"/>
  <PackageReference Include="NetEscapades.EnumGenerators.RuntimeDependencies" Version="1.0.0-beta21" />
</Project>

⚠️ When using the NetEscapades.EnumGenerators metapackage, it's important you don't set PrivateAssets=All. If you want to use PrivateAssets=All or ExcludeAssets="runtime" use NetEscapades.EnumGenerators.Generators for this scenario.

All of which means the package now supports many different scenarios, though with the associated complexity. All of which leads me to the final part of this post: how close are we to a stable release?

Open questions before a stable release

I've been very hesitant to put out a stable 1.0.0 release for the package, mostly because I don't really intend there to be a 2.0.0 unless there's a very good reason. The reason being is that I want consumers of the package to not have to worry about things breaking underneath them.

I have a separate rant about how I feel many NuGet library authors have misunderstood the increased speed of the .NET ecosystem in recent years as a license to churn out major versions with endless breaking changes. But that's for a completely different post😅

In terms of features, I'm pretty happy with where the generator is today compared to even a few months ago:

• Optional automatic support for System.Memory if it's available.
• Optional usage analyzers to encourage using the extension methods where possible.
• The extensible EnumParseOptions and SerializationOptions discussed in this post.
• Catering to "internal" usages of the generator (where you don't want downstream consumers to be aware you're using it).
• Support for enums defined in other libraries (including BCL enums)

Nevertheless, there are a few things I'm not entirely happy with. For example:

• The usage analyzers are IMO pretty cool, should they just be enabled as warnings by default?
  • Presumably, if you're adding this package, you want to preferentially use them, which would suggest yes, enable them.
  • But on the other hand, that would potentially be a big breaking change and annoy a bunch of people that just have a targeted usage
  • However they could just disable them in that case.
  • Unfortunately, you need to "preserve attribute usages" if you want the analyzers to work in "downstream" projects and ensure you don't exclude runtime assets, but that's maybe just a limitation we'll have to live with s🤷‍♂️
• The nested runtime dependency types are pretty ugly
  • Should they just be omitted/made private implementation details entirely?
  • My concern is that while they're a neat solution to a problem, maybe they should just be omitted (or made private, depending on complexity) to reduce confusion. The onboarding story becomes easier in this scenario: "add the NetEscapades.EnumGenerators.RuntimeDependencies package and get these extra features".
• You can't use "target-typed" new on the EnumParseOptions and SerializationOptions overloads due to ambiguity with existing methods (shown below)
  • This is really annoying, but I think the only way to "solve" it is to remove the overloads that contain the same number of parameters, which is again, pretty disruptive, and those methods are there for a reason (convenience)!
  • The biggest concern is EnumExtensions.Parse(string name, bool ignoreCase), as that bool is more convenient than using EnumExtensions.Parse(string name, EnumParseOptions options) if you just want to ignore case.
  • But maybe it's worth seriously entertaining making that break sooner rather than later.

Anyway, those are my current thoughts with the library. I need to figure out the answer to these questions before pushing out the 1.0.0, but I'd love for feedback from any users of the package that would be impacted by any of the changes above. Let me know your thoughts here or in the issue on Github. And hopefully, hopefully, we can get a stable version soon 😅

In the mean time, do try out the latest version, 1.0.0-beta21, read the package notes, and let me know about any issues or thoughts. There's extra goodness in there I haven't talked about in this post, so give it a try!

Summary

In this post I described the recent architectural to the NetEscapades.EnumGenerators package (which is now a metapackage) to support more scenarios. If you're a "standard" user of the package, there's nothing to worry about, you can simply add a reference to NetEscapades.EnumGenerators and everything should work smoothly.

However, if you want to control how the package flows transitively to consumers of your project then you may want to reference NetEscapades.EnumGenerators.Generators directly, and optionally add the NetEscapades.EnumGenerators.RuntimeDependencies package so you can use cleaner APIs.

I'd love any feedback you have about these recent changes; whether they're too confusing, if you can't work out which path you should take, or if you have any thoughts about the specific points I raised above. Just drop a comment here or in the issue on Github.

So far in this series I've described how to create and consume metrics using dotnet-counters, how to create each of the different Instrument types exposed by the System.Diagnostics.Metrics APIs, and how to use a source generator to produce values. In this post, I look at how to consume the stream of values produced by Instrument implementations in-process, using the MeterListener type.

I start by describing the scenario of an app that wants to record and process a specific subset of metrics exposed via the System.Diagnostics.Metrics APIs. We'll create a simple app that generates some load, use MeterListener to listen for Instrument measurements, and display the results in a table using Spectre.Console (because everyone loves Spectre.Console)!

Note that I'm not suggesting you use MeterListener directly in your production apps. In production, you'll likely want to use a solution like OpenTelemetry or Datadog that does all this work for you!

Creating the test ASP.NET Core app

As described above, for the purposes of this post, I created a simple "hello world" ASP.NET Core app using dotnet new web, and tweaked it so that it will send requests to itself, as long as the app is running:

using Microsoft.AspNetCore.Hosting.Server;
using Microsoft.AspNetCore.Hosting.Server.Features;

// Very basic hello-world app
var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

app.MapGet("/", () => "Hello World!");

var task = app.RunAsync();

// Grab the address Kestrel's listening on
var address = app.Services.GetRequiredService<IServer>()!
        .Features.Get<IServerAddressesFeature>()!
        .Addresses.First();

try
{
    // Run 4 loops in parallel, sending HTTP requests continuously
    // until the app gets the shut down notification
    await Parallel.ForAsync(0, 4, app.Lifetime.ApplicationStopping, async (i, ct) =>
    {
        var httpClient = new HttpClient()
        {
            BaseAddress = new Uri(address),
        };

        // Just keep hammering requests!
        while (!ct.IsCancellationRequested)
        {
            string _ = await httpClient.GetStringAsync("/");
        }
    });
}
catch (OperationCanceledException)
{
    // expected on shutdown
}

// Wait for the final cleanup
await task;

The code above isn't particularly pretty, but it does the following:

• Creates a "hello world" minimal API ASP.NET Core app.
• After the app starts up, it starts 4 parallel jobs
• Each job has its own HttpClient and continuously makes HTTP requests to the app
• ctrl+c in the console stops the requests and shut's down the app.

Now that we have this app, we can start grabbing some metrics out of it. We're aiming for something like the following, which shows the majority of metrics from my previous post in a live-updating Spectre.Console table:

                                  ASP.NET Core Metrics                                  
┌────────────────────────────────────────────┬─────────────────────────┬─────────────┐
│ Metric                                     │ Type                    │       Value │
├────────────────────────────────────────────┼─────────────────────────┼─────────────┤
│ aspnetcore.routing.match_attempts          │ Counter                 │     250,428 │
│ dotnet.gc.heap.total_allocated             │ ObservableCounter       │ 849,743,376 │
│ http.server.active_requests                │ UpDownCounter           │           4 │
│ dotnet.gc.last_collection.heap.size (gen0) │ ObservableUpDownCounter │   2,497,080 │
│ dotnet.gc.last_collection.heap.size (gen1) │ ObservableUpDownCounter │     774,872 │
│ dotnet.gc.last_collection.heap.size (gen2) │ ObservableUpDownCounter │   1,219,120 │
│ dotnet.gc.last_collection.heap.size (loh)  │ ObservableUpDownCounter │      98,384 │
│ dotnet.gc.last_collection.heap.size (poh)  │ ObservableUpDownCounter │      65,728 │
│ process.cpu.utilization                    │ ObservableGauge         │         36% │
│ http.server.request.duration               │ Histogram               │     0.011ms │
│ http.server.request.duration (count)       │ Histogram               │     250,425 │
└────────────────────────────────────────────┴─────────────────────────┴─────────────┘

To record the values from these metrics, we're going to use the MeterListener type.

Recording metrics with MeterListener

In my previous post I discussed how Instruments have both a consumer and a producer side. To consume the output of Instruments inside your app you must subscribe to them using a MeterListener. To manage all this configuration, we'll create a helper type called MetricManager.

Creating a wrapper MetricManager for working with metrics

To encapsulate the collection and aggregation of metrics emitted by the System.Diagnostics.Metrics APIs, I'm going to create a type called MetricManager. This is entirely optional, it's just helpful for my scenario. The public API for this type is shown below, which we'll be fleshing out shortly.

public class MetricManager : IDisposable
{
    public void Dispose();
    public MetricValues GetMetrics();
}

The MetricManager is responsible for interacting with the System.Diagnostics.Metrics APIs. And when you call GetMetrics(), the manager returns the values for each of the Instruments we listed above:

public readonly record struct MetricValues(
    long TotalMatchAttempts,
    long TotalHeapAllocated,
    long ActiveRequests,
    long HeapSizeGen0,
    long HeapSizeGen1,
    long HeapSizeGen2,
    long HeapSizeLoh,
    long HeapSizePoh,
    double CpuUtilization,
    double AverageDuration,
    long TotalRequests);

Just to reiterate, this is not required. It's just how I've chosen in this post to expose the interactions with the System.Diagnostics.Metrics APIs.

Note also that I'm creating a very well-defined API here. If you want to have more of a "generalised" listener, that can listen to all metrics, and records all the tags for those metrics, I strongly recommend looking at OpenTelemetry instead!

So we have our basic public API, now let's create a MeterListener and hook it up.

Creating a MeterListener and configuring callbacks

One of the design tenants of the System.Diagnostics.Metrics APIs is that they should be high performance. Commonly for .NET, that mostly means "you don't need additional allocations". That shows up in some of the design of the MeterListener as you'll see shortly.

The code below shows how we would extend MetricManager to create a MeterListener, initialize it, and configure callbacks:

public class MetricManager : IDisposable
{
    private readonly MeterListener _listener;

    public MetricManager()
    {
        // Create a MeterListener, and configure the method to call
        // when a new instrument is published in the application
        _listener = new()
        {
            InstrumentPublished = OnInstrumentPublished
        };

        // Configure the callbacks to invoke when an Instrument emits a value.
        // In this case, we know that the .NET runtime instruments we listen to only
        // produce long or double values, so that's all we listen for here
        _listener.SetMeasurementEventCallback<long>(OnMeasurementRecordedLong);
        _listener.SetMeasurementEventCallback<double>(OnMeasurementRecordedDouble);

        // Start the listener, which invokes OnInstrumentPublished for already-published Instruments
        _listener.Start();
    }

    // Call Dispose on the listener to prevent further callbacks being invoked
    public void Dispose() => _listener.Dispose();

    // Callback invoked whenever an instrument is published
    private void OnInstrumentPublished(Instrument instrument, MeterListener listener)
    {
        // ...
    }

    // Callback invoked whenever a `long` measurement is recorded
    private static void OnMeasurementRecordedLong(Instrument instrument, long measurement,
        ReadOnlySpan<KeyValuePair<string, object?>> tags, object? state)
    {
        // ...
    }

    // Callback invoked whenever a `double` measurement is recorded
    private static void OnMeasurementRecordedDouble(Instrument instrument, double measurement,
        ReadOnlySpan<KeyValuePair<string, object?>> tags, object? state)
    {
        // ...
    }
}

I've heavily commented the code above, but I'll highlight some interesting points.

Firstly, the OnInstrument callback allows the listener to choose which Meters and Instruments it wants to subscribe to. This callback is invoked once for each existing Instrument when you call MeterListener.Start(), and is then subsequently invoked whenever a new Meter or Instrument is subsequently registered.

In addition, we have the SetMeasurementEventCallback<T>() method. This is a generic method, because it allows you to register a different callback for each type of Instrument measurement you might receive. Instruments can be created with byte, short, int, long, float, double, and decimal types, so it's recommended that you register a callback for each of these types.

Note that if you use a generic argument that isn't one of these types, you'll get an exception at runtime.

This kind of API might seem a little unusual; having to register virtually identical callbacks for each different type feels a bit clumsy. But it's written this way for performance reasons. By having a dedicated callback for each supported T, you can avoid any allocation or overhead that would come from having a "generic" callback that would only work with object.

Also note that the callback you register doesn't have to be different methods like I have used above. You could also have a single generic method with a signature like this:

static void OnMeasurementRecorded<T>(
    Instrument instrument,
    T measurement,
    ReadOnlySpan<KeyValuePair<string, object?>> tags,
    object? state);

However, you would still need to call SetMeasurementEventCallback<T> once for each measurement type you want to handle, for example:

_listener.SetMeasurementEventCallback<long>(OnMeasurementRecorded);
_listener.SetMeasurementEventCallback<double>(OnMeasurementRecorded);

We are yet to implement these measurement callbacks, but before we get to that, we'll take a look at the OnInstrumentPublished() callback.

Selecting which Instruments to listen to

The MeterListener is "connected" to all of the Meters and Instruments in the application, but it won't automatically receive measurements from all of them unless you enable each one. For this demo, we only care about a subset of Meters and Instruments, so our OnInstrumentPublished() callback uses a switch expression to check for specific values of Instrument.Name and Meter.Name:

private void OnInstrumentPublished(Instrument instrument, MeterListener listener)
{
    string meterName = instrument.Meter.Name;
    string instrumentName = instrument.Name;

    // Is this a Meter and Instrument we care about?
    var enable = meterName switch
    {
        "Microsoft.AspNetCore.Routing" => instrumentName == "aspnetcore.routing.match_attempts",
        "System.Runtime"               => instrumentName is "dotnet.gc.heap.total_allocated"
                                                            or "dotnet.gc.last_collection.heap.size",
        "Microsoft.AspNetCore.Hosting" => instrumentName is "http.server.active_requests"
                                                            or "http.server.request.duration",
        "Microsoft.Extensions.Diagnostics.ResourceMonitoring" => instrumentName == "process.cpu.utilization",
        _ => false
    };

    if (enable)
    {
        // If yes, enable measurements, and pass the `MetricManager` as "state"
        listener.EnableMeasurementEvents(instrument, state: this);
    }
}

To enable measurements, you call MeterListener.EnableMeasurementEvents(), passing in the Instrument to listen to. One interesting point here is that we're also passing the MetricManager as the state variable. This variable is passed in to our OnMeasurementRecorded callbacks and is a way of avoiding closures or expensive lookups in the callback events. You'll see how it's used shortly.

Note that if we were creating a generic implementation that listened to all Insturments emitted by the app, we could implement this method very simply:

private void OnInstrumentPublished(Instrument instrument, MeterListener listener)
    => listener.EnableMeasurementEvents(instrument, state: this);

So at this point we've enabled the instruments, we've called MeterListener.Start(), and it's time to start receiving some measurements!

Triggering ObservableInstruments to emit measurements

Now that we've subscribed to the instruments, the OnMeasurementRecorded callbacks are invoked whenever an Instrument emits a value. For "standard" Instruments, that happens immediately, whenever a value is recorded: add a value to a Counter<long>, and our OnMeasurementRecorded callback is immediately called. But that's not how it works for observable instruments.

In my previous post, I described how observable instruments don't emit any values until the consumer asks them to. Well, the consumer here is MeterListener, and it needs to ask all the Instruments it is interested in to emit values when GetMetrics() is called:

public MetricValues GetMetrics()
{
    // This triggers the observable metrics to go and read the values and
    // then invoke the OnMeasurementRecorded callback to send the values to us
    _listener.RecordObservableInstruments();

    // ...
}

Calling RecordObservableInstruments() triggers all the observable instruments that we enabled to emit a measurement (by invoking their associated callbacks, such as those described in my previous post). These measurements are then reported via the callbacks registered with the MeterListener.

Our MeterListener is now completely configured, so it's time to flesh out the OnMeasurementRecorded callbacks.

Recording Instrument measurements

Whenever a measurement is recorded by an Instrument, the registered callback of the appropriate type is invoked (if you haven't registered an appropriate callback, none will be invoked). Exactly what you should do with that metric depends on how you want to aggregate your data.

The following implementation of the OnMeasurementRecordedLong method shows one way to aggregate the data, focusing on displaying long running totals for the duration of the app:

private static void OnMeasurementRecordedLong(Instrument instrument, long measurement,
    ReadOnlySpan<KeyValuePair<string, object?>> tags, object? state)
{
    var handler = (MetricManager)state!;
    switch (instrument.Name)
    {
        // Counter
        case "aspnetcore.routing.match_attempts":
            Interlocked.Add(ref handler._matchAttempts, measurement);
            break;

        // ObservableCounter
        case "dotnet.gc.heap.total_allocated":
            Interlocked.Exchange(ref handler._totalHeapAllocated, measurement);
            break;

        // UpDownCounter
        case "http.server.active_requests":
            Interlocked.Add(ref handler._activeRequests, measurement);
            break;

        // ObservableUpDownCounter
        case "dotnet.gc.last_collection.heap.size":
            foreach (var tag in tags)
            {
                if (tag is { Key: "gc.heap.generation", Value: string gen })
                {
                    switch (gen)
                    {
                        case "gen0": Interlocked.Exchange(ref handler._heapSizeGen0, measurement); break;
                        case "gen1": Interlocked.Exchange(ref handler._heapSizeGen1, measurement); break;
                        case "gen2": Interlocked.Exchange(ref handler._heapSizeGen2, measurement); break;
                        case "loh": Interlocked.Exchange(ref handler._heapSizeLoh, measurement); break;
                        case "poh": Interlocked.Exchange(ref handler._heapSizePoh, measurement); break;
                    }
                }
            }

            break;
    }
}

The first step is to cast the state object back to the MetricManager instance that we passed in when calling EnableMeasurementEvents(). We then switch based on the instrument name, and handle the measurement value differently depending on the instrument type:

• For Counter and UpDownCounter, the callback is invoked once for every time a new value is recorded, with the measurement value as the increment. To create a running total of values, you must add the new measurement to the current running total.
• For ObservableCounter and ObservableUpDownCounter, the callback is only invoked when you call RecordObservableInstruments(). The measurement value in these cases aren't incremental values, but rather they're the "final" current value, so you can use the value "as is" for the current running total.

You can see these rules applied in the above method, where the Counter and UpDownCounter metrics are aggregated using Interlocked.Add(), whereas the ObservableCounter and ObservableUpDownCounter metrics are "aggregated" by using Interlocked.Exchange.

Another interesting aspect is the handling of tags. The "dotnet.gc.last_collection.heap.size" is an ObservableUpDownCounter, so the values are emitted only when you call RecordObservableInstruments(). In this case, we receive one invocation of the callback per generation, with the gc.heap.generation tag indicating to which generation the current measurement applies.

In addition to the OnMeasurementRecordedLong callback, we also have the OnMeasurementRecordedDouble callback, which we use to record the ObservableGauge and Histogram metrics:

private static void OnMeasurementRecordedDouble(Instrument instrument, double measurement,
    ReadOnlySpan<KeyValuePair<string, object?>> tags, object? state)
{
    var handler = (MetricManager)state!;
    switch (instrument.Name)
    {
        // ObservableGauge
        case "process.cpu.utilization":
            Interlocked.Exchange(ref handler._cpuUtilization, measurement);
            break;

        // Histogram
        case "http.server.request.duration":
            Interlocked.Increment(ref handler._totalRequestCount);
            lock (handler._durationLock)
            {
                handler._intervalRequests++;
                handler._totalDuration += measurement;
            }

            break;
    }
}

The structure for this callback is very similar to the previous one:

• We cast the state variable to our MetricManager instance that we passed in when registering the callback.
• For the ObservableGauge (as for all of the observable instruments), we replace our recorded value, using Interlocked.Exchange()
• For the Histogram, there are many different ways we could aggregate the data, especially considering that these measurements contain a lot of high cardinality tags. I chose to calculate just two values from this data:
  • The total number of requests since app start, stored in _totalRequestCount.
  • The average request duration in the current time interval. This requires recording the number of requests (_intervalRequests) during the interval, and the sum of the durations of requests during the interval (_totalDuration). We'll use these values to calculate the average shortly.

Some of these measurements may be recorded concurrently with when while we are reading the values, which is why I've used Interlocked where possible, to make updates atomic. Where I couldn't use Interlocked, I used a lock for simplicity, though you should be careful about this in practice; in high performance applications it might be possible to run into lock contention, if many requests are trying to increment these values.

Now that all of our Instruments are recording values, both standard and observable, it's time to report the results.

Reporting the results from GetMetrics

I have already partially shown the GetMetrics() implementation, in so far as it's where we called RecordObservableInstruments(). Other than triggering the observable measurements to be taken, all GetMetrics() does is read the values recorded in the fields, calculate the average duration, and return a MetricValues instance:

public MetricValues GetMetrics()
{
    // This triggers the observable metrics to go and read the values
    // and then call the OnMeasurement callbacks to send the values to us
    _listener.RecordObservableInstruments();

    // Read all of the values from the fields and return a MetricValues object
    return new MetricValues(
        TotalMatchAttempts: Interlocked.Read(ref _matchAttempts),
        TotalHeapAllocated: Interlocked.Read(ref _totalHeapAllocated),
        ActiveRequests: Interlocked.Read(ref _activeRequests),
        HeapSizeGen0: Interlocked.Read(ref _heapSizeGen0),
        HeapSizeGen1: Interlocked.Read(ref _heapSizeGen1),
        HeapSizeGen2: Interlocked.Read(ref _heapSizeGen2),
        HeapSizeLoh: Interlocked.Read(ref _heapSizeLoh),
        HeapSizePoh: Interlocked.Read(ref _heapSizePoh),
        CpuUtilization: Volatile.Read(ref _cpuUtilization),
        AverageDuration: ComputeAndResetAverageDuration(),
        TotalRequests: Interlocked.Read(ref _totalRequestCount)
    );

    double ComputeAndResetAverageDuration()
    {
        long count;
        double sum;
        lock (_durationLock)
        {
            // Grab the current values
            count = _intervalRequests;
            sum = _totalDuration;
            // Reset the values
            _intervalRequests = 0;
            _totalDuration = 0;
        }

        // Do the calculation
        return count > 0 ? sum / count : 0;
    }
}

And with that, the implementation of MetricManager and its usage of MeterListener is complete. All that remains is to plug the listener into our app.

Creating a service to display the results

To view the metrics being collected by MetricManager and its MeterListener, I created a BackgroundService that would render a Spectre.Console live table to the console, and update it periodically:

using MyMetrics;
using Spectre.Console;

internal class MetricDisplayService : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        using var manager = new MetricManager();
        
        var table = new Table()
            .Title("[bold]ASP.NET Core Metrics[/]")
            .Border(TableBorder.Rounded)
            .AddColumn("Metric")
            .AddColumn("Type")
            .AddColumn(new TableColumn("Value").RightAligned());

        table.AddRow("aspnetcore.routing.match_attempts", "Counter", "0");
        table.AddRow("dotnet.gc.heap.total_allocated", "ObservableCounter", "0");
        table.AddRow("http.server.active_requests", "UpDownCounter", "0");
        table.AddRow("dotnet.gc.last_collection.heap.size (gen0)", "ObservableUpDownCounter", "0");
        table.AddRow("dotnet.gc.last_collection.heap.size (gen1)", "ObservableUpDownCounter", "0");
        table.AddRow("dotnet.gc.last_collection.heap.size (gen2)", "ObservableUpDownCounter", "0");
        table.AddRow("dotnet.gc.last_collection.heap.size (loh)", "ObservableUpDownCounter", "0");
        table.AddRow("dotnet.gc.last_collection.heap.size (poh)", "ObservableUpDownCounter", "0");
        table.AddRow("process.cpu.utilization", "ObservableGauge", "0%");
        table.AddRow("http.server.request.duration", "Histogram", "0.000ms");
        table.AddRow("http.server.request.duration (count)", "Histogram", "0");

        await AnsiConsole.Live(table).StartAsync(async ctx =>
        {
            // This is the update loop, where we poll the `MetricManager`
            while (!stoppingToken.IsCancellationRequested)
            {
                await Task.Delay(TimeSpan.FromSeconds(1), stoppingToken);
                RenderMetricValues(table, ctx, manager.GetMetrics());
            }
        });
    }

    private void RenderMetricValues(Table table, LiveDisplayContext ctx, in MetricManager.MetricValues values)
    {
        table.UpdateCell(0, 2, values.TotalMatchAttempts.ToString("N0"));
        table.UpdateCell(1, 2, values.TotalHeapAllocated.ToString("N0"));
        table.UpdateCell(2, 2, values.ActiveRequests.ToString("N0"));
        table.UpdateCell(3, 2, values.HeapSizeGen0.ToString("N0"));
        table.UpdateCell(4, 2, values.HeapSizeGen1.ToString("N0"));
        table.UpdateCell(5, 2, values.HeapSizeGen2.ToString("N0"));
        table.UpdateCell(6, 2, values.HeapSizeLoh.ToString("N0"));
        table.UpdateCell(7, 2, values.HeapSizePoh.ToString("N0"));
        table.UpdateCell(8, 2, $"{values.CpuUtilization:F0}%");
        table.UpdateCell(9, 2, $"{values.AverageDuration * 1000:F3}ms");
        table.UpdateCell(10, 2, values.TotalRequests.ToString("N0"));
        ctx.Refresh();
    }
}

Most of this code is simply setting up the table, the "important" part in terms of the interaction with the MetricManager all takes place in the AnsiConsole.Live block:

// As long as the app keeps running...
while (!stoppingToken.IsCancellationRequested)
{
    // ...wait 1 second...
    await Task.Delay(TimeSpan.FromSeconds(1), stoppingToken);
    // ...and then grab the metrics, and render them
    RenderMetricValues(table, ctx, manager.GetMetrics());
}

All that remains is to plug our background service into our app:

using Microsoft.AspNetCore.Hosting.Server;
using Microsoft.AspNetCore.Hosting.Server.Features;

var builder = WebApplication.CreateBuilder(args);

// Register the MetricDisplayService as an `IHostedService`
builder.Services.AddHostedService<MetricDisplayService>();

// Add the ResourceMonitoring package so that we can retrieve "process.cpu.utilization"
builder.Services.AddResourceMonitoring();
var app = builder.Build();

app.MapGet("/", () => "Hello World!");

app.Run();

and that's it! If we run the app, and generate some load, we'll see our metrics being reported to the console 🎉

┌────────────────────────────────────────────┬─────────────────────────┬─────────────┐
│ Metric                                     │ Type                    │       Value │
├────────────────────────────────────────────┼─────────────────────────┼─────────────┤
│ aspnetcore.routing.match_attempts          │ Counter                 │     250,428 │
│ dotnet.gc.heap.total_allocated             │ ObservableCounter       │ 849,743,376 │
│ http.server.active_requests                │ UpDownCounter           │           4 │
│ dotnet.gc.last_collection.heap.size (gen0) │ ObservableUpDownCounter │   2,497,080 │
│ dotnet.gc.last_collection.heap.size (gen1) │ ObservableUpDownCounter │     774,872 │
│ dotnet.gc.last_collection.heap.size (gen2) │ ObservableUpDownCounter │   1,219,120 │
│ dotnet.gc.last_collection.heap.size (loh)  │ ObservableUpDownCounter │      98,384 │
│ dotnet.gc.last_collection.heap.size (poh)  │ ObservableUpDownCounter │      65,728 │
│ process.cpu.utilization                    │ ObservableGauge         │         36% │
│ http.server.request.duration               │ Histogram               │     0.011ms │
│ http.server.request.duration (count)       │ Histogram               │     250,425 │
└────────────────────────────────────────────┴─────────────────────────┴─────────────┘

And with that we reach the end. Our app is able to report metrics about itself, and report those in any way it sees fit. In this example we just blindly report them to the console, but you could do anything with them. That said, if you're thinking of doing anything serious with these metrics, you should likely consider using the OpenTelemetry libraries instead!

Summary

In this post I describe the scenario of an app that wants to record and process a specific subset of metrics exposed via the System.Diagnostics.Metrics APIs. I then show a simple app that generates some load, use MeterListener to listen for Instrument measurements, and display the results in a table using Spectre.Console. Along the way I show the difference between the standard Instrument and ObservableInstrument measurements, show how to trigger observable measurements to be reported, and discuss performance aspects, such as passing state to the callback functions.

In the first post in this series I provided an introduction to the System.Diagnostics.Metrics APIs introduced in .NET 6. I initially introduced the concept of "observable" Instruments in that post, but didn't go into more details. In this post, we'll understand what being "observable" means, and how these Instruments differ from non-observable Instruments.

I start the post with a quick refresher on the basics of the System.Diagnostics.Metrics APIs, such as the different types of instruments available. I then show how you can create each of the instrument types and produce values from them.

System.Diagnostics.Metrics APIs

The System.Diagnostics.Metrics APIs were introduced in .NET 6 but are available in earlier runtimes (including .NET Framework) by using the System.Diagnostics.DiagnosticSource NuGet package. There are two primary concepts exposed by these APIs: Instrument and Meter:

• Instrument: An instrument records the values for a single metric of interest. You might have separate Instruments for "products sold", "invoices created", "invoice total", or "GC heap size".
• Meter: A Meter is a logical grouping of multiple instruments. For example, the System.Runtime Meter contains multiple Instruments about the workings of the runtime, while the Microsoft.AspNetCore.Hosting Meter contains Instruments about the HTTP requests received by ASP.NET Core.

There are also (currently, as of .NET 10) 7 different types of Instrument:

• Counter<T>
• ObservableCounter<T>
• UpDownCounter<T>
• ObservableUpDownCounter<T>
• Gauge<T>
• ObservableGauge<T>
• Histogram<T>.

To create a custom metric, you need to choose the type of Instrument to use, and associate it with a Meter. I'll discuss the differences between each of these instruments shortly, but first we'll look at the difference between "observable" instruments, and "normal" instruments.

What is an Observable* instrument?

When using the System.Diagnostic.Metrics APIs there's a "producer" side and a "consumer" side. The producer of metrics is the app itself, recording values and details about how it's operating. The consumer could be an in-process consumer, such as the OpenTelemetry libraries, or it could be an external process, such as dotnet-counters or dotnet-monitor.

The differences between a "normal" instrument and an "observable" instrument stem from who controls when and how a value is emitted:

• For "normal" instruments, the producer emits values as they occur. For example, when a request is received, ASP.NET Core emits the http.server.active_requests metric, indicating a new request is in-flight.
• For "observable" instruments, the consumer side asks for the value. For example, the dotnet.gc.pause.time metric returns "The total amount of time paused in GC since the process has started", but only when you ask for it.

In general, observable instruments are used when you have an effectively continuous value that you wouldn't make sense for the consumer to actively emit, such as the dotnet.gc.pause.time above, or where emitting all of the intermediate values would be too expensive from a performance point of view.

Technically, you could potentially emit this metric every time the GC pauses, but given that these values are more fine-grained than you would likely want anyway, it's much more efficient to allow the consumer to "poll" the values on demand, and therefore it makes the most sense as an observable instrument.

Now we understand the difference between observable and normal instruments, let's walk through all the instrumentation types and see how they're used in the .NET base class libraries.

Understanding the different Instrument types

So far in this series we've used a simple Counter<T> that records every time a given event occurs. In this post we'll look at each of the possible Instruments in turn, showing how you create an instrument of that type to produce a given metric. Where possible, I'm showing places within the .NET or ASP.NET Core libraries that use each of these instruments, to give "real world" versions of how these are used.

Counter<T>

The Counter<T> instrument is one of the simplest instruments conceptually. It is used to record how many times a given event occurs.

For example, the aspnetcore.diagnostics.exceptions metric is a Counter<long> which records the "Number of exceptions caught by exception handling middleware."

_handlerExceptionCounter = _meter.CreateCounter<long>(
    "aspnetcore.diagnostics.exceptions",
    unit: "{exception}",
    description: "Number of exceptions caught by exception handling middleware.");

Every time the ExceptionHandlerMiddleware (or DeveloperExceptionHandlerMiddleware) catches an exception, it adds 1 to this counter, first constructing an appropriate set of tags, and then calling Add(1, tags):

 private void RequestExceptionCore(string exceptionName, ExceptionResult result, string? handler)
{
    var tags = new TagList();
    tags.Add("error.type", exceptionName);
    tags.Add("aspnetcore.diagnostics.exception.result", GetExceptionResult(result));
    if (handler != null)
    {
        tags.Add("aspnetcore.diagnostics.handler.type", handler);
    }
    _handlerExceptionCounter.Add(1, tags);
}

As this Counter<T> is tracking a number of occurrences, you're always adding positive values, never negative values, though you can increase by more than 1 at a time if needs be.

ObservableCounter<T>

The ObservableCounter<T> is conceptually similar to a Counter<T>, in that it records monotonically increasing values. Being an "observable" instrument, it only records the values when "observed" (we'll look at how to observe the instruments in your own code in a subsequent post).

For example, the dotnet.gc.heap.total_allocated metric is an ObservableCounter<long> which records the "The approximate number of bytes allocated on the managed GC heap since the process has started":

s_meter.CreateObservableCounter(
    "dotnet.gc.heap.total_allocated",
    () => GC.GetTotalAllocatedBytes(),
    unit: "By",
    description: "The approximate number of bytes allocated on the managed GC heap since the process has started. The returned value does not include any native allocations.");

When observed, the lambda included in the definition is called, which invokes GC.GetTotalAllocatedBytes(). Note that this value steadily increases during the lifetime of the app, so it's not returning the difference since last invocation, it's returning the current running total.

UpDownCounter<T>

The UpDownCounter<T> is similar to the Counter<T>, but it supports reporting positive or negative values.

For example, the http.server.active_requests metric is an UpDownCounter<T> that records the "Number of active HTTP server requests.":

_activeRequestsCounter = _meter.CreateUpDownCounter<long>(
    "http.server.active_requests",
    unit: "{request}",
    description: "Number of active HTTP server requests.");

When a request is started, the server calls Add() and increments the value of the counter:

public void RequestStart(string scheme, string method)
{
    // Tags must match request end.
    var tags = new TagList();
    InitializeRequestTags(ref tags, scheme, method);
    _activeRequestsCounter.Add(1, tags);
}

private static void InitializeRequestTags(ref TagList tags, string scheme, string method)
{
    tags.Add(HostingTelemetryHelpers.AttributeUrlScheme, scheme);
    tags.Add(HostingTelemetryHelpers.AttributeHttpRequestMethod, HostingTelemetryHelpers.GetNormalizedHttpMethod(method));
}

Similarly, when the request ends, the server calls Add() to decrement the value of the counter:

public void RequestEnd(string protocol, string scheme, string method, string? route, int statusCode, bool unhandledRequest, Exception? exception, List<KeyValuePair<string, object?>>? customTags, long startTimestamp, long currentTimestamp, bool disableHttpRequestDurationMetric)
{
    var tags = new TagList();
    InitializeRequestTags(ref tags, scheme, method);

    // Tags must match request start.
    if (_activeRequestsCounter.Enabled)
    {
        _activeRequestsCounter.Add(-1, tags);
    }

    // ...
}

Consequently, the UpDownCounter<T> receives a series of increment/decrement values representing the movement of the metric.

ObservableUpDownCounter<T>

The ObservableUpDownCounter<T> is similar to the UpDownCounter<T> in that it reports increasing or decreasing values of a metric. The difference is that it returns the absolute value of the metric when observed, as opposed to a stream of deltas.

For example, the dotnet.gc.last_collection.heap.size metric is an ObservableUpDownCounter<long> that reports "The managed GC heap size (including fragmentation), as observed during the latest garbage collection":

s_meter.CreateObservableUpDownCounter(
    "dotnet.gc.last_collection.heap.size",
    GetHeapSizes,
    unit: "By",
    description: "The managed GC heap size (including fragmentation), as observed during the latest garbage collection.");

When observed, the GetHeapSizes() method is invoked and returns a collection of Measurements, each tagged by the heap generation name:

private static readonly string[] s_genNames = ["gen0", "gen1", "gen2", "loh", "poh"];
private static readonly int s_maxGenerations = Math.Min(GC.GetGCMemoryInfo().GenerationInfo.Length, s_genNames.Length);

private static IEnumerable<Measurement<long>> GetHeapSizes()
{
    GCMemoryInfo gcInfo = GC.GetGCMemoryInfo();

    for (int i = 0; i < s_maxGenerations; ++i)
    {
        yield return new Measurement<long>(gcInfo.GenerationInfo[i].SizeAfterBytes, new KeyValuePair<string, object?>("gc.heap.generation", s_genNames[i]));
    }
}

This returns the size of each heap at the last GC collection, the value of which may obviously increase or decrease.

Gauge<T>

The Gauge<T> is used to record "non-additive" values whenever they occur. These values can go up and down, and be positive or negative, but the point is that they "overwrite" all previous values.

Interestingly, this Instrument type was only added in .NET 9, and I couldn't find a single case of Gauge<T> being used in the .NET runtime, ASP.NET Core, or the .NET extensions packages 😅 So I made one up: for example, consider a gauge that reports the current room temperature when it changes:

var instrument = _meter.CreateGauge<double>(
    name: "locations.room.temperature",
    unit: "°C",
    description: "Current room temperature"
);

Then when the temperature of the room changes, you would report the new value:

public void OnOfficeTemperatureChanged(double newTemperature)
{
    instrument.Record(newTemperature, new KeyValuePair<string, object?>("room", "office"));
}

The gauge values are record whenever the temperature changes.

ObservableGauge<T>

Conceptually the ObservableGauge<T> is the same as a Gauge<T>, except that it only produces a value when observed. ObservableGauge<T> was added way back in .NET 6, and there are some examples of its use in this case.

For example, the process.cpu.utilization metric is an ObservableGauge<double> instrument which reports "The CPU consumption of the running application in range [0, 1]".

_ = meter.CreateObservableGauge(
    name: "process.cpu.utilization",
    observeValue: CpuPercentage);

When observed, the CpuPercentage() method is invoked, which returns a single value for the CPU usage as a value between 0 and 1.

private double CpuPercentage()
{
    // see above link for implementation
}

This Instrument is exposed in the Microsoft.Extensions.Diagnostics.ResourceMonitoring meter, and implemented in the Microsoft.Extensions.Diagnostics.ResourceMonitoring NuGet package.

Histogram<T>

The final instrument type is Histogram<T>, which is used to report arbitrary values, that you will typically want to aggregate using statistics.

For example, the http.server.request.duration metric is a Histogram<double> which records the "Duration of HTTP server requests.". Durations and latencies are a classic example of where you might want to use a histogram, so that you can calculate the p50, p90, p99 etc latencies, or to record all the values and plot them as a graph.

_requestDuration = _meter.CreateHistogram<double>(
    "http.server.request.duration",
    unit: "s",
    description: "Duration of HTTP server requests.",
    advice: new InstrumentAdvice<double> { HistogramBucketBoundaries = MetricsConstants.ShortSecondsBucketBoundaries });

The example above also shows our first example of InstrumentAdvice<T>. This type provides suggested configuration settings for consumers, indicating the best settings to use when processing Instrument values. In this case, the advice provides a suggested set of histogram bucket boundaries: [0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1, 2.5, 5, 7.5, 10], which can be useful for consumers to know how best to plot the metric values.

The _requestDuration histogram instrument is called whenever an ASP.NET Core request ends, recording the duration of the request, and a large associated number of tags. I've reproduced all the code below for completeness (expanding tag constants for clarity) but it's basically just building up a collection of tags which are recorded along with the duration of the request.

public void RequestEnd(string protocol, string scheme, string method, string? route, int statusCode, bool unhandledRequest, Exception? exception, List<KeyValuePair<string, object?>>? customTags, long startTimestamp, long currentTimestamp, bool disableHttpRequestDurationMetric)
{
    var tags = new TagList();
    InitializeRequestTags(ref tags, scheme, method);

    if (!disableHttpRequestDurationMetric && _requestDuration.Enabled)
    {
        if (HostingTelemetryHelpers.TryGetHttpVersion(protocol, out var httpVersion))
        {
            tags.Add("network.protocol.version", httpVersion);
        }
        if (unhandledRequest)
        {
            tags.Add("aspnetcore.request.is_unhandled", true);
        }

        // Add information gathered during request.
        tags.Add("http.response.status_code", HostingTelemetryHelpers.GetBoxedStatusCode(statusCode));
        if (route != null)
        {
            tags.Add("http.route", RouteDiagnosticsHelpers.ResolveHttpRoute(route));
        }

        // Add before some built in tags so custom tags are prioritized when dealing with duplicates.
        if (customTags != null)
        {
            for (var i = 0; i < customTags.Count; i++)
            {
                tags.Add(customTags[i]);
            }
        }

        // This exception is only present if there is an unhandled exception.
        // An exception caught by ExceptionHandlerMiddleware and DeveloperExceptionMiddleware isn't thrown to here. Instead, those middleware add error.type to custom tags.
        if (exception != null)
        {
            // Exception tag could have been added by middleware. If an exception is later thrown in request pipeline
            // then we don't want to add a duplicate tag here because that breaks some metrics systems.
            tags.TryAddTag("error.type", exception.GetType().FullName);
        }
        else if (HostingTelemetryHelpers.IsErrorStatusCode(statusCode))
        {
            // Add error.type for 5xx status codes when there's no exception.
            tags.TryAddTag("error.type", statusCode.ToString(CultureInfo.InvariantCulture));
        }

        var duration = Stopwatch.GetElapsedTime(startTimestamp, currentTimestamp);
        _requestDuration.Record(duration.TotalSeconds, tags);
    }
}

It's an interesting point to note that while the histogram is strictly about request durations, the presence of the many tags could enable you to derive various other metrics. For example, you could determine the number of "successful" requests, the number of requests to a particular route, or with a given status code.

And that's it, we've covered all of the Insturment types currently available in .NET 10. Note that there's no ObservableHistogram<T> type, as that generally wouldn't be practical to implement.

We now know how to create all the different types of Instrument, and in the first post of this series I showed how to record the metrics using dotnet-counters. In the following post in this series, we'll look at how to record these values in-process instead.

Summary

In this post, I described each of the different Instrument<T> types exposed by the System.Diagnostics.Metrics APIs. For each type I described when you would use it and provided an example of both how to create the Instrument<T>, and how to record values, using examples from the .NET base class libraries and ASP.NET Core. In the next post we'll look at how to record values produced by Instrument<T> types in-process.

In my previous post I provided an introduction to the System.Diagnostics.Metrics APIs introduced in .NET 6. In this post I show how to use the Microsoft.Extensions.Telemetry.Abstractions source generator, explore how it changes the code you need to write, and explore the generated code.

I start the post with a quick refresher on the basics of the System.Diagnostics.Metrics APIs and the sample app we wrote last time. I then show how we can update this code to use the Microsoft.Extensions.Telemetry.Abstractions source generator instead. Finally, I show how we can also update our metric definitions to use strongly-typed tag objects for additional type-safety. In both cases, we'll update our sample app to use the new approach, and explore the generated code.

You can read about the source generators I discuss in this post in the Microsoft documentation here and here.

Background: System.Diagnostics.Metrics APIs

The System.Diagnostics.Metrics APIs were introduced in .NET 6 but are available in earlier runtimes (including .NET Framework) by using the System.Diagnostics.DiagnosticSource NuGet package. There are two primary concepts exposed by these APIs; Instrument and Meter:

• Instrument: An instrument records the values for a single metric of interest. You might have separate Instruments for "products sold", "invoices created", "invoice total", or "GC heap size".
• Meter: A Meter is a logical grouping of multiple instruments. For example, the System.Runtime Meter contains multiple Instruments about the workings of the runtime, while the Microsoft.AspNetCore.Hosting Meter contains Instruments about the HTTP requests received by ASP.NET Core.

There are also multiple types of Instrument: Counter<T>, UpDownCounter<T>, Gauge<T>, and Histogram<T> (as well as "observable" versions, which I'll cover in a future post). To create a custom metric, you need to choose the type of Instrument to use, and associate it with a Meter. In my previous post I created a simple Counter<T> for tracking how often a product page was viewed.

Background: sample app with manual boilerplate

In this post I'm going to start from where we left off in the previous post, and update it to use a source generator instead. So that we know where we're coming from, the full code for that sample is shown below, annotated to explain what's going on; for the full details, see my previous post

using System.Diagnostics.Metrics;
using Microsoft.Extensions.Diagnostics.Metrics;

var builder = WebApplication.CreateBuilder(args);

// 👇 Register our "metrics helper" in DI
builder.Services.AddSingleton<ProductMetrics>();

var app = builder.Build();

// Inject the "metrics helper" into the API handler 👇 
app.MapGet("/product/{id}", (int id, ProductMetrics metrics) =>
{
    metrics.PricingPageViewed(id); // 👈 Record the metric
    return $"Details for product {id}";
});

app.Run();


// The "metrics helper" class for our metrics
public class ProductMetrics
{
    private readonly Counter<long> _pricingDetailsViewed;

    public ProductMetrics(IMeterFactory meterFactory)
    {
        // Create a meter called MyApp.Products
        var meter = meterFactory.Create("MyApp.Products");

        // Create an instrument, and associate it with our meter
        _pricingDetailsViewed = meter.CreateCounter<int>(
            "myapp.products.pricing_page_requests",
            unit: "requests",
            description: "The number of requests to the pricing details page for the product with the given product_id");

    }

    // A convenience method for adding to the metric
    public void PricingPageViewed(int id)
    {
        // Ensure we add the correct tag to the metric
        _pricingDetailsViewed.Add(delta: 1, new KeyValuePair<string, object?>("product_id", id));
    }
}

In summary, we have a ProductMetrics "metrics helper" class which is responsible for creating the Meter and Instrument definitions, as well as providing helper methods for recording page views.

When we run the app and monitor it with dotnet-counters we can see our metric being recorded:

Now that we have our sample app ready, lets explore replacing some of the boilerplate with a source generator.

Replacing boiler plate with a source generator

The Microsoft.Extensions.Telemetry.Abstractions NuGet package includes a source generator which, according to the documentation, generates code which:

…exposes strongly typed metering types and methods that you can invoke to record metric values. The generated methods are implemented in a highly efficient form, which reduces computation overhead as compared to traditional metering solutions.

In this section we'll replace some of the code we wrote above with the source generated equivalent!

First you'll need to install the Microsoft.Extensions.Telemetry.Abstractions package in your project using:

dotnet add package Microsoft.Extensions.Telemetry.Abstractions

Alternatively, update your project with a <PackageReference>:

<ItemGroup>
  <PackageReference Include="Microsoft.Extensions.Telemetry.Abstractions" Version="10.2.0" />
</ItemGroup>

Note that in this post I'm using the latest stable version of the package, 10.2.0.

Now that we have the source generator running in our app, we can put it to use.

Creating the "metrics helper" class

The main difference when you switch to the source generator is in the "metrics helper" class. There's a lot of different ways you could structure these—what I've shown below is a relatively close direct conversion of the previous code. But as I'll discuss later, this isn't necessarily the way you'll always want to use them.

As is typical for source generators, the metrics generator is driven by specific attributes. There's a different attribute for each Instrument type, and you apply them to a partial method definition which creates a strongly-typed metric, called PricingPageViewed in this case:

private static partial class Factory
{
    [Counter<int>("product_id", Name = "myapp.products.pricing_page_requests")]
    internal static partial PricingPageViewed CreatePricingPageViewed(Meter meter);
}

The example above uses the [Counter<T>] attribute, but there are equivalent versions for [Gauge<T>] and [Histogram<T>] too.

This creates the "factory" methods for defining a metric, but we still need to update the ProductMetrics type to use this factory method instead of our hand-rolled versions:

// Note, must be partial
public partial class ProductMetrics
{
    public ProductMetrics(IMeterFactory meterFactory)
    {
        var meter = meterFactory.Create("MyApp.Products");
        PricingPageViewed = Factory.CreatePricingPageViewed(meter);
    }

    internal PricingPageViewed PricingPageViewed { get; }

    private static partial class Factory
    {
        [Counter<int>("product_id", Name = "myapp.products.pricing_page_requests")]
        internal static partial PricingPageViewed CreatePricingPageViewed(Meter meter);
    }
}

If you compare that to the code we wrote previously, there are two main differences:

• The [Counter<T>] attribute is missing the "description" and "units" that we previously added.
• The PricingPageViewed metric is exposed directly (which we'll look at shortly), instead of exposing a PricingPageViewed() method for recording values.

The first point is just a limitation of the current API. We actually can specify the units on the attribute, but if we do, we need to add a #pragma as this API is currently experimental:

private static partial class Factory
{
    #pragma warning disable EXTEXP0003 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.

                                                        //   Add the Unit here 👇
    [Counter<int>("product_id", Name = "myapp.products.pricing_page_requests", Unit = "views")]
    internal static partial PricingPageViewed CreatePricingPageViewed(Meter meter);
}

The second point is more interesting, and we'll dig into it when we look at the generated code.

Updating our app

Before we get to the generated code, lets look at how we use our updated ProductMetrics. We keep the existing DI registration of our ProductMetrics type, the only change is how we record a view of the page

using System.Diagnostics.Metrics;
using System.Globalization;
using Microsoft.Extensions.Diagnostics.Metrics;

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddSingleton<ProductMetrics>();
var app = builder.Build();

app.MapGet("/product/{id}", (int id, ProductMetrics metrics) =>
{
    // Update to call PricingPageViewed.Add() instead of PricingPageViewed(id)
    metrics.PricingPageViewed.Add(value: 1, product_id: id);
    return $"Details for product {id}";
});

app.Run();

As you can see, there's not much change there. Instead of calling PricingPageViewed(id), which internally adds a metric and tag, we call the Add() method, which is a source-generated method on the PricingPageViewed type. Let's take a look at all that generated code now, so we can see what's going on behind the scenes.

Exploring the generated code

We have various generated methods to look at, so we'll start with our factory methods and work our way through from there.

Note that in most IDEs you can navigate to the definitions of these partial methods and they'll show you the generated code.

Starting with our Factory method, the generated code looks like this:

public partial class ProductMetrics 
{
    private static partial class Factory 
    {
        internal static partial PricingPageViewed CreatePricingPageViewed(Meter meter)
            => GeneratedInstrumentsFactory.CreatePricingPageViewed(meter);
    }
}

So the generated code is calling a different generated type, which looks like this:

internal static partial class GeneratedInstrumentsFactory
{
    private static ConcurrentDictionary<Meter, PricingPageViewed> _pricingPageViewedInstruments = new();

    internal static PricingPageViewed CreatePricingPageViewed(Meter meter)
    {
        return _pricingPageViewedInstruments.GetOrAdd(meter, static _meter =>
            {
                var instrument = _meter.CreateCounter<int>(@"myapp.products.pricing_page_requests", @"views");
                return new PricingPageViewed(instrument);
            });
    }
}

This definition shows something interesting, in that it shows the source generator is catering to a pattern I was somewhat surprised to see. This code seems to be catering to adding the same Instrument to multiple Meters.

That seems a little surprising to me, but that's possibly because I'm used to thinking in terms of OpenTelemetry expectations, which doesn't have the concept of Meters (as far as I know), and completely ignores it. It seems like you would get some weird duplication issues if you tried to use this source-generator-suggested pattern with OpenTelemetry, so I personally wouldn't recommend it.

Other than the "dictionary" aspect, this generated code is basically creating the Counter instance, just as we were doing before, but is then passing it to a different generated type, the PricingPageViewed type:

internal sealed class PricingPageViewed
{
    private readonly Counter<int> _counter;
    public PricingPageViewed(Counter<int> counter)
    {
        _counter = counter;
    }

    public void Add(int value, object? product_id)
    {
        var tagList = new TagList
        {
            new KeyValuePair<string, object?>("product_id", product_id),
        };

        _counter.Add(value, tagList);
    }
}

This generated type provides roughly the same "public" API for recording metrics as we provided before:

public class ProductMetrics
{
    // Previous implementation
    public void PricingPageViewed(int id)
    {
        _pricingDetailsViewed.Add(delta: 1, new KeyValuePair<string, object?>("product_id", id));
    }
}

However, there are some differences. The generated code uses a more "generic" version that wraps the type in a TagList. This is a struct, which can support adding multiple tags without needing to allocate an array on the heap, so it's generally very efficient. But in this case, it doesn't add anything over the "manual" version I implemented.

So given all that, is this generated code actually useful?

Is the generated code worth it?

I love source generators, I think they're a great way to reduce boilerplate and make code easier to read and write in many cases, but frankly, I don't really see the value of this metrics source generator.

For a start, the source generator is only really changing how we define and create metrics. Which is generally 1 line of code to create the metric, and then a helper method for defining the tags etc (i.e. the PricingPageViewed() method). Is a source generator really necessary for that?

Also, the generator is limited in the API it provides compared to calling the System.Diagnostics.Metrics APIs directly. You can't provide a Description for a metric, for example, and providing a Unit needs a #pragma…

What's more, the fact that the generated code is generic, means that the resulting usability is actually worse in my example, because you have to call:

metrics.PricingPageViewed.Add(value: 1, product_id: id);

and specify an "increment" value, as opposed to simply being

metrics.PricingPageViewed(productId: id);

(also note the "correct" argument names in my "manual case"). The source generator also seems to support scenarios that I don't envision needing (the same Instrument registered with multiple Meter), so that's extra work that need not happen in the source generated case.

So unfortunately, in this simple example, the source generator seems like a net loss. But there's an additional scenario it supports: strongly-typed tag objects

Using strongly-typed tag objects

There's a common programming bug when calling methods that have multiple parameters of the same type: accidentally passing values in the wrong position:

Add(order.Id, product.Id); // Oops, those are wrong, but it's not obvious!

public void Add(int productId, int orderId) { /* */ }

One partial solution to this issue is to use strongly-typed objects to try to make the mistake more obvious. For example, if the method above instead took an object:

public void Add(Details details) { /* */ }

public readonly struct Details
{
    public required int OrderId { get; init; }
    public required int ProductId { get; init; }
}

Then at the callsite, you're less likely to make the same mistake:

// Still wrong, but the error is more obvious! 😅
Add(new()
{
    OrderId = product.Id,
    ProductId = order.Id,
});

It turns out that passing lots of similar values is exactly the issue you run into when you need to add multiple tags when recording a value with an Instrument. To help with this, the source generator code can optionally use strongly-typed tag objects instead of a list of parameters.

Updating the holder class with strongly-typed tags

In the examples I've shown so far, I've only been attaching a single tag to the PricingPageViewed metric, but I'll add an additional one, environment just for demonstration purposes.

Let's again start by updating the Factory class to use a strongly-typed object instead of "manually" defining the tags:

private static partial class Factory
{
    // A Type that defines the tags 👇
    [Counter<int>(typeof(PricingPageTags), Name = "myapp.products.pricing_page_requests")]
    internal static partial PricingPageViewed CreatePricingPageViewed(Meter meter);
    // previously:
    // [Counter<int>("product_id", Name = "myapp.products.pricing_page_requests")]
    // internal static partial PricingPageViewed CreatePricingPageViewed(Meter meter);
}

public readonly struct PricingPageTags
{
    [TagName("product_id")]
    public required string ProductId { get; init; }
    public required Environment Environment { get; init; }
}

public enum Environment
{
    Development,
    QA,
    Production,
}

So we have two changes:

• We're passing a Type in the [Counter<T>] attribute, instead of a list of tag arguments.
• We've defined a struct type that includes all the tags we want to add to a value.
  • This is defined as a readonly struct to avoid additional allocations.
  • We specific the tag name for ProductId. By default, Environment uses the name "Environment" (which may not be what you want, but this is for demo reasons!).
  • We can only use string or enum types in the tags

The source generator then does its thing, and so we need to update our API callsite to this:

app.MapGet("/product/{id}", (int id, ProductMetrics metrics) =>
{
    metrics.PricingPageViewed.Add(1, new PricingPageTags()
    {
         ProductId = id.ToString(CultureInfo.InvariantCulture),
         Environment = ProductMetrics.Environment.Production,
    });
    return $"Details for product {id}";
});

In the generated code we need to pass a PricingPageTags object into the Add() method, instead of individually passing each tag value.

Note that we had to pass a string for ProductId, we can't use an int like we were before. That's not great perf wise, but previously we were boxing the int to an object? so that wasn't great either😅 Avoiding this allocation would be recommended if possible, but that's out of the scope for this post!

As before, let's take a look at the generated code.