The idea is pretty solid at its core: an MCP server provides a common API for agents to query services and data, and for services to advertise what actions are available to the agents. The MCP server sits in the middle to facilitate the whole shebang. For this to work, we need two distinct authorization contexts connected by the MCP server. Thankfully, OAuth gives us a great set of tools to address this, and Aaron Parecki wrote a fantastic piece about how OAuth can be applied to help solve this problem, I consider that article required reading for anyone in this space.

As it turns out, though, the MCP deployment pattern defies some of the assumptions about how OAuth ought to work.

Two Worlds

The proxy setup splits the world into an MCP Protocol space, where the agent connects to an MCP server, and what we’ll call the upstream service space, where the MCP server connects to some other service that does the actual work. This is where OAuth starts to come into play.

If we look at MCP as an OAuth-protected API, we can pretty easily see how we can split out the AS and RS roles inside the MCP server space. In the general case, it’s easy to see how the AS portion can facilitate the user authorizing the agent. The agent gets an OAuth token to call the MCP server, which maps to some set of credentials upstream. We don’t want to just pass through the MCP client’s token, though — there’s no guarantee the upstream service even uses OAuth, let alone the same set of credentials. So while we will need a mapping between these sides, this pattern allows us tons of flexibility in how we roll this out.

In the enterprise case, we can use existing AS policy to authenticate users to the MCP server and map their agent’s access to whatever sets of services that user can use. The integration between the service and the MCP server can be handled by the enterprise, without users needing to do any extra work. We can even allowlist this connection so that users don’t have to see a consent screen, as long as all the right policy conditions are in play. If the integrated service uses OAuth itself, we could even apply token exchange between the MCP client’s access token and the service’s required access token, to limit exposure. The enterprise can even lock down the MCP server to use only pre-registered, pre-approved MCP clients, already a common pattern among corporate OAuth deployments.

On the other end of the deployment spectrum, we might have an agent, MCP server, and upstream service all deployed and owned by completely separate entities with no prior relationship. This might sound crazy, but it’s not really any different from how desktop email clients work with the IMAP protocol. For most IMAP use cases, the mail client stores the user’s credentials and impersonates them on the API. With OAuth, we can probably do better than that, but OAuth was built to connect websites together in a world where things are more predictable and stable.

So how can we do this in a wildly dynamic MCP space?

Intertwined Credentials

OAuth classically requires the user to log in to the AS and approve the client, which is registered with the AS, to act on their behalf to call a resource. While we can apply that pattern to the MCP proxy, and as we saw in the enterprise case it can make a lot of sense, I propose that we can learn more from the world of email clients.

An email server isn’t going to know anything about a particular instance of email software ahead of time, and the server probably isn’t even going to know anything about a class of email software. A user could, if they chose, implement IMAP from scratch and use it with the server — that’s the promise of interoperability for protocols like IMAP. The server only cares if the protocol is implemented correctly and if the mail client can authenticate to the user account. If the authentication works, then the client is valid.

I argue that it’s much the same in the MCP proxy case. The identity of a particular instance of client software is less important because it should always be mapped to a particular set of access rights upstream. And where does the MCP server get those access rights? From the user authorizing the MCP server somehow. This could be yet another OAuth flow, it could be storing the user’s credentials, or it could be something very un-OAuth like accessing a local socket connection. If the MCP server can make a valid connection to the upstream service in the context of the user setting up the connect to their agent, then that’s all an average MCP server should really care about. The token that it issues to the agent will get mapped to the authenticated context used to call the upstream service. When the MCP server’s AS issues a token for the agent, the AS can store a mapping to the authentication needed for the upstream service. This is not dissimilar from mapping an OAuth access token to the user who was logged in to the AS at the time, and making that information available to the RS. In this case, our RS also needs to make a client call to something else, and that’s the information we make available.

I could be true that we might not even need client IDs in the traditional OAuth sense. The only security artifact that matters is the access token and its mapping to the upstream credential set. If I need to re-connect my upstream API, my agent can get a new token as a result and just use that. The MCP server might not even care about who I am, so long as I can successfully connect to the upstream system. After all, this is exactly the argument used for the design of OAuth clients in general: if they can get to the resources they need, they don’t need to know who the user is in many cases. (And if they do, there’s OIDC for that.)

Weaving Webs

This work is bringing to light some of the limitations and assumptions of the OAuth protocol. Some of these are solved by things that we built into GNAP, especially the notion of ephemeral clients, but even with GNAP it’s not a simple world.

For example, if the upstream service requires a static OAuth registration and then allowlists a client after a user authorizes it, does this leave open a door for attackers to exploit? Is there anything that the MCP server, sitting in the middle as a proxy, needs to do to help this? What about cases where the MCP server has no user interface at all?

There are many questions still left to be answered, but I, for one, am excited that they’re being asked and discussed right now. I look forward to being part of the conversation, and I hope you can join in. Maybe we’ll even invite the bots to help.

I’m really proud of what we’ve built in GNAP. One of the core tenets of GNAP was to look at the world of OAuth and surrounding technologies and figure out how we could do a lot of that better. It’s been great to see GNAP getting applied in a bunch of places over the web, from payments to key management, and especially in places where OAuth doesn’t reach as well. While OAuth remains deeply entrenched over the world, and likely will be for some time, the community has learned many things from GNAP. Alot of things that started in GNAP have been making their way back to the OAuth ecosystem in some form.

The most obvious of this is RFC9396: OAuth Rich Authorization Requests. This replacement of OAuth’s scope parameter was a direct and intentional backport of what became GNAP’s resource access rights, which also acronyms to RAR. In the OAuth world, we don’t get some of the clean features of GNAP, like being able to substitute strings for objects as a shorthand, but a lot of the core enhancements are there.

We’re also seeing yet another intent registration addition to OAuth 2 (on top of the pushed authorization request, device grant type, and CIBA extensions), and this one mimics a lot of the flexibility of GNAP’s interaction system. It’s a more narrow use case in the OAuth specification, but it’s clear that the pattern that GNAP was built on is here to stay.

And then there’s RFC9421: HTTP Message Signatures. This is work that started independently from GNAP but grew up around the same time, and GNAP utilizes HTTP Message Signatures as a core security function. I don’t think we’d have gotten the signing spec to be as robust as it is without some of the GNAP key proofing use cases driving the discussion.

And finally, the GNAP Resource Servers document has just passed IESG review and is on its own way to becoming an RFC as well. This document represents key abstractions in how and RS and AS relate to each other, and I hope we can continue to build this out and pull the best ideas out into the world.

The GNAP working group is shutting down now that its core work is done, but GNAP is far from over. I look forward to seeing it grow into its spaces, and serve as a beacon of how a delegation protocol can be engineered and built.

Data Synchronization

When multiple independent systems live together in the world and share data, inevitably that data is going to get out of sync. In a purely heirarchical system, we’re mostly concerned with building a consensus around the correct state of the shared data. We see this approach in distributed ledger systems, where nodes eventually come to a common view of what the shared state of the world ought to be.

But we don’t have that in a bubble-based architecture, because we don’t expect everyone to have the same view. Instead, we expect many different independent views to contribute to each other in a distributed fashion. This effectively means that for each bubble, it can send updates in and out to other systems. In most cases, there is a directionality to the data flow: one side is going to be treated as more authoritative than the other for a given context. What a bubble does when it’s on either end of that gap changes how we view the synchronization.

Account Updates From Above

When changes are pushed to us from an authoritative source, the simplest thing is to overwrite everything. After all, if it’s authoritative, why wouldn’t we just take that statement as truth? But the reality is that we’ve likely augmented our record of that user with additional details, overrides, and other localized changes that we don’t want to lose.

In these cases, we can shadow the data. In other words, we keep a copy of the source’s data separate from our own local view. When we get an update from that source, we can update our copy of the source’s data with wild abandon. We can then decide, by local policy, whether we want to adjust our overrides based on the update. Importantly, this decision is separate from accepting and processing the updated data from the source. This setup allows us to keep local information in the bubble at the same time that we sync from elsewhere.

Account Updates From Below

An authoritative bubble is sometimes going to want to pull updated information from the bubbles that live downstream. These are systems that we’ve sent accounts out to, and those systems might have something to tell us about our users. Maybe there’s an additional accreditation that’s been earned, or something to represent additional access details outside of our bubble, or even just an update to one of the core fields we sent down.

In any case, the downstream bubble is sending us additional data about a user, and we now have the chance to do something about it. If nothing else, we can store it and note it. If we want to, we can update the user’s record that we hold locally, and even go so far as to propagate that downward again to other bubbles.

Changing Trust

It’s not just user data that we can pass around, though that’s the most common piece we’d expect to see. The bubbles can also pass about other bubbles to each other, and incorporate what they learn into their own systems.

For many situations that fit the bubble architecture patterns, we expect to meet new peers in the field and make new decisions based on local context and requirements. These newly-discovered bubbles can then be propagated through the network, along with potentially interesting information about what the presenting bubble trusts it for.

Audit Logs

Now that we can identify users and the systems they come from, we can start to do one of the most interesting and perhaps complex jobs of a reconnected bubble: audit logs. When a bubble gets provisioned, that provisioning authority is likely to want to know what happens in that bubble during the disconnected spells. The bubble can package up the relevant audit log history and pass it along to the authorities that need it.

But auditing can go further than that: for any user that enters our bubble from somewhere else, we probably want to report back to the system that sent them to us. Since we know where we originally learned of them, and we know how to identify that system as a whole, we can filter and target the information we’re sending over. And we can do this while processing the updates they’re sending us about the account.

Conclusion

The bubble pattern sits between the always-connected and always-separate models, but the moment of reconnection gives us a profound opportunity to process user and trust data in ways that we may have never considered.

With that model in mind, from the perspective of our bubble, we’ve got three distinct identity processing systems that all need to come together to make things work: the local identity management system for our bubble, something to process inbound accounts, and something to package accounts up for outbound transmission to somewhere else.

The Local Bubble

Within the bubble itself, we are using a cohesive IdAM system and are almost certainly using federation technology to connect out to a set of RP’s within the bubble. All of these systems can look towards one authoritative IdP within the bubble for the source of all account information.

Inside the bubble, we have tons of freedom for how we want to connect our users to our systems. While we probably want to use current best-of-class technologies like OpenID Connect and passkeys, we only really need to be compatible internally, using whatever makes the most sense for our environment.

The important thing here is that each user has an account that is accessible within the bubble at all times, and is not dependent on reaching out to anything outside the bubble for local authentication.

Inbound Processing

Most of the users in a bubble probably came from somewhere. If we onboard an account from an external system, it means that we’re creating an account based on a set of attributes from a known source. These attributes can come in with an assertion, credential, certificate, API call, or some other technology. The important thing, for us, is that we can now tie these attributes to a known account, and we can cache the attributes as we received them. A lot of these are going to be immensely useful — we won’t have to have every user type in all their attributes every time they connect into a new bubble.

But it’s not enough that we’re just making a cached copy. In many cases, we’ll want to override or update these attributes locally, but we don’t necessarily want to lose the data from the source when we do that override. After all, we don’t control the data source, and we want to know where all of our information came from.

We can use an overlay style data structure that lets us keep both updated data and the source data at the same time. Let’s say, for instance, that Patty O’Sullivan gets an account onboarded into the system, but it turns out that everyone inside the bubble just calls her Sully. We can create a local value that overrides the official value, but the official value doesn’t go away: it’s still sitting in its own structure. If we don’t have an override, when we look up an attribute we can follow a pointer to an upstream source and get it directly without having to copy it.

The approach also allows us to very efficiently take care of cases where we don’t have a need for referencing an attribute that was handed to us, or that we need to create a brand new attribute that doesn’t exist at the source. And in fact, this pattern can be applied up the chain, since our source might have gotten its information from somewhere else in the first place.

And we can just keep copying this pattern, even pointing at multiple sources at the same time. We can optimize this graph structure for both storage size and lookup efficiency, but more importantly it allows us to keep the data sources separate from each other in a meaningful fashion. We can tell where we’re getting each attribute value from, and we can differentiate between local updates and data copied from elsewhere.

This also means that we can put restrictions on data from different layers. For example, maybe we want a policy that needs an update on a cached value every so often. Or if I’m doing a local override of an important attribute, like one that gets used in security decision making, then I need to check that the override is still valid after a certain timeout. This can avoid a class of configuration errors that we see in the field, where something gets changed in order to solve an immediate problem, but never gets changed back when things de-escalate.

Outbound Packaging

And of course, we also want our bubble to be able to act as the source for some downstream receivers as well. In order to do that, we need to be able to package up our accounts and assert them outbound.

But wait a moment — isn’t that the same exact thing we’re doing inside of the bubble for our apps? Aren’t we already going through a federation process to connect on the inside? Shouldn’t we just use that same IdP again, since it’s already set up and has all the same accounts?

While it would be possible to re-use the same component, it makes more sense to have a dedicated IdP that only speaks to external receivers. This separation allows us to deliberately control which information we share and with whom, and without it being conflated with local policy, changes, overrides, and other concerns. When we’re talking to an external receiver, we likely want to give a very specific view of an account in this context, especially considering that we want to minimize the transmission of sensitive data across boundaries.

Stacking the Three Pieces Together

Each identity system we’ve talked about here has a distinct role to play. In this way, the three parts of a bubble system — inbound, local, and outbound — can work together to create a cohesive path for an account, its attributes, and the person who’s using it.

Through all of that, several major concepts have risen to the surface, and I’ll be looking to tackle these in a few different posts — questions like “can’t we just copy the user store?” and “isn’t this just like an interfederation?” get brought up quickly each time. But I wanted to start with something very concrete before getting into the what and why: how would you make a bubble?

It’s Not New Technology

The central idea behind a bubble in this world is that it’s an internally-cohesive network of systems, with clear boundaries and processes to cross those boundaries. I think we can start building bubbles today out of tech that we’re already using for similar and related purposes.

An important corollary to that is that I deeply believe that this concept is not conducive to a single technology stack. So many times through tech history, we’ve been told that if the whole world would just adopt this one particular way of doing things, then all the problems would be solved. This line is usually delivered by a person selling the new way of doing things, or at the very least the picks and shovels to make it happen.

For bubbles, though? I think we’ve got all the most important parts already. What’s fundamentally different is how we use everything, and the assumptions we make around the moving parts and how we stick them together.

Crossing The Borders

In order to create and update accounts in the bubble, we often want to pull that information from elsewhere. Whether it’s an authoritative source that created the bubble in the first place, or it’s a peer we’re bumping up against in the field, we want to be able to copy identifiers and other attributes into our local account.

For the more structured cases, SCIM gives us a really powerful system for propagating user objects across systems. X.509 certificates and Verifiable Credentials also give us a way to carry a stack of user information into our system, while also providing a convenient key-proofing mechanism with the delivery.

But not everything is that structured, since we’ll also want to be talking to peers in the field about their accounts. We need to be able to do this without going all the way back up our peer’s hierarchy, and so we can do just-in-time provisioning based on a federation protocol as needed.

When that stack of user attributes gets to us, it becomes an input into an account management system — but just one input among potentially many. Instead of just overwriting or overriding a record we might already have, the incoming information feeds into the account in a way that makes sense for the local environment.

When we need to send updates about changes to others in the system, the shared signals and events frameworks give us a really solid base to build on. But what SSE, CAEP, and RISC are missing is a semantic layer that can talk about the kinds of dynamic accounts in distributed systems that we expect in a bubble environment.

It’s Familiar on the Inside

Within a bubble, everything is local. Because of that, we want to have a clear notion of a single account for each user. We can use federation technology like OpenID Connect (and the OAuth 2 it’s built on) to connect that one account to a variety of applications, devices, APIs, and whatever the bubbled system needs. This is a wholly separate federation protocol from the onboarding and outward facing processes we talked about above. We can also use SCIM to transmit internal user attributes and updates proactively, or we can just rely on the federation transactions to carry good-enough propagation of these attributes to our apps.

We aren’t going to be using external federation or similar technologies once the onboarding has taken place. For logging in to the IdP itself, we really should be using passkeys everywhere. Since we control the onboarding process, we get to control how we associate the accounts with the authenticators. Sometimes, this means we’ll hand someone their shiny new authenticator at the moment we make their account active. Sometimes, we’ll have them plug it in and bind it when we set things up. Sometimes, a record of their authenticator might come across the wire with them.

And if we’ve got applications, users, or circumstances that make some authenticators unworkable sometimes? Since the account is local, we have the ability to manage this in a way that makes sense in our environment. For example, a firefighter wearing heavy gloves is not going to be able to use a fingerprint reader in the field, but they could probably use one when back at HQ, not to mention all of the other users in the system that don’t have the same constraints. In other words, we can adapt as we need to because we are close to the environment that requires the adaptation.

Addressing The World

As we collect information about an account, we need to record not only what the information is, but also where we got it. Our view of that account is the amalgamation of all of our information sources, plus all of the local information about that account. In order for this view to make sense, we need to have a reasonable way to talk about where something came from.

Traditional federation models like to use hostnames for this, but not everything in our environment is going to be addressable on a stable, publicly-accessible URL. We can’t rely on a common data fabric (e.g., assuming everyone uses the same blockchain), and we can also be pretty sure that keys will change over time for different parties and circumstances, so we can’t just use the keys directly when we need a record.

OpenID Connect Federation brings a solution that works well for the online, connected world, but would need to be adapted for a space where the federation domains and their availability are much more dynamic. The SPIFFE project also brings us the concept of trust bundles, which tie a set of keys to identifiers in a way that can be passed between different domains. While not an exact analogue to the more macro problem here, there are some key similarities to what we’re seeing in the workload space.

Pulling it Together

The final solution isn’t going to be a single protocol, or even a single technology stack. Interoperability in this space is going to be defined by a complicated and contextual set of decisions. Two bubbles might not always be able to talk in a given dimension — one might speak OIDC outbound and another might only take in VC’s — but it’s going to be important that they can still speak in other dimensions. In the end, it’s people that make the technologies work, and we need to embrace the dirt and uncertainty of the world if we want any hope of surviving in it.

A Larger Scope

Scopes work pretty well in OAuth, but they’re ultimately just a set of strings. It’s worlds better than the all-or-nothing access that OAuth 1 or API keys give you, and if you have an API, they allow you to easily separate read and write access. But what if you want to be more specific? What if you want to, say, let someone access a specific account for certain features for a certain amount of time? Or read in one place but write to another? And what if you want to compose that set of features differently, such as any account for an unlimited time but only if the user’s online? The ways of describing API access are as varied as the APIs being accessed.

When faced with this problem, the first thing that many people realize is that because scopes are just strings, their own API can impose semantics and syntax on those strings. So people add parameters, or define composition rules, or even define a whole query language embedded into the scope string. But those approaches require a lot of complex processing and lack interoperable structure that would let you easily combine protection of multiple kinds of APIs.

What RAR gives you, instead, is an array of JSON objects to describe your API access. So instead of trying to cram everything into a single string, you can now put together an object that defines exactly what you want.

[
   {
      "type": "payment_initiation",
      "actions": [
         "initiate",
         "status",
         "cancel"
      ],
      "locations": [
         "https://example.com/payments"
      ],
      "instructedAmount": {
         "currency": "EUR",
         "amount": "123.50"
      },
      "creditorName": "Merchant A",
      "creditorAccount": {
         "iban": "DE02100100109307118603"
      },
      "remittanceInformationUnstructured": "Ref Number Merchant"
   }
]

This object is specific to the API in question and carries with it all the detail that is needed for successful processing. Each kind of API can define its own type value, which in turn defines what’s allowed to go into the rest of the object. And if you need more than one view of things, like read access to A but write access to B, then you can pass in multiple objects in the same structure.

One important question arises out of this: who needs to know this level of detail?

Who Cares About RAR

In this regard, RAR really is built on top of the concept of a scope. In an OAuth delegation, there are four parties. The client, resource owner, authorization server (AS), and resource server (RS).

These parties have particular relationships, and each of them might care about a RAR object or a scope in a slightly different way. However, the more important question is about which relationship is in play.

Client -> AS: Requesting Access

When requesting an access token, the client needs to be able to describe to the AS what it wants. RAR allows the client to get VERY specific, if the client knows what details it wants ahead of time. Maybe the client has prompted the resource owner for an account identifier, or has learned through some other protocol where the target system is located, or it’s just been configured to know that it needs to ask for specific objects in order to do specific things. In all of these cases, the client can send RAR objects to the AS just like it would a scope, in the hopes of getting an access token that can do what it asks for.

AS -> Client: Granting Access

When the access token is granted, the AS can tell the client which RAR objects have been applied to the token. While this information is no substitute for an API discovery protocol, this approach can let the client differentiate what an access token is good for in different dimensions. For example, a client can ask for a token for an available signing service, and then be granted a token for use at a specific signing service, indicated through the locations field in the RAR object.

AS -> Resource owner: Gathering Authorization

During the delegation process, the AS often needs to prompt the resource owner to see if they’re OK with what’s being delegated. While this starts as the Client->AS request, RAR gives the AS an opportunity to fine-tune the access by asking the resource owner to be specific, or even filling in values that get put into the resulting RAR object. Maybe the client is asking for account access but the resource owner stipulates that it’s only good for the next five minutes. This does come at a usability cost, since it’s much easier to display a list of scope strings with checkboxes. But experience has shown that this list is not a great security measure anyway, since most users won’t change the checkboxes, and often don’t understand the differentiated access being granted.

AS -> RS: Describing Access

The access token itself represents a certain set of rights that have been granted. These can be described in the metadata of the token, available through either a structured token field or an introspection response. In this way, the RS can learn what an access token is good for, and apply its policies appropriately. Does the token grant access for the HTTP GET command on the resource at /photo/123-fda1d? Is this token even good at this specific RS, or is it meant for somewhere else? The RAR object can be used to describe all of this.

Not Everything Has To Match

Finally, it’s important to note that the all of these different branches need not match each other in a single transaction. In one of the applications where I’ve personally deployed RAR, the client never sees the RAR objects. The client knows to ask for a specific scope, and the AS knows that when it sees that scope, the resulting token needs to apply to a whole set of things represented by the current user’s access within the system. The downstream APIs know nothing about users or accounts, but they do know the resources they protect.

As a consequence, the AS translates the client’s incoming scope request to a set of RAR objects that the APIs understand. The APIs never see or care about the scope, and the client never sees or cares about the RAR. In this way, internal API details stay internal and do not leak unnecessarily into the wider system.

However, a different client in this same ecosystem does have insight into the details of the API structure, and therefore its requests do specify RAR objects that target the APIs. These objects are processed in exactly the same way by the API servers, which gives us a powerful parallelism and profound code reuse in production.

GNAP Native

In GNAP, one of our main goals was to see what an OAuth-style system would look like without the constraints and history of OAuth, and one such constraint includes scopes. Consequently, GNAP’s native access rights is an array of objects that look suspiciously like RAR objects. This design is, of course, intentional, and in many ways RAR is the backport of GNAP’s access rights system to work on top of OAuth 2. While GNAP doesn’t have scopes in the same way, GNAP’s reference-based approach to its API design does allow for the use of a simple string to stand in for the objects in question, allowing a request to have both shortcut and fully specified items in the same request.

"access": [
    {
        "type": "photo-api",
        "actions": [
            "read",
            "write"
        ],
        "locations": [
            "https://server.example.net/",
            "https://resource.local/other"
        ],
        "datatypes": [
            "metadata",
            "images"
        ],
        "geolocation": [
            { lat: -32.364, lng: 153.207 },
            { lat: -35.364, lng: 158.207 }
        ]
    },
    {
        "type": "financial-transaction",
        "actions": [
            "withdraw"
        ],
        "identifier": "account-14-32-32-3",
        "currency": "USD"
    },
    "dolphin-metadata",
    "some other thing"
]

How Can I Use RAR?

RAR support is starting to show up across different vendors, though it’s not universal yet. One of the companies I work for, Authlete, supports RAR natively. Other products can often have RAR grafted on top, since it takes the form of an extra parameter to be processed by an extension or module.

The real value is that we are starting to see API access defined in terms of RAR objects, replacing the awkward and error-prone string composition practices of the past. RAR may seem complex, but when you look at how APIs are defined and scopes are used, the power of that complexity really starts to show its value.

In my experience, there are a few common ways to get there.

Conformance

The easiest way to achieve interoperability is for there to be no choice from the implementations. If there’s only one option, and implementations are motivated to follow that option, then interoperability is much easier to count on. If a specification has a MUST, and means it, you can realistically rely on that being followed by well-meaning developers.

This zero-level interoperability is not without its confusion, though. In any specification, there are a lot of assumptions that lead up to a MUST being placed. Changes in this context could make that MUST behave differently in between implementations. For example, taking the byte value of a data structure, even a simple one like a string or number, assumes an encoding and order for that data structure. What’s most dangerous about this kind of problem is that it’s easy for multiple developers to make the same assumptions and therefore assure themselves that the MUST as written is sufficient, until someone else comes along with different assumptions and everything breaks in spite of seeming to be conformant.

To give a concrete example, try asking anyone for a USB cable and chances are you’ll get a few different varieties back, from type A, to type C, to micro-B, to power-only varieties. All of them are USB and conform to all aspects of the cabling standard, but “USB” is not sufficient to ensure compatibility on a physical level.

Even with its limitations, it’s still a good idea for specifications to be specific as much as possible. But the world can’t always make a single choice and stick to it. Things start to get more interesting when there’s choice between options, though, so how do we handle that?

Discovery

If my system can ask your system which options it supports ahead of time, then ostensibly I should be able to pick one of those options and expect it to work. Many standard internet APIs are based around this concept, with an initial discovery phase that sets the parameters of interoperability for future connections.

This pattern works fairly well, at least for common options and happy paths. If your system supports some or all of what my system wants, then I can probably figure out how to connect to you successfully. If your system doesn’t support what I need, then at least I know I can’t start the process. OpenID Connect usually works from a discovery-based process, where the RP fetches the IdP’s discovery document prior to connecting.

The discovery pattern is predicated on an agreement of how to do discovery in the first place. I need to at least know how to make an initial call in order to figure out what the options are for all future calls. This is expected to be out of band for the rest of the protocol, but is often built on the same underlying assumptions. Does the protocol assume HTTP as a transport? Then discovery can use that, also.

Discovery is generally done without context, though. The existence of something in a discovery step does not guarantee that it will be usable in the context of a real request. A server might support seven different cryptographic algorithms, but might only allow some of them to specific clients or request types. That kind of detail is hard to capture through discovery.

For a physical example, let’s say that before you ask for a USB cable, you can check a list of all the available types that the person you’re asking has available. That way when you ask for a specific cable, you’ll at least know that they had it as an option. But maybe they only had one and already lent it out to someone else, or they only hand out power-only cables to people they haven’t met before, in case the cable goes walkabout.

Negotiation

If we can instead bake the discovery process into the protocol itself, we can end up with a negotiation pattern. One party makes a request that includes the options that they’re capable of, and the other party responds with their own set of options, or chooses from the first set. From there, both parties now know the parameters the need to connect.

This kind of option works well with connection-focused protocols, and it has the distinct advantage of avoiding an additional round trip to do discovery. There’s also no longer a need to specify a separate process for discovery, since it’s baked in to the protocol itself. Content negotiation in HTTP, algorithm selection in TLS, and grant negotiation in GNAP all follow this pattern.

Negotiation falls short when decisions have to be made about the initial connection, much like when there’s a separate discovery call. The protocol can be built to robustly account for those failures, such as a content type being unavailable in HTTP, but the ability to negotiate does not guarantee satisfactory results. Negotiation can also end up with less than ideal results when there’s not a clear preference order, but in such cases it’s possible for a negotiation to continue over several round trips.

If you need a USB cable, you can walk up to someone and say “Hi, can I borrow a USB cable? I need it to be USB-A or USB-C and I need it for a data connection.” The person you’re asking can then see if they have anything that fits your criteria, and choose appropriately from their supply. If they hand you something that’s less than ideal, you can clarify “I’d really prefer USB-C if you have it, but this will work if not”.

Configuration

On a simpler level, many developers simply want to choose an option and run with it, and if the other side makes a compatible choice, this can short-circuit any kind of discovery or negotiation process in a positive way. This might seem magical, but it happens way more often than many software architects and standards authors like to admit. It’s not uncommon for two developers to make similar assumptions, or for libraries to influence each others’ implementations such that they end up doing the same thing even without any external agreement to do so.

If a developer codes up something based on an example, and it works, that developer is not likely to look beyond the example. Why would they? The goal is to get something to connect, and if it does, then that job is done and they can move on to more interesting problems. And if it doesn’t work? Chance are they’ll tweak the piece that doesn’t work until it does work.

JSON works this way in practice, with well-formed JSON being the interoperability expectation and anything else being, effectively, schema-by-fiat. While there are schema languages on top of JSON, the practical truth is that applications apply their own internal schema-like expectations to the JSON by looking for a field with a certain name in a certain place with a data value that parses how they expect it to. Anything that runs afoul of that is an error not of JSON but for the application to deal with. This is a far cry from the days of XML, which expected processing of namespaces and schemas to make sense of tags at the parsing level. Was it more robust? Arguably, yes. But it was also far too heavy for an average developer to care about. JSON’s approach lets us get to data exchange simply by letting us get it right by accident most of the time, and ignoring things that don’t make sense.

If you want a USB-C cable but just ask someone for a USB cable, and they hand you a USB-C cable, everyone’s happy. You may have been accidentally interoperable with your request and response, but there’s power in that kind of accident and the frequency with which it happens.

Practicality

All of these interoperability methods have merit, and most systems are built out of a combination of all of them in one way or another. When we’re defining what interoperability means, we always need to take in the context of what is interoperable, for whom, and when.At the end of the day, practical interoperability means that things connect well enough to get stuff done. We should endeavor to build our standards and systems to allow for robust discovery and negotiation, but always keep in mind that developers will find the best path for them to connect.

Interoperability is a grand goal indeed, and while a lot of the time we stumble backwards into it, there are well-trodden paths for getting there.

Maybe it’s time to rethink what accounts and federation even mean. And for that, I’ve been thinking lately about bubbles.

Bubbles

A bubble is an interesting physical construct. A thin boundary that surrounds its contents. What’s part of the bubble is separated from what’s around the bubble by that boundary, creating a sub-entity that only exists and makes sense in a larger space.

In classical computer network systems, we liked putting hard boundaries up around things. Firewalls and even physical network cable separation let us define what was “in” and “out”. If you were on the network, you had to be good, because we made sure of that before you were allowed on the network.

While these approaches still have their uses, today we know that it’s not enough. Devices cross network boundaries all the time, and even our trusted API services dart around cloud data centers, pulling pieces and components and data from places we can’t fully predict and plan ahead of time. So the world has moved to a zero-trust approach, where we build security into every part of the architecture instead of assuming it’ll get stopped at the door. That is a very good thing, but it also turns out to not quite be enough.

Organizations want to be able to control policies in a central fashion, to define accounts and access in one dashboard. But on the other hand, cloud services need to make extremely local decisions in potentially ephemeral environments — do I start X image on Y network to process Z request? This dichotomy creates a tension in the system that only increases when you start to cross network and trust domain boundaries.

We need to address this, and that’s why I’ve been thinking about bubbles. Bubbles provide context, boundaries, and differentiation, and that’s what I think we need to consider in our identity and security systems.

Within a bubble, there’s a certain degree of trust, like the set of services within a pod on a shared virtual network. I can make hyper-local decisions about access and security, and I can do it in a way that doesn’t require me to talk outside the bubble. I can authenticate users directly, I can check policies that only apply to my systems, I can identify software and hardware, and I can do that all within the comfort of the bubble. It’s a small world that I can see and control. The bubble offers safety and familiarity, and we like that.

Foam

Bubbles don’t exist on their own, though, and that’s where the classical network thinking breaks down. It would be easy to say that a bubble is just a regular local network, disconnected from the world, but that doesn’t solve the most interesting problems. A single bubble might be a helpful concept, but it’s when we get a lot of bubbles together that things really get exciting, if you ask me.

Let’s take a user account as our example. We tend to think of an account as having a specific home, an authoritative source that binds the attributes and authenticators to a person (or other entity, if we’re feeling spicy). But those attributes and authenticators often come from somewhere, and that’s where the bubble concept really starts to shine.

I’m not just talking about just-in-time provisioning, where a central account database points to a new bubble and says “here is everyone that’s supposed to be in that bubble”. I do think that this kind of push is an important tool, but it can’t be the only one. Any developer will tell you that even the best distributed and delegated systems tend to accrete things like local accounts, group accounts, admin passwords, service keys, and other techniques that solve specific problems.

Instead of trying to engineer those kinds of things away, the bubble concept embraces the local-ness of them. Within the bubble, we want to authenticate and make our decisions locally. This lets us be fast and robust, and build in the kinds of things that we only care about here. But how do we balance that need against usability and account control?

When I’ve got a user in my system, they’ve got an account that exists only within the bubble. They can authenticate locally to an IdP or AS that is in charge of all the services within that bubble. The account has attributes and access controls that might only make sense to systems inside the bubble. But it would be obnoxious for a user to create a new local account by hand every time they needed to do something, even though that’s how we have solved this kind of thing in the past. This is the strength of federation technologies like OpenID Connect and credentialing systems like Verifiable Credentials — a user can show up and point back to an authoritative source for their attributes, and get an authenticated session out of the deal. We can use these to get the user inside the bubble, but instead of using these to log in every time, these technologies can be used to instantiate and bind an account within the bubble. From that point forward, the user can authenticate locally. If at any time in the future we need to verify the account with its source, we can re-run the same items we used at ingest.

And importantly, a bubble should be allowed to trust any other bubble as a source for information. There can be no strict hierarchy in reality. When my bubble is out in the world, I might have a very clear and immediate business need to trust people, information, and services from another bubble that I didn’t know about a few minutes ago. I should be able to make a local decision to trust a user that came from that bubble and bind them to an account in my own bubble.

This trust is also not linear. A user could have accounts, credentials, and artifacts from multiple places. It’s a common requirement in identity proofing to present evidence from multiple sources that corroborate your claims. In the same fashion, a user might show up with a VC from one issuer and an OIDC login from a different place. The combination of those things can be meaningful to the bubble in a unique way.

As the foam of distinct bubbles grows, it’s important to be able to trace provenance from these different bubbles. In our local policies, we need a way to say that “This is user A, they came to us from cloud B, but cloud B said they originally came from both cloud C and cloud D”, and be able to verify that chain. And since our bubble could be the authoritative source for some other bubble, we need a way to talk about that kind of chain downstream. These kinds of durable provenance artifacts aren’t simple, and they bring with them a whole host of privacy concerns — can an adversary use this to track an individual through a system that doesn’t want to be tracked? Can I troll around the network of bubbles and correlate all these points? It’s clear that being able to selectively disclose the provenance chain, as well as the data about the user themselves.

Turtles

A bubble can provide context for another bubble to exist. The bubbles can share part of their barriers, allowing specific trusted information to pass freely between them in a way that the external walls don’t, but stopping other information.

A bubble could also exist entirely within another bubble, acting as a sub-division. If we really do care about zero-trust security architectures, we’ll make our bubbles as small as possible and composable so we can stack concerns.

We’re starting to have this conversation on the IETF’s new Workload Identity for Multi-System Environments (WIMSE) mailing list. So if that kind of thing interests you, please join us there.

Flow

Bubbles offer us another helpful metaphor, in that they aren’t static. When you’ve got a bunch of bubbles floating around, they really don’t like to stay still. They move, connect, disconnect, and eventually even pop.

In a classical multi-lateral federation, the focus is always on getting all the online systems to talk to each other in a way that makes sense for the moment. A new university just signed the agreement? Get them into the directory! We’ve got a new deployable unit that’s heading into the field? Write down the serial number on that router and out the door you go.

But once we’re up and running, things change. New parties could get brought on locally, new services get pulled in as needed, and the entities that we once relied on cease to exist.

Disaster response gives us a great example here. In this case, we want to be able to stand up a new bubble of services in response to a specific event, time, and place. Specialists come in with qualifications. Some of these we can verify — you’re a doctor, I can see your medical license is good right now, triage tent is that way. You’re an electrician, your union card looks good, go jump on that truck. But sometimes people show up to help, and their presence in the situation is enough to warrant giving them access. You used to be a firefighter, ok grab an axe and make a firebreak up on that hill. You can answer phones and direct calls? Great, have a seat, here’s the switchboard and directory. We have a mix of accounts that get provisioned from the outside — the doctors and union electricians — and accounts that get provisioned locally — the firefighter and switchboard operator. All of these accounts are valuable for different reasons, and our systems need to have that level of flexibility.

And eventually, the disaster is over, and we need to clean up our data as much as the physical mess of the disaster. That firefighter went and accessed a bunch of stuff, were they supposed to see that? Those union electrical workers, were they actively in the union with valid licenses while they were down here, or had some of them been kicked out? And depending on what they did, does that matter to us? We need auditability and accountability for dynamic systems like this. We need to be able to call back to the union and say “hey someone used their credential from you and we let them do the following kinds of things, are you OK with that and are we OK with that?” It’s not an easy set of questions to answer, and it gets even more complex when we start chaining our systems together in unexpected ways.

These bubbles can also disconnect and re-connect to the greater foam. This is the value of the hyper-local decisions — once you’re on board, I don’t need to see your IdP all the time in order for you to log in. So if we go offline for a while, or your IdP goes offline for a while, that’s OK. But once we’re back online, I might want to check in with your IdP, especially if you’ve done something fishy inside my bubble. Cross-domain trust should come with cross-domain accountability.

It’s Not A Technology

I truly believe that no one technology will solve this, for the simple reason that we will never get the online world to agree to one standard to address things, no matter how good it is or how much nerds love it. Any solution that requires us all to speak the same single protocol is doomed to failure.

Reality is heterogeneous, and we need to build for that heterogeneous world. The real value, then, comes in defining the interconnects. Interoperability occurs for a purpose and in a context. I believe that we can use a family of standards and technologies in a common pattern to build our the future of internet connectivity.

As a consequence, in this space I see room for OpenID Connect, Verifiable Credentials, SCIM, Shared Signaling, TLS, SPIFFE, FIDO, and many other moving parts. The bubbles should provide a common set of connection points into the larger foam in which they exist. Not every bubble is going to use the same connection points, but each point provides a specific set of functionality and addresses a specific problem. Even inside the bubbles there’s room for a lot of flexibility and innovation — how do I connect and verify my internal services, how do I spin up subsystems, how do I know who’s there in the first place?

Some of you reading might be expecting the bottom of this article to be a pitch of my new stealthy start-up that solves all these problems with some magic product I’m trying to sell you, but I’m sorry to disappoint your CTO that it’s not going to just come off the shelf. In all truth, I don’t know exactly what this solution looks like, but I’m eager to start building it to see what’s out there.

How It Started

Back in what feels like the long-distant past, I started a research project at my then-employer to investigate federated identity and how it could be applied in our environments.

We started out by building a very slapdash but functional implementation of OpenID 2.0 (and its various extensions, like SREG and AX), and we called it MITREid because I’m not great at coming up with good names. This little experimental server got hooked into our enterprise identity management system, and it did a pretty good job at giving us a single-sign-on experience that wasn’t just prompting for the user’s password and checking against LDAP, like every other app at the time had been doing.

When OAuth 2.0 and OpenID Connect started development, I proposed a research project that would get us involved in the creation of those standards and come up with an implementation of them that we could use ourselves and give back to the community as an open source project. The result of this was the MITREid Connect project, with its flagship OpenID Connect IdP and OAuth 2.0 AS built as a Java Spring application. And as you can see, my skill at naming projects continues to shine here.

How It Went

We built MITREid Connect to fit into much the same kind of environment that the original MITREid OpenID 2.0 server.

• No internal user management, this would be handled by the corporate identity management system and plugged in through a service layer
• Pluggable authentication, so we could adapt it to whatever the corporate system and our customer environments had in place already
• Strict adherence to the standards, and a decent attempt to implement as much of them as we could manage
• Deployable in a Java servlet container, which is the kind of deployment we had readily available to us at the time

All of these goals shaped the project that came out of the research effort, and it was decently successful for its time. We released several major revisions of the project, implemented new-fangled security mechanisms like PKCE, and deployed it to both our corporate network and a number of other systems. It gave us login, API access, and a common security layer right at a time when things were shifting away from HTTP Basic Auth for everything.

When the OpenID Foundation started to make their conformance testing available, we were among the first to certify and submit our results. I even have the t-shirt to prove this!

While I was still an employee at MITRE, the MITREid Connect project was funded largely through the research program that started it. When I first founded Bespoke Engineering and started consulting full time, I fully expected that support, development, and deployment of MITREid Connect was going to be one of my major business lines, and that this would help the project continue. This was slightly true at first, with several large clients bringing me on board to help work on larger projects that were making use of MITREid Connect. However, most of my time on these projects was spent not on the MITREid Connect code itself, but on separate or larger parts of the overall project.

Not long after I left MITRE, some folks at MIT offered to host the project as part of a new consortium. This was a good relationship, but they couldn’t offer the kind of support for an engineering team that the MITRE research program once afforded. That consortium eventually folded into something else, and the MITREid Connect project was left on a shelf.

Eventually, even these lines drifted away, and my time was taken up by other efforts, and MITREid Connect became something I engaged less and less with. The handful of people that had worked on it even had plans to re-brand the project away from its MITRE roots, but even that small step never came to fruition.

Somewhere in that time, the world of computing and security shifted radically. MITREid Connect’s model of a heavyweight Java servlet no longer fit with the cloud-native hosted or even semi-hosted services that people are interested in using. Where once it was common for a large company to have its own data centers for computation power, it now seems crazy to not use a hosted cloud environment for your infrastructure. MITREid Connect was made for an on-premises perimeter-driven world, and that world has largely gone.

How It’s Going

MITREid Connect never really made the transition out of that world, and it shows in the code. It’s more than just its packaging as a Java servlet, it’s how sessions and data storage are handled, how scaling would function, and so on and so on. The dependencies stopped being updated, it was never repackaged — at least not by the core development crew — and it was never really brought up to speed with the rest of the world.

These days, MITREid Connect is not actively developed or maintained by any of the core developers, including myself. The main reason for this is simple: the people who were willing to pay for the work stopped asking for it and turned to other solutions. Most groups, including MITRE, have turned to cloud-hosted solutions to handle their entire identity stack. Or they’ve moved to other implementations, both open source and proprietary.

Even so, I wouldn’t call the project abandoned, exactly. It still exists and it still runs, and people still use it. But the core project’s dependencies haven’t been updated in quite some time and there’s nobody in the queue to manage that. The project has been forked a number of times, and some of those forks look like they’re still active, but I honestly don’t know which one I’d point someone to.

In conclusion, if you’ve found MITREid Connect and it works for you, that’s awesome, go to town! It’s immensely customizable and configurable, and the core is stable, solid, and thoroughly tested in more implementations than I know of. But if you’re looking for it to do something new, or to be deployed in your cloud container, I have to say you might want to look towards something else for a solution.

Our Goals

GNAP is a big protocol, and there was no reasonable way for us to build out literally every piece and option of it in our limited timeframe. While GNAP’s transaction negotiation patterns make the protocol fail gracefully when two sides don’t have matching features, we wanted to aim for success. As a consequence, we decided to focus on a few key interoperability points:

• HTTP Message Signatures for key proofing, with Content Digest for protecting the body of POST messages.
• Redirect-based interaction, to get there and back.
• Dynamic keys, not relying on pre-registration at the AS.
• Single access tokens.

While some of the components built out did support additional features, these were the ones we chose as a baseline to make everything work as best as it could. We laid out our goals to get these components to talk to each other in increasingly complete layers.

Our goal of the hackathon wasn’t just to create code, we wanted to replicate a developer’s experience when approaching GNAP for the first time. Wherever possible, we tried to use libraries to cover existing functionality, including HTTP Signatures, cryptographic primitives, and HTTP Structured Fields. We also used the existing XYZ Java implementation of GNAP to test things out.

New Clients

With all of this in hand, we set about building some clients from scratch. Since we had a functioning AS to build against, focusing on the clients allowed us to address different platforms and languages than we otherwise had. We settled on three very different kinds of client software:

• A single page application, written in JavaScript with no backend components.
• A command line application, written in PHP.
• A web application, written in PHP.

By the end of the weekend, we were able to get all three of these working, and the demonstration results are available as part of the hackathon readout. This might not seem like much, but the core functionality of all three clients was written completely from scratch, including the HTTP Signatures implementation.

Getting Over the Hump

Importantly, we also tried to work in such a way that the different components could be abstracted out after the fact. While we could have written very GNAP-specific code to handle the key handling and signing, we opted to instead create generic functions that could sign and present any HTTP message. This decision had two effects.

First, once we had the signature method working, the rest of the GNAP implementation went very, very quickly. GNAP is designed in such a way as to leverage HTTP, JSON, and security layers like HTTP Message Signatures as much as it can. What this meant meant for us during implementation is that getting the actual GNAP exchange to happen was a simple set of HTTP calls and JSON objects. All the layers did their job appropriately, keeping abstractions from leaking between them.

Second, this will give us a chance to extract the HTTP Message Signature code into truly generic libraries across different languages. HTTP Message Signatures is used in places other than GNAP, and so a GNAP implementor is going to want to use a dedicated library for this core function instead of having to write their own like we did.

We had a similar reaction to elements like structured field libraries, which helped with serialization and message-building, and cryptographic functions. As HTTP Message Signatures in particular gets built out more across different ecosystems, we’ll see more and more support for fundamental tooling.

Bug Fixes

Another important part of the hackathon was the discovery and patching of bugs in the existing XYZ authorization server and Java Servlet web-based client code. At the beginning of the weekend, these pieces of software worked with each other. However, it became quickly apparent that there were a number of issues and assumptions in the implementation. Finding things like this is one of the best things that can come out of a hackathon — by putting different code from different developers against each other, you can figure out where code is weak, and sometimes, where the specification itself is unclear.

Constructing the Layers

Probably the most valuable outcome of the hackathon, besides the working code itself, is a concrete appreciation of how clear the spec is from the eyes of someone trying to build to it. We came out of the weekend with a number of improvements that need to be made to GNAP and HTTP Message Signatures, but also ideas on what additional developer support there should be in the community at large. These things will be produced and incorporated over time, and hopefully make the GNAP ecosystem brighter and stronger as a result.

In the end, a specification isn’t real unless you have running code to prove it. Even more if people can use that code in their own systems to get real work done. GNAP, like most standards, is just a layer in the internet stack. It builds on technologies and technologies will be built on it.

Our first hackathon experience has shown this to be a pretty solid layer. Come, build with us!