Lockstep

Mobile: +61 (0) 414 488 851
Email: swilson@lockstep.com.au

I just don't get Levels of Assurance

IDAM practitioners and government authentication policy makers have settled on a generic way to categorise transaction risk and match it to a broad measure of authentication quality. The idea is to characterise the seriousness of a transaction in terms of "Levels of Assurance" (LOAs) and then match the authentication 'level' of the party you're planning to do business with. LOA schemas are codified in NIST SP 800-63 and described (more loosely) in the Australian National Electronic Assurance Framework (NEAF).

The idea of LOAs can be traced to risk management methodologies and standards like ISO 31000. These approaches involve gauging both the severity and frequency of anticipated adverse events, and combining those metrics to create a rolled-up risk rating for each event on an ordinal scale, like {Negligible, Low, Medium, High, Extreme}. Examples given in the NEAF documentation use severity-frequency tables lifted straight out of the older ustraian risk management standard AS/NZ 4360; see Table 3, p15 of the NEAF Framework document (PDF)).

A powerful feature of modern risk management standards is that each enterprise is empowered (in fact expected) to customise the way it assesses adverse events, in the context of its particular environment. Severity can be gauged in different ways, for example by referencing monetary losses, health consequences, political impact and so on; the most appropriate frame will depend on the business environment. Organisations also set their own policies for what level of risk is acceptable for each anticipated threat. So some will not tolerate residual risks that are worse than Low, while others will live with Medium risks on a case-by-case basis with special contingency plans. Good risk management standards allow that different organisations have different risk appetites.

But what LOA advocates seem to forget is that, as a result, risk determinations made under ISO 31000 and the like are not transferable between organisations. Simply saying that a certain event (for example compromise to a user account) has a risk rating of “Medium” tells someone outside the organisation nothing at all about the details of the threat, its impacts and expected likelihood.

And yet the Levels of Assurance paradigm has us pick and choose externally issued identities based on a generic ratings of LOA 1, 2, 3 or 4. There cannot be any certainty that all "LOA 3" credentials for instance are equivalent, nor that they will satisfy the detailed needs of all Relying Parties conducting "LOA 3" transactions.

In other words, you cannot pigeon hole risk.

I've seen repeatedly a silly situation where Relying Parties looking for say LOA 2 aren't quite satisfied with a given Identity Provider's idea of LOA 2, and they seek to haggle over a special "level two and a bit". Generic LOAs are supposed to save time with generic levels, but in reality, RPs and IdPs still spend a great deal of time on local risk assessment, hammering out their authentication arrangement. That work is entirely appropriate, but the thing is, the idealised LOA bucket becomes irrelevant. Pigeon-holing risk doesn't save time, and it can't save anyone from having to do detailed case-by-case risk analysis.

Here's the source of the mismatch. The idea of discrete standardised LOAs was based on schemas designed to measure and speificy risk within organisations. These standards were never meant for communicating risk assessments between organisations. Discrete assurance levels are fundamentally misleading, for they lead people to oversimplify the necessary matching of Identity Providers's offerings to Relying Parties' needs.

Posted in Security, Internet, Identity, Fraud, Federated Identity

Comments

Andrew JohnstonFri 16 Aug 2013, 1:13pm

Agreed! Let us use dollars (or euros, or yuan, or ...) instead. IdP X asserts Y about this subject, indemnifying RP Z of resultant damages to a maximum of $Q.

This leaves the RP in a solid, quantifiable position to make its own risk choices.

This also provides a mechanism for the IdP to communicate its confidence in its assertions in a way that is independent of technology. This simplifies the interface between IdP and RP to a lingua franca that enables discussion with managers, accountants, lawyers and shareholders.

Ken DaggThu 24 Jul 2014, 1:39am

While I agree that "definitive assurance that all "LOA 3" credentials issued by all IdPs are not equivalent" and that "they will not satisfy the detailed needs of all Relying Parties conducting "LOA 3" transactions" I wonder what the alternative might be. In my opinion something is needed to kick start the process and get RPs and IdPs out of the point-to-point model that perpetuates the "I'm different and unique" mode of operation.

Again, in my opinion, the understanding is probably "better" in terms of general agreement within a defined federation rather tha universally.

This being said, I also believe that as the the use of IdPs by RPs evolves and matures that the definition of LOAs and possibly the number will also increase. However, other than the current "point-to-point" model, which in my opinion is not scalable, the use of a defined set of LOAs to first categorize risk and then describe mitigation mechanisms is the only way to get started.

I also believe that the use of LOAs will not start to happen until RPs in a federation get involved in their definition. The definitions provided in 800-63 were done for the US government federation. The Government of Canada as well as the UK government have developed their own definitions for their federations. I do not believe that these definitions are applicable, as much as some vendors would like to make them so, to federations universally.

Richard MardlingThu 24 Jul 2014, 5:49pm

An interesting set of arguments that could spawn many hours worth of valuable debate.

In the UK there is GPG45 (http://bit.ly/1ofIt3K)which sets out the requirements for LOAs. It's easier to impose the requirements within a single federation than across federations. An issue that I've seen quite often is the many and varied classifications of data within an RP, so what would require an LOA2 in one organisation is an LOA3 in another. This causes confusion at best to the user and frustration at the other end of the continuum.

My point in the tweet - http://bit.ly/1kdkfaT - was that the LOA just tells you the checks the person has undertaken and (hopefully) the strength of authentication credentials. What it doesn't tell you is where the person is, what device they're using, what network they're using, what is the time of day where they are connecting from, is their security clearance current, is their training record up to date etc.?

As such LOA is fairly coarse and to make a balanced grant/deny decision that is the gift of the RP as it's their data, I consider the LOA as just one of the contextual attributes to consider.

Stephen WilsonThu 24 Jul 2014, 6:34pm

Thanks Ken.

Just a quick response from me right now. [It looks like we may have lots of ways to debate this in coming months, if the new NSTIC CTO Paul Grassi does convene discussion and feedback on the excellent question he put to the Cloud Identity Summit: 'What if we got rid of LOAs?'; see https://twitter.com/Steve_Lockstep/statuses/490925951627169792]

You say RPs and IdPs need to get out of the "I'm different and unique" mode of operation. I'd like to agree, but LOAs are too big a first step, for they mean in effect that "We are almost all the same".

The story of enterprise IT risk management for twenty years - culminating in ISO 31000 - has been the philosophy that risk management begins at home. All big projects are supposed to do a TRA; all TRAs are supposed to start with custom analysis of the information assets, threats and risks that are particular to the business. So the philosophy is that all organisations really ARE different!

If we are going to move towards RPs being more similar, then we need to make that transition much more slowly than imposing four generic risk bins.

And really, we all know LOAs are not working. Haven't we all seen IdPs and RPs arguing whether 'My LOA 3 equals your LOA 2.25?' [Ref: Ian Glazer's CIS 2014 keynote, https://www.tuesdaynight.org/2014/07/23/do-we-have-a-round-wheel-yet-musings-on-identity-standards-part-1.html]. In my view, as soon as RPs and IdPs start negotiating LOAs, the game is up! That negotiation is tantamount to Policy Mapping, and people thought federation frameworks and LOAs were the END of policy mapping.

Several years ago, one of NIST's PKI doyens said to me: "Cross-certification and policy mapping has been a rat hole that has sucked up vast amounts of energy better spent elsewhere". See also http://lockstep.com.au/library/pki/public_key_superstructure.

For LOAs to work, they need to seriously save people from more policy mapping. It proves a lot easier and cheaper to continue with bilateral IdP-RP arrangements (yes, each one a "special case" if you will) than it is to create generic multilateral arrangements when the policy mapping cannot be put to bed.

Ken DaggThu 24 Jul 2014, 10:05pm

I fully endorse the use of TRAs for each service that a RP makes available to a client whether that service is digital or manual. It puts the onus on the RP to identity the risk if something goes wrong with their use of attributes to deliver the right amount of service to an eligible client.

That being said, and to Richards point, the assertions provided by an IdP are one mechanism to offset that risk. In my opinion the assertion should NOT be the only mechanism. Mechanisms that perform other checks like location, network used, etc can also be used at the RP's discretion. An assertion at a LOA is not a silver bullet solution to all a RP's service delivery / security risks.

I'm not clear which people you are referencing with respect to additional policy mapping - I will assume it is the RP. If so, I disagree that they shouldn't have to do the policy mapping. They need to determine how much residual risk exists and how they are going to mitigate that risk.

When I was with the Governemnt of Canada we only provide assertions on tokens at LOA2. RPs were only allowed to use that assertion and had to implement any other mechanisms they felt were required in order to meet their TRA requirements. These mechanisms included checking additional "what you know" and "who you are" factors.

Finally, the fact that one organization says that they need a LOA2 assertion and another says they need a LOA3 is reality (whether digital or manual service delivery) even if, to the client, both organizations are collecting the same data or even if they appear to be offering the same service. That is the RP's call on one side and the client's decision on the other. As far as I am aware clients can, other than for government services, choose which supplier to obtain services from.

Stephen WilsonFri 25 Jul 2014, 2:06am

I don't mind that "one organization says that they need a LOA2 assertion and another says they need a LOA3". Not at all. My criticism is that when two organisations both settled on LOA 2, they have different ideas what that entails. That is, an RP works out it needs LOA 2 in a certain scheme like the UK IDAP, and then it short-lists the IdPs in that scheme providing credentials at LOA 2, and then the RP looks into the fine print, and becomes uncomfortable with the actual proofing processes of an IdP, and asks for more. I have actually seen RPs ask for "LOA 2.5".

So maybe you're right - the RP should accept the LOA 2 credential for what it is and then look for extra signals if more assurance is warranted.

My beef is this is not what federation has been sold to be. It's said to be much simpler. The executive slide decks and business cases tell businesses that they (as RPs) will be able to accept strangers based on "identities" issued by "IdPs" at generic Levels of Assurance. It's a terrible over-simplification. LOAs do not remove the need for:
1. local TRAs by RPs,
2. Policy mapping of what a given IdP's ID proofing processes really are in detail against what the RP believes it needs
3. extra authentication steps (or "trust elevation").

When you have to do all this work, the meaning of "LOA" rather disappears.

Stephen WilsonFri 25 Jul 2014, 2:21am

Thanks Ken.

Just a quick response from me right now. [It looks like we may have lots of ways to debate this in coming months, if the new NSTIC CTO Paul Grassi does convene discussion and feedback on the excellent question he put to the Cloud Identity Summit: 'What if we got rid of LOAs?'; see https://twitter.com/Steve_Lockstep/statuses/490925951627169792]

You say RPs and IdPs need to get out of the "I'm different and unique" mode of operation. I'd like to agree, but LOAs are too big a first step, for they mean in effect that "We are almost all the same".

The story of enterprise IT risk management for twenty years - culminating in ISO 31000 - has been the philosophy that risk management begins at home. All big projects are supposed to do a TRA; all TRAs are supposed to start with custom analysis of the information assets, threats and risks that are particular to the business. So the philosophy is that all organisations really ARE different!

If we are going to move towards RPs being more similar, then we need to make that transition much more slowly than imposing four generic risk bins.

And really, we all know LOAs are not working. Haven't we all seen IdPs and RPs arguing whether 'My LOA 3 equals your LOA 2.25?' [Ref: Ian Glazer's CIS 2014 keynote, https://www.tuesdaynight.org/2014/07/23/do-we-have-a-round-wheel-yet-musings-on-identity-standards-part-1.html]. In my view, as soon as RPs and IdPs start negotiating LOAs, the game is up! That negotiation is tantamount to Policy Mapping, and people thought federation frameworks and LOAs were the END of policy mapping.

Several years ago, one of NIST's PKI doyens said to me: "Cross-certification and policy mapping has been a rat hole that has sucked up vast amounts of energy better spent elsewhere". See also http://lockstep.com.au/library/pki/public_key_superstructure.

For LOAs to work, they need to seriously save people from more policy mapping. It proves a lot easier and cheaper to continue with bilateral IdP-RP arrangements (yes, each one a "special case" if you will) than it is to create generic multilateral arrangements when the policy mapping cannot be put to bed.

Ken DaggMon 28 Jul 2014, 6:48am

I think we are, for the most part, in agreement.

The difference, I believe, is in what an RP is expecting from an assertion at a specific LOA. The process that I believe we both believe should be followed starts with a RP undertaking a TRA. From that assessment the RP identifies their identity authentication requirements. The RP can, if it so desires at this point, classify this requirement as a specific LOA. In my mind, this classification is purely for communication purposes (especially to executives when trying to justify the expenditure required to implement a solution).

As we have discussed before the assertion provided by an IdP should only be looked at as one of the mechanisms the RP uses to mitigate the authentication requirements they have identified. In looking at how well the assertion meets their requirements the RP needs to understand the checks that the IdP has performed in order to be able to make the assertion they are making. Use of a Trust Framework such as the Kantara IAF makes this some what easier as the Service Assessment Criteria are well documented. In some cases a RP may, after examining the checks that have been performed, determine that the IdP's assertion is all they require. In other cases they may determine that they need additional mechanisms. Regardless, the discussion around liability is, in my opinion, easier if this model is used. The IdP is required to perform the checks they say they have performed - nothing more and nothing less. If they can be shown to have not performed the checks then they are liable for damages. If they have performed the checks the the RP is liable for damages.

The emphasis in the approach I just described is on the RP to drive the process. I agree with you that RP executives may not have been told the whole story or may be misinterpreting what they need to do when they use an assertion from an IdP. I don't know who has been leading them astray. Regardless, since the story has been misinterpreted, expectations need to be reset. I know, when I worked for the Government of Canada, we spent a significant effort in communicating the approach I described above. It took a while for the message to be understood but, once it was, they understood how they could use the mandatory LOA2 token assertion that the government had contracted and what they still had to do to ensure that their identity authentication requirements are being satisfied.

Robin WiltonSun 5 Oct 2014, 7:31pm

LOAs are what happens when actuaries get in on the authentication game...

Post a comment

If you are a registered user, Please click here to Sign In

Your Name*

Your Email Address* required, but won't be displayed on this site

To help prevent spam in our blog comments, please type in "Levels" (without the quotation marks) below*