Updated August 17, 2022.
IDAM practitioners and government authentication policy makers have settled on a generic way to categorise transaction risk and match it to a broad measure of authentication quality. The idea is to characterise the seriousness of a transaction in terms of “Levels of Assurance” (LOAs) and then match the authentication ‘level’ of the party you’re planning to do business with. LOA schemas are codified in NIST SP 800-63 and described (more loosely) in Australia’s National Electronic Assurance Framework (NEAF, archived here).
The idea of LOAs can be traced to risk management methodologies and standards like ISO 31000.These approaches involve gauging both the severity and frequency of anticipated adverse events, and combining those metrics to create a rolled-up risk rating for each event on an ordinal scale, like {Negligible, Low, Medium, High, Extreme}.Examples given in the NEAF documentation use severity-frequency tables lifted straight out of the older Australian risk management standard AS/NZ 4360; see Table 3, p15 of the NEAF Framework document (PDF)).
A powerful feature of modern risk management standards is that each enterprise is empowered (in fact expected) to customise the way it assesses adverse events, in the context of its particular environment. Severity can be gauged in different ways, for example by referencing monetary losses, health consequences, political impact and so on; the most appropriate frame will depend on the business environment. Organisations also set their own policies for what level of risk is acceptable for each anticipated threat. So some will not tolerate residual risks that are worse than Low, while others will live with Medium risks on a case-by-case basis with special contingency plans. Good risk management standards allow that different organisations have different risk appetites.
But what LOA advocates seem to forget is that, as a result, risk determinations made under ISO 31000 and the like are not transferable between organisations. Simply saying that a certain event (for example compromise to a user account) has a risk rating of “Medium” tells someone outside the organisation nothing at all about the details of the threat, its impacts and expected likelihood.
And yet the Levels of Assurance paradigm has us pick and choose externally issued identities based on a generic ratings of LOA 1, 2, 3 or 4.There cannot be any certainty that all “LOA 3” credentials for instance are equivalent, nor that they will satisfy the detailed needs of all Relying Parties conducting “LOA 3” transactions.
In other words, you cannot pigeon hole risk.
I’ve seen repeatedly a silly situation where Relying Parties looking for say LOA 2 aren’t quite satisfied with a given Identity Provider’s idea of LOA 2, and they seek to haggle over a special “level two and a bit”. Generic LOAs are supposed to save time with generic levels, but in reality, RPs and IdPs still spend a great deal of time on local risk assessment, hammering out their authentication arrangement. That work is entirely appropriate, but the thing is, the idealised LOA bucket becomes irrelevant. Pigeon-holing risk doesn’t save time, and it can’t save anyone from having to do detailed case-by-case risk analysis.
Here’s the source of the mismatch. The idea of discrete standardised LOAs was based on schemas designed to measure and specify risk within organisations. These standards were never meant for communicating risk assessments between organisations. Discrete assurance levels are fundamentally misleading, for they lead people to oversimplify the necessary matching of Identity Providers’ offerings to Relying Parties’ needs.