Why SLAs fall short – and what can be done about it? (Part 2 of 2)

In part one of this post, we examined the challenge of discussing IT demand in terms meaningful to our internal customers. That accomplished, the CIO’s organization must next fulfill that demand by acquiring, integrating and delivering the appropriate service(s), whether sourced internally or from the marketplace. Imagine for a moment the perfect world, one in which we would be able order the supply-side components of an IT solution where each provider would stand behind the complete realization of an intended outcome (for example, a provider of midrange server operations would put its fees at risk if the total IT solution didn’t, say, increase inventory turns). Back on the ground here in Kansas, however, we recognize that no party providing much less than a total solution (business process and underlying capabilities such as people and technology) would be willing to sign up for a business result. Furthermore, the current trend towards multi-sourcing puts such a total solution (and the business outcome coverage) even farther out of reach. And if the provider of one solution component would commit to the entire business outcome, would we really have faith in that guarantee anyway?

So on the supply side, we tend to be left with “traditional” service level agreements (SLAs), measuring the elements IT performance. Now, that’s not all bad. If we understand (and the provider can perform to) such SLAs, we should in theory be able to architect a solution based on the sum of those individual components. But theory seems to be failing miserably… so why don’t SLAs work as well as they should?

While there can be many shortcomings to SLAs, some are not so obvious. Most SLAs, take server availability, for example, adequately describe the level of quality we require for that part of the solution and are also useful in measuring the performance of the provider. But what if the production problem was a failure of redundant load balancers (yes, this really happened)? Oops –didn’t think to make that an SLA (hey, it was redundant!) — and the service provider gets off scot-free while the customer is angry and frustrated. Or how about the ability of the service provider to onboard the right project resources (e.g., skills, seniority) in the right timeframe. Believe it or not, I know of a case where the timeframe for getting a particular resource is over a year … and counting! Did the customer have that SLA in the contract? No, but access to specialized skills is a realistic expectation of a Tier 1 service provider and a factor that materially contributes to successfully meeting demand. The point is we can’t measure (and don’t want to try to measure) all aspects of a provider’s performance that may possibly impact the IT service (and hence, the business outcome). What we want from our providers is just a good, reliable service – what we signed up for. So what can we do to get the results we expect, when the dimensions of performance are as often qualitative as they are quantitative? Some different thinking is in order!

One method I’ve used when structuring agreements is to create a “CIO Satisfaction SLA” that carries a significant financial consequence. Your first reaction might be that there is no way a service provider would agree to put revenues at risk under such a construct, because it’s too subjective. Certainly, I can understand that a skeptical account manager might imagine his/her CIO customer on the hunt for Q4 cost savings, “we need to reduce spend now… I’ll use the CIO Satisfaction Survey, whack service provider with financial club (thud) and mission accomplished.” That’s why we structure such SLAs carefully; if done on a quarterly basis, they relate only to the performance of the previous quarter…and they must be backed by data, whether balanced scorecard results, customer satisfaction survey trends, or service delivery reports. The CIO must meet with the service provider to present these results and discuss the rationale for applying the SLA credit.

Another aspect of the CIO Satisfaction SLA worth noting is that I use it also as the “earnback” mechanism for recovering previous SLA credits. Think about it, if you’re going to “excuse” a previous performance miss, is the service provider’s ability to deliver that same SLA for the next 6 months sufficient justification for allowing an earnback? After all, hitting the service level was what you paid for in the first place – why should that be the basis for a reward?! No, I would rather offer an incentive to my service provider for going beyond the call of duty. If I, as a CIO, get in a jam because I suddenly have to deliver an emergency change to meet a business need, and my service provider pulls out the stops to make it happen – that feels like the partnership behavior I would like to reward. The CIO Satisfaction SLA gives me the means to show my appreciation and foster a more collaborative spirit.

There are other techniques for improving the alignment of service provider performance to business outcomes that I could discuss (for example, using term extensions as a performance incentive), but in my dealings with clients, it’s frequently the SLAs that they don’t – or really can’t – measure that make the difference in whether or not IT delivers true business value. So I encourage you remember the importance of addressing the qualitative side performance, whether you use the CIO Satisfaction SLA or some other creative mechanism in your performance management toolkit.