Transforming IT Blog
Join us in the Transforming IT HP Blog where we will discuss reinventing IT to overcome obstacles and take advantage of Instant on Enterprise opportunities.

Should an SLA define what the customer wants or what you can measure?

I was involved in a discussion about SLAs on Twitter recently. 

 

Some people felt very strongly that if you can’t measure something then it should not be included in a service level agreement (SLA), even if the customer says that this is the most important thing that they want from the service. I disagreed because I think the most important thing to put in an SLA is a description of what your customer wants, even if you can’t measure it.

 

How can we resolve this, so that we provide SLAs with measureable targets that still address the unmeasureable things that customers have asked for?

 

The worst possible approach is to insist on an IT-centric view of services, where everything is defined in strictly measureable terms. I have been involved in a number of escalations where the IT service provider has met all of their targets, but the customer has not been happy. This has almost always been due to the customer having signed up to an SLA that didn’t really describe what they wanted, because the outcomes they cared about could not be easily measured.

 

One way of dealing with this issue was described in a blog by David Cannon (@ITILSO) recently. David’s suggestion is that you should start by defining the desired outcome (what the customer wants). Then identify the factors that will make that outcome possible, then define what is needed for those factors to be achieved, and keep going till you have a list of factors that you can measure and control. This approach is certainly better than ignoring the required outcome and just defining measureable IT metrics, but (as David notes) “You will discover some factors that cannot be controlled due to physical or business limitations” and also you will discover some factors that can’t be easily broken down into the exact conditions required to achieve them.

 

When I was thinking about this issue recently, I remembered similar situations from when I was bringing up my children. I expected them to behave properly, but I could not possibly list every single thing that I expected of them – even if I spent many hours on this they were quite capable of coming up with some new and unexpected behavior that I hadn’t thought to ban! This didn’t mean that they could do anything they wanted so long as they met the measureable criteria that I had defined. I told them that they had to behave properly and then gave them examples of the kinds of evidence that would demonstrate this. When we came to discuss their behavior we could look at the evidence, but the thing we had to agree about was the overall behavior, not the specific things that I had, or had not, told them to do. I never told my kids that they weren’t allowed to set fire to their beds, but that doesn’t mean that it wasn’t a requirement!

 

This same approach can be used to define service levels.

 

  1. Ask the customer what they want and write that down. For example “IT failures will not have a significant impact on the business” or “Responses to requests for new service features will be flexible and try to meet our changing needs”. These may not be measureable, but if the customer agrees that you have achieved them then they will be satisfied with the service.
  2. Think about what you can measure that could be used as evidence that you achieved the things the customer cares about. For example “Priority 1 incidents will be resolved within 4 hours” or “Requests for new service features will be responded to within 5 working days with an approximate price”. Discuss these targets with the customer and make sure that they agree that if you achieve the measureable targets they would find this acceptable. Your SLA now has two different types of statement: the things the customer really wants, that you can’t measure, and for each one of these a set of things you can measure that will provide evidence that you have delivered what was agreed. So far this is quite similar to David Cannon’s approach.
  3. Measure the agreed targets and provide data for use in customer reviews. These reviews typically take place once a month. Present the data about your achievements against measureable targets and ASK the customer “are you satisfied that we delivered the outcome for which this is the agreed evidence”. In other words you should be discussing the customer outcome, and using the measureable data as evidence to show what you achieved. The key point is that it is achievement of the outcome that matters; the measured data is just evidence. Sometimes the customer may be dissatisfied with the service even if you achieved all the targets. As a service provider you should use this as an opportunity to understand why the customer is not satisfied, and whether the targets (and the service) need to be improved. On other occasions the customer may be satisfied even though the numeric targets were not met. This is also an opportunity to improve the targets. The important points are:
  • Spend most of the time in customer review meetings talking about the agreed outcomes, not about the measureable targets.
  • If you failed to meet customer expectations for an outcome then accept this and work with the customer to understand how you can meet their expectations in future. Don’t hide behind the data and tell the customer they are wrong!

 

There has been a big change in IT Service Management over the last few years. It is no longer acceptable to take an IT-centric view of services. We must all understand how our services create value for our customers, and how everything we do contributes to that value. Based on that understanding we can make sure that we keep satisfied customers by delivering the services that they really want, not just the services that we know how to measure.

 

Don’t think that this doesn’t apply to you because of the type of services you offer, or the type of service provider you are. It applies equally to everyone. I have heard people working for outsourcing organizations say that they should never deliver more than they have specified in the measurable targets, and that to do so would undermine negotiations for upgrades and renewals. I think that this is exactly the wrong way to think about it. If you want your customers to renew their contracts, if you want them to recommend your services to others, if you want to win more of their business, then you absolutely must focus on customer outcomes, and not on measureable targets. I’m not saying that you should always agree to deliver more and better service for the same price, but that you should worry more about the customer outcomes than about the numbers that you can use as evidence that you met these.

 

I would like to measure how effective this blog is, so rather than counting page views and reply counts I am going to count the number of replies where people say that they have read this blog and it will influence how they define or measure their services – please reply to this blog if it has made a difference to your thinking about SLAs.

 

Learn more about HP Consulting Services and how HP can help you shift your focus from operation to innovation.

Stuart Rance

If you want ideas about how to start thinking strategically, then read some of my other blogs:

 

 

For more info about me and what I can do for your organization, see my profile on our Technology Services Experts page.

 

 

Follow StuartRance on Twitter.jpg  Tweet about this article.jpg  Share this article on LinkedIn.jpg

 

Tags: ITSM| SLA
Comments
Ian Robinson(anon) | ‎01-03-2013 10:47 PM

'Over delivering' may be frowned upon by some. However is this where we are seen to add the most value until it becomes the norm?

 

Should IT departments or IT practitioners define or measure the services they provide? Or should these be written and  measured by the customer? Who is the customer? The user? The organisation? The shareholders? The end consumer? And what do customers mean by value, or more importantly 'adding value' and how do customers measure it?

 

If we are going to measure something we should clearly define what we are measuring and what we are going to compare it to. Then we can investigate areas of concerns, make predictions and try to continually improve.

 

However to deliver real visible value through the best use of IT and people we need 'everyone' on board from the top down. Leave the measuring and reporting of business outcomes to those accountable and responsible within the business. Leave the IT professionals to measure and report on their own professionalism and how they contributed (and at what cost) to the bottom line. Or is there a better way?

 

 

 

 

Stuart_Rance ‎01-03-2013 10:52 PM - edited ‎01-04-2013 12:56 PM

Ian,

 

When I talk about customers, I mean the people who negotiate and pay for the service, and who the IT organization regularly report to.

 

I see many reports full of statistics that mean nothing to customers, and should have been kept as internal IT data to help the various IT managers manage their process improvement activities. Sometimes this is because the customer has insisted on inappropriate metrics being included in the SLA, but more often it is because IT people are scared to include what the customer really wants because they aren't sure how to deliver it, or how to measure that they have delivered what was wanted.

 

 

Stephanie Truckenmiller(anon) | ‎01-04-2013 02:17 PM

This was very insightful in how to go about measuring the customer experience. Our SLAs are very IT centric, up or down during hours that systems are agreed to be up. The approach to negotiating what to measure for evidence is definitely one I plan to leverage as we continue to build out new and review existing agreements.

It is difficult for IT people to to go from a fact based to a more feeling based approach. The idea of agreeing on what evidence can be used to help determine service value is simple and brilliant.

Stuart_Rance | ‎01-04-2013 02:19 PM

Stephanie,

 

Thank you for the feedback. I'm looking forward to running a workshop for you later this month, where we can discuss this idea further.

mtrofimiuk | ‎01-04-2013 03:43 PM

Stuart,
It is insightful and practical also for field guy who is spending quite some time on escalations for targets which were met.

 

+1 to your evidence file

Stuart_Rance | ‎01-04-2013 03:48 PM

mtrofimiuk

 

"+1 to your evidence file"

 

Thank you, I'm counting them all.

urbano.freitas | ‎01-04-2013 06:01 PM

In my perspective the alignment between IT and business is something people from both departments must always try to achieve.

IT don't exist alone, it's an enabler of the business, so it makes, to me, we must try have measures of the outcomes the IT allow to business. And this, definitely can only be achieved if a constant conversation between the both parts are conducted, so the necessary taxonomy, trust and comprehension can be develop.

 

However, sometimes business want SLAs of things that aren't only an IT outcome, so in my perspective, although as you propose, we, the IT guys, should work to give them, a SLA that expresses the most possible what they want, we can't/shouldn't assume the commitment of providing an SLA measurement that can be effectively measure, or which is not totally controlled/managed by IT.

David Cannon(anon) | ‎01-04-2013 06:04 PM

Good points Stuart!

 

As far as including things in the SLA that we cannot measure:

  • The customer WILL find a way to measure your service - reagrdless of what you include in your SLA
  • If what you include in your SLA is different from what they are measuring, the SLA will no longer be relevant and your credibility will no longer be intact - doesn't matter how much you argue about what metrics are feasible
  • It is impossible to determine whether a service is valuable if you cannot measure it.  Therefore, the customer had to have had an idea of what the service would contribute to them before they invested in it.  If you don't know what that is, the discussion around what should be in the SLA is academic
  • There is nothing wrong with putting the onus for measurement on the customer - in fact they are the only ones who can measure certain aspects of the service.  Your SLA should specify the customer's role in measuring the service.  This is nothing new and unprecedented.  In fact this is the way every service situation works.  Can a hotel measure how well you slept?  No.  But is that the way we measure our hotel experience?  Absolutely.  Personally, I know that if I am near the ice machine or elevator, I will not be sleeping well that night.  So it's up to me to make the hotel aware of those preferences.  In fact, I have it in my frequent guest profile so that the hotel and I have the best opportunity to satisfy my needs for a restful night.  Customer / Service Provider relationships work best when they both contribute to the relationship
Roger Fitch(anon) | ‎01-04-2013 09:57 PM

 

Currently my performance is measured on KPIs (customer satisfaction surveys),  and SLAs are used as a vehicle to set 'reasonable expectations' between internal customers and my team. There will always be unforeseen exceptions to any process and I believe my best value add in this role is to be able to triage my team workload against such events with my customers, using the back drop of a SLA i.e. we know what is reasonable , we know why we are being asked to exceed that, and it is known we are working harder to meet an unusual need.

 

I think metric laden SLAs only work when the metrics used are at a level both parties clearly understand e.g. Between an IT dept and an external Service Provider. A previous role saw me in 2 hour long conference calls every morning battling each SLA breach because of financial penalties. In that role I was prepared every day, knew each statistic, and the value I could add there was balancing the response to each customer challenge with customer need/relationship vs. financial penalties.

 

SLAs should be appropriate to context and expectations - there is no one size fits all solution. We/I/You should be looking to where we add value accordingly.

Sergey_Z(anon) | ‎01-04-2013 10:26 PM

SLA is a legal document.

 

It has to define a  customer and a service provider obligations. If a customer requires something specific, it should be not simply stated, but clearly described how that specific service is to be delivered.

 

On my experience, the  best SLA has to have a defining part (declaring a subject of service, scope of service, terms of service etc), a delivery description part (who is authorised for the service on the customer side, how and to whom to address the financial and technical questions, standard/non standard delivery service time, escalation procedures, priorities of the assistance request resolutions etc.) and a  reference part (the  delivery organisation key staff and responsibilities according to a SLA and customer employees SLA's responsibilities and way of contact)

 

It is obvious that SLA is a working document which should be adjusted every time when support contract is offered. The indication how SLA match to reality is the customer's feedback  which must be obtained by delivering organisation after the SLA expiration. Having the honest customer's satisfaction feedback would  let you to  improve a SLA itself, but more importantly, to improve service delivery procedures of the entire  delivery organisation.

Yvonne_Bentley | ‎01-04-2013 11:02 PM

Stuart,

 

Thanks for shining light on the concept of building relevant SLAs with jointly-agreed measures that, traced back from factors that contribute to customer-desired outcomes, serve as evidence in demonstrating successful service delivery.  This idea of metrics as evidence showing contribution to a customer-desired end instead of being an end in themselves is an important shift towards customer-focused and away from IT- focused services.

 

Yvonne

Stuart_Rance ‎01-04-2013 11:50 PM - edited ‎01-05-2013 12:41 AM

My goodness what a lot of responses this blog has generated in just one day. There has also been lively discussion on the #back2ITSM facebook group.

 

urbano,

 

I do agree that sometimes customers want outcomes that IT can only partially influence, but I still think you need to write these down, and then document the kind of evidence that would show that IT had contributed to these outcomes.

 

David,

 

I love your comment that "The customer WILL find a way to measure your service - reagrdless of what you include in your SLA". this is why it's best to get the things the customer wants into the SLA, not just the things that IT can measure. I also agree that it's up to the customer to make IT aware of their preferences, but we could do so much more to make this easy for them.

 

Roger,

 

We often use measureable KPIs within IT to help us manage our processes, and our people. There's nothing wrong with that. Especially for KPIs like customer satisfaction which relate directly to how well we are delivering what customers want.  These KPIs will often be different to the measureable targets in the SLA.

 

You are of course correct that when an SLA includes penalties it drives the most unhelpful behaviours from both parties. It's amazing that we continue to create this sort of SLA when all the evidence shows that they really don't achieve what they are intended to. I also agree with your comment that SLAs should not be "one size fits all".

 

Sergey,

 

I don't agree that "SLA is a legal document". Sometimes an SLA is embedded into a legal contract for delivery of services, and this type of SLA is in my experience the worst for including IT metrics rather than customer outcomes. See my comment about penalties above.

 

Yvonne,

 

Thanks for your feedback, please let me know if you make use of these ideas - you can find me on LinkedIn if you want to send me a message about how you use SLAs.

 

James Finister(anon) | ‎01-05-2013 11:48 AM

This blog has generated some discussion on the Facebook "Back2ITSM" group. One point Stuart and I thought worth mentioning back here at the source is the possibility of explicity labelling certain indicators as "proxy indicators."

 

This means making it very clear that what you are measuring is not actually the thing you really want to measure, but  an indication of the likelihood that the thing you would measure if you could is being met. Dave Bremer also made the excellent point on the Facebook decion that too often we forget the I in KPI stands for Indicator, nothing more.

 

An example of a proxy measure is monitoring performance of a test machine processing dummy web transactions rather than monitoring the actual performance being experienced by end users over the internet. This is a proxy in two ways. The first is that it could produce perfectly good results whilst one or more real life users are sufferrign from slow transactions, and of course the speed of a transaction over the web is impacted by a multitude of factors, not just the speed of application and hardware under the control of the IT department.

JoshuaBrusse | ‎01-07-2013 07:00 AM

Hi Stuart, nice work...and it triggered good responses also :smileyhappy: both here and on Facebook https://www.facebook.com/#!/groups/back2itsm/...

 

I have always advocated that an SLA is just a piece of paper and not really relevant in the eyes of any customer (regardless if it is the "pay-the-bill-customer" or the "hand-on-the-keyboard-customer") is value is not delivered. Afterall it’s the experience that counts and VALUE is (and always will be) in the eye of the beholder. J…thus I do believe that the only way to describe and measure a service from a customer POV is to describe in terms of VALUE and measure that, nothing else. And - as David Cannon wrote – the customer knows how to determine value and there is no point arguing about that. All other measures, metrics and KPI are secondary and should be used to figure out why value was not delivered and not to make any other point whatsoever.  

 

What can (and should be) discussed though with the customer is what is wanted…of better… what is needed. Very often the customer needs advice what he/she needs and when that doesn’t come they make up what the want. And “want” and “need” are often not the same. The most important competency an IT organization should therefore have is “Business Engagement Management” (and note that I do not write Business Relationship Management)…because then you can ensure that – over the lifecycle of a service – the customers will get what they need…

 

So…to the point…a SLA should define what a customer needs (which – again - is not always what it wants) J

 

 

Stuart_Rance | ‎01-07-2013 07:59 AM

Joshua,

 

Thank you for your comment. Do you have a link or other information to show what you mean by Business Engagement Management that is different to Business Relationship Management.

Ian Robinson(anon) | ‎01-07-2013 09:55 PM

Stuart

 

There have been some very good comments here including "So…to the point…a SLA should define what a customer needs (which – again - is not always what it wants)" Joshua.

 

Can a SLA be updated timeously to keep up to date with the changing requirements of customers and user expectations?

 

As previously mentioned only the end consumer can measure the personal value they get from consuming a service. The same service may be valued differently by the organisation . 

Stuart_Rance | ‎01-07-2013 11:14 PM

Ian,

 

I agree that the SLA should define what the customer needs, rather than what the customer wants, and even more I would say that it should define what the customer is prepared to pay for.

 

You can update an SLA as often as you and your customer want. If you are doing daily updates to the SLA then I suspect you have something wrong, but to update the SLA as the customer needs change seems entirely reasonable. There aren't any laws or regulations about how you do IT Service Management, you just have to do what is right for you and your customers.

 

Joe Albano PhD(anon) | ‎01-17-2013 10:00 PM

There is a lot of good thinking here and I’m happy to see that the idea of the customer/user perspective seems to be getting discussed more in IT circles.

 

I’d like to add another perspective to the conversation. It seems to me that we often find ourselves trying to measure the results of the actions that we take. Perhaps it would be useful to give equal attention to the motivation behind the results. Many IT organizations seem to be making the transition from “keepers of the technology” to “participants in developing business value”. If IT organizations continue to focus solely on the outputs of activities rather than balancing our attention between what we do and why we do it, shifting from technology to business partner will be difficult.

 

 Some of my clients think of it this way: Good organizations know what they do. Great organizations know who they are.

Stuart_Rance | ‎01-17-2013 10:38 PM

Joe,

 

Thank you for your comment, I always enjoy reading your observations on these things.

 

I think you are absolutely correct, we need to get away from the inside-out SLAs based on an IT-centric view of services and move to an outside-in type of SLA that talks about customer outcomes, and how IT will help to facilitate those.

Joe Albano PhD(anon) | ‎01-17-2013 11:21 PM

Thanks Stuart.

 

I believe that there is a point worth clarifying in my position. I believe that better SLAs (that talk about customer outcomes) are part of a larger solution that shifts the role of IT in the organization. Until IT is able to more the conversation to purpose, not product, IT will continue to be a utility, a commodity.

 

I’m curious, how many CIOs have a seat at the executive round-table? Is IT helping to define the possible in the enterprise – as well as bringing the possible to practice? 

Stuart_Rance | ‎01-17-2013 11:23 PM

Joe,

 

Yes, of course. An SLA is just a reflection of the processes that surround it - you can't fix the whole of IT by just attending to one isolated part.

 

Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the community guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
Search
Showing results for 
Search instead for 
Do you mean 
About the Author
I help clients use service management to create business value for themselves and their customers. I am a senior ITIL examiner and I have wr...


Follow Us
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.