Customer Expectation is a Difficult Way to Judge Calls

Through the years we've had a few clients who asked their QA analysts to use a scale that judged call quality this way (example based on an actual QA scale):

Element: CSR used the customer's name in the call.
(choose one)

• Exceeded Customer Expectations
• Met Customer Expectations
• Fell Below Customer Expectations
• Needs improvement

While I applaud the intent of this approach (it asks the analyst to consider the customer's point-of-view), there are several problems that will undermine the objectivity and effectiveness of the QA process:

1. Do you really know what your customer expects? Asking the analyst to judge based on customer expectation assumes that each analyst understands what those expectations are. We've seen this approach used, but when we ask how the client arrived at determining just what the customer's expectations are, there is no supporting data. (typical response: "It's just common sense. We're all customers.") If there's not supporting data then each analyst is left to decide what "meets" or "exceeds" expectations. Because each analyst may have different thoughts or opinions, your resulting scores will be all over the map.
   
2. Is exceeding customer expectation always important? It might seem obvious, but it is actually a faulty notion to believe that exceeding customer expectation in every area is necessary to improve customer satisfaction. Certain service dimensions are what researchers call "penalty variables". If you exceed the customer's expectation in that dimension of service, you don't receive a proportionate boost in customer satisfaction (you will be penalized if you fall below customer expectation, however). It doesn't pay to exceed customer expectation in that particular area. Other dimensions are "reward variables", in which case you will be rewarded for exceeding customer expectation. The above scale presumes that every area of the phone call is a reward variable. This is not the case.
   
3. Can you objectively differentiate between meeting and exceeding expectations on any given behavior? In the example given, how do you differentiate between "met expectations" and "exceeded expectations" when using the customer's name? Do you give a number of times the name must be used (e.g. use the name three times in the call)? But, what if the call is only 30 seconds long? Does using the caller's first or last name count differently? It's difficult to make a consistent, objective choice based on what the customer's expectations may (or may not) be.
   
4. In almost every case we've seen this approach used, there are two negative options. In the scale I cited above the QA analyst was asked to choose between "fell below customer expectations" and "needs improvement". If, however, the scale is based on the notion that exceeding expectations is what you want, than wouldn't anything below "met expectation" mean that it needs improvement? How is an analyst to decide that an element was below expectation, yet it doesn't need to be improved?

Creating a QA scale can be an arduous task, but there are principles that, when followed, will allow you to maximize objectivity and make your scale both effective and efficient.