QAQNA

Double Dinging and Domino Dinging

In the world of Quality Assessment (QA) there are few injuries more heinous to a Customer Service Representative (CSR) than to "double ding" them. In the parlance of Call Center QA, this means that you mark them down on two different QA elements for the same behavioral infraction. For example, the CSR leaves the caller in a looooooooong period of unexplained silence (also known as "dead air"). The supervisor or QA analyst marks the CSR down on the QA form for "leaving the caller in dead air" and "not matching the caller's pace." The result is that the one behavior is being scored in two different places on the QA form. Thus, the dreaded "double ding."

Note: In many cases, "double dinging" result from poorly defined elements on the scale. The person analyzing the call doesn't know where a behavioral infraction "fits" in the scale, so they mark down in a couple of places. If you've got people "double dinging" you might want to take a look at your scale.

There are times, however, when one behavioral infraction results in a "domino effect." For example, the CSR pulls up the customer's account and finds that the previous CSR did not handle the customer's issue correctly. Rather than simply correcting the mistake, the CSR proceeds to complain to the customer how their coworker made such a bone-headed mistake. The CSR rambles on about how the company rushes people through training and puts them on the phone before they are ready. In this scenario, the CSR's disparaging comments result in the QA analyst "dinging" him for "tearing down the customer's confidence" but also for "not managing the length of the call." In effect, the CSR's inappropriate rant had a domino effect. He not only tore down the customer's confidence in the company with his negative comments (ding one), but his incessant rambling also resulted in keeping the customer on the phone longer than necessary (ding two).

When analyzing calls, it's important to make a distinction between "double dinging" (penalizing the CSR twice for the same behavior) and "domino dinging" (recognizing that one behavior had a domino effect, resulting in multiple infractions).

Creative Commons photo courtesy of Flickr and pixiesticks23

Praise or Criticism? What Works Best?

It's a classic debate in the world of call center quality assessment (QA). Do you use QA to praise Customer Service Representatives (CSRs) so as to encourage them and build their self-esteem? Do you use QA to be critical and hold CSRs accountable to keep them honest? Is there a happy medium, and if so, where is it?

When giving seminars, I often use the word pictures of the "QA Nazi" (who uses QA as a means of beating CSRs into submission) and the "QA Hippie" (who uses QA to give CSRs smiley faces and make their world a "happier place") to represent the extremes on both sides of the spectrum.

My coworker recently forwarded an article to me from NY Times Magazine about some research that's being done on the power of praise and criticism with children. While the research focuses on parents and their children, I would submit that there are some lessons for us all to learn in the QA, training and coaching arena.

The most recent research is finding that undue praise can actually have a negative effect. Those who are constantly and generally praised tend to become more competitive, less motivated and less willing to put out effort towards improvement.

Does this mean that praise isn't important? Not at all. What the research is discovering is that praise is a powerful force when it is specific and sincere.

I'm sure the debate will never end, and I'm not sure that it should. A professor of mine said, "truth lies at the tension between the two extremes," and I've found it apt in many situations. Finding that right balance between praise and accountability is elusive, but one to which all QA teams should strive. 

I continually come back to a few key tenets:

• Know what drives your customer's satisfaction, by asking them
• Define specific, desirable behaviors that will meet & exceed those expectations
• Measure those specific behaviors
• Give consistent, honest, data-led feedback to CSRs telling them which behaviors they are consistently performing, and which behaviors theyare inconsistently demonstrating
• Train and coach CSRs toward improvement
• Praise CSRs for the specific, documented acheivements and improvements
• Hold CSRs accountable for specific, documented lack of performance

Why Are We Here?

When I present workshops and seminars about setting up a Quality Assessment program (QA), I always start by asking the participants to discuss why the purpose of the QA program and what the real focus of the program is going to be:

CSR focused QA programs often exist to motivate and build up the self-esteem of the front-line CSR. It becomes a way to give folks a pat on the back.

Corporate focused QA programs usually exist to hold the corporate line and hold CSRs accountable to comply with programs, policies and procedures. It becomes a leverage tool.

Customer focused QA programs most often attempt to measure interactions from the customer's perspective. It becomes a microscope.

Existential QA programs generally exist for no apparent reason other than for a company to say "we have one, therefore we must care about customer service." It becomes meaningless.

Of course, you may detect traces of all four within your existing QA program. Nevertheless, I find that most programs will have a determined bent in one of the four directions.

Which way does your QA program bend? What gives it away? Which would you prefer? What could you change to make it what you think it should be? Please share!

Measure What You Know, Not What You Perceive

I am a patient person. Nevertheless, I've learned that I can also be an emotional volcano. I am very slow to anger, and I rarely erupt, but there is a limit to how much frustration I will bear before the explosion is bound to occur. I'm not saying this is a good thing, but it is true about me. Because of this, family, friends, and service providers will often misread me and my responses. I don't look angry. I'm not screaming and yelling. So, they conclude, everything is just fine when it's not. In reality, there's an eruption brewing just below my calm exterior.

You can't always tell what a person is thinking and feeling.

When creating criteria for your Quality Assessment (QA) scale or monitoring form, it's best to clearly define the behaviors you're listening for from the Customer Service Representative (CSR) on the phone. The easiest way to stay objective is to measure that which you can hear and know. Keep your criteria limited what the CSR says to the person on the other end of the conversation.

It's quite common to find companies or individuals basing their assessment on what they perceive the customer thinks or feels. I've seen QA scales that are based on how well the CSR met or exceeded the customer's expectations. However, unless you interview each customer, you're making specious judgements about what that customer thought of the experience. In addition, some customers will never be satisfied. It would not be appropriate to rate the CSR's effort based on an uncontrollable outcome.

People also like to make arguments based on the perceived response or lack of response from the customer. "I shouldn't be penalized for not saying 'please,'" a CSR might argue, "because the customer clearly didn't care whether I said it or not." But, you don't know what that particular customer thought, felt or perceived. Just like my friends thinking that I'm perfectly calm when there is an eruption brewing beneath the surface. QA is not intended to be a crystal ball that looks into the mind of each customer.

We can know, for a fact, what drives our customers' satisfaction on the whole. A good customer satisfaction survey will provide us with this information and it can be critically important in defining the elements we expect as part of our QA criteria. But to try and judge an individual call based on perception of the customer's response is an exercise in futile subjectivity.

We can't control or accurately read every customer's mind, but we can control what we say to each customer and how we say it.

Creative Commons photo courtesy of Flickr and nancee_art

Check the Scores for Consistency

Most companies who monitor calls and do Quality Assessment (QA) have some form of calibration. This usually follows some pattern of listening to a call with a group of people and analyzing it together to find out where you agree and disagree on an evaluation. It can be a painful process.

You can often learn just as much, if not more, by simply comparing a large sample of evaluations scored by different analysts.

For example, I was recently pouring over a comparison of quarterly data for one of our clients. I have a member of my team run the raw data on a regular basis and provide me with a comparison report. In this particular project, we have four analysts scoring a couplehundred calls per month. There are roughly 65 different behavioral elements we analyze that are rolled up into 14 corresponding attributes. There is a score for each attributeand a corresponding Overall Service score on a scale from 0 to 100.

First, I compared the average Overall Service score for each of the four analysts. The four analysts were within a half-point of the overallaverage for the group. This told me that I didn't have anyone who was particularly lenient or harsh in their analysis. Our overall service numbers were very similar. If an analyst had an overall service score that was much higher or much lower, it would have motivated me to dig into the underlying data to find out why. So far, so good.

Next, I compared the average scores for each of the 14 Attributes. Because some Attributes rarely apply, there are much higher deviations. Keeping this in mind, I focused on the attributes that apply most often and have the greatest impact on the Overall Service score. Once again, our scores were very similar.

Finally, I looked at the average number of times each analyst marked a behavior "yes," "no," or "not applicable" for each of the behavioral elements. In this instance, I found one of my new analysts who had marked a particular behavior as applicable on 100% of the calls analyzed while the other analysts had it applicable on less than half. Because it's an element within an attribute that the client normally scores very well - it didn't show up in the corresponding score. Nevertheless, it could eventually make a difference, and we were clearly not calibrated in scoring this particular behavior. By looking at the data, I was able to address the issue with the analyst and coach them on how to more accurately measure that particular behavior. From this point forward, we should be more closely calibrated.

Sometimes, you've got to let the data show you the way.

"...and a 7.5 from the Soviet judge."

I'm a child of the Cold War era, when the United States and the Soviet Union were two world super powers locked in an on-going political struggle across the globe. As a child watching the Olympics, it became a running joke to watch politics creep into the judges scores in competition like gymnastics and ice skating. No matter how well an athlete from the U.S. performed, we felt that the scores from the Soviet judges and their allies were always slightly lower than they should have been. Perhaps the Soviet children watching felt the same thing happening from Western judges.

Much like a gymnastics judge watching a routine and trying to place an objective measurement on what took place, call monitoring and quality assessment attempt to objectively measure what took place in an interaction between the company and the customer. There are ways to largely drive subjectivity out of the process and make the assessment as objective as possible. In the end, however, it is a human process. Mistakes will be made. People will see the same thing differently.

This doesn't mean, however, that you throw the baby out with the bath water. I will sometimes hear people argue that QA is completely unreliable because mistakes sometimes occur. If we apply the same logic to athletics than we should just make all the Olympic gymnasts, skaters, and divers hand back their medals and throw the sports out of the competition. That doesn't make much sense, does it?

When done properly, QA can provide a highly accurate picture of the service experience over a period of time. Because it is a human endeavor mistakes will occur. The benefits of listening, assessing, training and coaching far outweigh the risks of sticking your head in the sand and hoping that your team is doing a good job on the phone.

Creative Commons photo courtesy of Flickr and Bryangeek

Personality and Quality Assessment

I think a few people in my pre-conference workshop were surprised when I started the session with a brief personality test. After all, what do personality types have to do with QA in our call center?! Believe me, the longer I spend in this field the more I come to understand that personality often plays a huge, unrecognized role in a company's QA program.

The structure and function of Quality Assessment appears to be a nuts and bolts proposition complete with data, metrics, charts, graphs, sample sizes, and statistical methodology. The goal of QA is to objectively measure what takes place in phone calls. A good QA scale will allow you to quantify these "moments of truth" in very clear, systematic ways. Nevertheless, the program is built and administered by people. The results are communicated by people and used to coach and train people. Quality Assessment is about people, and the personalities of those people affect QA in a very real sense from beginning to end. If you're not careful, personalities can skew or derail the QA process altogether. A good QA program will provide a balanced measurement that minimizes the effect personalities may have on the process.

How are the personalities of your management team, your call center manager, your supervisors, and your CSRs affecting your QA process?

Creative Commons image courtesy of Flickrand Combined Media

Cost Savings in QA: Rethink Your Sample Sizes

In this economy, everyone is looking to save a buck. Companies are slashing budgets and trying to salvage the bottom line.

So where is the savings within the QA program?

The biggest cost in most QA programs are the time, energy and resources it takes to sit down, listen to, and properly analyze a phone call. It requires man hours to do the task.

So start by looking at your sample sizes and crunch the nubers. Depending on the goals of your program, you generally don't need to analyze hundreds of phone calls to get an accurate reflection of a CSRs service. If you have a well designed, behaviorally anchored QA scale, then a small sample of randomly selected calls will do the trick. I have been in call centers who will measure hundreds of phone calls for a given CSR. It's overkill.

Consider the pollsters who can accurately guage the opinions of 175 million Americans by talking to 1,000. It's statistically possible to do so with a small margin of error, if you do it right. QA works in the same way. You can get an accurate reflection of a CSRs service over thousands of calls by listening to just a handful, if you do it right.

Creative Commons photo courtesy of Flickrand Aaron Kyle

Join Me at the ICMI Call Center Demo Feb 25th in Miami

I'm going to be presenting a half-day pre-conference workshop at the ICMI Call Conference Demo in Miami on Building a Successful QA Program. The workshop will be 8 a.m. until noon on Wednesday, February 25th. I would love to have you join me for the workshop. It's always a pleasure and privilege to meet QAQnA readers.

The workshop will be great for anyone who is working on structuring a new QA program as well as those who would like to find new ideas for restructuring or improving their existing programs.

I hope to see you in Miami!

Angry Customers Part 5 - Empathy Resolution Statement

I have written on different occasions about the importance of both empathy and resolution when the customer's expectations have not been met, and about the importance of an apology. When dealing with an angry customer, it's important to focus on resolving the issue at hand, to the best of your ability. However, if you don't show any empathy or acknowledgment for the customer's frustration, you still aren't going to provide an optimal customer experience.

I hear many CSRs say, "the customer tells me 'I don't want your apology!' so I never apologize." I have heard customers say this (though data shows it is far less seldom than usually reported), and whenever I hear the statement made it is almost always followed with something like: "I want my issue resolved!"

The problem is not that the customer wants no empathy, the problem is that the customer feels that all he/she has received is empathy and no one is fixing the problem. When the customer says, "I don't want your apology!' it should be a red flag telling you that empathy/resolution is out of balance and you need to focus on resolving the issue.

To avoid getting these out of balance, I recommend what I call a "empathy resolution statement." You offer a simple apology for the problem followed by a statement of what you are going to do to resolve it. Once done, you can usually move on to focusing on resolution knowing that you've adequately expressed empathy.

"I'm sorry that (describe the unmet expectation). What I can/will do is (describe what action you will take)."

For example:

Customer was supposed to receive a call back and didn't.
"I'm sorry we didn't call you back. What I will do is pull up your account and find out what I can do to resolve this for you."

Order didn't arrive.
"I apologize that you didn't get your order as expected. I can check the order for you and let you know what the tracking information tells us."

Customers called several times with the same issue and it hasn't been resolved.
"I'm sorry we've let you down on this issue. I will do everything in my power to get this resolved."

A simple empathy resolution statement will provide the customer with a balanced approach that leads to focus on resolution, but doesn't completely ignore the customer's need for empathy. As with all service skills, each CSR needs to find ways to make the statement conversational and incorporate a wording that is natural and comfortable. If both the empathy and resolution portions of the statement are clearly communicated, you will often set yourself up for a successful service experience.