Benchmarking Basics
There are hundreds of formulas and methodologies
that could be combined using different weightings to arrive at an
overall Composite Score, from which it could be determined which
carrier is “best.” In all likelihood, these diverse
combinations could each arrive at contradictory conclusions. Different
parameters such as network performance, network footprint, service
pricing, customer support, and even handset selection affect, to
varying degrees, customers’ overall opinions of a wireless
carrier. These statistical unknowns are complicated even more by
the immense amount of logistical and hardware complexities involved
in benchmarking competing carriers in a mobile environment.
The first step to tackling these issues is
to provide focus. GWS’s benchmarking services determine which
carrier performed best by testing network reliability – the
single most valuable factor affecting subscriber attitudes, according
to a 2002 Yankee Group study. The next step is to realize the need
for objectivity in determining a composite score. GWS commissioned
an independent study to analyze subscriber attitudes of key performance
indicators and their effect on a subscriber’s opinion of network
reliability. In this way the composite score reflects how a conglomeration
of typical customers would collectively rate each carrier’s
network performance. With this composite score formula, GWS can
determine which carrier a population of customers would rate the
best, and how the remaining competition rates.
For Voice Benchmarking, each carrier’s
composite score is based on key performance indicators (“KPIs”),
which include Service Coverage, Access Failure Rate, Retainability,
Voice Quality, and the Access Time of the network. Each is described
below.
Service Coverage: Is service even available? If a
carrier does not even attempt to provide service where customers travel
on a daily basis, those customers will become frustrated very quickly.
Service Coverage is defined as the percentage of call attempts that
fail due to no service.
Access Failure Rate: Are calls getting blocked where customers
know there is normally service? Access Failures provide a minor annoyance
compared to the daily frustration experienced by service coverage
limitations. After all, in most access failure cases, pressing the
SEND button a second time will resolve the issue from the customer’s
perspective. The Access Failure Rate is defined as the percentage
of call attempts that fail in areas where there is service.
Retainability: How often are customers rudely interrupted
by dropped calls? Retainability reflects the percentage of good terminations.
Carriers want to see retainabilities above 98%. Retainability is defined
as the percentage of calls that were terminated by the customer rather
than the network.
Voice Quality: Even if your call did not drop, how often
did you strain to understand your coversation (bad downlink quality)?
How many times did your colleague ask you to repeat yourself (bad
uplink quality)? Either situation can negatively shift the focus of
the customer’s conversation to the wireless service. Voice Quality
is the percentage of voice scores that occurred that would be classified
as acceptable to a vast majority of users.
Access Time: Do customers notice a delay before their
calls started ringing? Access Time is the average amount of time elapsed
between pressing the SEND button on the phone and hearing the first
ringback tone. Sometimes this delay is considerable enough that the
customer consciously notices it. This is by far the least important
factor, but can be a determinant between two otherwise equal competitors.
Similarly, the Data Benchmarking Report
provides a Composite Score reflective of how a typical customer
would rate each carrier’s data network performance. The weighted
Score is based on the Service Coverage, Accessibility, Retainability,
and Throughput of the network. The first three KPIs have been outlined
above.
Throughput: How much time
will it take to download or upload a file? Throughput is the average
rate at which a network transfers the data contained within a file.
The faster the average throughput, the quicker you can transfer
that file and get on with life.
The KPIs listed above only provide a small
subset of the competitive information provided in GWS’s Voice
and Data Benchmarking Reports. There is a host of information relayed
to you to provide guidance in how to alleviate problem areas particular
to your network.
|