Contact center and IVR performance testing

Contact centers have replaced traditional call centers. Who wants to waste human resources on something that can be easily managed by software? Today, contact centers are an integral part of retail and service providers.

Unlike call centers, which can only interact with customers via the public switched telephone network (PSTN), contact centers include services that improve customer support and offer a wide range of tools for the most convenient communication and interaction.

The capabilities of basic telephone interaction have expanded thanks to interactive voice response (IVR). Managing customer requests in automatic mode and using speech recognition algorithms and smart navigation are only two of the methods that have made it possible to automate the process of customer interaction to a great extent.

Obviously, the higher the complexity of a contact center’s structure, the higher the level of hardware, and the higher the likelihood of unsatisfactory performance of the system or its components.

Given that the quality of the services provided through a contact center directly affects user experience and, as a result, consumer attitude toward a product, service, or company as a whole, there is hardly anything more important than timely identification of possible problems.

Contact center and IVR performance testing intro

Contact centers performance issues

The central issue behind contact center productivity is the technical capabilities of a company’s IT infrastructure. If they are not sufficient to meet the needs of your business, you can end up with additional costs and unhappy customers.

The top 4 IVR performance problems are:

  • poor sound quality;
  • audio signal delay;
  • poor or interrupted connection;
  • slow IVR navigation response.

If one or more of these problems are present, customers may misunderstand the message and hold false expectations or may have to ask the assistant to repeat the message, which would increase the total duration of the call. Either scenario would result in a poor customer experience.

Why is call duration so crucial?

Obviously, with an increase in the average duration of calls, the number of operators required to process them also increases. For example, a typical call lasting 3 minutes and 15 seconds can increase by 12 seconds if there are communication issues. The cumulative effect is equivalent to an additional 6 contact center operators for every 100.

ivr load testing

Load and communication quality

Perceptual Evaluation of Speech Quality (PESQ), an international standard for objective voice quality testing, measures voice transmission quality using a mean opinion score (MOS):

MOS Quality Distortion

5

Excellent

Imperceptible

4

Good

Noticeable but not annoying

3

Ok

Somewhat annoying

2

Bad

Annoying

1

Terrible

Very annoying

The lower the MOS, the worse the voice quality is perceived and the more difficult it is for customers to interact with operators.

Although some performance problems are obvious, mere observations without measurements of the overall load level and system functioning quality are not enough, especially when it comes to a large customer base or wide coverage.

On a regular business day, the call flow gradually increases. Accordingly, the quality of voice transmission gradually deteriorates. In the model graph below, you can see what can happen at 6 pm. A sale in retail, or any emergency, leads to an increased flow of calls and, ultimately, to system failure.

A call flow increase leads to a deterioration in the quality of voice transmission

Situations like this can arise from a wide range of IVR performance problems that cannot be identified in advance without load testing.

Testing contact center IVR with LoadRunner and Data Synergy Voice

Load testing is an effective solution to the abovementioned problems. If you don’t want to waste time, it is important to run the tests.

PFLB specialists use some of the best IVR QA testing tools available: Micro Focus’s well-known LoadRunner and the DS Voice plugin developed by our Australian partner Data Synergy. LoadRunner emulates thousands of user interactions with the tested system, while DS Voice interacts with digital telephony communication channels.

ivr load testing application under load

Using both LoadRunner and DS Voice, you can emulate and process interactions between customers and contact center operators. In particular, you can:

  • generate incoming and outgoing voice traffic over SIP channels;
  • measure the quality of voice communication (delays, distortion, packet loss);
  • transmit and process DTMF codes;
  • record, play, and recognize an audio stream;
  • convert text to audio.
ivr load testing architecture scheme
With our IVR system testing solution, you will be able to:
  • identify problems that users face during busy hours;
  • analyze a wide range of application and infrastructure parameters under different load levels;
  • find performance bottlenecks at all architectural levels;
  • measure the scalability parameters of the contact center;
  • reate a load both from inside and outside the contact center’s local network.

IVR performance testing: our approach

Modern contact centers are often complex systems that involve close interaction between IVR, customer relation management (CRM), and many other services. Therefore, besides testing the performance of IVR itself and telephone communication channels, PFLB offers comprehensive testing services that allow you to evaluate the performance of both overall and individual services and to identify the influence of system components on each other.

ivr load testing scheme data synergy

How do we test IVR systems?

There are 4 necessary stages plus an optional one:
  • developing an IVR testing methodology;
  • developing IVR performance testing tools;
  • running the tests;
  • analyzing the system and compiling a report;
  • retesting (optional).

Have a Project in Mind?

We have been working on performance testing projects since 2008.

Drop us a line to find out what our team can do for you.

1. Developing an IVR testing methodology

The testing methodology is created by PFLB specialists after analyzing the technical documentation, business processes configured in the IVR, and contact center platforms, as well as interviewing the client company’s specialists. What we do is as follows:
  • collect and analyze industrial environment statistics;
  • determine load generation methods and generator locations;
  • coordinate performance requirements;
  • define business processes and load scenarios;
  • describe interactions with external systems.

Then, we present the chosen methods to the client for approval.
When creating the testing method, we pay special attention to determining the load profile. The load profile is a set of operations with specified intensities that can be determined based on statistical data or an analysis of the system requirements. As a rule, several load profiles are selected, each of which differs in the set of operations or intensity. The greater the load on the system, the higher the quality of the load testing.

Developing an IVR testing methodology

2. Developing IVR performance testing tools

Creating a test model includes 6 steps:
  • developing load scripts and load scenarios;
  • developing external system emulators (if necessary);
  • creating scripts;
  • generating/depersonalizing the database (if necessary);
  • preparing test data;
  • configuring monitoring tools and running trial tests.

This is the most challenging stage from a technical point of view. PFLB engineers develop and debug the necessary IVR performance testing tools.

As part of this stage, PFLB offers its clients the opportunity to use its own technology, PLUS, which can help cut costs and improve the testing quality using ready-made performance testing tools, such as Data Synergy Voice, SOAP/REST processing emulator ISO 8583, SOAP emulator of a credit broker, WebSphere log analyzer, MS SQL database depersonalization module, and Visa and Mastercard clearing test data generator.

3. Running the tests

This stage comprises not only running the tests according to the approved load scenarios but also processing the results and evaluating the validity of the tests performed. For the duration of the testing, PFLB specialists are given access to load generation stations.

Running the tests

4. Analyzing the system and compiling a report

PFLB specialists analyze the results and share their conclusions and recommendations with the client. They pay special attention to finding performance bottlenecks, preparing recommendations on optimizing the code, architecture, and infrastructure of the system or developing appropriate organizational regulations. This is when they also determine whether the channel width is sufficient for the IVR, what the operator’s real response time is, what the level of communication quality is, etc. Then, they compile a report with the test results, detected problems, and recommendations.

5. Retesting (optional)

After the client performs the recommended optimizations, tests can be run again to confirm the improvement in system performance and to identify any new factors that limit performance.

We can also develop a plan for the client to keep the performance testing solutions up-to-date for a future retest.

PFLB offers services for setting up the support process, which can include both comprehensive maintenance of the created solution and training of the client’s staff engineers.

About the company

PFLB is a leading provider of independent software testing and quality assurance for enterprise clients. The company has successfully carried out more than 250 major projects and is certified according to ISO 20000.

PFLB employs more than 400 specialists, and its clients include some of the biggest European banks, retail companies, telecommunications providers, and many other organizations. Clients that have used PFLB’s IVR load testing with DS Voice include IKEA and The Coca-Cola Company.