Siebel CRM Testing for Bank's Products

We were once approached by a customer from Pochta Bank with a request to perform
load testing on a Siebel CRM system.

aleksander makarov crm testing company
Aleksander Makarov

Head of Load and Performance Testing

pochta bank sibel CRM load testing

One of the largest banks, it serves 10 million customers with more than
18,000 branches of service.

This happened during a transformation, in which they expected demand for the Bank's products to increase, leading to more user operations and higher load on the system. As a result, we needed to use CRM load testing to determine the Siebel system's performance during peak season.

The system being tested is a credit approval service based on Siebel CRM. The system provides functionality for working with card and credit products, making it possible to conduct the full range of credit-servicing operations. The system also features a contact center module.

Our team accepted the challenge and decided to take a create approach to the CRM software testing problem.

To begin, we studied how users work with the system. This turned out to be not so simple process, since real user scenarios involve many stages. In fact, for Siebel CRM testing we had to build diagrams showing status transitions for requests and use statistics to determine the percentage of transitions at each stage.

To analyze user behavior and compose CRM testing checklist, we had to crunch statistics and build different load profiles depending on the time of day and peak periods.

All the profiles and a description of the load model were summarized in a detailed document — the load testing methodology.

Modeling user behavior
modeling user behavior crm test

Find more deliverables from us

See our case studies to have detailed information about the projects we have worked on.
Take deep into the tasks we managed to solve and implemented solutions.

Read all cases

Determining the load testing tool

Simultaneously with the methodology, we began to work out the approach to writing scripts and choose the most suitable CRM testing tool for the system.

In the process of creating the load testing model for Oracle Siebel, we encountered a large number of difficulties that significantly increase the cost of testing.

When you're deploying a live system, it is constantly receiving a huge number of improvements, fixes for blocking bugs, layout fixes, patches, and hot fixes. And everything would be fine, but with Siebel there is one nuance: the traffic is extremely sensitive to changing the interface, which means that, in addition to the fact that the scripts are extremely difficult to write due to the huge number of element correlations and identifiers, they almost instantly lose their relevance and become invalid.

For any action, a large amount of traffic is created in the system (more than 90 requests for opening an arbitrary page)

Siebel traffic
siebel traffic siebel crm load testing

Existing systems (JMeter, LoadRunner) cannot determine which parameters need to be passed between requests. As a result, all traffic must be processed manually.

To solve this problem, CRM testing company Performance Lab created a FiddlerToJMeter plugin that makes it possible to automate the routine operations of recording and parameterizing the Siebel system's traffic.

JMeter Siebel plugin
jmeter siebel plugin siebel crm load testing

The CRM tester only needs to specify which values to parameterize. In a single click, the plugin then prepares a ready-to-run JMeter script with recorded traffic.

Thus, the time spent on creating load scripts is significantly diminished, reducing the cost of testing without sacrificing quality.

Siebel traffic parameterization
siebel traffic parameterization siebel crm load testing

Developing emulators for
external systems

To emulate integration, we decided to use a test environment for functional testing, on which the external system was deployed. But after a series of CRM tests, the test bench proved to be weak for the external system and could not load Siebel. So we developed an external system emulator that handles requests with delays characteristic of the production environment.

Testing Siebel CRM
testing siebel crm load testing

Results of the project

During the course of the project:

- We converted more than 25 iterations of load testing.

- We found 5 issues affecting performance.

- We conducted a system analysis of the database and performed optimizations improved performance by 2.5 times.

- We defined performance indicators that would allow the business to scale up to 10,000 simultaneous users.

The customer was satisfied, and we gained good experience in testing systems based on Siebel CRM and a tool for fast correlation of complex traffic in JMeter scripts.

This plugin is now part of our load testing platform bumq.io. So, if you have a problem that involves testing Siebel-based systems, we can help.

Got a project in mind?

There's no better place for a QA solution than Performance Lab.
Drop us a line to find out what our team can do for you.

Request a quote