Siebel CRM Load Testing
We were once approached by a customer with a request to perform load testing on a Siebel CRM system.
This happened during a transformation, in which they expected demand for the Bank’s products to increase, leading to more user operations and higher load on the system. As a result, we needed to use load testing to determine the Siebel system’s performance during peak season.
The system being tested is a credit approval system based on Siebel CRM. The system provides functionality for working with card and credit products, making it possible to conduct the full range of credit-servicing operations. The system also features a contact center module.
Want to Learn More about Our Performance Testing Services?
Find out what’s included and how to start working with us
We accepted the challenge and decided to take a create approach to the problem. To begin, we studied how users work with the system. This turned out to be not so simple, since real user scenarios involve many stages. In fact, we had to build diagrams showing status transitions for requests and use statistics to determine the percentage of transitions at each stage.
To analyze user behavior, we had to crunch statistics and build different load profiles depending on the time of day and peak periods.
All the profiles and a description of the load model were summarized in a detailed document — the load testing methodology.
Determining the load testing tool
Simultaneously with the methodology, we began to work out the approach to writing scripts and choose the most suitable tool for the system.
In the process of creating the load testing model for Oracle Siebel, we encountered a large number of difficulties that significantly increase the cost of testing.
When you’re deploying a live system, it is constantly receiving a huge number of improvements, fixes for blocking bugs, layout fixes, patches, and hot fixes. And everything would be fine, but with Siebel there is one nuance: the traffic is extremely sensitive to changing the interface, which means that, in addition to the fact that the scripts are extremely difficult to write due to the huge number of element correlations and identifiers, they almost instantly lose their relevance and become invalid.
For any action, a large amount of traffic is created in the system (more than 90 requests for opening an arbitrary page)
Existing systems (JMeter, LoadRunner) cannot determine which parameters need to be passed between requests. As a result, all traffic must be processed manually.
To solve this problem, PFLB created a FiddlerToJMeter plugin that makes it possible to automate the routine operations of recording and parameterizing the Siebel system’s traffic.
The tester only needs to specify which values to parameterize. In a single click, the plugin then prepares a ready-to-run JMeter script with recorded traffic.
Thus, the time spent on creating load scripts is significantly reduced, reducing the cost of testing without sacrificing quality.
Developing emulators for external systems
To emulate integration, we decided to use a test environment for functional testing, on which the external system was deployed.
But after a series of tests, the test bench proved to be weak for the external system and could not load Siebel. So we developed an external system emulator that handles requests with delays characteristic of the production environment.
Results of the project
- We converted more than 25 iterations of load testing;
- We found 5 issues affecting performance;
- We conducted a system analysis of the database and performed optimizations improved performance by 2.5 times;
- We defined performance indicators that would allow the business to scale up to 10,000 simultaneous users.
The customer was satisfied, and we gained good experience in testing systems based on Siebel CRM and a tool for fast correlation of complex traffic in JMeter scripts.
This plugin is now part of our load testing tool PFLB. So, if you have a problem that involves testing Siebel-based systems, we can help.