Load testing SAP systems
Our customer is one of the largest retail chains selling electronics and household appliances.
The company implemented a full-fledged omnichannel approach to sales — this means a unified product assortment, price, and service, both when buying in stores and online. The stores have a uniform format and special design concept. In addition to an effective retail format and a store concept that is customer-oriented, the company also offers high-quality customer service.
Performance Lab was hired to test an integrated set of SAP systems, particularly with respect to system performance. These are the integrated systems:
SAP PI/PO — Integration bus;
SAP ERP — Enterprise resource planning management system (procurement, sales, distribution, accounting, tax, and management accounting and reporting, balances,
basic data, etc.);
SAP POS DM — Store data management system;
SAP TM — A system for planning and operational management of transportation along
pre-defined routes (mainline transportation), and inside the city, where the route is
not clearly defined and only the start, end and intermediate points are known.
Got a project in mind?
There is no better place for a QA solution than Performance Lab.Drop us a line to find out what our team can do for you.
Our team faced the following tasks:
- Determine whether the systems being tested meet the performance requirements before and after the transformation;
- Check whether performance worsened after the transformation;
- Identify processes that need to be optimized (if any);
- Check whether performance worsened as a result of optimization.
How did we do it?
To achieve the objectives associated with the project, we conducted tests in which we emulated traffic generated by users and external systems, and background jobs were launched.
To emulate user load on the SAP ERP system, we used the SAP GUI protocol. The system included tasks that corresponded closely to the scheduling in the production environment.
The main load on the SAP PI system comes from data streams from other systems. They were emulated by sending XML files via the SOAP protocol and uploading files to an FTP server.
The load was emulated according to the test scenarios with the specified intensity. The composition of the tested operations was established based on the customer’s expert opinion. The operations were chosen according to the following criteria:
- The operation is business critical (performed daily and affects sales or logistics operations in the system);
- The operation significantly loads the system;
- The transformation is expected to result in increased load from this operation.
Information about the target load and SLAs was obtained during interviews with functional consultants.
After each test, we analyzed the results. The load was applied using HP LoadRunner and Apache JMeter.
The intensity of scenario execution by each virtual user depends on the scenario, system response times, and the amount of delay between two consecutive iterations. In the process of testing, the total intensity of scenario execution by all virtual users (the simulated load on the information system) was changed by changing the number of virtual users performing the scenarios and changing the amount of delay between successive iterations.
What problems did the team encounter along the way?
Correctly achieving the objectives required us to first carefully study the system. The customer’s consultants did not always have answers to all of our questions. We were able to accomplish some of the objectives thanks to our rich experience in load testing for other projects.
The test bench on which the work was carried out was significantly less powerful than the production environment. Working together, we leveled it out to half-capacity, but we had to cut the load profile in half.
The load testing environment was used by a large number of different teams. This led
to changing configurations and distorted test results. After talking with all the teams, we introduced a testing schedule.
What results were achieved?
We identified a problem with budget control (found a database locking problem).
The customer optimized the program;
We identified a overflow problem involving ActiveMQ;
With our help, they debugged and fine-tuned the SAP PI system;
We found a critical error in a TSA patch that almost went into production;
Based on the test results, a table of product comparison materials and receipt
indices was added — without this optimization, the tests caused SAP POS DM to