Load testing is necessary when deploying any new system or redistributing data streams among existing systems. I think everybody knows this or at least would guess this is the case.
Everything seems simple: grab your load testing tool, record some traffic, get some scripts, run your tests, and enjoy the process. At least that’s the theory…
Approaches to loading
Actually, in most cases engineers are faced with testing information systems that have a classic three-tiered architecture:
The standard approach to performance testing is to emulate user traffic from clients to the back end. This works, but only if the end client applications use standard protocols such as HTTP/HTTPS, SOAP, ODBC/JDBC, SAP GUI, and a few others.
But what do you do if the exchange between client and server uses a closed protocol or, even worse, a proprietary protocol, and the traffic cannot be recorded using standard tools?
You can try to figure out the protocol, decompile the application, reverse engineer it, and develop a plug-in or sampler based on the data received. But the likelihood of success is extremely small. And if you also consider the time required and translate that to money, then the entire undertaking is very difficult to justify.
In other words, the task is far from trivial and not always solvable. In fact, this is akin to hacking. It’s like a TV show where the players being told the story before being asked what’s in the black box. But you may not even have access to the story (read: source code for the application’s client end).
Of course, you can use various traffic analyzers (Wireshark, TcpDump, and others), but the volume of traffic may be so large that you’ll need months to analyze it!
In our experience, it is not uncommon situation when a customer starts to think about load testing only one week before a scheduled release.
Of course, you still have to write a wrapper so it works correctly with the chosen loading tool.
Well, we can only salute those who choose this path and succeed. Your results will be truly grand if you can overcome all of the “stumbling blocks”, but we in general we do not recommend this approach.
What can be done if you aren’t willing to expend so many resources on such lengthy investigations?
At this point, an experienced load tester will, of course, say, “Hey, guys, try CITRIX!”
Using Citrix virtualization technologies
One way to handle this task is to use Citrix virtualization products, which may it possible to gain remote access to an application on almost any device. What’s at the heart of this approach?
The client application that end users interact with is published on a dedicated Citrix server (farm).
Unlike the first approach where the loading tool is “inserted” into the client-server link and parametrized traffic is used, here our tool sends commands to copies of the client application using the ICA protocol.
To tell the truth, we actually manage the applications through a terminal server.
This lets us emulate not the behavior of the client application through replaying traffic, but rather the behavior of a large number of users of the client application. It’s no secret that Citrix has been used in performance testing long enough to emulate user behavior on a client application. But by no means can every loading tool work with this protocol.
The most popular loading tool for working with the Citrix protocol is HP’s LoadRunner.
It’s popularity is a result of the fact that there is no other choice. Looking at JMeter, we see nothing, empty space, anguish, pain, and somebody’s forgotten broken crutches on an overgrown path through a forest that leads who knows where.
Let’s returning to Load Runner. This product recommends itself as massive and functional. However, for all of its pluses there are minuses.
The most obvious is the product’s cost and complexity when expanding functionality.
Anyway, we’ve chosen a tool, so everything ought to work out, but now we count up the cost of the licenses. In current market conditions, when converting dollar prices to rubles, the cost of licenses for testing may exceed the value of the solution being tested.
What’s more, the closest analog, Apache JMeter, is entirely free and easy to extend, thanks to open source code and a simple, understandable API for writing plugins.
So what am I driving at? Yes, Load Runner is good, but our customers’ desire for a more budget-friendly tool prompted us to develop our own JMeter plugin to link with CITRIX, OCR, and synchronization.
Our plugin work produced a highly versatile and easy-to-use tool for working with Citrix, which almost entirely covers HP LoadRunner’s functionality when working with this protocol, and in some instances surpasses it. But the most important advantage for our customers is the fact that it is free.
Differences from LoadRunner-based solutions
If you’ve ever written terminal scripts for LR, then you are familiar with the agony of debugging, which requires the synchronization of every little thing, every image, and where being off by a couple of pixels can break everything. Tedious, slow, unreliable, devilishly torturous – these are the epithets you want to cry out during coding and debugging. The pain and suffering literally permeate you when you have to support this model.
Knowing this, when developing the plugin to work with Citrix we placed great emphasis on achieving the following benefits:
The most important difference is that it is free of charge for our customers. Using JMeter instead of LoadRunner makes considerable savings possible, especially during lengthy testing, because using HP LoadRunner requires the purchase of licenses whose cost is based on the number of virtual users and the duration of testing.
And now we’ll briefly explain how it all works.
Anybody not directly associated with load testing can skip this section in order to avoid brain trauma.
What you need to get started
You’ll need the following in order for the plugin to work properly:
Installing the plugin
I won’t describe how to install JMeter and JRE. I assume everybody is already familiar with these procedures. Installing Citrix Receiver is also straightforward, so we’ll get right to installation of the plugin. To install the plugin itself, simply copy the plugin’s .jar files and com4j library files to the lib folder in JMeter’s root directory. But to be able to record and playback you also need to add a registry entry (the path given is for Windows x64; for 32-bit versions, remove the Wow6432Node subdirectory from the path):
This text can be saved in a text file with the .reg extension and then added to the registry by executing the file.
To record scripts, you must add the Citrix ICA Recorder element (located in the Non-Test Elements category) to WorkBench and there indicate the correct connection parameters:
The recording parameters must also be specified:
Recording a session
Once all of the settings have been entered, you can connect to the a session and start recording. To start/stop recording, use the Recording controls button group in the recorder. The Connect button simply connects to the session. This does not start recording. You can start recording later. The Record button connects and immediately starts recording the selected actions. The Stop button stops recording and disconnects from the session. The session itself (Citrix Receiver window) is not closed. Moreover, when the session is exited (for example, by closing the application) recording is stopped automatically based on the LogOut event.
The Step controls button group is for managing steps. It becomes available once recording has begun. The Add Step button saves the current step and adds it to the selected controller. Additionally, the recording begins to be received by a new sampler. The Restart Step button simply clears all actions from the current step (helpful if you make a mistake). The Start Tag button inserts a tag so that all commands recorded until the End Tag is pressed will be ignored and instead the text entered by the user will be inserted. Accordingly, the End Tag button inserts a tag that terminates the effect of the Start Tag command. The Synchronize button creates a screenshot of the Citrix session’s entire workspace and brings up a dialog with synchronization settings, which I would like to discuss in more detail.
There are two 2 synchronization options available at present: screenshot-based synchronization and text-based synchronization (using OCR).
To synchronize using screenshots, select the synchronization area (red semitransparent rectangle) and specify the right parameters.
This type of synchronization involves comparing checksums of the specified screenshot areas during recording and playback. As a result, changing the color of a single pixel during playback will change the screenshot’s checksum, which will cause the comparison to fail and the area will be considered missing.
For text-based synchronization, you also must choose the area of the screen in which text will be recognized. You also need to specify two parameters: OCR config handle – A string indicated in the OCR Config test element in the OCR Handle field (more about this later), used to select the customized OCR engine to be used for text recognition; and OCR result variable – The name of a variable used to record the result. If nothing is recognized, the variable will be created and initialized with an empty string.
Preparing for playback
Once the script has been recorded, you should have a test plan with a controller and Citrix ICA Samplers inside. At least one Citrix ICA Config element is required for playback.
A Citrix ICA config looks very similar to a Citrix ICA Recorder. It has exactly the same connection parameters, so there’s no point in describing them here. I’ll only mention that all fields in the connection configuration can be parametrized, and these values will be used when initializing the session, i.e. when executing the first Citrix ICA Sampler encountered. For example, don’t be afraid to use CSV Data Set Config to parametrize fields. The Citrix Config Handle field must be unique for all such configs. The samplers use it to determine which connection settings to use, and to connect to the session to continue the test.
The settings to playback sessions differ from those in the recorder. I’ll point out the differences:
If text recognition actions were recorded during recording, then an OCR Config element must also be added. It is quite simple and has only two parameters:
The training images are images with sequentially printed symbols (in a line or in a table). The range of codes for the specified symbols must be uninterrupted and in increasing order. A single range should be indicated for each image.
Description of samplers
In addition to the standard names and comments, the Citrix ICA Sampler has two fields. First, Citrix Config Handle indicates which settings to use to start a session or the pool from which to take an existing active session. The second field contains a list of all the commands it will execute. This field may be fully parametrized using JMeter variables.
To playback a recorded and parametrized script, simply run the test. After each sampler has run, you can view the result of each command in Response Data.
If an .ica file is used to initialize sessions, it’s best to parametrize the field that corresponds to it. Because the file is single-use, the second stream (iteration) cannot be started using the same file. To solve this problem, you can get these files immediately before starting a session, e.g. using a web protocol.
If a sampler is unable to gain control of a session within 120 seconds, it will attempt to logout and disconnect the session for another 60 seconds. Then another attempt is connect will be made automatically.
If in rare instances an error occurs during screenshot-based synchronization, don’t worry. Try adding sampler the checksum obtained during playback (you can view the checksum on the Response Data tab in the View Results Tree, if the error occurred during synchronization).
If one of several actions is required based on what is displayed on the screen, you can use synchronization without a timeout and conditional controllers in the cycle in order to execute the desired action.
When using Citrix Receiver v4.4+, you cannot start a session with “direct” connection parameters. Only through .ica files.
Advantages of using Citrix during performance testing
Using Citrix technologies in performance testing has its advantages over traditional approaches.
During this type of testing, interaction with the system takes place through the client application. Data sent from the client to the server are not emulated, but rather the user’s actions on the client. As a result, there is no need to analyze the details of the client’s interaction with the system. And this increases information security, makes it possible to test systems with an extremely complex/closed protocol, and saves time on development of test scenarios. This approach also provides the ability to assess the client application’s perfomance (for example, the time require to render information, the time required to write data to disk, etc.). What’s more, you can test the scalability of the Citrix farm itself.
The main deficiency is that to achieve universality you have to pay for load bench capacity and a Citrix farm. This technology works well when you have hundreds or thousands of users, but when there are tens of thousands or hundreds of thousands — alas, you have to think of something on the protocol level. (We’ll tell you about this another time.)
Choosing a performance testing tool is crucial when creating a testing strategy. The individual specifics of the system being tested, the protocols being used, and the load on the system are highly important.
If a testing project involves Citrix, then we can boldly state that we can perform the testing using our in-house tools just as well as HP LoadRunner, which can yield significant savings on testing costs.
This extension is a truly “fresh” solution, which was developed at the end of 2015 but has already proven itself well in performance testing at a major bank.
A comparison with other solutions revealed no material disadvantages, so if the difference is trivial, why pay more?
About the author
Alex Makarov is an IT expert with more than 15 years of experience. Alex is the Head of load and performance testing department at Performance Lab, a global company specializing in software testing and quality assurance. His main focus is to ensure that complex information systems perform well and to deploy new technological solutions in the area of software testing.