Go back to all articles

What is a Regression Test? Definition & Examples

May 17, 2019
9 min read

During website or application maintenance, developers are often forced to change their code or add new features. Unfortunately, after code modifications, the software does not always work as it did before – system collapses and crashes are a challenge to watch out for.

Moreover, one code change can slow down the performance of the entire project, reduce page load time or increase the usage of system resources. To prevent negative effects after a software change, development teams run regression testing.

Read this blog post to find out what  regression testing is, why it is important, and what it consists of. We will also share some tips and tools for successful regression testing.

What is Regression Testing?

IT professionals define regression testing as a part of software testing designed to determine if a system is crash-resistant and functional after a code change. At this stage, a tester re-executes a set of cases they ran during the initial development stage to ensure that there was no negative impact. It’s important to test not just the part of the code that has been modified or a newly added feature, but the entire system.

To cut the cost and the number of hours needed for full regression testing, most companies run automated test sessions. This way, they can maintain higher precision and reduce the number of man-made errors, as well as run uninterrupted testing 24/7.

Regression Testing in Software

Regression testing is a crucial part of software maintenance. Its main purpose is to find bugs in the overall system that have been overlooked after the introduction of a new feature. Here is an example of regression testing in software:

Example
App A is a database management tool. There are three basic functions – Add, Save, and Delete – that allow users to enter data or delete a row. In a new build, an ‘Update’ feature has been introduced to allow users to edit the changes and save the input. During regression testing, a QA specialist will have to determine if the introduction of a new feature has impacted the way the ‘Add’, ‘Save’, and ‘Delete’ buttons work.

There are several approaches to regression testing. Here is a brief rundown of the most widely used techniques.

  • 01 Retest everything
    This approach implies that all the tests of the system should be re-executed. While it’s the safest way to ensure the project is bug-free, it takes a lot of time and commitment to run a full suite of tests. That’s why the ‘retest everything’ practice is rarely used among testers and, in the case where a team decides to go with it, the sessions will most likely be automated.
  • 02 Regression test selection
    By selecting a subset of existing test cases, a QA specialist can cut operating costs tremendously, compared to retesting the entire system. There are several practices testers use to select a case of regression test sessions.

    To start with, you can only test a suite that yields coverage to the modified section of the original program. Another popular approach is a Safe Technique where a tester works with a number of cases that expose one or multiple faults in the modified program. Other approaches to test selection include the Data Flow Coverage Technique and the Random Technique.
  • 03 Prioritization of test cases
    This approach allows a QA specialist to focus on testing the most frequently used functionalities and cases that have a crucial business impact while temporarily putting all the secondary features aside. By prioritizing test cases, you will cut the size of the testing suite tremendously and have more time to thoroughly assess the performance of the crucial parts of the system.

Who Needs Regression Testing?

Unfortunately, it’s hard to imagine a product that would never need to change. To stay relevant and attract more users, developers have to upgrade their projects with new features, change the back end to make the tool’s performance more effective, and adapt to managing a bigger amount of incoming traffic.

Maintaining a software product without regression testing will result in massive tech debt and a decrease in the level of user satisfaction.

For developers, regression testing usually means the following:

  • 01
    Ensuring a bug-free performance after changing the back-end code. Introducing a new feature can impact the entire system – and not necessarily in a good way. Testers run regression testing sessions in order to make sure that the performance of the system didn’t take a hit after as much as a small code modification.
  • 02
    Testing the performance of the application after adding a new feature. Regression testing allows developers to know that new functionalities align well with the old ones, that the infrastructure of the product is capable of executing more complex actions without losing in load time speed or crashing, and so on.
  • 03
    Detecting and fixing performance issues. Even if you did no major code changes, it’s still wise to run a regression testing session in case a performance defect has been recorded. Coming back to retest the original test suite allows developers to save time as they don’t have to write new cases.
  • 04
    Defining the parts of an application that are at the highest risk of failure. Regression testing will help the development team understand which parts of the system are most vulnerable to changes and have the highest odds of crashing. Testers will know to pay more attention to the maintenance of these features in the future.
Have a Project in Mind?​
We have been working on performance testing projects since 2008.
Drop us a line to find out what our team can do for you.
Get a quote You’ll hear back from our tech account manager in one day if not sooner

Types of Regression Testing

There is no single approach to regression testing. Apart from the techniques discussed above (those that have to do with the size of the test suite), there are a few more types of regression testing. Let’s take a look at some widespread approaches:

  • 01 Corrective regression testing
    This approach is used if the code hasn’t been changed significantly. For such sessions, developers rarely write new test cases – instead, they prefer reusing the old ones.
  • 02 Progressive regression testing
    This approach is used to test the impact of a new component on the system. To use progressive regression testing, team members should be well aware of the exact number and the nature of code changes. For this type of testing, new cases are written.
  • 03 Selective regression testing
    In this case, a tester chooses a range of test cases to speed up the progress. While this approach is a good way to put less money and effort into retesting, it’s quite challenging for developers to set the conditions between the experiment and the range of covered program elements.
  • 04 Complete regression testing
    This approach is normally used when the development team struggles to define the number of changes made or the impact of these modifications. Complete regression testing gives a QA professional a complete snapshot of a system as a whole. Normally, this is deployed in the final stages of development before the release of a build.

How to do Regression Testing

Creating a strategy during the early stages of development and aligning with it until the product release is a good way to do regression testing. The good news is, building a testing framework is relatively straightforward. Here are the steps QA specialists normally take to get started.

Step 1. Gather tests for execution

The first step in designing a regression test strategy is collecting all cases a QA specialist intends to re-execute. Here are a few tips on smart test selection:

  • 01
    Include cases in error-prone areas of the program, as they are likely to be most vulnerable to system changes as well.
  • 02
    Add cases that verify the main functions of the product. This includes the homepage, the login page, the checkout gateway, etc.
  • 03
    Include complex cases, such as GUI event sequences.

Step 2. Estimate the time for test cases execution

Be sure to estimate the time needed to test every chosen feature. Keep in mind that, apart from a session, your testers might need to take some time to get to know the range of tools used to execute and report particular tests and add them to the schedule. Here are a few other factors that can influence the amount of estimated time for testing:

  • 01
    Creation of test data
  • 02
    Regression test planning (especially for a beginning QA specialist)
  • 03
    Reviewing test cases

Step 3. Outline which tests can be automated

Automated tests are faster and more reliable than manual ones. In the long run, you’ll be able to reuse such scripts for your next project – it improves the efficiency of software maintenance and creates a set of standards within the team.

When it comes to regression testing, developers tend to automate most cases. However, if you’re looking at a complex sequence of events – it’s better to execute a manual check. The same stays true for all GUI-related cases – here, manual testing is often the only option.

Dividing manual and automated tests into two separate groups is the best way to avoid miscommunication within the team and keep reports in order.

Step 4. Prioritize test cases

It’s always helpful for a tester to determine which cases are the most relevant for the program and focus on executing them as a first priority. To manage sessions productively, it is crucial to prioritize. Here’s a simple framework you can follow while grading the value of test cases.

  • Priority 0
    All the sanity test cases fall into the category. The tests of the basic functionality of the product and pre-system acceptance are the first a QA specialist should concentrate on, as they provide the most value both for users and engineers.
  • Priority 1
    If your program has features that are crucial but not core (in other words, a tool would still work without them but the performance wouldn’t be satisfactory), the cases to test them fall under Priority 1 and are to be handled as soon as all the scenarios labeled as Priority 0 are checked.
  • Priority 2
    Includes test cases that are not providing high project value but are crucial to avoid tech debt and complications for developers. On a user’s side, the impact of these features is not noticeable.

Step 5. Use tools to speed up the testing process

There is a wide range of tools for regression testing that help QA specialists handle planning, preparation, and reporting. Using these off-the-shelf solutions allows the team to speed up the process and use the best practices of regression testing.

Here are some tools developers can consider using to improve the efficiency of testing:

  • Selenium – a portable framework for web application testing;
  • Watir – a tool that enables automated testing of Ruby-based apps;
  • Rational Functional Tester – an automated testing tool that skillfully mimics the actions of a human tester.

Examples of Regression Tests

There is no single approach to regression testing. Apart from the techniques discussed above (those that have to do with the size of the test suite), there are a few more types of regression testing. Let’s take a look at some widespread approaches:

  • 01
    Bug regression – a tester checks if a specific bug that has allegedly been fixed was in fact eliminated.
  • 02
    General functional regression – a range of broad tests across all areas of the app aimed at ensuring that recent changes have not resulted in code destabilization.
  • 03
    Conversion and port testing – a suite of test cases executed to ensure that the application has been successfully ported to a new platform.
  • 04
    Localization testing – in case a program has been modified and rewritten in a new programming language, a tester assesses the performance of the interface and ensures that the application follows its new set of cultural rules. To execute such a test, you may have to modify old cases taking the change of a programming language into account, or even write new ones.
  • 05
    Build verification testing – a series of small tests aimed at verifying if a build is worth fixing or if the damage is irreparable. A failed test would result in a build rejection.

Regression Testing at PFLB

In case you’re looking for a team of qualified regression testing specialists, consider contacting PFLB. We have a team of ISTQB-certified testers experienced in creating coverage for regression testing, prioritizing and executing test cases, and giving a comprehensive report regarding the ways to improve the build.

We use a large number of tools to run test cases – the list includes (but is not limited to) TestLink, JIRA, HP ALM, Microsoft TFS, and many more. To provide companies with agile maintenance, at PFLB we run continuous system monitoring of nightly and weekly builds.

Take a look at the full list of services offered by PFLB. If you want to work with a team of experienced testers, be sure to contact us.

Table of contents
Let us know about your needs
We can provide multiple performance testing services and a lot more than that if the situation needs a far more complex approach.
Get a quote You’ll hear back from our tech account manager in one day if not sooner

Related insights in blog articles

Explore what we’ve learned from these experiences
11 min read

Roles and Responsibilities of Performance Tester

performance testing roles and responsibilities in a nutshell
Apr 9, 2024

The core of efficient performance testing is an experienced, certified and well-structured team of professionals, who have already worked in similar projects, and learned all the peculiarities of QA testing types, and protocols.  If consciously chosen, they can evaluate a product, test its behavior under a load, check its response time, etc., and thus, empower […]

11 min read

Tips for Efficient Web Application Performance Testing

tips for efficient web application performance testing
Apr 4, 2024

Performance testing is one of the most challenging components of web application testing. But it is worth it: if done right, performance testing can forecast product behavior and its response to user’s actions with an impressive amount of detail and precision. Irrelevant web performance test data or wrong metrics chosen during testing cost companies a lot of money and effort, while not monitoring your web application performance at all can lead directly to a crash, say, on Black Friday, if you are not prepared to handle an increased number of concurrent users. Business or product owners needn’t know exactly how to performance test a website, but it’s useful to have a basic understanding of procedures that you pay for. From this post, you can learn what web application performance testing is, when it should be done, and how. Then, you will be able to make an informed choice whether to hire testing specialists or run testing sessions on your own.

15 min read

Top Tools for Developers in 2024

top developers tools
Apr 2, 2024

The ratings of the best software development tools are time-sensitive: new tools emerge every now and then. Monitoring and mastering them is a full-time job, and sometimes a waste of time, too, since some oldies but goldies remain chosen by the team.  Having done all the hard choices for you, we are sharing our list […]

4 min read

Big Data Europe 2024

software testing conferences preview
Mar 22, 2024

Big Data Conference Europe is a four-day conference with technical talks in the fields of AI, Cloud and Data. The conference will take place both on-site and online, providing the opportunity for everyone to participate in their preferred format.

  • Be first to know

    Once a month we’ll be sending you a letter with all the useful insights that we could find and analise