By Alex Wissing
QA Engineer II
As Agile methodologies and CI/CD (Continuous Integration and Continuous Development) systems have become much more common and are structured around shorter delivery cadences, there is less time for a QA (Quality and Assurance) department to test software and ensure it is production-ready.
Many teams have solved this problem by automating tests that otherwise would have been manually performed by a member of the QA department. Manual testing can be tedious, time-consuming, error-prone, and in more complex projects, unattainable. Automated testing is an increasingly valuable tool for teams, and there are many different strategies for implementing test automation on a project. Let’s look at high-level overviews of two popular CI/CD pipelines.
For those not familiar with CI/CD pipelines, this is a helpful analogy: think of the CI/CD pipeline as a series of locked gates. Let’s say you have created a new navigation feature for your project. You need the right combination of keys to unlock all of the gates and have your new feature be available on your production project. Each step in the pipeline will either fail and stop you from proceeding or pass and allow you to continue to the next gate.
Pipeline example #1 is simple, but the popular structure, especially for smaller companies or teams. In this method, code is written, unit tested, deployed for testing, undergoes functional testing, and is then released to the production environment. Unit tests can be automated but are written by the developer and only test the code one function or “unit” at a time. The real benefit of automated testing comes into play during the functional testing step. Automated functional tests are built to verify that the intended functionality of the new feature works correctly. How is this accomplished? By using a testing framework that programmatically mimics how a user would interact with the application. Modeling the testing framework after typical human behavior ensures that the tests cover the most common user journeys.
The next crucial phase is the regression test step. The goal here is to verify that integrating the new feature has not negatively affected other areas of the site/project. This structure works well for companies or projects that do not have a large number of third party dependencies or micro-services that need to be mocked up in a test environment. After all, building something new while breaking a handful of other existing parts of your codebase is a classic example of taking one step forward and three steps backward.
Pipeline example #2 is well suited for larger companies/projects that have many third party dependencies and/or microservices. It consists of two sequential pipelines: the Build Pipeline and the Release Pipeline. The build pipeline is similar to the previous example but does not deploy directly to production. Instead, it triggers the release pipeline for the newly checked-in code to be tested in multiple “release management” environments. These release management environments can then be used to validate the new code/feature against the project’s third-party dependencies or microservices.
Generally, there are three environments associated with this release pipeline structure: Local/Development, Staging/Testing/QA, and Production. These are all just versions of the same project built for different testing purposes.
The Development environment step is where new features are first built and integrated with the rest of the project. This environment usually has its own database and server instances which have the same structure as the production instances, but different test or “fake” data. At this stage, developers also verify unit tests and ensure there is sufficient code coverage.
The Quality Assurance environment will also closely mimic the production environment. This ensures that any functional and/or integration tests performed are as accurate and as meaningful as possible. One benefit of this pipeline strategy is that if there is an issue with the new feature, it is easier to isolate and troubleshoot the issue in the QA environment before it is promoted to production. This is also where regression testing comes into play to ensure the new feature did not adversely impact any existing functionality. When performed manually, this type of test can be incredibly expensive because of the high number of potential test path variables. This is a great example of how test automation can be a huge timesaver.
The Production environment is where real users will interact with your project. The new feature has passed through all the gates of the pipeline and has been released into the wild! If your project is a website, the production environment is the actual site that the public interacts with! There are further strategies for testing in the production environment, such as feature flagging, which allows you to control which users have access to new features. This can be hugely beneficial in situations where it’s important to validate the viability of a new product feature or to ensure that that your infrastructure can bear an increased amount of traffic.
The major benefits of a CI/CD pipeline with automated testing are the speed of development, isolation of environments, and keeping production data clean while ensuring a high-quality end product. When implemented correctly, creating a CI/CD pipeline with automated testing can have positive rippling effects for a project’s success as well as optimize for faster delivery without sacrificing quality.
If you're interested in learning more about Techtonic please Contact Us today! Techtonic is the premier partner for developing onshore, commercial-grade software. We've also developed a unique method to simultaneously close the talent gap that exists within technology, by tapping into a new and highly diverse workforce. Our model is pioneering the way clients develop and hire top talent, and we'd love to work with you!