Innovative Test Global

PERFORMANCE ENGINEERING

Performance engineering is proactive, continuous, and end-to-end application performance testing and monitoring. It allows seamless collaboration between teams, tools, and processes through continuous feedback loops. Here, it’s not just testers who are responsible for quality assurance but developers, performance engineers, product owners, and business analysts as well.

“Performance engineering is a cultural shift in the way organizations view their essential processes. Performance engineering embraces practices and capabilities that build quality and performance throughout an organization." - Joe Colantonio of Test Guild Podcast: Six Top Trends for Performance in 2020 and Beyond

By leveraging right-sized tools spanning developers to performance engineers, Performance Engineering enables shift-left performance testing and shift-right application performance monitoring. It’s difficult to appreciate just how much of a departure performance engineering is from traditional performance testing without understanding classic performance testing is.

What is the Difference Between Performance Testing and Performance Engineering?

Performance Engineering

Classic performance testing is effectively a subset of performance engineering. It usually entails running a single round of load testing as part of the post-development quality assurance (QA) cycle. Performance testing involves checking the speed, reliability, scalability, stability, response time, and resource use of an application under the anticipated workload. Before we get into the differences between performance engineering and performance testing, we’ll first take a look at performance testing in isolation and why it’s, by itself, no longer sustainable.

First, the testing is viewed in isolation and treated as an afterthought that only begins at the end of functional testing.

Second, working in silos causes large communication gaps between project sub-teams and prevents the collaboration required to deliver a high quality product.
Third, by the time performance testing kicks in, the organization has already sunk substantial time, effort, and finances into the application’s design, development, and promotion. Fourth, performance testing is often treated as an afterthought and isn’t included in the ‘done’ criteria preceding release. So at this juncture, the business needs the app in production urgently and expects no delays. In this context, QA's feedback happens too late to fix completely before the release. Inevitably, a high number of performance problems would unnecessarily make it to the production environment just so the release remained within schedule. Fixing a defect in production is much more expensive and disruptive than doing so early in development.

Fifth, traditional performance testing may have been perfect for the Waterfall model but is out of place in today’s DevOps-centric world. DevOps reduces the failure rate of new releases by shrinking the time between when a change is committed to the system and when the change is placed in production. Continuous integration and continuous delivery (CI/CD) makes sure the software is always in a releasable state throughout its lifecycle. DevOps also focuses on realigning the organization to support end-to-end collaboration between stakeholders, functions and tools. To meet the rapid delivery demands of DevOps, software development needs a more evolved performance testing approach. That new approach is software performance engineering.

Now, let’s delve into the key differences between performance engineering vs performance testing.

First, performance testing is a quality check of the application’s load handling and responsiveness. It establishes how well the system will bear a production load and anticipates the issues that could come up during heavy load conditions. Performance engineering seeks to design the application from the start with performance metrics in mind and facilitate the discovery of issues early in development.

Second, performance testing is a QA process that usually takes place when a round of software development is complete. Performance engineering is a continuous process that is embedded in all phases of the software development cycle – from design, to development, and into the end-user experience. Third, performance testing is conducted by the QA team while performance engineering involves RND and QA.

Innovative Test Global

Performance Engineering Concepts

Through the following concepts, DevOps and performance engineering deliver consistent production performance results, allowing customers to deploy applications efficiently with more confidence and to roll out high performing, stable software that fulfil user expectations.

End-to-End Optimization

Performance engineering delivers end-to-end system optimization through a continuous testing and monitoring process. This shifts-left performance and load testing into the development process. That’s unlike traditional performance testing where the testing took place after functional testing was stabilized and the code released. Once the code is released, performance engineering tracks the app in production by tapping into application performance monitoring (APM) tools.

Cross-Functional Team of Performance Stakeholders

Performance engineering enables collaboration among project stakeholders – from business analysts to developers. Maintaining high performance levels that enhance the customer experience, keeping up with the pace of business, and managing end-to-end performance makes everyone, not just QA/Performance Engineers, a steward of product performance. Here’s how.

Testing Center of Excellence

The testing Center of Excellence (CoE) serves as the trusted testing advisor and the custodian of best practices. The CoE supports different business units, different testing methods (such as DevOps and Agile), and has the flexibility to recommend performance testing and testing tools as needed. To build better testing models and improve test quality, the CoE acts as the single point of consolidating and reusing test data that has been generated and collected across multiple business units over time.

Performance Engineer

The performance engineer provides a holistic view of all code in development to ensure performance testing criteria is comprehensive, encompasses the bigger picture, and factors all distinct pieces of code in development. The performance engineer is the primary user of performance testing tools and has a high degree of expertise in scripting, designing, running, and analyzing test results. Performance engineering brings the performance engineer to the early phase of development where they can provide performance metrics and scenarios required for code to be considered ready for release. Early involvement means the performance engineer can ensure the solution satisfies the performance expectations set out at the beginning of development. They also confirm that the architecture and design is consistent throughout development.

Software Developer

Developers are experts at coding but often light in both functional and performance testing. They work in their integrated development environment (IDE) and tend to use their preferred tools with little inclination toward learning new ones. Performance engineering shifts-left performance testing which brings it into the software developer’s area of responsibility. With input from the performance engineer, software developers can run performance testing while writing code. The developer doesn’t release their code before it passes performance testing criteria.

DevTester

The devtester doesn’t exist in classic performance testing since there are clear distinctions between software developers and performance engineers. With performance engineering, the devtester emerges as a stakeholder that connects performance engineering and developer teams. They bridge the gap by having solid coding and testing skills though not quite at the same level of expertise as the developer and the performance engineer. They can run tests quickly and have much greater flexibility than developers in using different tools as is needed.

Business Analysts and Application Engineers

By shifting-right the testing, performance engineering brings on board business analysts and application engineers. This guarantees that business and application performance requirements defining the quality of the user experience are incorporated into the performance criteria. These two roles monitor the app in production to ensure top-notch application performance at all times.