Functional Test Automation

Background

Every year dozens and dozens of major online retailers experience performance problems when it matters most back to school days, Black Friday, Cyber Monday, and the holiday shopping season. And it is not like they have not been performance testing for peak traffic loads since the summer.

Most often the problem is that they are encumbered by an old-fashioned, difficult-to-use, manual labor-intensive load, and performance testing solution. As peak loads get more intense and eCommerce systems get more complex, performance testing gaps widen and risks multiply.

• Their business priority was to reduce “Time to Market”, so the Testing Cycle needed to be reduced
• Testing had to be done across 13 Caribbean Counties, environments, and browsers
• The Performance suite size was increasing with each release due to new functionality leading to increased test effort

Our Solution

IGT introduced a dynamic Performance Testing approach aimed at expediting test execution and reducing cycle time, fostering an environment of enhanced efficiency. At the core of this approach was the meticulous examination of the complete system, ensuring comprehensive identification of every transaction per second (TPS). The LoadRunner Performance Testing automation tool served as the linchpin, enabling a seamless amalgamation of precision and automation.

This innovation was further fortified by the design of a hybrid automation framework, leveraging LoadRunner’s capabilities to execute identical scripts across multiple datasets and diverse environments. Compatibility was ensured through cross-browser testing, encompassing IE, Chrome, and Firefox. Maintenance was streamlined by only updating test data sheets, facilitating effortless Continuous Integration.

The strategic incorporation of customized reporting and remote execution provided real-time insights, crucial for informed decision-making. Significantly, concurrent script execution via LoadRunner drastically curtailed execution time, optimizing the testing process. Each release’s Performance test was rigorously benchmarked, ensuring consistent and evolving standards.

Furthermore, the comprehensive scope extended to include the performance evaluation of critical bug scenarios, underscoring IGT’s commitment to exhaustive testing. The synergy of meticulous planning, state-of-the-art tools, and strategic execution culminated in a streamlined, efficient, and effective Performance Testing solution.

Phase 1 –Planning and development of Performance scripts

• Load Runner Installation
• Knowledge transfer from the QA team
• Prepare a Performance Test plan
• Scripting Scenarios
• Business Process Strategy preparation
• VUser Specification document

Phase 2 – Execute, Bottleneck Identification

• Project downtime notification / Black Friday test
• Environment Preparation
• Site Administration Freeze
• Phased Backup of Project Database
• Execution of the scripts and results analysis
• Recommendation of fixes

Phase 3 – Re-execution of the Scenarios

• Environment Preparation
• Project downtime notification
• Site Administration Freeze
• Phased Backup of Project Database
• Re-execution of the scripts and results comparison
• Performance Summary Report preparation
• Handover

Tech Stack used: LoadRunner, Spotlight, Web HTTP/S, Selenium WebDriver, JAVA, Winium, Sikuli, Robot class/ TestNG/ Maven

Impact

By successfully resolving performance issues and overcoming major roadblocks, our proposed technical solution brought forth a multitude of benefits for the client. The approach involved meticulous tracing of transactions with extended response times, uncovering missing indexes in the database, optimizing code to enhance page load speed, and fine-tuning SQL queries. Additionally, we mitigated memory-related bottlenecks by reducing garbage collection time and increasing heap size.

The outcome was striking: the client saw substantial enhancement in transaction response times when comparing the results from initial and subsequent rounds of execution. Notably, manual testing productivity soared by 40%, and Performance test scripts unearthed a considerable number of critical and major defects. The scripts’ versatility allowed execution across various browsers and environments through simple URL parameter adjustments. Detailed HTML reports, highlighting, and snapshots facilitated effective defect tracking and exception management. This comprehensive solution operated seamlessly at an enterprise level, culminating in improved performance, heightened efficiency, and enhanced overall software quality.