Challenge

Every year dozens and dozens of major online retailer’s experience performance problems when it matters most back to school days, Black Friday, Cyber Monday, the holiday shopping season. And it is not like they have not been performance testing for peak traffic loads since the summer.

Most often the problem is that they are encumbered by an old-fashioned, difficult-to-use, manual labour-intensive load, and performance testing solution. As peak loads get more intense and eCommerce systems get more complex, performance testing gaps widen and risks multiply.

• Their business priority was to reduce “Go-To-Market Time”, so the Testing Cycle needed to be reduced
• Testing had to be done across Multiple client locations, environment, and browsers
• The Performance suite size was increasing with each release due to new functionality leading to increased test effort

Our Solution

IGT proposed Performance testing approach for faster test execution and reduced cycle time

• IGT propose complete system study to identify every transaction TPS
  LoadRunner Performance testing automation tool was used for automation.
• A hybrid automation framework was designed using LoadRunner.
• Execute the same script against multiple datasets and environment
• Compatibility Test – Run the same script on IE, Chrome & Firefox
• Only Test data sheets needs to be updated
• Easier maintenance and Continuous Integration
• Customized reports and remote execution
• Concurrent execution of test script using LoadRunner which reduces the execution time
• Every release Performance test has been benchmarked.
• In addition to this, performance of critical bugs related scenarios was also done

Phase 1 –Planning and development of Performance scripts

• Load Runner installation
• Knowledge transfer from QA team
• Prepare Performance Test plan
• Scripting Scenarios
• Business Process Strategy preparation
• VUser Specification document

Phase 2 – Execute, Bottleneck Identification

• Project downtime notification / Black Friday test
• Environment Preparation
• Site Administration Freeze
• Phased Backup of Project Database
• Execution of the scripts and results analysis
• Recommendation of fixes

Phase 3 – Re-execution of the Scenarios

• • Environment Preparation
  • Project downtime notification
  • Site Administration Freeze
  • Phased Backup of Project Database
  • Re-execution of the scripts and results comparison
  • Performance Summary Report preparation
  • Handover

The Change

Performance issue resolved, identified major roadblocks

Find below benefits of technical solution proposed to the client

• Traced the transactions taking long time to respond
• Found missing indexes on Database
• Code was optimized to reduce time taken to load page
• SQL queries were tuned
• Garbage Collection time reduced to free memory
• Heap size increased
• Clients saw significant improvement in the transaction response time with the implementation of changes when comparing the results from the first round of execution with the second round.
• Increased the productivity of the team for manual testing by 40%
• Good number of critical and major defects have been identified by the Performance test scripts
• Scripts are executed in different browsers and various environments by just changing the required URL parameters in spreadsheet.
• HTML Report logs, spotlight and snapshots are generated for tracking defects and exceptions
• Framework served at the enterprise level

Technologies

Technology Platform LoadRunner, Spotlight, Web HTTP/S, Selenium WebDriver, JAVA, Winium, Sikuli, Robot class/ TestNG/ Maven